Should nil COLLECTION element mean a "nil" candidate for completing-read? Alist doc.

Should nil COLLECTION element mean a "nil" candidate for completing-read? Alist doc.

[Drew Adams]

This change was introduced in Emacs 22. Dunno if it was deliberate. I wonder
whether it is a good change. I tend to think not, but perhaps there is some
special use case for it. In any case, if it is to be maintained, and not
considered a bug, then the doc is inadequate and confusing.

(completing-read "ff: " '(nil ("a") ("b)))
or
(completing-read "ff: " '(nil "a" "b))

The completion candidates are "nil", "a", and "b". The string "nil" is returned
if you choose that candidate (a string is always returned).

In Emacs prior to 22, the second form above was not possible, and the first form
produced only the candidates "a" and "b". That makes sense, to me.

The doc describing the COLLECTION arg to `completing-read', `try-completion',
etc. does not call out this special treatment of nil. In fact, it does not even
say that you can mix cons entries and string entries (let alone nil entries). It
says that the COLLECTION can be a list of strings or an alist - it is neither,
in these examples.

Not only that, the doc defining alists (Elisp manual, node Association Lists)
says that each entry is a cons - nil entries are not mentioned.

Well, after saying that the elements are conses, it says that atomic elements
don't raise an error - but they are ignored. IOW, an alist is a list of conses,
but non-conses are passed over in silence. They are certainly not treated as
actual alist elements (content) in any way.

But then the same node documents function `assoc-default' and its arg ALIST,
mentioning special treatment of atomic ALIST elements. That contradicts what is
said earlier in the same node.

If an alist is supposed to contain conses, and any non-conses are supposed to be
ignored (neither an error nor actual data), then both `completing-read' and
`assoc-default' should not speak of alist args.

In any case, besides pointing out the doc confusion, my main question is about
`completing-read': why the special treatment for nil entries (treating them as
if they were the string "nil")?

RE: Should nil COLLECTION element mean a "nil" candidate forcompleting-read? Alist doc.

[Drew Adams]

I forgot to point out that treating nil this way can lead to extra (ugly) code,
just to remove or prevent production of such nil entries, so they are not
available to the user.

IOW, I suspect that the number of times where this "feature" might be useful
(for what?) is less than the number of times where it is a nuisance.

A typical case would be mapping a function that can return nil over a list. With
the current behavior, either an extra, nil-removing pass is needed or an
iteration is needed that is a bit more complex (ugly) than a simple mapping.

Re: Should nil COLLECTION element mean a "nil" candidate forcompleting-read? Alist doc.

[Stefan Monnier]

> I forgot to point out that treating nil this way can lead to extra
> (ugly) code, just to remove or prevent production of such nil entries,
> so they are not available to the user.

A completion table can be a list of strings, a list of conses where the
car is a string, an obarry, a hashtable, or a function.  Your "table" is
none of those, so it's invalid.  Adding (delq nil ...)  before passing
that table to the completion functions doesn't seem like a big issue.

> IOW, I suspect that the number of times where this "feature" might be
> useful (for what?) is less than the number of times where it is
> a nuisance.

It's not a feature.


        Stefan

RE: Should nil COLLECTION element mean a "nil" candidateforcompleting-read? Alist doc.

[Drew Adams]

> > I forgot to point out that treating nil this way can lead to extra
> > (ugly) code, just to remove or prevent production of such 
> > nil entries,
> > so they are not available to the user.
> 
> A completion table can be a list of strings, a list of conses 
> where the car is a string, an obarry, a hashtable, or a function.
> Your "table" is none of those, so it's invalid.  Adding
> (delq nil ...)  before passing that table to the completion
> functions doesn't seem like a big issue.

First, it's not "my table". ;-)

More importantly:

1. Adding `delq' means a separate pass over the list (which can be big).
Preventing inclusion of nil, as an alternative, requires uglier, less clear code
(e.g. code that pushes an element to the list when it is non-nil).

2. Saying that users can do that as a workaround ("y'a qu'a") is a cop-out. This
is not documented - it's simply a gotcha. Users won't even know they must do
that, except through accidental discovery by trial and error (which is how I
discovered it). It is all too easy to create a list that might have some
extraneous nil elements - and not even be aware of them. Hence, this is also
error-prone.

> > IOW, I suspect that the number of times where this 
> > "feature" might be useful (for what?) is less than
> > the number of times where it is a nuisance.
> 
> It's not a feature.

Right; it's a bug - a regression wrt Emacs 21, in fact (you admit it wasn't
added intentionally, as a feature, and it is a user-visible change in behavior).
It is undocumented behavior that contradicts the documented behavior. The
general doc for alists says that nil entries will be ignored. Exceptions to that
should be documented.

You want to neither fix the code (so it will ignore nil entries) nor document
this change in behavior (since Emacs 21). That's not right.

I pose the questions: Should nil be ignored here? Should nil be treated like the
string "nil" here? Those are "should" questions. It's not sufficient to avoid a
response and take the view that the behavior is what it is and should not be
documented. I'm asking what the behavior _should_ be. And the doc should
describe what the function does - _especially_ considering any special
treatment, which is not obvious.

Re: Should nil COLLECTION element mean a "nil" candidateforcompleting-read? Alist doc.

[Stefan Monnier]

>> It's not a feature.
> Right; it's a bug - a regression wrt Emacs 21, in fact (you admit it wasn't

No.  It's a change in undocumented and unsupported use-case.
We sometimes try to preserve bug-compatibility, but not always.

> I pose the questions: Should nil be ignored here? Should nil be
> treated like the string "nil" here?

If we wanted to give it a defined behavior, I'd vote for "signal an
error".  But until proven otherwise, I think it's not worth the trouble.


        Stefan

Re: Should nil COLLECTION element mean a "nil" candidate for completing-read? Alist doc.

[Stefan Monnier]

> (completing-read "ff: " '(nil ("a") ("b)))

A nil there is simply an error.  We don't signal an error, but the
none of the behaviors you may see is considered as a feature.


        Stefan

RE: Should nil COLLECTION element mean a "nil" candidate for completing-read? Alist doc.

[Drew Adams]

> > (completing-read "ff: " '(nil ("a") ("b)))
> 
> A nil there is simply an error.  We don't signal an error, but the
> none of the behaviors you may see is considered as a feature.

What does that mean? Are you simply saying "don't do that" - users must ensure
that there are no nil elements in the COLLECTION arg?

Or are you saying that it is a `completing-read' bug that should be fixed - it
should ignore nil elements in its COLLECTION arg, instead of treating them as
equivalent to the string "nil"?

And what about the rest of my post, regarding the doc contradictions and
confusion wrt alist elements?

Re: Should nil COLLECTION element mean a "nil" candidate for completing-read? Alist doc.

[Stefan Monnier]

> What does that mean? Are you simply saying "don't do that" - users
> must ensure that there are no nil elements in the COLLECTION arg?

Yes.

> And what about the rest of my post, regarding the doc contradictions and
> confusion wrt alist elements?

Same idea:

> The doc describing the COLLECTION arg to `completing-read',
> `try-completion', etc. does not call out this special treatment of
> nil.  In fact, it does not even say that you can mix cons entries and
> string entries (let alone nil entries).

Indeed, it doesn't say you can do it.  So if you do it, you're on
your own.


        Stefan

RE: Should nil COLLECTION element mean a "nil" candidate for completing-read? Alist doc.

[Drew Adams]

> > What does that mean? Are you simply saying "don't do that" - users
> > must ensure that there are no nil elements in the COLLECTION arg?
> 
> Yes.

Too bad.

> > And what about the rest of my post, regarding the doc 
> > contradictions and
> > confusion wrt alist elements?
> 
> Same idea:
> 
> > The doc describing the COLLECTION arg to `completing-read',
> > `try-completion', etc. does not call out this special treatment of
> > nil.  In fact, it does not even say that you can mix cons 
> > entries and string entries (let alone nil entries).
> 
> Indeed, it doesn't say you can do it.  So if you do it, you're on
> your own.

The current behavior, which you apparently don't want to correct as a bug,
contradicts the behavior descibed by the doc. The doc says that alist treatment
ignores nil entries - and here it does not. Here nil is treated as if it were
the 3-character string "nil".

That special, undocumented behavior contradicts the existing doc. Based on the
doc, users have a right to expect nil elements in alists to be ignored. There is
nothing indicating that they should expect `completing-read' to treat nil as
"nil".

We are saying one thing and doing something quite different. Users of
`completing-read' have a right to a more faithful description of its behavior.
They shouldn't have to find out by trial and error how it treats certain
elements.

I propose that this be considered a bug (not a doc bug), and that it be fixed by
doing what the doc says: ignore nil entries. That will also simplify code -
users won't need to add extra code just to filter out nil or prevent its
inclusion. Currently, not only must such code be added, but it must be
commented, since this weird behavior that it works around isn't even documented.

The status quo is not good.

Re: Should nil COLLECTION element mean a "nil" candidate for completing-read? Alist doc.

[Stefan Monnier]

> The current behavior, which you apparently don't want to correct as
> a bug, contradicts the behavior descibed by the doc.

The behavior is necessarily different from the one applied to pure
alists, since the argument can also be a list of strings.

> The status quo is not good.

Your words don't carry much weight if they don't come with actual
concrete examples.


        Stefan

Re: Should nil COLLECTION element mean a "nil" candidate for completing-read? Alist doc.

[Kevin Rodgers]

Drew Adams wrote:

>>> The doc describing the COLLECTION arg to `completing-read',
>>> `try-completion', etc. does not call out this special treatment of
>>> nil.  In fact, it does not even say that you can mix cons 
>>> entries and string entries (let alone nil entries).
>> Indeed, it doesn't say you can do it.  So if you do it, you're on
>> your own.
> 
> The current behavior, which you apparently don't want to correct as a bug,
> contradicts the behavior descibed by the doc. The doc says that alist treatment
> ignores nil entries - and here it does not. Here nil is treated as if it were
> the 3-character string "nil".

The doc string and the info entry cited by Johan do not agree: the info
entry mentions the possibility of symbol elements.  And of course `nil'
is a symbol whose name is "nil".

> That special, undocumented behavior contradicts the existing doc. Based on the
> doc, users have a right to expect nil elements in alists to be ignored. There is
> nothing indicating that they should expect `completing-read' to treat nil as
> "nil".
> 
> We are saying one thing and doing something quite different. Users of
> `completing-read' have a right to a more faithful description of its behavior.
> They shouldn't have to find out by trial and error how it treats certain
> elements.
> 
> I propose that this be considered a bug (not a doc bug), and that it be fixed by
> doing what the doc says: ignore nil entries. That will also simplify code -
> users won't need to add extra code just to filter out nil or prevent its
> inclusion. Currently, not only must such code be added, but it must be
> commented, since this weird behavior that it works around isn't even documented.
> 
> The status quo is not good.

-- 
Kevin Rodgers
Denver, Colorado, USA

Re: Should nil COLLECTION element mean a "nil" candidate for completing-read? Alist doc.

[Johan Bockgård]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> (completing-read "ff: " '(nil ("a") ("b)))
>
> A nil there is simply an error.  We don't signal an error, but the
> none of the behaviors you may see is considered as a feature.

     If COLLECTION is an alist (*note Association Lists::), the
     permissible completions are the elements of the alist that are
     either strings, symbols, or conses whose CAR is a string or symbol.
     Symbols are converted to strings using `symbol-name'.  Other
     elements of the alist are ignored. (Remember that in Emacs Lisp,
     the elements of alists do not _have_ to be conses.)  In
     particular, a list of strings or symbols is allowed, even though
     we usually do not think of such lists as alists.

     -- (info "(elisp) Basic Completion")


(But this is not really true--the first element can't be a symbol other
than nil (or lambda).)

RE: Should nil COLLECTION element mean a "nil" candidate forcompleting-read? Alist doc.

[Drew Adams]

> >> (completing-read "ff: " '(nil ("a") ("b)))
> >
> > A nil there is simply an error.  We don't signal an error, but the
> > none of the behaviors you may see is considered as a feature.
> 
>      If COLLECTION is an alist (*note Association Lists::), the
>      permissible completions are the elements of the alist that are
>      either strings, symbols, or conses whose CAR is a string 
>      or symbol.
>      Symbols are converted to strings using `symbol-name'.  Other
>      elements of the alist are ignored. (Remember that in Emacs Lisp,
>      the elements of alists do not _have_ to be conses.)  In
>      particular, a list of strings or symbols is allowed, even though
>      we usually do not think of such lists as alists.
> 
>      -- (info "(elisp) Basic Completion")

I gotta say, this doc is poor, and on a very slippery slope (if not already
slipped to the bottom).

It's one thing to say that in Emacs Lisp a non-cons is tolerated in an alist.
More precisely, the doc says that it is not an error, but also that such
elements ARE IGNORED.

It's quite another thing to consider that alists are any lists whatsoever, that
it makes no difference what the elements are. 

A non-cons element is ignored is different from calling every list an "alist"
just because it is not an error for an alist element to be an atom. That's just
plain silly and confusing. Why have a notion of alist if every list is an alist?
Why say that atomic elements of alists are ignored if they are not?

> (But this is not really true--the first element can't be a 
> symbol other than nil (or lambda).)

Yes, indeed. There is so much undocumented special casing going on here that it
is a mine field for users. What's the point of documenting the behavior if that
doc is way off the mark and users need to discover the real behavior by trial
and error?