bug#4718: 23.1; C-h f gives doc for the wrong function

bug#4718: 23.1; C-h f gives doc for the wrong function

[Drew Adams]

emacs -Q
 
C-h f dired-byte-compile RET
 
The doc for a different function, `dired-do-byte-compile', is shown
instead.
 
Similarly, for other functions that do not have doc strings. E.g., it
shows the doc for `dired-compress-file' when you ask for
`dired-compress'.
 
This is silly. It was far better in Emacs 22 (21, 20, 19, 18,...),
where Emacs told you clearly that there was no doc for the function
requested.
 
It is too easy not to notice that Emacs is in fact giving you the
wrong thing - something different from what you requested. You assume
that Emacs DTRT and that what you see is the doc you requested.
 
Instead, you now need to pay close attention and double-check the
function name at the top of the *Help* buffer. This makes life harder
for users, not easier.
 
If a user asks about oranges, it's inappropriate to tell him/her about
apples, especially without even saying anything (We know nothing about
oranges, would you like to know about apples instead?).
 
This is a serious UI bug.
 
In GNU Emacs 23.1.1 (i386-mingw-nt5.1.2600)
 of 2009-07-29 on SOFT-MJASON
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (4.4)'
 

bug#4718: 23.1; C-h f gives doc for the wrong function

[Juanma Barranquero]

On Wed, Oct 14, 2009 at 01:49, Drew Adams <drew.adams@oracle.com> wrote:
> emacs -Q
>
> C-h f dired-byte-compile RET

  emacs -Q

  C-h f dired-byte-compile <RET> => [No match]

  M-x load-library <RET> dired <RET>
  C-h f dired-byte <TAB> => dired-do-byte-compile
  <RET>   => description for `dired-do-byte-compile', as expected.

  M-x load-library <RET> dired-aux <RET>
  C-h f dired-byte <TAB>  => dired-byte-compile
  <RET>  => description for `dired-byte-compile', as expected.

The difference being that `dired-do-byte-compile' is autoloaded, and
`dired-byte-compile' is not.

Which behavior were you expecting?

    Juanma

bug#4718: 23.1; C-h f gives doc for the wrong function

[Drew Adams]

> > emacs -Q
> > C-h f dired-byte-compile RET
> 
>   M-x load-library <RET> dired <RET>
>   C-h f dired-byte <TAB> => dired-do-byte-compile <RET>
>   => description for `dired-do-byte-compile', as expected.

I didn't hit TAB.  I hit RET:

M-x load-library RET dired RET
C-h f dired-byte-compile RET
=> description for `dired-do-byte-compile', NOT as expected.

I entered one entire function name. Emacs didn't complain that there was no such
function. Emacs instead silently gave me the doc for a different function.
That's totally inappropriate.

When I hit RET, Emacs should say `No match' and not accept my erroneous input,
as it used to do in Emacs 22 and before. If there is no function whose name is
`dired-byte-compile', then when I try to enter that name Emacs should tell me
that immediately: no such function.

In no case should Emacs go off and come back with the doc for some other
function - and without even showing me that other function's name first! That's
ridiculous. That is not user-friendly at all.

Imagine if you paste a complete URL in your browser and you get a totally
different Web site from what you request, the browser thinking that it is being
helpful because it notices some similarity between your URL and another that it
knew about.

Can you imagine your Web experience in that case? Imagine if your browser does
that each time you click a broken link: "helpfully" transforming the bad URL
into a different one that "works" - but that corresponds to an unrelated Web
site.

The idea is not to maximize your chances of getting to some Web site, any old
Web site. What you want is to be told that the URL is bad: `No such site'.

Entering a complete URL is very different from (1) typing part of a URL, (2)
asking the browser to help you find a relevant complete URL, and then (3)
hitting RET to show your agreement with its proposal.

It's also different if your browser first tells you that a complete URL you
entered is bad and then asks if you want it to try to guess another URL. In that
case (a) you are told there is no match and (b) you decide whether you want to
go off on a wild goose chase. You are in control in that case, not the browser.

I mention the browser analogy because everyone can relate to it. Its UI is
straightforward and commonsensical. What we're doing in Emacs now is not.

When you hit RET, you are telling Emacs, a browser, whatever: "This is what I
want. If you don't have that, then just say so."

Emacs has always allowed you, in some contexts (but not in others), to hit RET
to both complete and enter the completed text. But that becomes less appropriate
when the completion is not obvious from the input text (as is the case for
partial completion).

It's particularly problematic if the user's intention is that what s?he entered
be considered already complete. And we cannot know that intention for sure; we
can only suppose it because s?he chose to use RET, not TAB.

`C-h f' should either (a) forego `completion-styles' and use basic (prefix)
completion or (b) require confirmation when prefix completion fails and it moves
on to more exotic attempts to find something that matches.


[It is also unfriendly to users for Emacs TAB to perform such partial completion
by default, but TAB is a different story from RET. The mystery and
unexpectedness of the TAB behavior has already been documented as several other
bugs, and will no doubt continue to cause unwitting users to file bugs. I got a
mail from a user just yesterday who said with annoyance after discovering what
was happening, "suddenly you get lots of completion results which seemingly no
connection to what you've been typing". Indeed you do, indeed you do.]

bug#4718: 23.1; C-h f gives doc for the wrong function

[Stefan Monnier]

> I entered one entire function name. Emacs didn't complain that there
> was no such function.  Emacs instead silently gave me the doc for
> a different function.  That's totally inappropriate.

> When I hit RET, Emacs should say `No match' and not accept my
> erroneous input, as it used to do in Emacs 22 and before.

emacs22 -Q
C-h f dolis RET

will happily descrie the `dolist' function.  So, no, this is no strictly
new behavior in this respect.  The partial completion in Emacs-23 does
make it more likely that completion will find some function rather than
return "no match".  If someone wants to make this function use a `ask'
for `require-match', as is done in M-x, I won't object, tho I do not
think it's a big deal.

For what it's worth I have a local patch that indirectly changes this
behavior: it accepts any function name (even non-existing ones),
requires confirmation for non-existing ones, and then tries to guess
which file to load to find the function.
 

        Stefan

bug#4718: 23.1; C-h f gives doc for the wrong function

[Drew Adams]

> emacs22 -Q
> C-h f dolis RET
> 
> will happily descrie the `dolist' function.  So, no, this is 
> no strictly new behavior in this respect.

I already said the same thing:

 >> Emacs has always allowed you, in some contexts (but
 >> not in others), to hit RET to both complete and enter
 >> the completed text. But that becomes less appropriate
 >> when the completion is not obvious from the input text
 >> (as is the case for partial completion).

This change qualitatively alters what to expect from RET. Until now, you could
pretty much be sure of what RET was going to give you - the only case for
possible confusion was multiple names with the same prefix, and there you
typically got some help from the `Complete but not unique' feedback.

Now, you type `orange' and you find out afterward that you entered `apple'.
Qualitatively, we're no longer in the same ballpark.

> The partial completion in Emacs-23 does make it more likely
> that completion will find some function rather than
> return "no match".

That's it. And for RET, especially, that can be quite confusing. With TAB, you
see what you will get, at least.

> If someone wants to make this function
> use a `ask' for `require-match', as is done in M-x, I won't
> object, tho I do not think it's a big deal.

I hope someone will. I don't have the time now. It should have been done when we
introduced `completion-styles' and partial completion as the default behavior.

But we should not impose a regimental `ask' for this in general. The problem
does not exist for prefix completion. We should show you the sole completion and
ask for confirmation only when it does not correspond to prefix completion.
Non-basic completion is the only case where there is really an element of
surprise, confusion, and lack of understanding.

> For what it's worth I have a local patch that indirectly changes this
> behavior: it accepts any function name (even non-existing ones),
> requires confirmation for non-existing ones, and then tries to guess
> which file to load to find the function.

The problem is not non-existing functions. In that case, the current code would
still say `No match'. The problem is (a) treating additional patterns as matches
when combined with (b) RET.

As I said, with TAB it's one thing. With RET, you don't even get a chance to see
what the completion is (until you see the *Help* buffer, and then you're
unlikely to double-check the function name).

I don't even think this is specific to `C-h f'. We should probably do the same
thing most of the time: make RET confirm when the completion is not an obvious
one (i.e. a suffix).

bug#4718: 23.1; C-h f gives doc for the wrong function

[Stefan Monnier]

> But we should not impose a regimental `ask' for this in general.
> The problem does not exist for prefix completion.  We should show you
> the sole completion and ask for confirmation only when it does not
> correspond to prefix completion.  Non-basic completion is the only
> case where there is really an element of surprise, confusion, and lack
> of understanding.

I disagree, the same problem exists for prefix completion.  Maybe it's
less frequent, but it exists nevertheless.  Which brings us to the
reason why we don't currently ask: choosing the wrong name is harmless
because C-h f does not perform any dangerous operation that might lose
you some work.

>> For what it's worth I have a local patch that indirectly changes this
>> behavior: it accepts any function name (even non-existing ones),
>> requires confirmation for non-existing ones, and then tries to guess
>> which file to load to find the function.

> The problem is not non-existing functions.  In that case, the current
> code would still say `No match'.  The problem is (a) treating
> additional patterns as matches when combined with (b) RET.

Reread what I wrote: I said "indirectly".
It's related not for its functionality but because if we want to be able
to accept non-existing functions, then RET can't perform completion
any more.

> I don't even think this is specific to `C-h f'.  We should probably do
> the same thing most of the time: make RET confirm when the completion
> is not an obvious one (i.e. a suffix).

That's almost already the case: it's fairly rare for Emacs completion to
use this kind of strong `require-match'.


        Stefan

bug#4718: 23.1; C-h f gives doc for the wrong function

[Drew Adams]

> > But we should not impose a regimental `ask' for this in general.
> > The problem does not exist for prefix completion.  We 
> > should show you the sole completion and ask for confirmation
> > only when it does not correspond to prefix completion.
> > Non-basic completion is the only case where there is really
> > an element of surprise, confusion, and lack
> > of understanding.
> 
> I disagree, the same problem exists for prefix completion.  Maybe it's
> less frequent, but it exists nevertheless.

As I said a couple of times, this difference in degree leads to a qualitative
difference. It's not much of a problem for prefix completion, in practice - for
the reasons I gave.

> Which brings us to the reason why we don't currently ask:
> choosing the wrong name is harmless because C-h f does not
> perform any dangerous operation that might lose you some work.

No one claimed that it will start a thermonuclear explosion.

The point is that we make it harder, not easier, for users currently. A user now
has to really pay attention and double-check the name of the function at the top
of *Help*. This is error-prone and a time-waster for users. That extra burden on
users isn't necessary.

Does that really happen? Well, I reported it just as it happened to me. It took
me a while in fact to realize that I was studying the doc (args etc.) for the
wrong function - one that is similarly named.

And Juanma was of course right that that happened to me in this case because I
had loaded dired.el but not dired-aux.el (normally, I load them both from the
get-go). It's a good example of what can happen and how Emacs now throws
obstacles our way instead of making things easier.

My expectation was that I was providing the complete name of an existing
function. In Emacs 22, Emacs would have told me there was no such function, and
I would have immediately realized that I had not loaded dired-aux.el. In Emacs
23, Emacs silently substituted another function, and I wasted time studying the
wrong thing.

Will this problem cause a nuclear meltdown? Probably not.

> >> For what it's worth I have a local patch that indirectly 
> >> changes this behavior: it accepts any function name
> >> (even non-existing ones), requires confirmation for
> >> non-existing ones, and then tries to guess
> >> which file to load to find the function.
> 
> > The problem is not non-existing functions.  In that case, 
> > the current code would still say `No match'.  The
> > problem is (a) treating additional patterns as matches
> > when combined with (b) RET.
> 
> Reread what I wrote: I said "indirectly". It's related not
> for its functionality but because if we want to be able
> to accept non-existing functions, then RET can't perform
> completion any more.

I'm not arguing that RET should not perform completion anymore. In fact, I
stated that it should. What I proposed is that it not silently accept a
completion, without confirmation, unless it has the user's text as a prefix
(modulo directory, $, etc., as I said).

IOW, for prefix completion RET's behavior is not a problem, in practice. Let's
solve the real problem, and not generalize to "fix" something that isn't broken.

The problem is RET substituting a completion that you would never have guessed
and then accepting that, without ever showing it to you. It is only in such
cases that it would be good for RET to stop, show you what it is about to
accept, and let you confirm or cancel.

Can we recognize such cases? Maybe not in fine-tuned way, never erring one way
or the other. But simply deciding that prefix completion is OK for RET to do
what it's always done, and any other completion is a priori not OK, would help a
lot.

Even in the case of non-prefix completion, we could test if the particular
completion did in fact have the user text as a prefix (meaning that the result
was a prefix completion, even if the method used was not basic completion), and
if so treat it as prefix completion (no confirmation in a case such as C-h f).

bug#4718: 23.1; C-h f gives doc for the wrong function

[Stefan Monnier]

>> I disagree, the same problem exists for prefix completion.  Maybe it's
>> less frequent, but it exists nevertheless.

> As I said a couple of times, this difference in degree leads to
> a qualitative difference. It's not much of a problem for prefix
> completion, in practice - for the reasons I gave.

I understood what you said, but as written above I disagree: the same
"confusing" behavior can happen at times with the old code.  It's just
a lot less frequent.  So it's a question of frequency rather
than a qualitative difference.

>> Reread what I wrote: I said "indirectly". It's related not
>> for its functionality but because if we want to be able
>> to accept non-existing functions, then RET can't perform
>> completion any more.

> I'm not arguing that RET should not perform completion anymore.

Obviously not, I have no idea why you take the idea that I think you
said so.


        Stefan

bug#4718: 23.1; C-h f gives doc for the wrong function

[Juanma Barranquero]

On Wed, Oct 14, 2009 at 03:49, Drew Adams <drew.adams@oracle.com> wrote:

> I entered one entire function name. Emacs didn't complain that there was no such
> function.

In your example, you didn't even load dired, so I was trying to
understand what you did.

> Emacs instead silently gave me the doc for a different function.
> That's totally inappropriate.
>
> When I hit RET, Emacs should say `No match' and not accept my erroneous input,
> as it used to do in Emacs 22 and before.

Stefan already has answered that: emacs 22 did in some cases, too.

> Imagine if you paste a complete URL in your browser and you get a totally
> different Web site from what you request, the browser thinking that it is being
> helpful because it notices some similarity between your URL and another that it
> knew about.

Irrelevant. URL completion in most browsers is not similar to Emacs completion.

> Can you imagine your Web experience in that case? Imagine if your browser does
> that each time you click a broken link: "helpfully" transforming the bad URL
> into a different one that "works" - but that corresponds to an unrelated Web
> site.

Navigating to an unexpected URL could have security implications; not
so for symbol completion (at least, in most cases).

> Emacs has always allowed you, in some contexts (but not in others), to hit RET
> to both complete and enter the completed text. But that becomes less appropriate
> when the completion is not obvious from the input text (as is the case for
> partial completion).
>
> It's particularly problematic if the user's intention is that what s?he entered
> be considered already complete. And we cannot know that intention for sure; we
> can only suppose it because s?he chose to use RET, not TAB.

You're saying that you would rather it didn't work for `dolis' <RET>
either, then. You prefer to be asked. Fine. Personally, I kinda like
the way it works now. Certainly does not strike me as user-unfriendly.

    Juanma

bug#4718: 23.1; C-h f gives doc for the wrong function

[Drew Adams]

> You're saying that you would rather it didn't work for `dolis' <RET>
> either, then.

No, I'm not saying that. I have no problem with the behavior of Emacs 22 and
before.

Letting RET complete using prefix completion is not problematic in the way that
letting it do so with partial completion is. With only prefix completion, `dolis
RET' can only complete to something that has `dolis' as a prefix. When there is
only one such completion, it is not very hard to guess what that is.

That is, with prefix completion the gap between what is known (the input) and
what is unknown (the completion) is small and predictable. If you choose to hit
RET, it's because you pretty much know what you're going to get.

That's especially true, the longer the input. In the case I gave,
`dired-byte-compile', the input is already quite long for a function name. That
means both (a) it is unlikely that the sole completion would be much longer and
therefore hard to guess and (b) it is not unreasonable for both the program and
the user to consider the input as pretty much the whole function name.

IOW, RET, with the meaning "this is what I want" fits well here. RET in that
sense does not fit well with partial completion, where your input could complete
to pretty much anything. You have very little knowledge of what the completions
are. We've already seen bizarre examples of typing one thing and getting
something totally unforeseen as a result. I don't think anyone denies that.

The point is that that unknown is more or less controllable by users when TAB is
involved. They see the result before entering it or cancelling. RET is another
story altogether. You cannot have a good idea what will happen when you push
that big red button.

I suppose the end result will be that users will eventually learn to hit TAB RET
systematically instead of RET. I would rather see the program make them jump
through such a hoop (confirm) only when it's really needed. That is, only when
using partial completion, since prefix completion has never posed a problem in
this regard.

That's one solution I see: not ask for confirmation except when the completion
does not have the input as a prefix. (By input, I mean modulo directory name, $
for env vars, etc.) IOW, treat the problem only where it is - there is no
problem for the classic prefix completion.

bug#4718: 23.1; C-h f gives doc for the wrong function

[Juanma Barranquero]

On Wed, Oct 14, 2009 at 06:24, Drew Adams <drew.adams@oracle.com> wrote:

> No, I'm not saying that. I have no problem with the behavior of Emacs 22 and
> before.

Aha. Sorry for misrepresenting your point.

> Letting RET complete using prefix completion is not problematic in the way that
> letting it do so with partial completion is. With only prefix completion, `dolis
> RET' can only complete to something that has `dolis' as a prefix. When there is
> only one such completion, it is not very hard to guess what that is.

But there's not necessarily just one completion. If you have cl loaded,

  C-h f defu <RET>  => defun

but it could also be `defun*'.

> That is, with prefix completion the gap between what is known (the input) and
> what is unknown (the completion) is small and predictable. If you choose to hit
> RET, it's because you pretty much know what you're going to get.

I don't think so, because <RET> is also a form of completion:

  C-h f buffer-face <RET> => "buffer-face-"
  <RET> => "Possible completions are:"

> That
> means both (a) it is unlikely that the sole completion would be much longer and
> therefore hard to guess and (b) it is not unreasonable for both the program and
> the user to consider the input as pretty much the whole function name.

It is not unreasonable, of course. But neither it is unreasonable the
opposite: to understand RET as, "if there's only one completion, give
me that". I think it's useful.

> IOW, RET, with the meaning "this is what I want" fits well here. RET in that
> sense does not fit well with partial completion, where your input could complete
> to pretty much anything.

It's a matter of tastes, I think.

> That's one solution I see: not ask for confirmation except when the completion
> does not have the input as a prefix.

That seems reasonable, though surely there's people who will feel as
strongly about it as you feel about the current default behavior :-)

    Juanma

bug#4718: 23.1; C-h f gives doc for the wrong function

[Drew Adams]

> > Letting RET complete using prefix completion is not 
> > problematic in the way that letting it do so with
> > partial completion is. With only prefix completion, `dolis
> > RET' can only complete to something that has `dolis' as a 
> > prefix. When there is only one such completion, it is not
> > very hard to guess what that is.
> 
> But there's not necessarily just one completion.

I said, "When there is only one such completion."
That's the case in question in this bug report.

There is little problem when there are multiple completions - you see them, even
for partial-completion.

There is far less confusion with prefix completion, but there is no great
problem even for partial-completion when there are multiple completions.

For prefix completion, all of the completions have the same longest common
prefix, which is what you see. Especially with longer input, the guess wrt the
completions is a narrow gap; it is nothing like the case for partial completion,
where the possible completions are all over the map.

But even for partial completion, there is little problem when there are multiple
completions. You can see them, even if you hit RET.

The problem is when there is only one completion and you hit RET. For prefix
completion, you can pretty much guess what the completion is, especially for
long input. For partial completion you have no idea.

> > That means both (a) it is unlikely that the sole completion 
> > would be much longer and therefore hard to guess and (b)
> > it is not unreasonable for both the program and
> > the user to consider the input as pretty much the whole 
> > function name.
> 
> It is not unreasonable, of course. But neither it is unreasonable the
> opposite: to understand RET as, "if there's only one completion, give
> me that". I think it's useful.

But you don't know what that one completion is. That is the problem. You haven't
got a clue. So when you say "give me that one", you don't know what one it is
that you're committing to.

With prefix completion, there is really no comparison wrt the amount of
knowledge you have about what you're accepting - you generally know what you
will get.

So yes, RET's completing behavior can be useful - with prefix completion. With a
mix of completion styles, however, it's like playing darts in the dark.

It is precisely the fact that it is useful with prefix completion that I do not
want to see us simply institute a confirmation policy for RET in a blanket way.

That is why I suggested that RET confirmation is called for only when the sole
completion does not have the user's text as a prefix. IOW, only for the
"surprising" partial-completion case. And again, the problem I'm reporting is
particularly wrt a sole completion.

> > That's one solution I see: not ask for confirmation except 
> > when the completion does not have the input as a prefix.
> 
> That seems reasonable, though surely there's people who will feel as
> strongly about it as you feel about the current default behavior :-)

Well, there are always people who feel differently about everything in Emacs. We
have options when there are important differences in approach for users.

By default at least, Emacs users should not have to guess what's happening or
feel inadequate and at a loss because they don't understand what's going on -
why they get the results they get. Users should feel like they control Emacs.
They shouldn't be uncomfortably surprised or wondering WTF.

The new completion behavior is a WTF? behavior in many respects. The default
behavior should be tamed to be less surprising. Anyone who comes to feel
comfortable with getting an unknown, or who particularly likes playing darts in
the dark, can choose another value for the option that would control this.

bug#4718: 23.1; C-h f gives doc for the wrong function

[Stefan Monnier]

> The problem is when there is only one completion and you hit RET. For prefix
> completion, you can pretty much guess what the completion is, especially for
> long input.  For partial completion you have no idea.

[ I'm not sure what/why we're still discussing, but anyway.. ]

BTW, you may want to try icomplete-mode.


        Stefan

bug#4718: 23.1; C-h f gives doc for the wrong function

[Drew Adams]

> > The problem is when there is only one completion and you 
> > hit RET. For prefix completion, you can pretty much guess
> > what the completion is, especially for
> > long input.  For partial completion you have no idea.
> 
> BTW, you may want to try icomplete-mode.

I do use icomplete-mode. Always have.

I report the problem on behalf of Emacs and Emacs users, not especially on my
behalf as a particular user. What I use personally is not very relevant. What I
use wrt completion is in fact very different from vanilla Emacs, in any case.