bug#3746: M-r in comint mode should use isearch

bug#3746: M-r in comint mode should use isearch

[Dan Nicolaescu]

When using the minibuffer for example doing:

M-x
then
C-r
will use isearch to search the history.  This is nice because it shows
the items that are matched.

M-r should to the same thing when used in comint, right now it's hard to
tell if it matches anything before typing RET

bug#3746: M-r in comint mode should use isearch

[Juri Linkov]

> When using the minibuffer for example doing:
>
> M-x
> then
> C-r
> will use isearch to search the history.  This is nice because it shows
> the items that are matched.
>
> M-r should to the same thing when used in comint, right now it's hard
> to tell if it matches anything before typing RET

Do you mean implementing a bash-like history isearch
in comint/shell modes?  I think it would be nice.
But maybe M-r is not the best key to start a shell history
isearch in the comint buffer.  In the minibuffer M-r keeps
its old functionality, and C-s is used to isearch the
minibuffer history.

It seems we can't use C-s in the comint buffer to isearch the
shell history because searching in the output is useful as well.
Maybe we should let C-s to isearch the shell history from the M-r
minibuffer instead of the minibuffer history of the previous M-r input?

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3746: M-r in comint mode should use isearch

[Dan Nicolaescu]

Juri Linkov <juri@jurta.org> writes:

  > > When using the minibuffer for example doing:
  > >
  > > M-x
  > > then
  > > C-r
  > > will use isearch to search the history.  This is nice because it shows
  > > the items that are matched.
  > >
  > > M-r should to the same thing when used in comint, right now it's hard
  > > to tell if it matches anything before typing RET
  > 
  > Do you mean implementing a bash-like history isearch
  > in comint/shell modes?  I think it would be nice.

Yep.

  > But maybe M-r is not the best key to start a shell history
  > isearch in the comint buffer.  In the minibuffer M-r keeps
  > its old functionality, and C-s is used to isearch the
  > minibuffer history.

The problem with that is that C-s (and C-r) already have bindings in
comint modes, and changing them would not be liked by most users.
Doing M-r and then C-s/C-r seems a bit cumbersome. 
IMHO just using M-r for incremental history search will provide no
surprises for the user, and it seems like a natural extension of the
current behavior.

bug#3746: M-r in comint mode should use isearch

[Juri Linkov]

> The problem with that is that C-s (and C-r) already have bindings in
> comint modes, and changing them would not be liked by most users.
> Doing M-r and then C-s/C-r seems a bit cumbersome.
> IMHO just using M-r for incremental history search will provide no
> surprises for the user, and it seems like a natural extension of the
> current behavior.

Currently M-r typed in the shell buffer activates the minibuffer with
the prompt "Previous input matching (regexp): ".  I doubt that users
will like to see Isearch activated instead of this minibuffer.
But maybe we should try using M-s that is less useful in the
shell buffer.

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3746: M-r in comint mode should use isearch

[Dan Nicolaescu]

Juri Linkov <juri@jurta.org> writes:

  > > The problem with that is that C-s (and C-r) already have bindings in
  > > comint modes, and changing them would not be liked by most users.
  > > Doing M-r and then C-s/C-r seems a bit cumbersome.
  > > IMHO just using M-r for incremental history search will provide no
  > > surprises for the user, and it seems like a natural extension of the
  > > current behavior.
  > 
  > Currently M-r typed in the shell buffer activates the minibuffer with
  > the prompt "Previous input matching (regexp): ".  I doubt that users
  > will like to see Isearch activated instead of this minibuffer.

Why not?  It does provide the same functionality and it seems easier to
use.  Maybe the prompt would need to be adjusted, but, IMHO, it's
doubtful the users would be unpleasantly surprised by the behavior...

  > But maybe we should try using M-s that is less useful in the
  > shell buffer.
  > 
  > -- 
  > Juri Linkov
  > http://www.jurta.org/emacs/

bug#3746: M-r in comint mode should use isearch

[Juri Linkov]

>   > > The problem with that is that C-s (and C-r) already have bindings in
>   > > comint modes, and changing them would not be liked by most users.
>   > > Doing M-r and then C-s/C-r seems a bit cumbersome.
>   > > IMHO just using M-r for incremental history search will provide no
>   > > surprises for the user, and it seems like a natural extension of the
>   > > current behavior.
>   >
>   > Currently M-r typed in the shell buffer activates the minibuffer with
>   > the prompt "Previous input matching (regexp): ".  I doubt that users
>   > will like to see Isearch activated instead of this minibuffer.
>
> Why not?  It does provide the same functionality and it seems easier to
> use.  Maybe the prompt would need to be adjusted, but, IMHO, it's
> doubtful the users would be unpleasantly surprised by the behavior...

M-r has a different UI. Its main difference that it is a non-incremental
regexp search unlike C-s that is an incremental string search.  Both bash
and the Emacs minibuffer activate Isearch on C-s, so it would be natural
to do the same in shell buffers.  We could try the same context-dependent
dwim approach used by the option of `dired-isearch-filenames', i.e.
"When point is on a file name initially, search file names".  So when point
is on the shell prompt then run Isearch on the shell history, otherwise
run Isearch on the shell buffer.

What to do with the prompt is more difficult problem.  Bash replaces the
shell prompt with the search prompt.  Perhaps we should do the same.

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3746: M-r in comint mode should use isearch

[Drew Adams]

> So when point is on the shell prompt then run Isearch on
> the shell history, otherwise run Isearch on the shell buffer.

Why?

I don't really want to get involved in this thread. I will say though that the
approach just quoted is a good example of the weakness  of DWIM.

A better approach, generally, is to give users explicit control, without such a
dependence on fine-grained context or mode. IOW, in this case, have the two
different kinds of search be initiated by a user differently, using two
different commands/keys. That's the typical, simple, straightforward Emacs
approach.

The DWIM approach, far from always making things easier for users, complicates
the mental models they need to use to make sense of their interactions with the
program. They need to be aware (not necessarily consciously) of much more
contextual state. The behavior is more complex, not less.

That's not to say that users can't make use of it or that they need to be
consciously thinking about complex contextual issues all the time. Like riding a
bike or driving a car, this can soon become unconscious and natural (provided
they use it a lot).

The general objection is rather Occam's razor: _Why_ add such complexity? What
is gained? Occasionally the added convenience of having a single key to use
makes a DWIM command worth it. But usually not.

If the two behaviors and their triggering contexts are complementary and quite
separate, then it can sometimes be worth it. But just combining behaviors on the
same key binding whenever there is some degree of context difference is not
justified, on its own. Just because Isearch on the history and Isearch on the
buffer have no target overlap is not in itself a sufficient reason to combine
them on the same key.

Put another way, is it really worth it in this case to make the command behavior
more complex, just to avoid having two key bindings? What reasons can you give?

One can look throughout Emacs and find plenty of commands/keys that _could_ be
combined, pushing the decision of which command to apply from the user into the
combo-command, according to context. But very few commands merit such
combination. 

I'm sure you could combine some commands/keys in almost any major mode - from
Dired to *Buffer List* to view-mode to almost anything. There are commands that
make sense only in certain contexts and so could be combined. But _why_ do that?
You could make `U' in Dired do something different when no files are marked. You
could perhaps combine that with `#' to do something when there are no auto-save
files. But why?

You get the idea: Combining different commands on the same key, to take
advantage of their applicability in different contexts, makes the UI more
complex; it forces users to consider the context. All that's gained is a
keystroke or two. Generally, it's not worth it.

Some argue that having lots of commands and lots of key bindings is also
complex. Yes, but then the different features are more separate, explicit,
independent. They are not linked by state/mode changes. States and modes are
among the most complex UI critters for users to tame, and they are largely
invisible/virtual.

Each time there is a DWIM proposal like this, I'd like to see a little extra
justification - justification for the combination itself: why one complex
command instead of two simple commands? If this isn't supported by argument each
time, we might well find ourselves soon in a maze of DWIM, with people emulating
such tricky combination more and more.

Programmers like to be clever, and DWIM is a cute cleverness trap. Like trying
to eke out extra functionality from fewer bytes of assembler code, apparent
cleverness can have a price in ease of change, ease of use, and ease of
understanding.

bug#3746: M-r in comint mode should use isearch

[Juri Linkov]

> The general objection is rather Occam's razor: _Why_ add such complexity?
> What is gained? Occasionally the added convenience of having a single key
> to use makes a DWIM command worth it. But usually not.

This case is an exception.

C-r is the most frequent key I use in bash running in xterm because
it is very convenient to search for old commands in the shell history.
I suppose the same is true for other users.

Using a different key in Emacs for the same functionality
will cause too much trouble.  Just imagine when you have a habit
typing C-r to search on the shell history, typing it in the Emacs
shell buffer will not do what you mean.  This recalls that
Emacs has a different key M-r.  Then switching back to xterm
and typing M-r: Ahh, that M-r is valid only in Emacs,
but in xterm it is C-r.  Arrrgh!

I think having two different keys for the same functionality
(C-r and M-r for Isearch on the history) is worse than having
the same key for different contexts (the shell prompt or the
rest of the shell buffer).

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3746: M-r in comint mode should use isearch

[Lennart Borgman]

On Thu, Jul 9, 2009 at 1:27 AM, Juri Linkov<juri@jurta.org> wrote:
>> The general objection is rather Occam's razor: _Why_ add such complexity?
>> What is gained? Occasionally the added convenience of having a single key
>> to use makes a DWIM command worth it. But usually not.
>
> This case is an exception.
>
> C-r is the most frequent key I use in bash running in xterm because
> it is very convenient to search for old commands in the shell history.
> I suppose the same is true for other users.
>
> Using a different key in Emacs for the same functionality
> will cause too much trouble.  Just imagine when you have a habit

I do not think it is an exception. I think it is a way of thinking we
should always consider.

bug#3746: M-r in comint mode should use isearch

[Drew Adams]

> > The general objection is rather Occam's razor: _Why_ add 
> > such complexity? What is gained? Occasionally the added
> > convenience of having a single key to use makes a DWIM
> > command worth it. But usually not.
> 
> This case is an exception.

I leave that to your judgment; I can't speak to that.

> C-r is the most frequent key I use in bash running in xterm because
> it is very convenient to search for old commands in the shell history.
> I suppose the same is true for other users.
> 
> Using a different key in Emacs for the same functionality
> will cause too much trouble.  Just imagine when you have a habit
> typing C-r to search on the shell history, typing it in the Emacs
> shell buffer will not do what you mean.  This recalls that
> Emacs has a different key M-r.  Then switching back to xterm
> and typing M-r: Ahh, that M-r is valid only in Emacs,
> but in xterm it is C-r.  Arrrgh!
> 
> I think having two different keys for the same functionality
> (C-r and M-r for Isearch on the history) is worse than having
> the same key for different contexts (the shell prompt or the
> rest of the shell buffer).

The alternative is not necessarily to have two completely different keys. The
typical, traditional, and simpler approach is to use the same key but still
allow two behaviors and keep the behavior distinction under explicit user
control: Initiate one of the two behaviors using `C-u'. Depending on context is
far more complicated.

And depending on context doesn't necessarily mean fewer keystrokes (even though
you never need to hit `C-u'): If the cursor is not in the right place for the
particular behavior you want, then you must first move it there. I'm not sure
that anything significant is gained by the approach you propose, except added
complexity. `C-r' for history search and `C-u C-r' for normal search seems
reasonable, to me.

But it's your call, of course.

bug#3746: M-r in comint mode should use isearch

[Juri Linkov]

> And depending on context doesn't necessarily mean fewer keystrokes
> (even though you never need to hit `C-u'): If the cursor is not in the
> right place for the particular behavior you want, then you must first
> move it there. I'm not sure that anything significant is gained by the
> approach you propose, except added complexity. `C-r' for history
> search and `C-u C-r' for normal search seems reasonable, to me.

Generally, using C-u to modify behavior is a good thing but not for C-r
since its C-u argument runs a regular expression search instead.

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3746: M-r in comint mode should use isearch

[Drew Adams]

> > And depending on context doesn't necessarily mean fewer keystrokes
> > (even though you never need to hit `C-u'): If the cursor is 
> > not in the right place for the particular behavior you want,
> > then you must first move it there. I'm not sure that anything
> > significant is gained by the approach you propose, except
> > added complexity. `C-r' for history search and `C-u C-r' for
> > normal search seems reasonable, to me.
> 
> Generally, using C-u to modify behavior is a good thing but 
> not for C-r since its C-u argument runs a regular expression
> search instead.

Hm, I wonder why. ;-)

I wonder why they didn't choose to have the search behavior (regexp vs plain)
depend intelligently on the context (where the cursor is, what phase the moon is
in,...).  `C-u' seems so, well, primitive - actually leaving it up to the user
to indicate explicitly what s?he means! Surely there is some smart way to
second-guess the user in that case also. ;-)

bug#3746: M-r in comint mode should use isearch

[Juri Linkov]

>> Generally, using C-u to modify behavior is a good thing but
>> not for C-r since its C-u argument runs a regular expression
>> search instead.
>
> Hm, I wonder why. ;-)
>
> I wonder why they didn't choose to have the search behavior
> (regexp vs plain) depend intelligently on the context
> (where the cursor is, what phase the moon is in,...).
> `C-u' seems so, well, primitive - actually leaving it up
> to the user to indicate explicitly what s?he means!
> Surely there is some smart way to second-guess the user
> in that case also. ;-)

If we are going to go down this road, soon we will have
only one command:

  M-x dwim RET

;-)

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3746: M-r in comint mode should use isearch

[Drew Adams]

And then we'll need only one key: RET 

> If we are going to go down this road, soon we will have
> only one command: M-x dwim RET ;-)

bug#3746: M-r in comint mode should use isearch

[Dan Nicolaescu]

Juri Linkov <juri@jurta.org> writes:

  > >   > > The problem with that is that C-s (and C-r) already have bindings in
  > >   > > comint modes, and changing them would not be liked by most users.
  > >   > > Doing M-r and then C-s/C-r seems a bit cumbersome.
  > >   > > IMHO just using M-r for incremental history search will provide no
  > >   > > surprises for the user, and it seems like a natural extension of the
  > >   > > current behavior.
  > >   >
  > >   > Currently M-r typed in the shell buffer activates the minibuffer with
  > >   > the prompt "Previous input matching (regexp): ".  I doubt that users
  > >   > will like to see Isearch activated instead of this minibuffer.
  > >
  > > Why not?  It does provide the same functionality and it seems easier to
  > > use.  Maybe the prompt would need to be adjusted, but, IMHO, it's
  > > doubtful the users would be unpleasantly surprised by the behavior...
  > 
  > M-r has a different UI. Its main difference that it is a non-incremental
  > regexp search unlike C-s that is an incremental string search.  

Is there any reason not to have an incremental search for M-r too?

  > Both bash and the Emacs minibuffer activate Isearch on C-s, so it
  > would be natural to do the same in shell buffers.  We could try the
  > same context-dependent dwim approach used by the option of
  > `dired-isearch-filenames', i.e.  "When point is on a file name
  > initially, search file names".  So when point is on the shell prompt
  > then run Isearch on the shell history, otherwise run Isearch on the
  > shell buffer.

That sounds fine, but it's not exclusive with M-r being an incremental
search too.

We are at the beginning of a release cycle, so we can experiment, if the
users hate the change, there's enough time to change back.

  > What to do with the prompt is more difficult problem.  Bash replaces the
  > shell prompt with the search prompt.  Perhaps we should do the same.

Agreed.

bug#3746: M-r in comint mode should use isearch

[Juri Linkov]

>   > Both bash and the Emacs minibuffer activate Isearch on C-s, so it
>   > would be natural to do the same in shell buffers.  We could try the
>   > same context-dependent dwim approach used by the option of
>   > `dired-isearch-filenames', i.e.  "When point is on a file name
>   > initially, search file names".  So when point is on the shell prompt
>   > then run Isearch on the shell history, otherwise run Isearch on the
>   > shell buffer.
>
> That sounds fine, but it's not exclusive with M-r being an incremental
> search too.
>
> We are at the beginning of a release cycle, so we can experiment, if the
> users hate the change, there's enough time to change back.

Ok, since all approaches are shaky, let's try all of them.
I'll post a patch soon.

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3746: M-r in comint mode should use isearch

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]

Please see the attached code that implements isearch in comint mode.
It is not posted as a patch because it has one remaining problem:

`comint-previous-input' and `comint-next-input' don't signal an error
at the beginning/end of the history ring.  This is unlike the minibuffer's
commands `previous-history-element' and `next-history-element' that
signal an error at the beginning/end of the minibuffer history.
So Isearch on the minibuffer history can stop and wrap.  But in comint
mode Isearch with an unmatched string doesn't stop and goes into the loop.

Maybe we should change `comint-previous-input' and `comint-next-input'
to signal an error at the beginning/end of the history ring?


[-- Attachment #2: comint.history.isearch.el --]
[-- Type: application/emacs-lisp, Size: 7713 bytes --]

[-- Attachment #3: Type: text/plain, Size: 45 bytes --]


-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3746: M-r in comint mode should use isearch

[Stefan Monnier]

> Please see the attached code that implements isearch in comint mode.

What would be the "UI" (e.g. key-binding) for it?
Only the comint-history-isearch defcustom?

> Maybe we should change `comint-previous-input' and `comint-next-input'
> to signal an error at the beginning/end of the history ring?

That sounds fine to me, but pleast double check all uses of those
functions to make sure it won't introduce other problems.


        Stefan

bug#3746: M-r in comint mode should use isearch

[Juri Linkov]

>> Please see the attached code that implements isearch in comint mode.
>
> What would be the "UI" (e.g. key-binding) for it?
> Only the comint-history-isearch defcustom?

One way to use this feature is to configure `comint-history-isearch' to
a non-nil value.  But I think it should be nil by default because after
trying to use it with non-nil values I found it counterintuitive since
in comint mode on typing `C-r' I expect it to search in the comint output,
not in the input history.

There is no such problem in bash where C-r searches in the input history,
since there is no way to search in the bash output.

So we need another key to search in the history.  However, this is
a controversial issue, so I will defer to others to decide what key
to use for that.  (In the code I sent, there is a command
`comint-history-isearch-backward' that could be bound to any key.)

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3746: M-r in comint mode should use isearch

[Juri Linkov]

>> Maybe we should change `comint-previous-input' and `comint-next-input'
>> to signal an error at the beginning/end of the history ring?
>
> That sounds fine to me, but pleast double check all uses of those
> functions to make sure it won't introduce other problems.

To avoid possible problems, I installed a non-intrusive implementation
that signals an error inside `comint-history-isearch-search' before
calling `comint-next-input' or `comint-previous-input'.

-- 
Juri Linkov
http://www.jurta.org/emacs/

bug#3746: marked as done (M-r in comint mode should use isearch)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 871 bytes --]

Your message dated Mon, 30 Nov 2009 18:12:45 +0200
with message-id <87ocmknicw.fsf@mail.jurta.org>
and subject line Re: bug#3746: M-r in comint mode should use isearch
has caused the Emacs bug report #3746,
regarding M-r in comint mode should use isearch
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@emacsbugs.donarmstrong.com
immediately.)


-- 
3746: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=3746
Emacs Bug Tracking System
Contact owner@emacsbugs.donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 2720 bytes --]

From: Dan Nicolaescu <dann@ics.uci.edu>
To: bug-gnu-emacs <bug-gnu-emacs@gnu.org>
Subject: M-r in comint mode should use isearch
Date: Fri, 3 Jul 2009 06:26:41 -0700 (PDT)
Message-ID: <200907031326.n63DQfPA027629@godzilla.ics.uci.edu>


When using the minibuffer for example doing:

M-x
then
C-r
will use isearch to search the history.  This is nice because it shows
the items that are matched.

M-r should to the same thing when used in comint, right now it's hard to
tell if it matches anything before typing RET



[-- Attachment #3: Type: message/rfc822, Size: 3880 bytes --]

From: Juri Linkov <juri@jurta.org>
To: Stefan Monnier <monnier@IRO.UMontreal.CA>
Cc: 3746-done@emacsbugs.donarmstrong.com, Dan Nicolaescu <dann@ics.uci.edu>
Subject: Re: bug#3746: M-r in comint mode should use isearch
Date: Mon, 30 Nov 2009 18:12:45 +0200
Message-ID: <87ocmknicw.fsf@mail.jurta.org>

>> What would be the "UI" (e.g. key-binding) for it?
>> Only the comint-history-isearch defcustom?
>
> One way to use this feature is to configure `comint-history-isearch' to
> a non-nil value.  But I think it should be nil by default because after
> trying to use it with non-nil values I found it counterintuitive since
> in comint mode on typing `C-r' I expect it to search in the comint output,
> not in the input history.
>
> There is no such problem in bash where C-r searches in the input history,
> since there is no way to search in the bash output.
>
> So we need another key to search in the history.  However, this is
> a controversial issue, so I will defer to others to decide what key
> to use for that.  (In the code I sent, there is a command
> `comint-history-isearch-backward' that could be bound to any key.)

To help to select a key binding, below is a table of current key bindings:

* Bash:

M-p    non-incremental-reverse-search-history
M-n    non-incremental-forward-search-history

M-r    revert-line

C-r    reverse-search-history
C-s    forward-search-history

C-p    previous-history
C-n    next-history

* Emacs comint mode:

M-p    comint-previous-input
M-n    comint-next-input

M-r    comint-previous-matching-input
M-s    comint-next-matching-input

C-r    isearch-backward
C-s    isearch-forward

C-p    previous-line
C-n    next-line

So I now agree with the subject of bug#3746 that says

  "M-r in comint mode should use isearch"

I marked bug#3746 as closed with M-r bound to the new command that uses
Isearch, added corresponding menu items, removed M-s (that was
symmetrical to M-r) to expose the global key binding M-s for comint
with removing one problem from the list of problems in FOR-RELEASE
that used to say:

  3. Shell mode binds `M-s' to `comint-next-matching-input'.
     This key binding can be changed later when we will implement
     incremental search through the shell history.

-- 
Juri Linkov
http://www.jurta.org/emacs/