Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Stefan Monnier]

> +;;; Commentary:
> +
> +;; Enabling this package implements more dynamic interaction with the
> +;; *Completions* buffer to give the user a similar experience than
> +;; interacting with Zle from zsh shell.
> +
> +;; The package intents to implement such functionalities without using
> +;; hacks or complex functions.  And using the default Emacs Completion
> +;; infrastructure.

Please write a description that can be understood by someone who doesn't
know Zsh (or Helm or Ivy or ...).


        Stefan

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Ergus]

On Sun, Nov 15, 2020 at 01:41:34PM -0500, Stefan Monnier wrote:
>> +;;; Commentary:
>> +
>> +;; Enabling this package implements more dynamic interaction with the
>> +;; *Completions* buffer to give the user a similar experience than
>> +;; interacting with Zle from zsh shell.
>> +
>> +;; The package intents to implement such functionalities without using
>> +;; hacks or complex functions.  And using the default Emacs Completion
>> +;; infrastructure.
>
>Please write a description that can be understood by someone who doesn't
>know Zsh (or Helm or Ivy or ...).
>
>
>        Stefan
>
Done!

Maybe it needs some comments in the manual/NEWS too? If so, where?

Please check if it needs anything else to be merged into master. I am
reusing the highlight face... but maybe the package needs its own face
for the overlay?

How can I make that emacs find the file automatically? It needs to be
added in cus-<something>.el right?

Best,
Ergus

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Stefan Monnier]

>> Please write a description that can be understood by someone who doesn't
>> know Zsh (or Helm or Ivy or ...).
> Done!

Much better, thanks.

> Maybe it needs some comments in the manual/NEWS too? If so, where?

I haven't looked at it enough to know, yet, sorry.

> Please check if it needs anything else to be merged into master.  I am
> reusing the highlight face... but maybe the package needs its own face
> for the overlay?

Sorry haven't looked at the code yet.

> How can I make that emacs find the file automatically? It needs to be
> added in cus-<something>.el right?

What do you mean by "find the file automatically"?

BTW, this new feature reminds me of completion-cycle-threshold.


        Stefan

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Ergus]

Hi Stefan:

On Sun, Nov 15, 2020 at 06:32:08PM -0500, Stefan Monnier wrote:
>>> Please write a description that can be understood by someone who doesn't
>>> know Zsh (or Helm or Ivy or ...).
>> Done!
>
>Much better, thanks.
>
>> Maybe it needs some comments in the manual/NEWS too? If so, where?
>
>I haven't looked at it enough to know, yet, sorry.
>
>> Please check if it needs anything else to be merged into master.  I am
>> reusing the highlight face... but maybe the package needs its own face
>> for the overlay?
>
>Sorry haven't looked at the code yet.
>
:(

>> How can I make that emacs find the file automatically? It needs to be
>> added in cus-<something>.el right?
>
>What do you mean by "find the file automatically"?
>
Now I have to load the file with -l to be allowed to use the command.

>BTW, this new feature reminds me of completion-cycle-threshold.
>
More or less is the same basic functionality for tabs, but with a lot of
steroids.

1) When completion-cycle-threshold is nil (the default completion) tab
functionality ends when the *Completions* is visible. After that tab
does nothing else unless there are too many candidates and the list
needs to be scrolled.

2) On the other hand when completion-cycle-threshold is t the
completions buffer is never shown. So the completion system behaves more
like vim (ugh), the next completion is not easily predictable. And, of
course, the user can't know how many tabs will need to reach what he
wants.

3) If completion-cycle-threshold is a number then candidates are shown,
but when start cycling, the <tab> order is independent from the one in
*Completions* (this behavior IMO is even worst). Also, there is not
feedback between the current candidate and the visible completions list.

-----------------

This mode shows the completions buffer (like when
completion-cycle-threshold is nil), but then cycles there (like when
completion-cycle-threshold is non-nil). But over a visible list of
candidates and highlighting the current one.

Once cycling started it is easy to know what will be the next <tab>
candidate or how many tabs (or arrows) are needed to get to a
candidate. It is also possible to go to the candidate down, up, or the
previous one (left) if you accidentally passed it.

Another advantage is that the cursor (point) stays in the same
mini-buffer place after completion, so, in case the list is too long and
you want to reduce it (or a wrong letter was inserted); you only need to
type another letter(backspace) and tab again (like in bash or zsh or the
default completion engine to get a new list).

Finally the mode only changes the default completion behavior once the
completions are visible and another tab is pressed (Where by default Tab
did nothing before).

Hope this helps.
>
>        Stefan
>
BTW:

1) completion-cycle-threshold t: is somehow equivalent to bash:

bind "TAB:menu-complete"
bind "set show-all-if-ambiguous on"

2) completion-cycle-threshold nil: is somehow similar to default bash
completion. (tab shows the list and wait more letters)

3) This mode mimics zle from zsh. It is impossible to have this behavior
in bash because readline can't give this. zsh team needed to create zle
to have it.

There are known limitations like that some users will like the
candidates bellow the prompt and not over or have everything in the same
window, or C-g should hide completions instead of exiting the
mini-buffer... But adding them may be too invasive or collide with the
default behavior and may add some worthless complexity.

Best, Ergus.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Stefan Monnier]

>>> How can I make that emacs find the file automatically? It needs to be
>>> added in cus-<something>.el right?
>>What do you mean by "find the file automatically"?
> Now I have to load the file with -l to be allowed to use the command.

Oh, then you want to put a `;;;###autoload` cookie just above the entry
point(s).

> 3) If completion-cycle-threshold is a number then candidates are shown,
> but when start cycling, the <tab> order is independent from the one in
> *Completions* (this behavior IMO is even worst). Also, there is not
> feedback between the current candidate and the visible completions list.

Yes, that's the case I find similar.  The differences I can see are:
- In your code you get to see the other candidates (with
  `completion-cycle-threshold` the *Completions* is not necessarily
  shown).
- In your code, you get to see your selection highlighted in *Completions*.
- In your code, you have to hit TAB an extra time, whereas with
  `completion-cycle-threshold` you start cycling as soon as there are
  few enough candidates.
- In your code the threshold depends on the size of *Completions* (and
  the size of the completions themselves?) rather than being a fixed limit.
- The order of completions is different.

I think it might be a good idea to try and bring those two closer to
each other.  E.g. when cycling, make sure the *Completions* buffer, if
shown, displays the choices in the order in which they are cycled, and
highlight the chosen one.

I think the key [pun unintended] difference between the two is the extra
TAB which lets you interpret it as a request to enter a special mode
with special bindings to move between the different
displayed candidates.


        Stefan

RE: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Drew Adams]

> I think it might be a good idea to try and bring those two closer to
> each other.  E.g. when cycling, make sure the *Completions* buffer, if
> shown, displays the choices in the order in which they are cycled, and
> highlight the chosen one.

I thought that's what he described.  In any case, that's
what is the case for Icicles.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Ergus]

On Sun, Nov 15, 2020 at 10:56:54PM -0500, Stefan Monnier wrote:
>>>> How can I make that emacs find the file automatically? It needs to be
>>>> added in cus-<something>.el right?
>>>What do you mean by "find the file automatically"?
>> Now I have to load the file with -l to be allowed to use the command.
>
>Oh, then you want to put a `;;;###autoload` cookie just above the entry
>point(s).
>
>> 3) If completion-cycle-threshold is a number then candidates are shown,
>> but when start cycling, the <tab> order is independent from the one in
>> *Completions* (this behavior IMO is even worst). Also, there is not
>> feedback between the current candidate and the visible completions list.
>
>Yes, that's the case I find similar.  The differences I can see are:
>- In your code you get to see the other candidates (with
>  `completion-cycle-threshold` the *Completions* is not necessarily
>  shown).
>- In your code, you get to see your selection highlighted in *Completions*.
>- In your code, you have to hit TAB an extra time, whereas with
>  `completion-cycle-threshold` you start cycling as soon as there are
>  few enough candidates.
>- In your code the threshold depends on the size of *Completions* (and
>  the size of the completions themselves?) rather than being a fixed limit.
>- The order of completions is different.
>
>I think it might be a good idea to try and bring those two closer to
>each other.  E.g. when cycling, make sure the *Completions* buffer, if
>shown, displays the choices in the order in which they are cycled, and
>highlight the chosen one.
>
Yes, I tried that, but the resulting code was ugly because the candidate
in completion-cycle-threshold is taken from completion--do-completion
directly and totally unrelated with the *Completions* buffer
content.

Changing that would require some changes of completion--cycle-threshold
/ completion--do-completion in minibuffer.el... I am not a proficient
lisper to do that :(. I don't want to break the universe there.

>I think the key [pun unintended] difference between the two is the extra
>TAB which lets you interpret it as a request to enter a special mode
>with special bindings to move between the different
>displayed candidates.
>

This was intentional because I didn't want to change any default
behavior, so I overloaded the tab just when where unused (*Completions*
is shown and all completions are visible).

The use of the selection/scroll with tab is another effort to not change
any default behavior; because tab scrolls *Completions* when they are
too long. So my code was injected with minimal modification because
initially I wanted to discuss to enable this mode by default in the
future (yes, a man can dream ;p).

I find the current behavior intuitive (and removing the extra tab will
make it even simpler) but I prefer to sacrifice that detail if that
increases the probabilities to enable it by default. Almost only new
users use the *Completion* instead of helm, ivy, icomplete or ido; I
want to improve that first impression.

This is something that could be managed with a custom if you prefer. So
requiring the extra tab or not is easy to implement. Limit the
candidates by number may be a bit more tricky and I am not sure it worth
the effort. Simple is better than complicated. But if you think it worth
the effort I could try...
>
>        Stefan
>
>
Ergus

RE: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Drew Adams]

FWIW -

> This mode shows the completions buffer..., but then cycles
> there.... But over a visible list of candidates and
> highlighting the current one.

That's exactly what Icicles does, and has done for
a couple of decades now.

And the sort order in *Completions* is the same as
the cycling order.  And you can change the order
any time, on the fly.

> Once cycling started it is easy to know what will be the next <tab>
> candidate or how many tabs (or arrows) are needed to get to a
> candidate. It is also possible to go to the candidate down, up, or the
> previous one (left) if you accidentally passed it.

Of course.

And you can also go directly to candidates, without
cycling.

Cycling is actually a fairly inefficient, unconscious
way to navigate among candidates.  It's handy, but
dumb cycling shouldn't be the only or the main way to
get around.

Progressively narrowing the current set of matches is
much better.  Combine cycling with that: progressive
completion/filtering, matching multiple simple patterns.
Cycling & filtering work hand in hand.

> Another advantage is that the cursor (point) stays in the same
> mini-buffer place after completion, so, in case the list is too long
> and you want to reduce it (or a wrong letter was inserted); you only need
> to type another letter(backspace) and tab again (like in bash or zsh or
> the default completion engine to get a new list).

Again, like Icicles.
___

https://www.emacswiki.org/emacs/Icicles_-_Cycling_Completions

https://www.emacswiki.org/emacs/Icicles_-_Progressive_Completion

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

> Cycling is actually a fairly inefficient, unconscious
> way to navigate among candidates.  It's handy, but
> dumb cycling shouldn't be the only or the main way to
> get around.

I agree.  I tried the completions-highlight-modifications branch, but
unfortunately restricting a set of navigation keys limits the freedom of
using any navigation key for browsing completion candidates that is
available today.  What I mean is that currently it's possible in the
minibuffer to type <PageUp> (switch-to-completions), and then all keys
are available for selecting a completion: arrow keys, page scrolling
keys - even completion drop-down combo in browser's address bar allows
page scrolling.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Ergus]

On Mon, Nov 16, 2020 at 10:54:48AM +0200, Juri Linkov wrote:
>> Cycling is actually a fairly inefficient, unconscious
>> way to navigate among candidates.  It's handy, but
>> dumb cycling shouldn't be the only or the main way to
>> get around.
>
>I agree.  I tried the completions-highlight-modifications branch, but
>unfortunately restricting a set of navigation keys limits the freedom of
>using any navigation key for browsing completion candidates that is
>available today.  What I mean is that currently it's possible in the
>minibuffer to type <PageUp> (switch-to-completions), and then all keys
>are available for selecting a completion: arrow keys, page scrolling
>keys - even completion drop-down combo in browser's address bar allows
>page scrolling.
>
Hi Juri:

With the current implementation this is not supposed to change. Of
course there is space for improvement and there are bugs, but so far,
all navigation still works in *Completions* buffer as usual because I
just added some commands. If there is anything that breaks the normal
previous behavior somehow, please tell me; because I put special
attention to prevent that.

We could add a custom to disable the new bindings in the *Completions*
if you think is better; because IMO the most important thing is to
"navigate" with arrows the completions from the minibuffer without
leaving it; and the overlay.

WDYT??

The code is extremely simple, so, if you think you could improve
anything it is more tan welcome.

Ergus

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

> With the current implementation this is not supposed to change. Of
> course there is space for improvement and there are bugs, but so far,
> all navigation still works in *Completions* buffer as usual because I
> just added some commands. If there is anything that breaks the normal
> previous behavior somehow, please tell me; because I put special
> attention to prevent that.

Sorry, I was not clear enough: I meant that the new feature doesn't
allow in the minibuffer using the same navigation keys that are
already available in the *Completions* buffer, e.g. <PgDown> and <PgUp>
typed in the *Completions* buffer scroll it up and down.

For example, while in the minibuffer type 'TAB' that displays a very
long list of completions, then type 'C-M-v', and see how it scrolls the
*Completions* buffer.  Better yet try typing <M-PgDown> and <M-PgUp>
in the minibuffer, and see how it scrolls the *Completions* buffer
up and down.  I meant that a new feature could allow such page scrolling
without the 'M-' modifier, by just using <PgDown> and <PgUp> in the minibuffer
(and C-v/M-v as well) to scroll the *Completions* buffer.

> We could add a custom to disable the new bindings in the *Completions*
> if you think is better; because IMO the most important thing is to
> "navigate" with arrows the completions from the minibuffer without
> leaving it; and the overlay.

Maybe a custom could provide some DWIM behavior by default,
e.g. to activate these keys only when the *Completions* buffer
is displayed.

RE: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Drew Adams]

> Sorry, I was not clear enough: I meant that the new feature doesn't
> allow in the minibuffer using the same navigation keys that are
> already available in the *Completions* buffer, e.g. <PgDown> and <PgUp>
> typed in the *Completions* buffer scroll it up and down.
> 
> For example, while in the minibuffer type 'TAB' that displays a very
> long list of completions, then type 'C-M-v', and see how it scrolls the
> *Completions* buffer.  Better yet try typing <M-PgDown> and <M-PgUp>
> in the minibuffer, and see how it scrolls the *Completions* buffer
> up and down.  I meant that a new feature could allow such page
> scrolling without the 'M-' modifier, by just using <PgDown> and <PgUp>
> in the minibuffer (and C-v/M-v as well) to scroll the *Completions* buffer.

(By <PgDown> and <PgUp> I assume you really mean
the logical keys <next> and <prior>.)

I hope that you don't choose keys such as <next>
and <prior> for this.  Or <home> and <end>.

Such pairs are useful for other things.  Icicles
uses those two pairs of keys for cycling different
kinds of matches, for instance.

> (and C-v/M-v as well)

And I especially hope we don't assign _multiple_
pairs for this scrolling.  Icicles uses `C-v' and
`M-v' for it.  Aren't those two keys sufficient?
Why multiply things, here, reducing key binding
possibilities for other purposes?
___

Yes, I know that Emacs binds both pairs, C-v/M-v
and next/prior, in general.  But that was done
long ago, when keys were not so precious.  I'd
prefer that `next' and `prior' were left free of
any default key bindings.

I feel the same about C-e/C-b and end/home, FWIW.

I know that many applications outside Emacs use
end/home and next/prior for movement.  But I don't
think it's great that Emacs does that by default.

I do feel differently about C-n/C-p and down/up.
But I think that users outside Emacs are _much_
more likely to expect the arrow keys to move
down/up than they are to have strong expectations
about end/home and next/prior.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Ergus]

On Mon, Nov 16, 2020 at 10:23:47PM +0200, Juri Linkov wrote:
>> With the current implementation this is not supposed to change. Of
>> course there is space for improvement and there are bugs, but so far,
>> all navigation still works in *Completions* buffer as usual because I
>> just added some commands. If there is anything that breaks the normal
>> previous behavior somehow, please tell me; because I put special
>> attention to prevent that.
>
>Sorry, I was not clear enough: I meant that the new feature doesn't
>allow in the minibuffer using the same navigation keys that are
>already available in the *Completions* buffer, e.g. <PgDown> and <PgUp>
>typed in the *Completions* buffer scroll it up and down.
>
I didn't change any default current behavior because I tried to prevent
complains when proposing to enable this by default in a future (as I
said "A man can dream"). In the code there is a macro
`with-minibuffer-scroll-window' that execute any command in the
Completions buffer only when it is enabled but also the
completions-highlight-minibuffer-map is set in completion-setup-hook,
so, after the completions are visible only.

Basically we only need to add the command you propose like

(defun ...
     (with-minibuffer-scroll-window scroll-up-command))

And bind them... probably with a remap in
completions-highlight-minibuffer-map.

>For example, while in the minibuffer type 'TAB' that displays a very
>long list of completions, then type 'C-M-v', and see how it scrolls the
>*Completions* buffer.

When the list is too long the highlight is not enabled now. This can be
seen as an issue or a feature, because it keeps the previous behavior
and requires minimal changes in the minibuffer.el side.

>Better yet try typing <M-PgDown> and <M-PgUp>
>in the minibuffer, and see how it scrolls the *Completions* buffer
>up and down.  I meant that a new feature could allow such page scrolling
>without the 'M-' modifier, by just using <PgDown> and <PgUp> in the minibuffer
>(and C-v/M-v as well) to scroll the *Completions* buffer.
>

<PgUp> will break the current behavior... there will be complains if we
try that and we will waste precious time in this mailing list trying to
get an agreement to go nowhere (as usual). In any case all them are
trivial to implement and even simpler with a remap.

C-M-v only scrolls Completions in some conditions. If you move forth and
back from another buffer then that one will be the one scrolled instead
of Completions. IMO this is a confusing behavior, but try to change that
and for sure will be the end of the world for some users.... so we need
more customs :).

>> We could add a custom to disable the new bindings in the *Completions*
>> if you think is better; because IMO the most important thing is to
>> "navigate" with arrows the completions from the minibuffer without
>> leaving it; and the overlay.
>
>Maybe a custom could provide some DWIM behavior by default,
>e.g. to activate these keys only when the *Completions* buffer
>is displayed.
>

This is also trivial to do... but I prefer to know if there is an
agreement in favor of the current unobtrusive changes before extending
it more.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

> Basically we only need to add the command you propose like
>
> (defun ...
>     (with-minibuffer-scroll-window scroll-up-command))
>
> And bind them... probably with a remap in
> completions-highlight-minibuffer-map.

Yes, one command could do it all, for example:

#+begin_src emacs-lisp
(defun with-completions-selected-window ()
  (interactive)
  (with-selected-window (get-buffer-window "*Completions*" 0)
    (when-let ((command (lookup-key (current-active-maps)
                                    (this-single-command-keys))))
      (call-interactively command))))
(dolist (key '(up down left right next prior))
  (define-key minibuffer-local-map `[(,key)] 'with-completions-selected-window))
#+end_src

It redirects all navigation keys from the minibuffer to the *Completions* buffer
where they are bound to cursor motion commands.

Then it just needs to highlight the completion under point.
Have you tried using 'cursor-sensor-functions' to detect
when point is on a completion to highlight it?

It seems it could be possible to set 'cursor-sensor-functions'
to something like this:

#+begin_src emacs-lisp
(lambda (_w _x dir)
  ;; highlight only in non-selected windows
  (unless (eq (get-buffer-window) (selected-window))
    (when (eq dir 'entered)
      (display--update-for-mouse-movement
       (line-number-at-pos) (current-column)))))
#+end_src

to highlight the completion under point as if the mouse pointer hovered
over it.

This could be useful not only for this feature, but also
for tmm-menubar, to make it look more like a menu.

RE: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Drew Adams]

> > Basically we only need to add the command you propose like
> >
> > (defun ...
> >     (with-minibuffer-scroll-window scroll-up-command))
> >
> > And bind them... probably with a remap in
> > completions-highlight-minibuffer-map.
> 
> Yes, one command could do it all, for example:
> 
> (defun with-completions-selected-window ()
>   (interactive)
>   (with-selected-window (get-buffer-window "*Completions*" 0)
>     (when-let ((command (lookup-key (current-active-maps)
>                                     (this-single-command-keys))))
>       (call-interactively command))))
> (dolist (key '(up down left right next prior))
>   (define-key minibuffer-local-map `[(,key)] 'with-completions-
> selected-window))
> 
> It redirects all navigation keys from the minibuffer to the
> *Completions* buffer where they are bound to cursor motion commands.

I'll request again that you not bind `next' and
`prior' this way, so that they can be saved for
other uses.  (I described uses that Icicles makes.)

I'll also request that you not bind `left' and
`right' this way.  Those keys should be available
in the minibuffer for cursor movement there.

We should not be removing keys useful for ordinary
text editing from the minibuffer gratuitously.

`up' and `down', and `C-v' and `M-v', yes.  And
`wheel-(up|down)'.  Those make sense; others don't.

In addition, `C-v' and `M-v' for scrolling
*Completions* from the minibuffer should wrap.
And wrapping to the beginning should move point
past the help text to start of the candidates.

Icicles does these things, and has for a long time.
You're free to check the code it uses for them.
It's small and straightforward (see, e.g.,
`icicle-scroll-Completions-forward').

https://www.emacswiki.org/emacs/download/icicles-mcmd.el

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

> The code is extremely simple, so, if you think you could improve
> anything it is more tan welcome.

The branch contains many different features with different problems,
so here is a short summary for these 4 separate features:

1. feature of setting minibuffer-tab-through-completions-function to
   completions-highlight-minibuffer-tab-through-completions:

The problem is that such TAB cycling is limited only to the case
when the whole *Completions* buffer is visible.

The original behavior of multiple presses of TAB is limited too:
its browsing is uni-directional, there is no way to use S-TAB
to scroll completions backwards.

So I don't see how relying on TAB could improve navigation in completions.

2. completion-set-suffix sometimes doesn't work;

3. commands that dispatch navigation keys from the minibuffer to the
   *Completions* buffer: this doesn't cover other navigation keys.

Maybe like windmove-default-keybindings defines modifiers,
a similar function could set up modifiers for scrolling completions
from the minibuffer, then the users could choose whether to use
e.g. 'M-' modifier for scrolling completions, or no modifier at all.

4. the feature of using completions-highlight-overlay often fails to
   update highlighting when point is moved in the *Completions* buffer
   by other navigation commands.

To fix this, it could work like hl-line-mode, but instead of
highlighting the whole line, it could highlight only completions.

Then completion highlighting would be useful on its own.
A new defcustom e.g. 'completion-highlight' (disabled by default)
could add a post-command-hook like hl-line-mode does.
Then other completion commands could enable this variable locally.

RE: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Drew Adams]

Responding to some of what you say there, comparing
with Icicles, as food for thought.

With Icicles there are pair of keys for cycling
forward and backward: `down' and `up' arrows.

But you can also just continue to use `TAB' (which
completes, as usual), to cycle forward.  This is
handy.  When you want to cycle backward, use `up'.

[And no, `S-TAB' does not cycle backward.  In Icicles,
`S-TAB' and `TAB' act similarly, but provide different
kinds of completion.]
___

You say that such highlighting is limited to when
all candidates are visible in *Completions*.  That's
not the case in Icicles.  The current candidate, which
is the one that's highlighted, is always visible (if
*Completions* is shown).
___

In Icicles you need not show *Completions* to cycle
candidates.  Cycling still places candidates, one by
one, in the minibuffer.  (You can complete without
showing *Completions* by using `C-M-TAB' instead of
`TAB'.)

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Ergus]

Hi Juri:

I made some of the changes you suggested, I think the behavior is a bit
more reliable now. 

On Wed, Nov 18, 2020 at 09:43:28PM +0200, Juri Linkov wrote:
>> The code is extremely simple, so, if you think you could improve
>> anything it is more tan welcome.
>
>The branch contains many different features with different problems,
>so here is a short summary for these 4 separate features:
>
>1. feature of setting minibuffer-tab-through-completions-function to
>   completions-highlight-minibuffer-tab-through-completions:
>
>The problem is that such TAB cycling is limited only to the case
>when the whole *Completions* buffer is visible.
>
This was as I said intentional. And to change that could be relatively
easy but I am not 100% we should. Maybe with a custom in the future...

>The original behavior of multiple presses of TAB is limited too:
>its browsing is uni-directional, there is no way to use S-TAB
>to scroll completions backwards.
>
>So I don't see how relying on TAB could improve navigation in completions.
>
>2. completion-set-suffix sometimes doesn't work;
>
I think it is fixed now.

>3. commands that dispatch navigation keys from the minibuffer to the
>   *Completions* buffer: this doesn't cover other navigation keys.
>
>Maybe like windmove-default-keybindings defines modifiers,
>a similar function could set up modifiers for scrolling completions
>from the minibuffer, then the users could choose whether to use
>e.g. 'M-' modifier for scrolling completions, or no modifier at all.
>
I will try that when have some time.

>4. the feature of using completions-highlight-overlay often fails to
>   update highlighting when point is moved in the *Completions* buffer
>   by other navigation commands.
>
>To fix this, it could work like hl-line-mode, but instead of
>highlighting the whole line, it could highlight only completions.
>
Done.

I didn't use the post-command-hook before because I didn't know about it
when I started and then I forgot to fix that. Please give it a try now.

>Then completion highlighting would be useful on its own.
>A new defcustom e.g. 'completion-highlight' (disabled by default)
>could add a post-command-hook like hl-line-mode does.
>Then other completion commands could enable this variable locally.

I made this on the beginning, but I didn't want to add more code to
simple.el...

Maybe not a custom but a minor mode is better and simpler. It is just to
divide my current code in two minor modes instead of just one.

I will consider that...

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

> I made some of the changes you suggested, I think the behavior is a bit
> more reliable now.

Thanks.  Please see more comments.

>>The problem is that such TAB cycling is limited only to the case
>>when the whole *Completions* buffer is visible.
>>
> This was as I said intentional. And to change that could be relatively
> easy but I am not 100% we should. Maybe with a custom in the future...

To be able to merge the branch to master, all its features should work
without problems, but the TAB cycling feature is broken by design:
sometimes it scrolls completions window, sometimes moves to next completion.
This behavior is unpredictable to users.

So I suggest first to implement more straightforward features,
and leave such controversial features at the last thing to do.

>>3. commands that dispatch navigation keys from the minibuffer to the
>>   *Completions* buffer: this doesn't cover other navigation keys.
>>
>>Maybe like windmove-default-keybindings defines modifiers,
>>a similar function could set up modifiers for scrolling completions
>>from the minibuffer, then the users could choose whether to use
>>e.g. 'M-' modifier for scrolling completions, or no modifier at all.
>>
> I will try that when have some time.

This could also allow the user to select what keys to dispatch
to the *Completions* buffer.  For example, TAB and S-TAB move
to the next/previous completion in the *Completions* buffer,
so it makes sense to do the same from the minibuffer as well
(optionally).

>>4. the feature of using completions-highlight-overlay often fails to
>>   update highlighting when point is moved in the *Completions* buffer
>>   by other navigation commands.
>>
>>To fix this, it could work like hl-line-mode, but instead of
>>highlighting the whole line, it could highlight only completions.
>>
> I didn't use the post-command-hook before because I didn't know about it
> when I started and then I forgot to fix that. Please give it a try now.

Highlighting completion under point is useful on its own.
It makes sense for the users to enable it separately from other features,
like hl-line-mode is useful on its own.

> Maybe not a custom but a minor mode is better and simpler. It is just to
> divide my current code in two minor modes instead of just one.

hl-line-mode is a mode because it can be useful in any buffer,
but completion highlighting is useful only in the *Completions* buffer,
so it can be a user option.

>>Then completion highlighting would be useful on its own.
>>A new defcustom e.g. 'completion-highlight' (disabled by default)
>>could add a post-command-hook like hl-line-mode does.
>>Then other completion commands could enable this variable locally.
>
> I made this on the beginning, but I didn't want to add more code to
> simple.el...

Adding much more code to simple.el is undesirable indeed, but
for the completion highlighting feature it would take only
~20 lines of code and option/face definition in minibuffer.el.

All the remaining features need own package file indeed.
BTW, the file name completions-highlight.el is too ugly
as a package name.  An example of a good package/mode name is
icomplete.el.  A new package that allows navigation
of completions from the minibuffer could be like icomplete.el
in other regards too: define the mode/keymap in the same way,
use similar options/hooks, etc.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Ergus]

Hi Juri:

Thanks for your time ;)

On Thu, Nov 19, 2020 at 09:45:04AM +0200, Juri Linkov wrote:
>
>Thanks.  Please see more comments.
>
>
>To be able to merge the branch to master, all its features should work
>without problems, but the TAB cycling feature is broken by design:
>sometimes it scrolls completions window, sometimes moves to next completion.
>This behavior is unpredictable to users.
>
>So I suggest first to implement more straightforward features,
>and leave such controversial features at the last thing to do.
>
I agree. The current tab design is IMO broken by itself but that's not
something I intended to change. In any case the arrow keys are there in
a more consistent way to move... and as we know, there will be never an
agreement on this. So it is better if we find a way to set that
unilaterally to what we think is better and add a custom to disable
it.... More extensions, better alternatives and complains will come out
with the time whatever we do.
>
>This could also allow the user to select what keys to dispatch
>to the *Completions* buffer.  For example, TAB and S-TAB move
>to the next/previous completion in the *Completions* buffer,
>so it makes sense to do the same from the minibuffer as well
>(optionally).
>
I understand the idea, but IMO this is pointless. Because M-* or S-* is
not better than just pageup + whatever. I don't find the zsh behavior
confusing an that was my pattern from the beginning. I am worried
because adding more and more bindings (even with a prefix) may

a) Conflict with existing bindings/default current behavior

b) Lead some useless complex combinations like C-M-someting (that does
not work in the terminal very often BTW)

c) conflict with some frequently user-defined bindings. For example
S-arrow now is used to select the region and M-arrow to backward word

Maybe we should look more how zsh behaves... and try to mimic that as
much as possible. Because it is already pretty consisten
>
>Highlighting completion under point is useful on its own.
>It makes sense for the users to enable it separately from other features,
>like hl-line-mode is useful on its own.
>
>hl-line-mode is a mode because it can be useful in any buffer,
>but completion highlighting is useful only in the *Completions* buffer,
>so it can be a user option.
>
Yes, but this actually works as a hook, so we need to add/remove the
command to/from post-command-hook. That's easier and cleaner with a mode
(or a toggle-something command) than with a custom right?. (please,
remember I don't know the deep lisp secrets, so maybe there is actually
a better way)
>
>Adding much more code to simple.el is undesirable indeed, but
>for the completion highlighting feature it would take only
>~20 lines of code and option/face definition in minibuffer.el.
>
Indeed. I already did that on the beginning, but then I moved it
out. Actually If I can choose it will be better if all the *Completions*
stuff are moved to their own file... But that's a task for a good
developer, not me ;p. There is actuallu the completion.el I am not sure
what's there.

>All the remaining features need own package file indeed.
>BTW, the file name completions-highlight.el is too ugly
>as a package name.  An example of a good package/mode name is
>icomplete.el.  A new package that allows navigation
>of completions from the minibuffer could be like icomplete.el
>in other regards too: define the mode/keymap in the same way,
>use similar options/hooks, etc.

I agree, but we have display-fill-column, display-line-numbers... In any
case I don't care the name at all; so I will just ask here for names,
and when the war ends I just obey the winner orders.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

> c) conflict with some frequently user-defined bindings. For example
> S-arrow now is used to select the region and M-arrow to backward word

But don't arrow keys cause more conflicts?  They are used in the minibuffer
with their default bindings left/right-char, previous/next-line.

This is the reason why icomplete provides less natural keybindings
'C-.' and 'C-,' to move to the previous/next completion.

> Maybe we should look more how zsh behaves... and try to mimic that as
> much as possible. Because it is already pretty consisten

Does zsh use arrow keys to navigate in completions?
How then it's possible to get previous/next history item (up/down arrows),
and move the cursor in the command line (left/right arrows)?

>>Highlighting completion under point is useful on its own.
>>It makes sense for the users to enable it separately from other features,
>>like hl-line-mode is useful on its own.
>>
>>hl-line-mode is a mode because it can be useful in any buffer,
>>but completion highlighting is useful only in the *Completions* buffer,
>>so it can be a user option.
>>
> Yes, but this actually works as a hook, so we need to add/remove the
> command to/from post-command-hook. That's easier and cleaner with a mode
> (or a toggle-something command) than with a custom right?. (please,
> remember I don't know the deep lisp secrets, so maybe there is actually
> a better way)

With a defcustom it's easier to the user to enable completion highlighting
in the *Completions* buffer.  How the user could do the same using hooks?

>>Adding much more code to simple.el is undesirable indeed, but
>>for the completion highlighting feature it would take only
>>~20 lines of code and option/face definition in minibuffer.el.
>>
> Indeed. I already did that on the beginning, but then I moved it
> out. Actually If I can choose it will be better if all the *Completions*
> stuff are moved to their own file... But that's a task for a good
> developer, not me ;p. There is actuallu the completion.el I am not sure
> what's there.

completion.el provides dynamic-completion-mode.

>>All the remaining features need own package file indeed.
>>BTW, the file name completions-highlight.el is too ugly
>>as a package name.  An example of a good package/mode name is
>>icomplete.el.  A new package that allows navigation
>>of completions from the minibuffer could be like icomplete.el
>>in other regards too: define the mode/keymap in the same way,
>>use similar options/hooks, etc.
>
> I agree, but we have display-fill-column, display-line-numbers...

display-fill-column-indicator.el wins the prize for the longest file name
in Emacs repo, display-line-numbers.el is a forerunner :-)

> In any case I don't care the name at all; so I will just ask here for names,
> and when the war ends I just obey the winner orders.

Sorry, I have no enough fantasy to come up with a good name.
Maybe someone could help to invent a better name?

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

>>> Maybe we should look more how zsh behaves... and try to mimic that as
>>> much as possible. Because it is already pretty consisten
>>
>>Does zsh use arrow keys to navigate in completions?
>>How then it's possible to get previous/next history item (up/down arrows),
>>and move the cursor in the command line (left/right arrows)?
>
> Similar to what we are doing.
>
> 1) When NO completions shown: previous/next history and so on.
> 2) When completions are shown: navigate completions.
>
> Completions are only shown after a "tab request" 1->2 (unlike
> incomplete)

Trying to mimic the 'zsh' behavior is a good thing.  Using
'zsh' as a reference, what do you think about naming the package
'zcomplete'.

> If you type any letter completions disappear, inserts a space and then
> the letter, so
>
> backspace just hides completions (deletes the inserted space), so goes
> from 2->1 easily (unlike *Completions*).

If you want to emulate the 'zsh' behavior, then much simpler approach
would be to allow TAB to pop-up the completions buffer, i.e. to display
and switch to it.  Then all motion keys will be available by default.
To display the cursor in the minibuffer is still possible by using
'(cursor-in-echo-area t)'.  RET will insert the completion under point
to the minibuffer.  Any self-inserting key could switch back
to the minibuffer and insert its character.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Ergus]

On Sat, Nov 21, 2020 at 09:30:17PM +0200, Juri Linkov wrote:
>>>> Maybe we should look more how zsh behaves... and try to mimic that as
>>>> much as possible. Because it is already pretty consisten
>>>
>>>Does zsh use arrow keys to navigate in completions?
>>>How then it's possible to get previous/next history item (up/down arrows),
>>>and move the cursor in the command line (left/right arrows)?
>>
>> Similar to what we are doing.
>>
>> 1) When NO completions shown: previous/next history and so on.
>> 2) When completions are shown: navigate completions.
>>
>> Completions are only shown after a "tab request" 1->2 (unlike
>> incomplete)
>
>Trying to mimic the 'zsh' behavior is a good thing.  Using
>'zsh' as a reference, what do you think about naming the package
>'zcomplete'.
>
Ok

>> If you type any letter completions disappear, inserts a space and then
>> the letter, so
>>
>> backspace just hides completions (deletes the inserted space), so goes
>> from 2->1 easily (unlike *Completions*).
>
>If you want to emulate the 'zsh' behavior, then much simpler approach
>would be to allow TAB to pop-up the completions buffer, i.e. to display
>and switch to it.  Then all motion keys will be available by default.
>To display the cursor in the minibuffer is still possible by using
>'(cursor-in-echo-area t)'.  RET will insert the completion under point
>to the minibuffer.  Any self-inserting key could switch back
>to the minibuffer and insert its character.
>
The real issue with that is not setting, but reseting. Because
completions-list-map uses:

1. \e\e\e delete-completion-window
2. q quit-window
3. z kill-current-buffer

to exit. In principle quit-window-hook works for 2. But for 1 we need
an advise and rebind q and 3 is bind to z. so 

I am not clear what's the best approach or the consequences of changing
them. But my proposal will be then make:
q and z: self-insert (as they will call quit-window in the hook any way).

And rebind
C-g: to kill-current-buffer. (I don't really know the difference between
all them)

The hook for self-insert is trivial. But it won't cover commands like
delete-backward-char, backward-kill-word and so on.

The trick there may be to throw the read-only error and call 1, 2 or
3... to go to the minibuffer when any command tries to modify
"Completions* and try to execute there.

But I don't know if it is possible to do that for a whole buffer or a
map in a clean way and locally to a window/buffer only. Is it?

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

> The real issue with that is not setting, but reseting. Because
> completions-list-map uses:
>
> 1. \e\e\e delete-completion-window
> 2. q quit-window
> 3. z kill-current-buffer
>
> to exit. In principle quit-window-hook works for 2. But for 1 we need
> an advise and rebind q and 3 is bind to z. so I am not clear what's the
> best approach or the consequences of changing
> them. But my proposal will be then make:
> q and z: self-insert (as they will call quit-window in the hook any way).

I see no problem in treating q and z as self-inserting characters
in this special mode.

> And rebind
> C-g: to kill-current-buffer. (I don't really know the difference between
> all them)

Or maybe better to bind C-g to abort-recursive-edit.

> The hook for self-insert is trivial. But it won't cover commands like
> delete-backward-char, backward-kill-word and so on.

So it seems more keys need to be bound in the completions buffer
to redirect their effect to the minibuffer.

> The trick there may be to throw the read-only error and call 1, 2 or
> 3... to go to the minibuffer when any command tries to modify
> "Completions* and try to execute there.
>
> But I don't know if it is possible to do that for a whole buffer or a
> map in a clean way and locally to a window/buffer only. Is it?

There keybindings could be added by a new mode enabled in the
completions buffer.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Ergus]

On Sat, Nov 21, 2020 at 09:30:17PM +0200, Juri Linkov wrote:
>>>> Maybe we should look more how zsh behaves... and try to mimic that as
>>>> much as possible. Because it is already pretty consisten
>>>
>>>Does zsh use arrow keys to navigate in completions?
>>>How then it's possible to get previous/next history item (up/down arrows),
>>>and move the cursor in the command line (left/right arrows)?
>>
>> Similar to what we are doing.
>>
>> 1) When NO completions shown: previous/next history and so on.
>> 2) When completions are shown: navigate completions.
>>
>> Completions are only shown after a "tab request" 1->2 (unlike
>> incomplete)
>
>Trying to mimic the 'zsh' behavior is a good thing.  Using
>'zsh' as a reference, what do you think about naming the package
>'zcomplete'.
>
>> If you type any letter completions disappear, inserts a space and then
>> the letter, so
>>
>> backspace just hides completions (deletes the inserted space), so goes
>> from 2->1 easily (unlike *Completions*).
>
>If you want to emulate the 'zsh' behavior, then much simpler approach
>would be to allow TAB to pop-up the completions buffer, i.e. to display
>and switch to it.  Then all motion keys will be available by default.
>To display the cursor in the minibuffer is still possible by using
>'(cursor-in-echo-area t)'.  RET will insert the completion under point
>to the minibuffer.  Any self-inserting key could switch back
>to the minibuffer and insert its character.
>
Hi Juri:

Could you try it now please?

I am wondering that with this new behavior it is a bit more like zsh but
diverges a bit too much from the current default emacs completion, so
making this default will be virtually impossible. And my real objective
was to improve the default completion; not make a new one.

IMO we may reconsider if we should go in the direction of commit
60a1003aee or in the one of c7c47e78e6.

Best,
Ergus

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

> Could you try it now please?

Thanks, now this feature is getting into good shape, and it seems soon
your branch will be ready for merging.

> I am wondering that with this new behavior it is a bit more like zsh but
> diverges a bit too much from the current default emacs completion, so
> making this default will be virtually impossible. And my real objective
> was to improve the default completion; not make a new one.

Don't hold out too much hope on enabling this feature by default.
Even in zsh this completion style is not enabled by default,
only adding "zstyle ':completion:*' menu select" enables it.

Also in Emacs even less intrusive icomplete-mode is not enabled
by default.

For example, currently when the completions buffer is displayed,
C-r is still available for searching through the previous history.
But when this new mode is enabled, it's more useful to allow C-r
to search in the completions buffer, that works well with your
latest commit.

IOW, the main difference of this feature is that typing TAB
enables a special mode for selecting a completion, and disables
history navigation and motion keys in the minibuffer.

> IMO we may reconsider if we should go in the direction of commit
> 60a1003aee or in the one of c7c47e78e6.

Yes, these are two diverging roads.  So we need to try both, identify
their advantages/disadvantages to decide what path/patch to follow.

With c7c47e78e6, all present motion keys are still available in the
completions buffer, but are all keys that used to exit still handled?

Also setting buffer-local cursor-in-echo-area to t in the completions
buffer provides a better look and feel of editing in the minibuffer
(like you disabled mode-line).

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Ergus]

On Mon, Nov 23, 2020 at 11:14:47AM +0200, Juri Linkov wrote:
>> Could you try it now please?

IMO icomplete (and ido/ivy/helm/etc) are more invasive than this mode
(as I conceived it initially at least) because they diverged much more
from the *Completions* way to do and the <tab> philosophy. I would
actually prefer to remove features from this one if there is a chance to
add some of these improvements to the default completion system (after
the expected war of course). Maybe making it less "zsh-ich" and just
improving a bit the Completions interaction may be possible??

In case you really think that nothing of this may become default then I
prefer to add this code as a package to elpa instead; to not overload
the base code with potentially useless/unknown modes that nobody will
find useful/discover (once a user is capable to configure their init.el
they will go for icomplete/ido/fido/ivy/helm instead of this).

I think very few old users use the default completions system these days
and it gives a terrible first impression to new comers and make emacs
feel too "vintage".

That's what I want to improve. If the changes we do are conservative
enough (maybe just like the first version I proposed: add tabs, arrows
interaction and highlight after an extra tab) It may not bother old
users while the new will find the more interactive behavior after the
extra tab very handy.


>
>Also setting buffer-local cursor-in-echo-area to t in the completions
>buffer provides a better look and feel of editing in the minibuffer
>(like you disabled mode-line).

Ahh I forgot to mention.

In gui interface cursor-in-echo-area works fine (setting the cursor in
the "right" position) but in -nw the cursor is set at the end of the
line and leaves an extra space.

I am not sure if this is an issue or a feature IMO it is an issue
because behavior must be the same either in tui or gui.

I even considered to set an overlay but maybe this is something must be
properly fixed (if it is an issue of course)

Best,
Ergus

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Jean Louis]

* Ergus <spacibba@aol.com> [2020-11-23 14:58]:
> IMO icomplete (and ido/ivy/helm/etc) are more invasive than this mode
> (as I conceived it initially at least) because they diverged much more
> from the *Completions* way to do and the <tab> philosophy.

TAB philosophy is for line based completions. Anything else need not
use TAB. Ivy is not based on TAB it is based on visual selection.

Helm is specifically not based on TAB and information about TAB can
be found on Helm Wiki.  

Emacs Helm Wiki
https://github.com/emacs-helm/helm/wiki

> In case you really think that nothing of this may become default then I
> prefer to add this code as a package to elpa instead; to not
> overload

Please give me reference that I can try that package to see how well
it fits in my needs.

> the base code with potentially useless/unknown modes that nobody will
> find useful/discover (once a user is capable to configure their init.el
> they will go for icomplete/ido/fido/ivy/helm instead of this).

For your research:

icomplete/ido/fido are built-in completions that I never use, simply
never. In fact I find them dangerous as unclear usage and bad design
(personal view and case of course) can easily lose files or complete
something else what is not meant to be.

ivy is in GNU ELPA and that is great, I am using ivy to visually
locate specific sets in the database or in database editing functions.

helm I use also for database work where visual selections and
filtering is necessary.

For switch to buffer I do use often either ivy or helm, it does not
matter.

But majority of my time I do not use those non built-in
completion. So majority of the time I am not using completions as the
Emacs built-in completion is equally fast, more precise and less
confusing!

If I would count number of keys I have to press in ivy to complete to
specific directory I think it is even more than necessary, same for
helm.

> I think very few old users use the default completions system these days
> and it gives a terrible first impression to new comers and make emacs
> feel too "vintage".

I do not share that idea, quite contrary I find Emacs built-in
standard completion very fine and pretty fast feature. When you are
familiar with it, it becomes easy. To users in my group so far I do
not even recommend anything else but the built-in.

Often I switch from one buffer to other, it is trivial to come back by
using built-in completion.

There are cases where visual completion really helps, but it does not
necessarily make things faster!

> I even considered to set an overlay but maybe this is something must be
> properly fixed (if it is an issue of course)

Any completion shall work well in console mode.

Please send me that package that I try it out.
Thank you.

Jean

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Eli Zaretskii]

> Date: Mon, 23 Nov 2020 17:13:54 +0300
> From: Jean Louis <bugs@gnu.support>
> Cc: emacs-devel@gnu.org, Stefan Monnier <monnier@iro.umontreal.ca>,
>  Drew Adams <drew.adams@oracle.com>, Juri Linkov <juri@linkov.net>
> 
> TAB philosophy is for line based completions. Anything else need not
> use TAB.

And yet Bash completion uses TABs, but displays multi-line completion
lists.  And so does GDB (actually, Readline).

So I don't agree with your philosophical assessment.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Jean Louis]

* Eli Zaretskii <eliz@gnu.org> [2020-11-23 22:13]:
> > Date: Mon, 23 Nov 2020 17:13:54 +0300
> > From: Jean Louis <bugs@gnu.support>
> > Cc: emacs-devel@gnu.org, Stefan Monnier <monnier@iro.umontreal.ca>,
> >  Drew Adams <drew.adams@oracle.com>, Juri Linkov <juri@linkov.net>
> > 
> > TAB philosophy is for line based completions. Anything else need not
> > use TAB.
> 
> And yet Bash completion uses TABs, but displays multi-line completion
> lists.  And so does GDB (actually, Readline).
> 
> So I don't agree with your philosophical assessment.

No need to agree. Let me express me better:

When I write words to select from a list of candidates in helm, I can
write anything like for the file sat_media I can write: at me

I am not referring to how completion shows its candidates but that by
using TAB the line is being expanded to completion. TAB completion
cannot be expanded by writing anything by common implementations. Some
new implementation could do that too, it is just not common or
non-existent.

To expand the file sat_media what is common is to start typing begin
of the file in Bash like saTAB to get sat_media.

But I cannot write mediaTAB to get sat_media, so TAB expands lines.

That it displays multiline completion lists is fine and good, but it
will not search by parts of the words like non-TAB completions. Though
it could.

Helm does not expand by TAB, it expands by anything, if I write
~/tmp/at me I would get ~/tmp/sat_media so it is not line based
completion (line is not expanded) it is visual choice completion and
the select key does not matter much, one could select TAB as well in
that case.

Emacs enhances line based completion that one can use at least joker
*, so I could write *atTAB to get maybe sat_media.

But I cannot write ~/tmp/*at m to get ~/tmp/sat_media though
implementation could be quite different and it could be possible.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Dmitry Gutov]

On 23.11.2020 13:46, Ergus wrote:
> In case you really think that nothing of this may become default then I
> prefer to add this code as a package to elpa instead; to not overload
> the base code with potentially useless/unknown modes that nobody will
> find useful/discover (once a user is capable to configure their init.el
> they will go for icomplete/ido/fido/ivy/helm instead of this)

That's a very good point.

(I haven't tried the proposed feature yet, sorry.)

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Ergus via Emacs development discussions.]

[-- Attachment #1: Type: text/plain, Size: 1182 bytes --]

Maybe the question to do now is: 

Which improvements do you thing could be accepted in the Default completions to modernize it a bit and improve the interaction/look. I mean specially for the first impression and to improve the functionality when running with -Q.

I thought that the first versions of the patch may be more or less good enough because it was just extending the default behavior a little bit when tabs where unused. With minimal interaction with the default behavior but easy to discover.

Best,
Ergus


On November 23, 2020 9:54:13 PM GMT+01:00, Dmitry Gutov <dgutov@yandex.ru> wrote:
>On 23.11.2020 13:46, Ergus wrote:
>> In case you really think that nothing of this may become default then
>I
>> prefer to add this code as a package to elpa instead; to not overload
>> the base code with potentially useless/unknown modes that nobody will
>> find useful/discover (once a user is capable to configure their
>init.el
>> they will go for icomplete/ido/fido/ivy/helm instead of this)
>
>That's a very good point.
>
>(I haven't tried the proposed feature yet, sorry.)

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 1581 bytes --]

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Dmitry Gutov]

On 24.11.2020 01:27, Ergus wrote:
> Maybe the question to do now is:
> 
> Which improvements do you thing could be accepted in the Default 
> completions to modernize it a bit and improve the interaction/look. I 
> mean specially for the first impression and to improve the functionality 
> when running with -Q.

FWIW, this zsh-inspired completion looks better to me than the current 
default one. It's just snappier.

And I'm not even a zsh user.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

>> Maybe the question to do now is:
>> Which improvements do you thing could be accepted in the Default
>> completions to modernize it a bit and improve the
>> interaction/look. I mean specially for the first impression and to
>> improve the functionality when running with -Q.
>
> FWIW, this zsh-inspired completion looks better to me than the current
> default one. It's just snappier.

I like zcomplete as well, but the problem is how to activate it unobtrusively?
So that before activation navigation keys retain their normal meanings,
but after activation they navigate in the completions list.  zsh uses
TAB TAB to activate completions navigation, but in the Emacs minibuffer
TAB TAB browses the completions list pagewise.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

> I would actually prefer to remove features from this one if there is
> a chance to add some of these improvements to the default completion
> system (after the expected war of course). Maybe making it less
> "zsh-ich" and just improving a bit the Completions interaction may be
> possible??

There are already features in your branch that would be useful
in the default completion system.

Please see a new API developed on emacs-devel with subject
"on helm substantial differences" that allows adding
a prefix to completions.  When the option 'completions-detailed'
is enabled, then e.g. 'C-h o' adds a prefix to completions.

The problem is that the natural thing to do is to move point
down to the needed completion in the *Completions* buffer
and type RET.  But currently it raises an error:

  Debugger entered--Lisp error: (error "No completion here")
    signal(error ("No completion here"))
    error("No completion here")
    choose-completion(13)
    funcall-interactively(choose-completion 13)
    call-interactively(choose-completion nil nil)
    command-execute(choose-completion)

This is because currently 'choose-completion' is too limited.
It doesn't accept the selected completion when point is on its prefix.
Your function 'zcomplete-select-near' could help in this case.
Could you please move it from your branch to minibuffer.el.

> If the changes we do are conservative enough (maybe just like the
> first version I proposed: add tabs, arrows interaction and highlight
> after an extra tab) It may not bother old users while the new will
> find the more interactive behavior after the extra tab very handy.

To be able to enable this mode by default, it should be less invasive.
Maybe it should be activated by pressing some key other than TAB TAB?
Or some option could be added to choose how it should be activated?

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Stefan Monnier]

>>Adding much more code to simple.el is undesirable indeed, but
>>for the completion highlighting feature it would take only
>>~20 lines of code and option/face definition in minibuffer.el.
> Indeed. I already did that on the beginning, but then I moved it
> out. Actually If I can choose it will be better if all the *Completions*
> stuff are moved to their own file...

Hmm... minibuffer.el *is* "their own file".


        Stefan

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Stefan Monnier]

>>>>Adding much more code to simple.el is undesirable indeed, but
>>>>for the completion highlighting feature it would take only
>>>>~20 lines of code and option/face definition in minibuffer.el.
>>> Indeed. I already did that on the beginning, but then I moved it
>>> out. Actually If I can choose it will be better if all the *Completions*
>>> stuff are moved to their own file...
>>Hmm... minibuffer.el *is* "their own file".

> Yes, of course, But not *Completions*. All *Completions* related functions
> are in simple.el including all the completion formatting and so on.

Oh you mean the functions around completion-list-mode?
Yes, these should arguably move from simple.el to minibuffer.el.


        Stefan

RE: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Drew Adams]

> > Cycling is actually a fairly inefficient, unconscious
> > way to navigate among candidates.  It's handy, but
> > dumb cycling shouldn't be the only or the main way to
> > get around.
> 
> I agree.  I tried the completions-highlight-modifications branch, but
> unfortunately restricting a set of navigation keys limits the freedom
> of using any navigation key for browsing completion candidates that is
> available today.  What I mean is that currently it's possible in the
> minibuffer to type <PageUp> (switch-to-completions), and then all keys
> are available for selecting a completion: arrow keys, page scrolling
> keys - even completion drop-down combo in browser's address bar allows
> page scrolling.

I expressed that (and meant it) generally.  But in
particular I meant that the real power of completion
(matching/filtering) is your friend.  Cycling is
handiest when combined with that.  (And in particular
when combined with progressive completion.)
___

As for navigating among candidates, cycling is fine
if you also have an easy way to get directly to a
given candidate (and so restart cycling from there).

As for scrolling, in Icicles the usual `C-v' and
`M-v' scroll *Completions*.  It was always misguided
for vanilla Emacs to use TAB for that, IMO.
___

And yes, there should be keys to toggle the input focus.

Icicles uses `C-insert' to move the cursor back and
forth between the minibuffer and *Completions*.  It
moves between the current candidate in the minibuffer
(from cycling) to/from the same candidate in
*Completions*.

Likewise, to move back & forth between the minibuffer
and the buffer from which you activated the minibuffer.
Icicles uses the `pause' key for that.  This lets you
interrupt completion to search, edit, etc. in the
original buffer, without needing a recursive edit.
___

https://www.emacswiki.org/emacs/Icicles_-_Moving_Between_the_Minibuffer_and_Other_Buffers

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

> Icicles uses `C-insert' to move the cursor back and
> forth between the minibuffer and *Completions*.  It
> moves between the current candidate in the minibuffer
> (from cycling) to/from the same candidate in
> *Completions*.

What the default completions implementation is still missing
is a command/key to insert the selected completion
from the the *Completions* buffer to the minibuffer
without exiting the minibuffer.

RE: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Drew Adams]

> > Icicles uses `C-insert' to move the cursor back and
> > forth between the minibuffer and *Completions*.  It
> > moves between the current candidate in the minibuffer
> > (from cycling) to/from the same candidate in
> > *Completions*.
> 
> What the default completions implementation is still missing
> is a command/key to insert the selected completion
> from the the *Completions* buffer to the minibuffer
> without exiting the minibuffer.

Does it have a key that moves from a completion candidate
in the minibuffer (e.g. from cycling) to the same candidate
in *Completions*?  That's also missing, AFAIK, in addition
to placing the current completion in the minibuffer as you
cycle.  No?

The same key should be used for both operations: minibuffer
to *Completions* and *Completions* to minibuffer.  And it
should go to the "current" completion candidate in each case.

And it should be a key that's not ordinarily used for editing,
if it's made available in the minibuffer.  For example, we
shouldn't use a key such as `insert'.
___

It's already the case that perfectly fine self-inserting keys
(`?', `SPC') have been excluded, in the minibuffer, from their
ordinary behavior (self-inserting).

That was misguided, IMO.  It took us decades to finally get
`SPC' to be self-inserting, and even then, it's only for
`read-file-name', not `completing-read'.  Why it was assumed
that completion candidates would/should not contain space
chars or question marks, I don't know.

Re: feature/completions-highlight-modifications e3c5b99 3/6: Add completions-highlight-mode initial implementation.

[Juri Linkov]

> What the default completions implementation is still missing
> is a command/key to insert the selected completion
> from the the *Completions* buffer to the minibuffer
> without exiting the minibuffer.

Actually this is easily customizable by such snippet, so no problem.

#+begin_src emacs-lisp
(defun choose-completion-no-auto-exit ()
  (interactive)
  (let ((completion-no-auto-exit t))
    (call-interactively 'choose-completion)))

(define-key completion-list-mode-map [C-return] 'choose-completion-no-auto-exit)
#+end_src