RE: on helm substantial differences

RE: on helm substantial differences

[Drew Adams]

> Features wanted in ivy or icomplete or anything inside
> of Emacs or in GNU ELPA:
> 
> - full frame completion with mode line staying where it is
> - split window completion as helm does it, with mode line
>   staying where it should be

And you mentioned vertical presentation of candidates.

FWIW, Icicles offers that.  You can show candidates
vertically, in a single column *Completions*, if you set
option `icicle-Completions-max-columns' to 1.

There are many kinds of completion, and many kinds of
completion candidates.  For short candidates, a single
column wastes an incredible amount of screen real estate.

That may be fine, if your only purpose is to narrow to
a particular candidate.  But if you're operating on or
otherwise examining the _set_ of current matches in some
way, then you likely want to see more than just a few of
them, and being able to show more than one on a single
line helps a lot.

On-the-fly filtering is about many things.  What's
really involved is manipulating a candidate _set_.
For that, it helps to see most or all of the set.

The design of Icicles recognizes this high degree of
variability in candidates (including display size) and
in uses of filtering and manipulations of a set of
candidates.

In particular, you can change the *Completions* display
on the fly, if you also use library Do Re Mi, just by
using `C-x w' or `C-x |' in the minibuffer, followed by
arrow keys or mouse wheel to increment and decrement the
values of options `icicle-inter-candidate-min-spaces' and
`icicle-candidate-width-factor'.  That is, you can
incrementally change how many columns of candidates are
displayed (`C-x w') and the space used between candidates
(`C-x |').

If all kinds of candidates looked basically similar, then
there'd be no need to be able to tweak the display on the
fly.  And yes, many kinds of candidates do look basically
similar (function names, buffer names, etc.), so you don't
need to change the display often.  But when it makes sense
to do so, you can easily do it, immediately.

https://www.emacswiki.org/emacs/Icicles_-_Completions_Display

Re: on helm substantial differences

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2020-11-12 20:43]:
> > Features wanted in ivy or icomplete or anything inside
> > of Emacs or in GNU ELPA:
> > 
> > - full frame completion with mode line staying where it is
> > - split window completion as helm does it, with mode line
> >   staying where it should be
> 
> And you mentioned vertical presentation of candidates.
> 
> FWIW, Icicles offers that.  You can show candidates
> vertically, in a single column *Completions*, if you set
> option `icicle-Completions-max-columns' to 1.

I am testing it.

Re: on helm substantial differences

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2020-11-12 20:43]:
> > Features wanted in ivy or icomplete or anything inside
> > of Emacs or in GNU ELPA:
> > 
> > - full frame completion with mode line staying where it is
> > - split window completion as helm does it, with mode line
> >   staying where it should be
> 
> And you mentioned vertical presentation of candidates.
> 
> FWIW, Icicles offers that.  You can show candidates
> vertically, in a single column *Completions*, if you set
> option `icicle-Completions-max-columns' to 1.

Our conversation helped me to make some conclusions.

What I need is how you and others pointed out is real time dynamic
filtering or searching. It is filtering when a present display of
hyperlinks has to be filtered.

It is searching when completion encompasses all nodes that are not
only in the current selection. In that case it positions the cursor in
possibly different set of the tree.

Completion as such, as it works in built-in Emacs or any other, like
ivy, icicles, helm, is working well for that functionality to locate
the node or to find other set with searched node.

Additional feature that would spare much time for user is real time
dynamic filtering for tabulated-set-mode. I will see how that can be
implemented.

By functionality it would be similar to completing-read, though not
same.

> There are many kinds of completion, and many kinds of
> completion candidates.  For short candidates, a single
> column wastes an incredible amount of screen real estate.

That may be true for various other use cases. For this particular
usage candidates are mixture of very long lines and sometimes short
lines. Also good for thinking about that.

tabulated-list-mode does allow wide or widening of candidates I think
indefinitely. That helps the user to read what is really there as
completion candidate or to obtain more information. For example I can
go with C-e to the far left side of the text, there could be even
right screen or several screens from left to right which give various
information pertaining to the hyperlink while priority information is
shown on first screen only.

completing-read functions would mostly be limited in that regard, I
would need to shorten information to accommodate completing-read. New
feature shall rather accommodate the program.

> That may be fine, if your only purpose is to narrow to
> a particular candidate.

That is one usage case. completing-read finds the candidate, but then
displays all surrounding candidates in the set of the selected
candidate. In the next step is possible to either examine or activate
the hyperlink or do other actions.

> But if you're operating on or otherwise examining the _set_ of
> current matches in some way, then you likely want to see more than
> just a few of them, and being able to show more than one on a single
> line helps a lot.
> 
> On-the-fly filtering is about many things.  What's
> really involved is manipulating a candidate _set_.
> For that, it helps to see most or all of the set.

Yes.

> The design of Icicles recognizes this high degree of
> variability in candidates (including display size) and
> in uses of filtering and manipulations of a set of
> candidates.

For now icicles are functional and I can use it. In general I will
leave to user which completing-read or package to use. 

Overall discussion helped me to find various other similar
projects.

As the words I used were not confirming to understanding of other
people and I have got various other references that relate to
HyperScope.

GNOWSYS: A Kernel for Semantic Computing!
https://www.gnu.org/software/gnowsys

gstudio
https://gitlab.com/gnowledge/gstudio

Re: on helm substantial differences

[Juri Linkov]

> In particular, you can change the *Completions* display on the fly

It would be nice if in addition to already existing 'annotation-function'
'completing-read' also supported 'format-function' or 'display-function',
i.e. a function that would return a completion candidate to display
in the *Completions* buffer.  This will allow completions to use
the same format that is typical to a package's default format, e.g.
typing TAB in the 'C-x d' minibuffer could display completions in the
the *Completions* buffer in Dired format:

  Click on a completion to select it.
  In this buffer, type RET to select the completion near point.
  Possible completions are:
  -rw-rw-r-- 1   185,205 2020-11-14 21:52 minibuffer.el
  -rw-rw-r-- 1     8,386 2020-11-14 21:52 minibuf-eldef.el
  …

'M-x package-install RET a TAB' could display completions in 'package' format:

  Click on a completion to select it.
  In this buffer, type RET to select the completion near point.
  Possible completions are:
  ada-mode       7.1.4     available   gnu   major-mode for editing Ada sources
  ada-ref-man    2020.1    available   gnu   Ada Reference Manual 2012
  …

'C-x b' - completions with the format of 'list-buffers', etc.

Re: on helm substantial differences

[Stefan Monnier]

>> In particular, you can change the *Completions* display on the fly
> It would be nice if in addition to already existing 'annotation-function'
> 'completing-read' also supported 'format-function' or 'display-function',
> i.e. a function that would return a completion candidate to display
> in the *Completions* buffer.

Agreed: the `annotation-function` is much too limited.
I don't think "in addition" is right tho: the new functionality should
aim to replace the current `annotation-function`.

Tangentially related are the :company-doc-buffer, :company-docsig,
and :company-location properties, which aren't documented (and were
designed simply to match what `company-mode` needed), and might deserve
to be more properly added to the official API.


        Stefan

RE: on helm substantial differences

[Drew Adams]

> > It would be nice if in addition to already existing 'annotation-
> > function' 'completing-read' also supported 'format-function' or
> > 'display-function', i.e. a function that would return a completion
> > candidate to display in the *Completions* buffer.
> 
> Agreed: the `annotation-function` is much too limited.
> I don't think "in addition" is right tho: the new functionality should
> aim to replace the current `annotation-function`.

I'm not sure I agree.  Candidate display can be open to
all kinds of programmatic control (in Icicles, at least).

And there's the question of the relation between a "display
candidate" (what you see) and the "real" candidate that you
can _act_ on.  For example, a relative file name is shown
but you act on an absolute file name (or the file).

And there can be a difference between what is displayed and
what you _match_ against.  For example, your input is not
matched against a candidate annotation - that's just extra.

All of these possibilities are available.  Sometimes you
want matching to go against all parts of a display candidate,
sometimes you don't.  Sometimes you want to be able to match
also against some candidate parts that are not displayed
(the display is kept simple, or matching those unshown parts
is optional).

There are lots of possibilities.  (Icicles handles these in
its version of `completing-read', while keep that function's
argument list the same as vanilla.)

Re: on helm substantial differences

[Juri Linkov]

>> It would be nice if in addition to already existing 'annotation-function'
>> 'completing-read' also supported 'format-function' or 'display-function',
>> i.e. a function that would return a completion candidate to display
>> in the *Completions* buffer.
>
> Agreed: the `annotation-function` is much too limited.
> I don't think "in addition" is right tho: the new functionality should
> aim to replace the current `annotation-function`.

Then maybe to differentiate one from another: when the string
returned from `annotation-function` has text properties then
use it to display a complete candidate, otherwise append it
as an annotation - for backward compatibility.

RE: on helm substantial differences

[Drew Adams]

> > the `annotation-function` is much too limited.  I don't
> > think "in addition" is right tho: the new functionality
> > should aim to replace the current `annotation-function`.
> 
> Then maybe to differentiate one from another: when the string
> returned from `annotation-function` has text properties then
> use it to display a complete candidate, otherwise append it
> as an annotation - for backward compatibility.

That would be extremely limiting.  It would mean you
couldn't use text properties other than to mean that.

I already gave reasons against the general "replacement"
idea.  But this more restrictive idea is especially not
good.  If some such thing were introduced it should be
based on a particular (new) text property, not just the
presence of any text properties.

Re: on helm substantial differences

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: Jean Louis <bugs@gnu.support>,  Gregory Heytings <ghe@sdf.org>,
>   spacibba@aol.com,  andreyk.mad@gmail.com,  emacs-devel@gnu.org,
>   rudalics@gmx.at,  monnier@iro.umontreal.ca,  Eli Zaretskii <eliz@gnu.org>
> Date: Sun, 15 Nov 2020 22:07:47 +0200
> 
> It would be nice if in addition to already existing 'annotation-function'
> 'completing-read' also supported 'format-function' or 'display-function',
> i.e. a function that would return a completion candidate to display
> in the *Completions* buffer.  This will allow completions to use
> the same format that is typical to a package's default format, e.g.
> typing TAB in the 'C-x d' minibuffer could display completions in the
> the *Completions* buffer in Dired format:
> 
>   Click on a completion to select it.
>   In this buffer, type RET to select the completion near point.
>   Possible completions are:
>   -rw-rw-r-- 1   185,205 2020-11-14 21:52 minibuffer.el
>   -rw-rw-r-- 1     8,386 2020-11-14 21:52 minibuf-eldef.el
>   …
> 
> 'M-x package-install RET a TAB' could display completions in 'package' format:
> 
>   Click on a completion to select it.
>   In this buffer, type RET to select the completion near point.
>   Possible completions are:
>   ada-mode       7.1.4     available   gnu   major-mode for editing Ada sources
>   ada-ref-man    2020.1    available   gnu   Ada Reference Manual 2012
>   …
> 
> 'C-x b' - completions with the format of 'list-buffers', etc.

This will have to be an opt-in feature.  It presents completion
candidates in such a radically different form that people will be
confused; I would be.  We cannot change the default behavior so
radically, and on top of that make the display different in each mode.

Thanks.

Re: on helm substantial differences

[Stefan Monnier]

>> Agreed: the `annotation-function` is much too limited.
>> I don't think "in addition" is right tho: the new functionality should
>> aim to replace the current `annotation-function`.
> Then maybe to differentiate one from another: when the string
> returned from `annotation-function` has text properties then
> use it to display a complete candidate, otherwise append it
> as an annotation - for backward compatibility.

That's a possibility, yes.  Tho we could also use another
property and deprecate the `annotation-function`.

Before knowing what's the best approach, I think we should clearly
decide what would be the "ideal" new API.  E.g. should it return "any
string" and then it'd be up to the infrastructure code to store
side-info about what is the corresponding candidate's actual text?
Or should it instead return a string with additional text-properties
explaining which parts are annotations and which part is the actual
candidate's text?  Or should it return a list of strings? ...


        Stefan

Re: on helm substantial differences

[Juri Linkov]

> Before knowing what's the best approach, I think we should clearly
> decide what would be the "ideal" new API.  E.g. should it return "any
> string" and then it'd be up to the infrastructure code to store
> side-info about what is the corresponding candidate's actual text?
> Or should it instead return a string with additional text-properties
> explaining which parts are annotations and which part is the actual
> candidate's text?  Or should it return a list of strings? ...

An additional question is at what stage to add faces like
'completions-common-part'?  Currently annotations are appended
after all commonality highlighting is done.  But with returning
a completion string, I guess such faces should be added
on the returned string at the last stage.

Re: on helm substantial differences

[Juri Linkov]

> This will have to be an opt-in feature.  It presents completion
> candidates in such a radically different form that people will be
> confused; I would be.  We cannot change the default behavior so
> radically, and on top of that make the display different in each mode.

Like there are already possible choices 'horizontal' and 'vertical'
in the user option 'completions-format', it could also support
a new choice like 'rich'.

RE: on helm substantial differences

[Drew Adams]

> > This will have to be an opt-in feature.  It presents completion
> > candidates in such a radically different form that people will be
> > confused; I would be.  We cannot change the default behavior so
> > radically, and on top of that make the display different in each
> mode.
> 
> Like there are already possible choices 'horizontal' and 'vertical'
> in the user option 'completions-format', it could also support
> a new choice like 'rich'.

`rich' is a poor choice, IMO - it doesn't mean much.

(Even Donald Trump pretends to be rich.)

Re: on helm substantial differences

[Stefan Monnier]

>> Before knowing what's the best approach, I think we should clearly
>> decide what would be the "ideal" new API.  E.g. should it return "any
>> string" and then it'd be up to the infrastructure code to store
>> side-info about what is the corresponding candidate's actual text?
>> Or should it instead return a string with additional text-properties
>> explaining which parts are annotations and which part is the actual
>> candidate's text?  Or should it return a list of strings? ...
> An additional question is at what stage to add faces like
> 'completions-common-part'?  Currently annotations are appended
> after all commonality highlighting is done.  But with returning
> a completion string, I guess such faces should be added
> on the returned string at the last stage.

The way faces like `completions-common-part` are applied is sometimes
problematic, indeed, but in this particular case I don't see what's
problematic about it: (as long as the matching is done on the
un-annotated candidates) I can't see why you're prefer to apply faces
like `completions-common-part` after adding annotations rather
than before.


        Stefan

Re: on helm substantial differences

[Juri Linkov]

>> Like there are already possible choices 'horizontal' and 'vertical'
>> in the user option 'completions-format', it could also support
>> a new choice like 'rich'.
>
> `rich' is a poor choice, IMO - it doesn't mean much.

This is a reference to the package https://github.com/Yevgnen/ivy-rich
Ivy supports a richer format of completions, and there no reason
not to do something similar in the default completion framework.

RE: on helm substantial differences

[Drew Adams]

> >> Like there are already possible choices 'horizontal' and 'vertical'
> >> in the user option 'completions-format', it could also support
> >> a new choice like 'rich'.
> >
> > `rich' is a poor choice, IMO - it doesn't mean much.
> 
> This is a reference to the package ivy-rich

Is it?  What says that?

The name itself doesn't say that.  If that's important
then call it `ivy-rich', not `rich'.  But that just
punts, no?  Then someone needs to find that package
and see what it says "rich" means here.

Just because some other package uses a vague name like
that, does that mean Emacs shouldn't do better?

Might as well call it `enhanced', `special', `super',
or `wonderful'.  Does "rich" have a meaning that
suggests something here other than "full" or "great"?

> Ivy supports a richer format of completions,

Which means what, exactly?  Doesn't mean anything to
me, except to suggest that it uses a different format
that's somehow better than the other formats (how so?).

> and there no reason not to do something similar
> in the default completion framework.

I'd suggest the opposite: there's no reason to be as
unclear in the default framework.

If you call it `ivy-rich' then OK.  Someone can
search for what that might really mean.

Just one opinion.  Doesn't really matter to me what
you call it.

Re: on helm substantial differences

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: drew.adams@oracle.com,  bugs@gnu.support,  ghe@sdf.org,
>   spacibba@aol.com,  andreyk.mad@gmail.com,  emacs-devel@gnu.org,
>   rudalics@gmx.at,  monnier@iro.umontreal.ca
> Date: Mon, 16 Nov 2020 22:41:06 +0200
> 
> > This will have to be an opt-in feature.  It presents completion
> > candidates in such a radically different form that people will be
> > confused; I would be.  We cannot change the default behavior so
> > radically, and on top of that make the display different in each mode.
> 
> Like there are already possible choices 'horizontal' and 'vertical'
> in the user option 'completions-format', it could also support
> a new choice like 'rich'.

That'd be fine, I think.  But we should try to come up with a name
that is more descriptive, for example 'mode-specific' or something, to
make it more explicit that this is not just about arranging the
candidates on display.

Re: on helm substantial differences

[Juri Linkov]

>> > `rich' is a poor choice, IMO - it doesn't mean much.
>>
>> This is a reference to the package ivy-rich
>
> Might as well call it `enhanced', `special', `super',
> or `wonderful'.  Does "rich" have a meaning that
> suggests something here other than "full" or "great"?

If you don't like "rich" do you think "wise" is better?

Whereas current completions are column-oriented,
completions in the proposed format are line-wise.

Re: on helm substantial differences

[Protesilaos Stavrou]

On 2020-11-16, 16:04 -0800, Drew Adams <drew.adams@oracle.com> wrote:

>> >> Like there are already possible choices 'horizontal' and 'vertical'
>> >> in the user option 'completions-format', it could also support
>> >> a new choice like 'rich'.
>> >
>> > `rich' is a poor choice, IMO - it doesn't mean much.
>> 
>> This is a reference to the package ivy-rich
>
> Is it?  What says that?
>
> The name itself doesn't say that.  If that's important
> then call it `ivy-rich', not `rich'.  But that just
> punts, no?  Then someone needs to find that package
> and see what it says "rich" means here.
>
> Just because some other package uses a vague name like
> that, does that mean Emacs shouldn't do better?
>
> Might as well call it `enhanced', `special', `super',
> or `wonderful'.  Does "rich" have a meaning that
> suggests something here other than "full" or "great"?

What 'ivy-rich' does is complement the candidates of 'ivy-mode' with
pertinent information.  For example, standard Ivy shows a vertical list
consisting only of buffer names, while 'ivy-rich' adds to them their
major mode and file system path, filling up the empty space.

Couched in those terms, "rich" could be replaced with something like
"detailed", "descriptive", "informative"...  Or, if you want to be
precise, prepend "vertical-" to those options.

-- 
Protesilaos Stavrou
protesilaos.com

RE: on helm substantial differences

[Drew Adams]

> >> > `rich' is a poor choice, IMO - it doesn't mean much.
> >>
> >> This is a reference to the package ivy-rich
> >
> > Might as well call it `enhanced', `special', `super',
> > or `wonderful'.  Does "rich" have a meaning that
> > suggests something here other than "full" or "great"?
> 
> If you don't like "rich" do you think "wise" is better?

No, not I.  "Wise" doesn't tell me anything about it.
Nor would "DWIM".  Nor would "smart".  Just one opinion.

I agree with Eli that the name can better describe the
thing.  And I already said that, if no better name can
be found, then `ivy-rich' would be OK, if "ivy-rich"
is in fact what it's called outside vanilla Emacs and
users can find a description by googling for "ivy rich"
or "ivy-rich".

> Whereas current completions are column-oriented,
> completions in the proposed format are line-wise.

Sorry, I don't know what it means for completions to be
column-oriented or line-oriented.  But if you're saying
that that's the distinction, then sure, maybe a name
that covers that would be appropriate.

RE: on helm substantial differences

[Drew Adams]

> What 'ivy-rich' does is complement the candidates of 'ivy-mode' with
> pertinent information.  For example, standard Ivy shows a vertical list
> consisting only of buffer names, while 'ivy-rich' adds to them their
> major mode and file system path, filling up the empty space.
> 
> Couched in those terms, "rich" could be replaced with something like
> "detailed", "descriptive", "informative"...  Or, if you want to be
> precise, prepend "vertical-" to those options.

Yes, please.  Thank you.  Some name such as "detailed",
which gets across that the candidate includes additional
info of some sort.

Re: on helm substantial differences

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]

> Before knowing what's the best approach, I think we should clearly
> decide what would be the "ideal" new API.  E.g. should it return "any
> string" and then it'd be up to the infrastructure code to store
> side-info about what is the corresponding candidate's actual text?

After trying different variants, it turned out that the most convenient
is to use annotation-function that can contain a placeholder for the
completion candidate.  Currently "%s" is used as a placeholder,
for example, "prefix %s suffix" but it can be any string.
Or maybe better to allow returning a list of two separate strings
for prefix and suffix from annotation-function?

Here is an example used for testing:


[-- Attachment #2: display-function.png --]
[-- Type: image/png, Size: 98453 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: annotation-function.patch --]
[-- Type: text/x-diff, Size: 4234 bytes --]

diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 9d57a817b2..b98033753d 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -83,7 +80,6 @@
 
 ;; - add support for ** to pcm.
 ;; - Add vc-file-name-completion-table to read-file-name-internal.
-;; - A feature like completing-help.el.
 
 ;;; Code:
 
@@ -1754,13 +1753,22 @@ completion--insert-strings
             (if (not (consp str))
                 (put-text-property (point) (progn (insert str) (point))
                                    'mouse-face 'highlight)
-              (put-text-property (point) (progn (insert (car str)) (point))
-                                 'mouse-face 'highlight)
-              (let ((beg (point))
-                    (end (progn (insert (cadr str)) (point))))
-                (put-text-property beg end 'mouse-face nil)
-                (font-lock-prepend-text-property beg end 'face
-                                                 'completions-annotations)))
+              (let* ((split (split-string (cadr str) "%s"))
+                     (prefix (when (cadr split) (car split)))
+                     (suffix (or (cadr split) (car split))))
+                (when prefix
+                  (let ((beg (point))
+                        (end (progn (insert prefix) (point))))
+                    (put-text-property beg end 'mouse-face nil)
+                    (font-lock-prepend-text-property beg end 'face
+                                                     'completions-annotations)))
+                (put-text-property (point) (progn (insert (car str)) (point))
+                                   'mouse-face 'highlight)
+                (let ((beg (point))
+                      (end (progn (insert suffix) (point))))
+                  (put-text-property beg end 'mouse-face nil)
+                  (font-lock-prepend-text-property beg end 'face
+                                                   'completions-annotations))))
 	    (cond
 	     ((eq completions-format 'vertical)
 	      ;; Vertical format
diff --git a/lisp/help-fns.el b/lisp/help-fns.el
index 170f497541..00603272bf 100644
--- a/lisp/help-fns.el
+++ b/lisp/help-fns.el
@@ -127,16 +127,30 @@ help-enable-completion-autoload
   :version "26.3")
 
 (defun help--symbol-completion-table (string pred action)
-  (when help-enable-completion-autoload
-    (let ((prefixes (radix-tree-prefixes (help-definition-prefixes) string)))
-      (help--load-prefixes prefixes)))
-  (let ((prefix-completions
-         (and help-enable-completion-autoload
-              (mapcar #'intern (all-completions string definition-prefixes)))))
-    (complete-with-action action obarray string
-                          (if pred (lambda (sym)
-                                     (or (funcall pred sym)
-                                         (memq sym prefix-completions)))))))
+  (if (eq action 'metadata)
+      '(metadata
+	(annotation-function
+         . (lambda (c)
+             (let* ((s (intern c))
+                    (doc (condition-case nil (documentation s) (error nil)))
+                    (doc (and doc (substring doc 0 (string-match "\n" doc)))))
+               (format "%s %%s%s"
+                       (propertize (cond ((commandp s) "c")
+                                         ((fboundp s) "f")
+                                         ((boundp s) "v")
+                                         " ")
+                                   'face 'shadow)
+                       (if doc (propertize (format " -- %s" doc) 'face 'shadow) ""))))))
+    (when help-enable-completion-autoload
+      (let ((prefixes (radix-tree-prefixes (help-definition-prefixes) string)))
+        (help--load-prefixes prefixes)))
+    (let ((prefix-completions
+           (and help-enable-completion-autoload
+                (mapcar #'intern (all-completions string definition-prefixes)))))
+      (complete-with-action action obarray string
+                            (if pred (lambda (sym)
+                                       (or (funcall pred sym)
+                                           (memq sym prefix-completions))))))))
 
 (defvar describe-function-orig-buffer nil
   "Buffer that was current when `describe-function' was invoked.

Re: on helm substantial differences

[Juri Linkov]

> Couched in those terms, "rich" could be replaced with something like
> "detailed", "descriptive", "informative"...  Or, if you want to be
> precise, prepend "vertical-" to those options.

Thanks for the suggestion, "detailed" is a good one.  This is
like in some file managers with List view and Detailed view.

Re: on helm substantial differences

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 1086 bytes --]

>> Before knowing what's the best approach, I think we should clearly
>> decide what would be the "ideal" new API.  E.g. should it return "any
>> string" and then it'd be up to the infrastructure code to store
>> side-info about what is the corresponding candidate's actual text?
>
> After trying different variants, it turned out that the most convenient
> is to use annotation-function that can contain a placeholder for the
> completion candidate.  Currently "%s" is used as a placeholder,
> for example, "prefix %s suffix" but it can be any string.

Actually, I discovered that unfortunately annotation-function
is unsuitable for formatting data in a columnar format.
It needs to receive a complete list of completions to be able
to adjust their widths for more optimal overall columnar layout.

So a new function is needed that is like annotation-function
but accepts a list of all completions.  There is an existing
similar function 'display-sort-function' that could be abused
to add prefix/suffix annotations, but maybe a separate function
would be preferable for cleaner API?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: display-function.patch --]
[-- Type: text/x-diff, Size: 2288 bytes --]

diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index e18c5c1372..c671750e9a 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -117,6 +117,9 @@ completion-metadata
 - `annotation-function': function to add annotations in *Completions*.
    Takes one argument (STRING), which is a possible completion and
    returns a string to append to STRING.
+- `display-function': function to display entries in *Completions*.
+   Takes one argument (COMPLETIONS) and should return a list
+   of completions with a placeholder that separates prefix/suffix.
 - `display-sort-function': function to sort entries in *Completions*.
    Takes one argument (COMPLETIONS) and should return a new list
    of completions.  Can operate destructively.
@@ -1885,6 +1888,9 @@ completion-extra-properties
    completion).  The function can access the completion data via
    `minibuffer-completion-table' and related variables.
 
+`:display-function': Function to display completions.
+   The function must accept one argument, a list of completions.
+
 `:exit-function': Function to run after completion is performed.
 
    The function must accept two arguments, STRING and STATUS.
@@ -1971,6 +1977,9 @@ minibuffer-completion-help
                        (plist-get completion-extra-properties
                                   :annotation-function)
                        completion-annotate-function))
+             (dfun (or (completion-metadata-get all-md 'display-function)
+                       (plist-get completion-extra-properties
+                                  :display-function)))
              (mainbuf (current-buffer))
              ;; If the *Completions* buffer is shown in a new
              ;; window, mark it as softly-dedicated, so bury-buffer in
@@ -2017,6 +2026,9 @@ minibuffer-completion-help
                                         (let ((ann (funcall afun s)))
                                           (if ann (list s ann) s)))
                                       completions)))
+                      (when dfun
+                        (setq completions
+                              (funcall dfun completions)))
 
                       (with-current-buffer standard-output
                         (set (make-local-variable 'completion-base-position)

Re: on helm substantial differences

[Jean Louis]

* Juri Linkov <juri@linkov.net> [2020-11-17 22:33]:
> > Before knowing what's the best approach, I think we should clearly
> > decide what would be the "ideal" new API.  E.g. should it return "any
> > string" and then it'd be up to the infrastructure code to store
> > side-info about what is the corresponding candidate's actual text?
> 
> After trying different variants, it turned out that the most convenient
> is to use annotation-function that can contain a placeholder for the
> completion candidate.  Currently "%s" is used as a placeholder,
> for example, "prefix %s suffix" but it can be any string.
> Or maybe better to allow returning a list of two separate strings
> for prefix and suffix from annotation-function?

If I may comment on (not related to above paragraph) the attached .png
picture is that minibuffer enlarged very much over the screen. What
would happen if the screen would be split in two windows before
completion, would then the very narrow scratch buffer on top be
divided with another mode line? In that case both divided windows
would be very narrow and not readable. Sometimes at least the focused
window should be readable during completion as completion is very
general for programmers who may need reference in the window they are
working in.

Large minibuffer window practically splits window in 2 parts, one very
narrow, minibuffer very high. In this case your *scratch* buffer,
would it have more text, would not be usable at all as it little would
be visible.

My suggestion is that minibuffer rather makes proper split window by
default or that it does not enlarge over the screen more then
the calculated half of the full window, as that way user would have
visible buffer.

My suggestion for Emacs that already has 2 split windows is that
minibuffer completion does not resize the focused window as that is
where user works and expects (probably) to maybe insert something into
that buffer or use it as reference during completion. Rather
minibuffer should use the unfocused other window (from two or more
split windows) to show its completions then enlarging itself over the
screen and effectively disturbing the visual static interface.

Re: on helm substantial differences

[Juri Linkov]

> If I may comment on (not related to above paragraph) the attached .png
> picture is that minibuffer enlarged very much over the screen. What
> would happen if the screen would be split in two windows before
> completion, would then the very narrow scratch buffer on top be
> divided with another mode line? In that case both divided windows
> would be very narrow and not readable. Sometimes at least the focused
> window should be readable during completion as completion is very
> general for programmers who may need reference in the window they are
> working in.
>
> Large minibuffer window practically splits window in 2 parts, one very
> narrow, minibuffer very high. In this case your *scratch* buffer,
> would it have more text, would not be usable at all as it little would
> be visible.
>
> My suggestion is that minibuffer rather makes proper split window by
> default or that it does not enlarge over the screen more then
> the calculated half of the full window, as that way user would have
> visible buffer.
>
> My suggestion for Emacs that already has 2 split windows is that
> minibuffer completion does not resize the focused window as that is
> where user works and expects (probably) to maybe insert something into
> that buffer or use it as reference during completion. Rather
> minibuffer should use the unfocused other window (from two or more
> split windows) to show its completions then enlarging itself over the
> screen and effectively disturbing the visual static interface.

I suggested you to customize this easily by one-liner:

  (push '("\\`\\*Completions\\*\\'" nil (window-height)) display-buffer-alist)

Re: on helm substantial differences

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 810 bytes --]

> Actually, I discovered that unfortunately annotation-function
> is unsuitable for formatting data in a columnar format.
> It needs to receive a complete list of completions to be able
> to adjust their widths for more optimal overall columnar layout.
>
> So a new function is needed that is like annotation-function
> but accepts a list of all completions.  There is an existing
> similar function 'display-sort-function' that could be abused
> to add prefix/suffix annotations, but maybe a separate function
> would be preferable for cleaner API?

It turned out that 'display-function' is not a suitable name either.
What a new function should do is to prepend/append affixes
on both sides of a completion, accept a list of all completions,
and return a list of two separate strings for prefix and suffix:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: affix-function.patch --]
[-- Type: text/x-diff, Size: 4866 bytes --]

diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 9d57a817b2..3f700803b6 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -121,6 +117,9 @@ completion-metadata
 - `annotation-function': function to add annotations in *Completions*.
    Takes one argument (STRING), which is a possible completion and
    returns a string to append to STRING.
+- `affix-function': function to prepend/append a prefix/suffix to entries.
+   Takes one argument (COMPLETIONS) and should return a list
+   of completions with a completion, its prefix and suffix.
 - `display-sort-function': function to sort entries in *Completions*.
    Takes one argument (COMPLETIONS) and should return a new list
    of completions.  Can operate destructively.
@@ -1689,8 +1688,7 @@ completion--insert-strings
     (let* ((length (apply #'max
 			  (mapcar (lambda (s)
 				    (if (consp s)
-					(+ (string-width (car s))
-                                           (string-width (cadr s)))
+                                        (cl-reduce #'+ (mapcar #'string-width s))
 				      (string-width s)))
 				  strings)))
 	   (window (get-buffer-window (current-buffer) 0))
@@ -1715,8 +1713,7 @@ completion--insert-strings
           ;; FIXME: `string-width' doesn't pay attention to
           ;; `display' properties.
           (let ((length (if (consp str)
-                            (+ (string-width (car str))
-                               (string-width (cadr str)))
+                            (cl-reduce #'+ (mapcar #'string-width str))
                           (string-width str))))
             (cond
 	     ((eq completions-format 'vertical)
@@ -1754,13 +1751,21 @@ completion--insert-strings
             (if (not (consp str))
                 (put-text-property (point) (progn (insert str) (point))
                                    'mouse-face 'highlight)
-              (put-text-property (point) (progn (insert (car str)) (point))
-                                 'mouse-face 'highlight)
-              (let ((beg (point))
-                    (end (progn (insert (cadr str)) (point))))
-                (put-text-property beg end 'mouse-face nil)
-                (font-lock-prepend-text-property beg end 'face
-                                                 'completions-annotations)))
+              (let* ((prefix (when (nth 2 str) (nth 1 str)))
+                     (suffix (or (nth 2 str) (nth 1 str))))
+                (when prefix
+                  (let ((beg (point))
+                        (end (progn (insert prefix) (point))))
+                    (put-text-property beg end 'mouse-face nil)
+                    (font-lock-prepend-text-property beg end 'face
+                                                     'completions-annotations)))
+                (put-text-property (point) (progn (insert (car str)) (point))
+                                   'mouse-face 'highlight)
+                (let ((beg (point))
+                      (end (progn (insert suffix) (point))))
+                  (put-text-property beg end 'mouse-face nil)
+                  (font-lock-prepend-text-property beg end 'face
+                                                   'completions-annotations))))
 	    (cond
 	     ((eq completions-format 'vertical)
 	      ;; Vertical format
@@ -1880,6 +1885,9 @@ completion-extra-properties
    completion).  The function can access the completion data via
    `minibuffer-completion-table' and related variables.
 
+`:affix-function': Function to prepend/append a prefix/suffix to completions.
+   The function must accept one argument, a list of completions.
+
 `:exit-function': Function to run after completion is performed.
 
    The function must accept two arguments, STRING and STATUS.
@@ -1966,6 +1974,9 @@ minibuffer-completion-help
                        (plist-get completion-extra-properties
                                   :annotation-function)
                        completion-annotate-function))
+             (xfun (or (completion-metadata-get all-md 'affix-function)
+                       (plist-get completion-extra-properties
+                                  :affix-function)))
              (mainbuf (current-buffer))
              ;; If the *Completions* buffer is shown in a new
              ;; window, mark it as softly-dedicated, so bury-buffer in
@@ -2012,6 +2023,9 @@ minibuffer-completion-help
                                         (let ((ann (funcall afun s)))
                                           (if ann (list s ann) s)))
                                       completions)))
+                      (when xfun
+                        (setq completions
+                              (funcall xfun completions)))
 
                       (with-current-buffer standard-output
                         (set (make-local-variable 'completion-base-position)

Re: on helm substantial differences

[Basil L. Contovounesios]

Juri Linkov <juri@linkov.net> writes:

> -					(+ (string-width (car s))
> -                                           (string-width (cadr s)))
> +                                        (cl-reduce #'+ (mapcar #'string-width s))

Nit: If you're calling mapcar anyway, why not simply (apply #'+ 0 ...)?

Thanks,

-- 
Basil

Re: on helm substantial differences

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 558 bytes --]

>> -					(+ (string-width (car s))
>> -                                           (string-width (cadr s)))
>> +                                        (cl-reduce #'+ (mapcar #'string-width s))
>
> Nit: If you're calling mapcar anyway, why not simply (apply #'+ 0 ...)?

Thanks, fixed.  Now the patch also adds a new user option
'completions-detailed'.  An option separate from 'completions-format'
is needed in case when a completion function doesn't provide :affix-function,
so there is a need to fall back to the customized value of 'completions-format'.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: completions-detailed.patch --]
[-- Type: text/x-diff, Size: 9685 bytes --]

diff --git a/lisp/help-fns.el b/lisp/help-fns.el
index 170f497541..de4005d6cd 100644
--- a/lisp/help-fns.el
+++ b/lisp/help-fns.el
@@ -127,16 +127,45 @@ help-enable-completion-autoload
   :version "26.3")
 
 (defun help--symbol-completion-table (string pred action)
-  (when help-enable-completion-autoload
-    (let ((prefixes (radix-tree-prefixes (help-definition-prefixes) string)))
-      (help--load-prefixes prefixes)))
-  (let ((prefix-completions
-         (and help-enable-completion-autoload
-              (mapcar #'intern (all-completions string definition-prefixes)))))
-    (complete-with-action action obarray string
-                          (if pred (lambda (sym)
-                                     (or (funcall pred sym)
-                                         (memq sym prefix-completions)))))))
+  (if (eq action 'metadata)
+      (when completions-detailed
+        '(metadata
+	  (affix-function
+           . (lambda (completions)
+               (mapcar (lambda (c)
+                         (let* ((s (intern c))
+                                (doc (condition-case nil (documentation s) (error nil)))
+                                (doc (and doc (substring doc 0 (string-match "\n" doc)))))
+                           (list c (concat (cond ((commandp s)
+                                                  "c")
+                                                 ((eq (car-safe (symbol-function s)) 'macro)
+                                                  "m")
+                                                 ((fboundp s)
+                                                  "f")
+                                                 ((custom-variable-p s)
+                                                  "u") ; user option
+                                                 ((boundp s)
+                                                  "v")
+                                                 ((facep s)
+                                                  "a")
+                                                 ((and (fboundp 'cl-find-class)
+                                                       (cl-find-class s))
+                                                  "t") ; CL type
+                                                 (" "))
+                                           " ")
+                                 (if doc (format " -- %s" doc) ""))))
+                       completions)))))
+
+    (when help-enable-completion-autoload
+      (let ((prefixes (radix-tree-prefixes (help-definition-prefixes) string)))
+        (help--load-prefixes prefixes)))
+    (let ((prefix-completions
+           (and help-enable-completion-autoload
+                (mapcar #'intern (all-completions string definition-prefixes)))))
+      (complete-with-action action obarray string
+                            (if pred (lambda (sym)
+                                       (or (funcall pred sym)
+                                           (memq sym prefix-completions))))))))
 
 (defvar describe-function-orig-buffer nil
   "Buffer that was current when `describe-function' was invoked.
diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 9d57a817b2..7e1d806059 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -83,7 +80,6 @@
 
 ;; - add support for ** to pcm.
 ;; - Add vc-file-name-completion-table to read-file-name-internal.
-;; - A feature like completing-help.el.
 
 ;;; Code:
 
@@ -121,6 +117,9 @@ completion-metadata
 - `annotation-function': function to add annotations in *Completions*.
    Takes one argument (STRING), which is a possible completion and
    returns a string to append to STRING.
+- `affix-function': function to prepend/append a prefix/suffix to entries.
+   Takes one argument (COMPLETIONS) and should return a list
+   of completions with a completion, its prefix and suffix.
 - `display-sort-function': function to sort entries in *Completions*.
    Takes one argument (COMPLETIONS) and should return a new list
    of completions.  Can operate destructively.
@@ -1669,7 +1668,7 @@ completion-in-region--single-word
     (#b000 nil)
       (_     t))))
 
-(defface completions-annotations '((t :inherit italic))
+(defface completions-annotations '((t :inherit (italic shadow)))
   "Face to use for annotations in the *Completions* buffer.")
 
 (defcustom completions-format 'horizontal
@@ -1681,6 +1680,16 @@ completions-format
   :type '(choice (const horizontal) (const vertical))
   :version "23.2")
 
+(defcustom completions-detailed nil
+  "When non-nil, display completions vertically with one completion per row.
+This option overrides another related option `completions-format'.
+Some commands might provide a detailed view with more information added
+to completions.  When the used completion function doesn't provide
+a detailed view via `affix-function', then fall back to the value
+defined by `completions-format'."
+  :type 'boolean
+  :version "28.1")
+
 (defun completion--insert-strings (strings)
   "Insert a list of STRINGS into the current buffer.
 Uses columns to keep the listing readable but compact.
@@ -1689,8 +1698,7 @@ completion--insert-strings
     (let* ((length (apply #'max
 			  (mapcar (lambda (s)
 				    (if (consp s)
-					(+ (string-width (car s))
-                                           (string-width (cadr s)))
+					(apply #'+ (mapcar #'string-width s))
 				      (string-width s)))
 				  strings)))
 	   (window (get-buffer-window (current-buffer) 0))
@@ -1715,10 +1723,14 @@ completion--insert-strings
           ;; FIXME: `string-width' doesn't pay attention to
           ;; `display' properties.
           (let ((length (if (consp str)
-                            (+ (string-width (car str))
-                               (string-width (cadr str)))
+                            (apply #'+ (mapcar #'string-width str))
                           (string-width str))))
             (cond
+             ((and completions-detailed (= (length str) 3))
+              ;; Detailed view
+              ;; When `str' contains prefix and suffix this means
+              ;; that caller specified `affix-function'.
+              )
 	     ((eq completions-format 'vertical)
 	      ;; Vertical format
 	      (when (> row rows)
@@ -1754,14 +1766,27 @@ completion--insert-strings
             (if (not (consp str))
                 (put-text-property (point) (progn (insert str) (point))
                                    'mouse-face 'highlight)
-              (put-text-property (point) (progn (insert (car str)) (point))
-                                 'mouse-face 'highlight)
-              (let ((beg (point))
-                    (end (progn (insert (cadr str)) (point))))
-                (put-text-property beg end 'mouse-face nil)
-                (font-lock-prepend-text-property beg end 'face
-                                                 'completions-annotations)))
+              (let* ((prefix (when (nth 2 str) (nth 1 str)))
+                     (suffix (or (nth 2 str) (nth 1 str))))
+                (when prefix
+                  (let ((beg (point))
+                        (end (progn (insert prefix) (point))))
+                    (put-text-property beg end 'mouse-face nil)
+                    (font-lock-prepend-text-property beg end 'face
+                                                     'completions-annotations)))
+                (put-text-property (point) (progn (insert (car str)) (point))
+                                   'mouse-face 'highlight)
+                (let ((beg (point))
+                      (end (progn (insert suffix) (point))))
+                  (put-text-property beg end 'mouse-face nil)
+                  (font-lock-prepend-text-property beg end 'face
+                                                   'completions-annotations))))
 	    (cond
+	     ((and completions-detailed (= (length str) 3))
+	      ;; Detailed view
+	      (when (zerop row) (setq truncate-lines t))
+	      (insert "\n")
+	      (setq row (1+ row)))
 	     ((eq completions-format 'vertical)
 	      ;; Vertical format
 	      (if (> column 0)
@@ -1880,6 +1905,9 @@ completion-extra-properties
    completion).  The function can access the completion data via
    `minibuffer-completion-table' and related variables.
 
+`:affix-function': Function to prepend/append a prefix/suffix to completions.
+   The function must accept one argument, a list of completions.
+
 `:exit-function': Function to run after completion is performed.
 
    The function must accept two arguments, STRING and STATUS.
@@ -1966,6 +1994,9 @@ minibuffer-completion-help
                        (plist-get completion-extra-properties
                                   :annotation-function)
                        completion-annotate-function))
+             (xfun (or (completion-metadata-get all-md 'affix-function)
+                       (plist-get completion-extra-properties
+                                  :affix-function)))
              (mainbuf (current-buffer))
              ;; If the *Completions* buffer is shown in a new
              ;; window, mark it as softly-dedicated, so bury-buffer in
@@ -2012,6 +2043,9 @@ minibuffer-completion-help
                                         (let ((ann (funcall afun s)))
                                           (if ann (list s ann) s)))
                                       completions)))
+                      (when xfun
+                        (setq completions
+                              (funcall xfun completions)))
 
                       (with-current-buffer standard-output
                         (set (make-local-variable 'completion-base-position)

Re: on helm substantial differences

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 1027 bytes --]

> diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
> index 9d57a817b2..b98033753d 100644
> --- a/lisp/minibuffer.el
> +++ b/lisp/minibuffer.el
> @@ -83,7 +80,6 @@
>  
>  ;; - add support for ** to pcm.
>  ;; - Add vc-file-name-completion-table to read-file-name-internal.
> -;; - A feature like completing-help.el.
>  
>  ;;; Code:
>  

BTW, there is another TODO item in minibuffer.el:

;;   - indicate how to display the completions in *Completions* (turn
;;     \n into something else, add special boundaries between
;;     completions).  E.g. when completing from the kill-ring.

Such pre-processing can be performed by the caller.
For a long time I used the command 'yank-from-kill-ring'
bound to 'C-M-y' that relies on query-replace-descr
to display newlines as ^J and displays long completions
with an ellipsis at the end and an ellipsis on leading whitespace.

These pre-processed completions also needed to display
shorter strings in the minibuffer in icomplete-mode,
so this is usable in icomplete-mode as well:


[-- Attachment #2: yank-from-kill-ring.patch --]
[-- Type: text/x-diff, Size: 2774 bytes --]

diff --git a/lisp/simple.el b/lisp/simple.el
index bb28145502..076469b3d2 100644
--- a/lisp/simple.el
+++ b/lisp/simple.el
@@ -5449,6 +5449,55 @@ rotate-yank-pointer
 "With ARG, rotate that many kills forward (or backward, if negative)."
   (interactive "p")
   (current-kill arg))
+
+(defvar yank-from-kill-ring-history)
+(defun yank-from-kill-ring (string)
+  "Insert the kill-ring item selected from the minibuffer history.
+Use minibuffer navigation and search commands to browse the kill-ring
+in the minibuffer history before typing RET to insert the item,
+or use completion on the elements of the kill-ring."
+  (interactive
+   (list (let* ((history-add-new-input nil)
+                ;; Remove keymaps from text properties of copied string,
+                ;; because typing RET in the minibuffer might call
+                ;; an irrelevant command from the map of copied string.
+                (yank-from-kill-ring-history
+                 (mapcar (lambda (s)
+                           (remove-list-of-text-properties
+                            0 (length s)
+                            '(
+                              keymap local-map action mouse-action
+                              button category help-args)
+                            s)
+                           s)
+                         kill-ring))
+                (completions
+                 (mapcar (lambda (s)
+                           (let* ((s (query-replace-descr s))
+                                  (b 0))
+                             ;; Add ellipsis on leading whitespace
+                             (when (string-match "\\`[[:space:]]+" s)
+                               (setq b (match-end 0))
+                               (add-text-properties 0 b `(display "…") s))
+                             (when (> (length s) (- 40 b))
+                               (add-text-properties
+                                (min (+ b 40) (length s)) (length s)
+                                `(display "…") s))
+                             s))
+                         yank-from-kill-ring-history)))
+           (completing-read "Yank from kill-ring: "
+                            (lambda (string pred action)
+                              (if (eq action 'metadata)
+                                  ;; Keep sorted by recency
+	                          '(metadata (display-sort-function . identity))
+                                (complete-with-action action completions string pred)))
+                            nil nil nil
+                            'yank-from-kill-ring-history))))
+  (setq yank-window-start (window-start))
+  (push-mark)
+  (insert-for-yank string))
+
+(global-set-key "\M-\C-y" 'yank-from-kill-ring)
 \f
 ;; Some kill commands.
 

Re: on helm substantial differences

[Juri Linkov]

> +- `affix-function': function to prepend/append a prefix/suffix to entries.
> +   Takes one argument (COMPLETIONS) and should return a list
> +   of completions with a completion, its prefix and suffix.

BTW, using `affix-function' also makes it possible to group completions,
then the prefix of the first completion in the group could contain
a group header with group name.

select yank via completion

[Stefan Monnier]

> BTW, there is another TODO item in minibuffer.el:
>
> ;;   - indicate how to display the completions in *Completions* (turn
> ;;     \n into something else, add special boundaries between
> ;;     completions).  E.g. when completing from the kill-ring.
>
> Such pre-processing can be performed by the caller.
> For a long time I used the command 'yank-from-kill-ring'
> bound to 'C-M-y' that relies on query-replace-descr
> to display newlines as ^J and displays long completions
> with an ellipsis at the end and an ellipsis on leading whitespace.

Looks great, thanks.  Any reason you bind it to C-M-y instead
of to "M-y that doesn't immediately follow C-y"?


        Stefan

RE: on helm substantial differences

[Drew Adams]

> BTW, there is another TODO item in minibuffer.el:
> 
> ;;   - indicate how to display the completions in *Completions* (turn
> ;;     \n into something else, add special boundaries between
> ;;     completions).  E.g. when completing from the kill-ring.
> 
> Such pre-processing can be performed by the caller.

FWIW, Icicles does this.  If a completion candidate
is multi-line then it is separated from the following
candidate in *Completions* by a blank line (extra
newline).

Candidates are also distinguished in these ways:

* They have property `mouse-face'.  (Separator
  whitespace, including a newline, does not.)

* The "current" candidate (e.g. when cycling) is
  highlighted, which makes it and its limits stand
  out.

RE: select yank via completion

[Drew Adams]

> Looks great, thanks.  Any reason you bind it to C-M-y instead
> of to "M-y that doesn't immediately follow C-y"?

FWIW, Icicles uses `icicle-completing-yank' for `M-y'
not right after `C-y'.  It's also bound to `C-- C-y'.
(`C-y' is `icicle-yank-maybe-completing'.)

,----
| icicle-completing-yank is an interactive compiled Lisp function in
| 'icicles-cmd1.el'.
| 
| It is bound to menu-bar edit icicles icicle-completing-yank.
| 
| (icicle-completing-yank)
| 
| Yank an entry from a selection ring, choosing it using completion.
| By default, the selection ring used is the kill ring.
| 
| If you also use library `browse-kill-ring+.el' or library
| `second-sel.el' then an alternative selection ring is used if you
| provide a prefix argument: `browse-kill-ring-alternative-ring' or
| `secondary-selection-ring'.  This gives you a way to yank chosen items
| from two different sets of selections.
| 
| When the kill ring is used, this is similar to `yank', but this does
| not rotate the ring.  The mark is pushed first, so the yanked text
| becomes the region.
...
`----

Actually, if you use library `second-sel.el' then, by
default Icicles binds `M-y' at top level to command
`icicle-yank-pop-commands', which lets you choose a
selection from either kll-ring or secondary selection
ring, using completion against the ring entries.

,----
| icicle-yank-pop-commands is an interactive compiled Lisp function in
| 'icicles-cmd1.el'.
| 
| It is bound to M-insert, M-y.
| 
| (icicle-yank-pop-commands &optional ARG)
| 
| `yank-pop', `yank-pop-secondary', or `icicle-completing-yank'.
| Which of these is used depends on the previous command, as follows:
| 
|  * If the previous command was a yank-secondary command, then
|    `yank-pop-secondary'.
| 
|  * Else if the previous command was a yank command (i.e. using the
|    kill ring), then `yank-pop'.
| 
|  * Else `icicle-completing-yank'.
| 
| In the last case (`icicle-completing-yank'), during completion you can
| use:
| 
|  * `C-,' to sort the candidates to yank in different ways (repeat)
|  * `S-delete' to remove a candidate entry from the selection ring
|  * `C-S-return' to copy a candidate to the other selection ring
| 
| You need library `second-sel.el' for this command.
| 
`----

Re: select yank via completion

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 1169 bytes --]

>> BTW, there is another TODO item in minibuffer.el:
>>
>> ;;   - indicate how to display the completions in *Completions* (turn
>> ;;     \n into something else, add special boundaries between
>> ;;     completions).  E.g. when completing from the kill-ring.
>>
>> Such pre-processing can be performed by the caller.
>> For a long time I used the command 'yank-from-kill-ring'
>> bound to 'C-M-y' that relies on query-replace-descr
>> to display newlines as ^J and displays long completions
>> with an ellipsis at the end and an ellipsis on leading whitespace.
>
> Looks great, thanks.  Any reason you bind it to C-M-y instead
> of to "M-y that doesn't immediately follow C-y"?

Oops, I don't remember why I forgot about no-op M-y.

Here is a better patch.  There is still a small problem:
sometimes it's handy to slightly edit a previous string
from kill-ring before inserting it.  But with completing-read
it's not easy to insert the space character, e.g. after 'M-p'
while editing, typing 'SPC' errors with "[No matches]"
and doesn't insert a space character.

Could you recommend a more lightweight version of completing-read
that doesn't override the 'SPC' key?


[-- Attachment #2: yank-from-kill-ring.patch --]
[-- Type: text/x-diff, Size: 5000 bytes --]

diff --git a/lisp/simple.el b/lisp/simple.el
index bb28145502..32850b4b7c 100644
--- a/lisp/simple.el
+++ b/lisp/simple.el
@@ -5362,27 +5362,29 @@ yank-pop
 property, in the way that `yank' does."
   (interactive "*p")
   (if (not (eq last-command 'yank))
-      (user-error "Previous command was not a yank"))
-  (setq this-command 'yank)
-  (unless arg (setq arg 1))
-  (let ((inhibit-read-only t)
-	(before (< (point) (mark t))))
-    (if before
-	(funcall (or yank-undo-function 'delete-region) (point) (mark t))
-      (funcall (or yank-undo-function 'delete-region) (mark t) (point)))
-    (setq yank-undo-function nil)
-    (set-marker (mark-marker) (point) (current-buffer))
-    (insert-for-yank (current-kill arg))
-    ;; Set the window start back where it was in the yank command,
-    ;; if possible.
-    (set-window-start (selected-window) yank-window-start t)
-    (if before
-	;; This is like exchange-point-and-mark, but doesn't activate the mark.
-	;; It is cleaner to avoid activation, even though the command
-	;; loop would deactivate the mark because we inserted text.
-	(goto-char (prog1 (mark t)
-		     (set-marker (mark-marker) (point) (current-buffer))))))
-  nil)
+      (call-interactively 'yank-from-kill-ring)
+    (setq this-command 'yank)
+    (unless arg (setq arg 1))
+    (let ((inhibit-read-only t)
+          (before (< (point) (mark t))))
+      (if before
+          (funcall (or yank-undo-function 'delete-region) (point) (mark t))
+        (funcall (or yank-undo-function 'delete-region) (mark t) (point)))
+      (setq yank-undo-function nil)
+      (set-marker (mark-marker) (point) (current-buffer))
+      (insert-for-yank (current-kill arg))
+      ;; Set the window start back where it was in the yank command,
+      ;; if possible.
+      (set-window-start (selected-window) yank-window-start t)
+      (if before
+          ;; This is like exchange-point-and-mark, but doesn't activate the mark.
+          ;; It is cleaner to avoid activation, even though the command
+          ;; loop would deactivate the mark because we inserted text.
+          (goto-char (prog1 (mark t)
+                       (set-marker (mark-marker) (point) (current-buffer))))))
+    nil))
+
+(put 'yank-pop 'delete-selection t)
 
 (defun yank (&optional arg)
   "Reinsert (\"paste\") the last stretch of killed text.
@@ -5449,6 +5451,56 @@ rotate-yank-pointer
 With ARG, rotate that many kills forward (or backward, if negative)."
   (interactive "p")
   (current-kill arg))
+
+(defvar yank-from-kill-ring-history)
+(defun yank-from-kill-ring (string)
+  "Insert the kill-ring item selected from the minibuffer history.
+Use minibuffer navigation and search commands to browse the kill-ring
+in the minibuffer history before typing RET to insert the item,
+or use completion on the elements of the kill-ring."
+  (interactive
+   (list (let* ((history-add-new-input nil)
+                ;; Remove keymaps from text properties of copied string,
+                ;; because typing RET in the minibuffer might call
+                ;; an irrelevant command from the map of copied string.
+                (yank-from-kill-ring-history
+                 (mapcar (lambda (s)
+                           (remove-list-of-text-properties
+                            0 (length s)
+                            '(
+                              keymap local-map action mouse-action
+                              button category help-args)
+                            s)
+                           s)
+                         kill-ring))
+                (completions
+                 (mapcar (lambda (s)
+                           (let* ((s (query-replace-descr s))
+                                  (b 0))
+                             ;; Add ellipsis on leading whitespace
+                             (when (string-match "\\`[[:space:]]+" s)
+                               (setq b (match-end 0))
+                               (add-text-properties 0 b `(display "…") s))
+                             (when (> (length s) (- 40 b))
+                               (add-text-properties
+                                (min (+ b 40) (length s)) (length s)
+                                `(display "…") s))
+                             s))
+                         yank-from-kill-ring-history)))
+           (completing-read "Yank from kill-ring: "
+                            (lambda (string pred action)
+                              (if (eq action 'metadata)
+                                  ;; Keep sorted by recency
+	                          '(metadata (display-sort-function . identity))
+                                (complete-with-action action completions string pred)))
+                            nil nil nil
+                            'yank-from-kill-ring-history))))
+  (setq yank-window-start (window-start))
+  (push-mark)
+  (insert-for-yank string))
+
+(put 'yank-from-kill-ring 'delete-selection t)
+
 \f
 ;; Some kill commands.
 

RE: select yank via completion

[Drew Adams]

> There is still a small problem:
> sometimes it's handy to slightly edit a previous string
> from kill-ring before inserting it.  But with completing-read
> it's not easy to insert the space character, e.g. after 'M-p'
> while editing, typing 'SPC' errors with "[No matches]"
> and doesn't insert a space character.
> 
> Could you recommend a more lightweight version of 
> completing-read that doesn't override the 'SPC' key?

It's _always_ handy to be able to edit a kill-ring entry
before inserting it into the minibuffer.

It's high time that `completing-read' stop using `SPC' for
word completion.  `SPC' should be self-inserting (and the
same for `?').

There's no reason nowadays (and there was never a good
reason) to expect that completion candidates will not
include space chars.

Dunno why this is such a tooth-pull.  It took decades to
even get `SPC' to be self-inserting for `read-file-name'.
I raised the same issue for `completing-read' at that time
(and before then).

Word completion?  How much is it actually relevant now?
In any case, it's easy to allow for it.  Icicles uses
`M-SPC' for that.

Re: select yank via completion

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 1096 bytes --]

>> There is still a small problem:
>> sometimes it's handy to slightly edit a previous string
>> from kill-ring before inserting it.  But with completing-read
>> it's not easy to insert the space character, e.g. after 'M-p'
>> while editing, typing 'SPC' errors with "[No matches]"
>> and doesn't insert a space character.
>> 
>> Could you recommend a more lightweight version of 
>> completing-read that doesn't override the 'SPC' key?
>
> It's _always_ handy to be able to edit a kill-ring entry
> before inserting it into the minibuffer.

BTW, sometimes it's also handy to just browse the kill-ring
without immediately inserting an entry, to copy an edited entry
(that adds a new kill-ring entry) to paste later.  To allow
such browsing we need to remove "*" from the interactive spec of
'yank-pop'.

> It took decades to even get `SPC' to be self-inserting for
> `read-file-name'.  I raised the same issue for `completing-read' at
> that time (and before then).

So now a new patch let-binds (minibuffer-completing-file-name t)
around completing-read to allow inserting SPC, problem solved:


[-- Attachment #2: yank-from-kill-ring.patch --]
[-- Type: text/x-diff, Size: 5156 bytes --]

diff --git a/lisp/simple.el b/lisp/simple.el
index bb28145502..589b3648c2 100644
--- a/lisp/simple.el
+++ b/lisp/simple.el
@@ -5360,29 +5360,83 @@ yank-pop
 "This command honors the `yank-handled-properties' and
 `yank-excluded-properties' variables, and the `yank-handler' text
 property, in the way that `yank' does."
-  (interactive "*p")
+  (interactive "p")
   (if (not (eq last-command 'yank))
-      (user-error "Previous command was not a yank"))
-  (setq this-command 'yank)
-  (unless arg (setq arg 1))
-  (let ((inhibit-read-only t)
-	(before (< (point) (mark t))))
-    (if before
-	(funcall (or yank-undo-function 'delete-region) (point) (mark t))
-      (funcall (or yank-undo-function 'delete-region) (mark t) (point)))
-    (setq yank-undo-function nil)
-    (set-marker (mark-marker) (point) (current-buffer))
-    (insert-for-yank (current-kill arg))
-    ;; Set the window start back where it was in the yank command,
-    ;; if possible.
-    (set-window-start (selected-window) yank-window-start t)
-    (if before
-	;; This is like exchange-point-and-mark, but doesn't activate the mark.
-	;; It is cleaner to avoid activation, even though the command
-	;; loop would deactivate the mark because we inserted text.
-	(goto-char (prog1 (mark t)
-		     (set-marker (mark-marker) (point) (current-buffer))))))
-  nil)
+      (call-interactively 'yank-from-kill-ring)
+    (setq this-command 'yank)
+    (unless arg (setq arg 1))
+    (let ((inhibit-read-only t)
+          (before (< (point) (mark t))))
+      (if before
+          (funcall (or yank-undo-function 'delete-region) (point) (mark t))
+        (funcall (or yank-undo-function 'delete-region) (mark t) (point)))
+      (setq yank-undo-function nil)
+      (set-marker (mark-marker) (point) (current-buffer))
+      (insert-for-yank (current-kill arg))
+      ;; Set the window start back where it was in the yank command,
+      ;; if possible.
+      (set-window-start (selected-window) yank-window-start t)
+      (if before
+          ;; This is like exchange-point-and-mark, but doesn't activate the mark.
+          ;; It is cleaner to avoid activation, even though the command
+          ;; loop would deactivate the mark because we inserted text.
+          (goto-char (prog1 (mark t)
+                       (set-marker (mark-marker) (point) (current-buffer))))))
+    nil))
+
+(put 'yank-pop 'delete-selection t)
+
+(defvar yank-from-kill-ring-history)
+(defun yank-from-kill-ring (string)
+  "Insert the kill-ring item selected from the minibuffer history.
+Use minibuffer navigation and search commands to browse the kill-ring
+in the minibuffer history before typing RET to insert the item,
+or use completion on the elements of the kill-ring."
+  (interactive
+   (list (let* ((history-add-new-input nil)
+                (ellipsis (if (char-displayable-p ?…) "…" "..."))
+                ;; Remove keymaps from text properties of copied string,
+                ;; because typing RET in the minibuffer might call
+                ;; an irrelevant command from the map of copied string.
+                (yank-from-kill-ring-history
+                 (mapcar (lambda (s)
+                           (remove-list-of-text-properties
+                            0 (length s)
+                            '(
+                              keymap local-map action mouse-action
+                              button category help-args)
+                            s)
+                           s)
+                         kill-ring))
+                (completions
+                 (mapcar (lambda (s)
+                           (let* ((s (query-replace-descr s))
+                                  (b 0))
+                             ;; Add ellipsis on leading whitespace
+                             (when (string-match "\\`[[:space:]]+" s)
+                               (setq b (match-end 0))
+                               (add-text-properties 0 b `(display ,ellipsis) s))
+                             (when (> (length s) (- 40 b))
+                               (add-text-properties
+                                (min (+ b 40) (length s)) (length s)
+                                `(display ,ellipsis) s))
+                             s))
+                         yank-from-kill-ring-history))
+                ;; Allow ‘SPC’ to be inserted literally.
+                (minibuffer-completing-file-name t))
+           (completing-read "Yank from kill-ring: "
+                            (lambda (string pred action)
+                              (if (eq action 'metadata)
+                                  ;; Keep sorted by recency
+                                  '(metadata (display-sort-function . identity))
+                                (complete-with-action action completions string pred)))
+                            nil nil nil
+                            'yank-from-kill-ring-history))))
+  (setq yank-window-start (window-start))
+  (push-mark)
+  (insert-for-yank string))
+
+(put 'yank-from-kill-ring 'delete-selection t)
 
 (defun yank (&optional arg)
   "Reinsert (\"paste\") the last stretch of killed text.

Re: on helm substantial differences

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

> BTW, using `affix-function' also makes it possible to group completions,
> then the prefix of the first completion in the group could contain
> a group header with group name.

'mule--ucs-names-annotation' has this comment:

  ;; FIXME: It would be much better to add this annotation before rather than
  ;; after the char name, so the annotations are aligned.

With `affix-function' it's possible to add the char before the char name.
Adding groups is also simple (I recall that Richard asked to group by blocks).


[-- Attachment #2: chars-groups.png --]
[-- Type: image/png, Size: 47736 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: mule--ucs-names-by-group.patch --]
[-- Type: text/x-diff, Size: 2197 bytes --]

diff --git a/lisp/international/mule-cmds.el b/lisp/international/mule-cmds.el
index d361971a1f..f536c10a13 100644
--- a/lisp/international/mule-cmds.el
+++ b/lisp/international/mule-cmds.el
@@ -3092,6 +3092,37 @@ mule--ucs-names-annotation
   (let ((char (gethash name ucs-names)))
     (when char (format " (%c)" char))))
 
+(defun mule--ucs-names-by-group (names)
+  (let* ((names-chars
+          (mapcar (lambda (name) (cons name (gethash name ucs-names))) names))
+         (groups-names
+          (seq-group-by
+           (lambda (name-char)
+             (let ((script (aref char-script-table (cdr name-char))))
+               (if script (symbol-name script) "ungrouped")))
+           names-chars))
+         names-headers header)
+    (dolist (group groups-names)
+      (setq header t)
+      (dolist (name-char (cdr group))
+        (push (list (car name-char)
+                    (concat
+                     ;; header
+                     (if header
+                         (progn
+                           (setq header nil)
+                           (concat "\n" (propertize
+                                         (format "* %s\n" (car group))
+                                         'face 'header-line)))
+                       "")
+                     ;; prefix
+                     (if (cdr name-char) (format "%c" (cdr name-char)) " ")
+                     " ")
+                    ;; suffix
+                    "")
+              names-headers)))
+    (nreverse names-headers)))
+
 (defun char-from-name (string &optional ignore-case)
   "Return a character as a number from its Unicode name STRING.
 If optional IGNORE-CASE is non-nil, ignore case in STRING.
@@ -3138,8 +3169,10 @@ read-char-by-name
 	   prompt
 	   (lambda (string pred action)
 	     (if (eq action 'metadata)
-		 '(metadata
-		   (annotation-function . mule--ucs-names-annotation)
+		 `(metadata
+		   ,(if completions-detailed
+                        '(affix-function . mule--ucs-names-by-group)
+                      '(annotation-function . mule--ucs-names-annotation))
 		   (category . unicode-name))
 	       (complete-with-action action (ucs-names) string pred)))))
 	 (char

Re: select yank via completion

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  spacibba@aol.com,  Jean
>  Louis <bugs@gnu.support>,  andreyk.mad@gmail.com,  emacs-devel@gnu.org,
>   rudalics@gmx.at,  Gregory Heytings <ghe@sdf.org>,  Eli Zaretskii
>  <eliz@gnu.org>
> Date: Fri, 20 Nov 2020 10:53:39 +0200
> 
> BTW, sometimes it's also handy to just browse the kill-ring
> without immediately inserting an entry, to copy an edited entry
> (that adds a new kill-ring entry) to paste later.  To allow
> such browsing we need to remove "*" from the interactive spec of
> 'yank-pop'.

And then where do we bitch at the user if he/she does want to yank
into a read-only buffer?

Re: on helm substantial differences

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Date: Fri, 20 Nov 2020 11:24:24 +0200
> Cc: spacibba@aol.com, Jean Louis <bugs@gnu.support>, andreyk.mad@gmail.com,
>  emacs-devel@gnu.org, rudalics@gmx.at,
>  Stefan Monnier <monnier@iro.umontreal.ca>, Gregory Heytings <ghe@sdf.org>,
>  Eli Zaretskii <eliz@gnu.org>, Drew Adams <drew.adams@oracle.com>
> 
> > BTW, using `affix-function' also makes it possible to group completions,
> > then the prefix of the first completion in the group could contain
> > a group header with group name.
> 
> 'mule--ucs-names-annotation' has this comment:
> 
>   ;; FIXME: It would be much better to add this annotation before rather than
>   ;; after the char name, so the annotations are aligned.
> 
> With `affix-function' it's possible to add the char before the char name.
> Adding groups is also simple (I recall that Richard asked to group by blocks).

FWIW, I like the old behavior better, since it shows completions in
the "traditional" way: first the stuff I typed completed, then
whatever else is relevant.  So maybe this should be an optional
feature.

Re: on helm substantial differences

[Eli Zaretskii]

> Date: Fri, 20 Nov 2020 13:57:33 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: spacibba@aol.com, bugs@gnu.support, andreyk.mad@gmail.com,
>  emacs-devel@gnu.org, contovob@tcd.ie, rudalics@gmx.at,
>  monnier@iro.umontreal.ca, ghe@sdf.org, drew.adams@oracle.com
> 
> > From: Juri Linkov <juri@linkov.net>
> > Date: Fri, 20 Nov 2020 11:24:24 +0200
> > Cc: spacibba@aol.com, Jean Louis <bugs@gnu.support>, andreyk.mad@gmail.com,
> >  emacs-devel@gnu.org, rudalics@gmx.at,
> >  Stefan Monnier <monnier@iro.umontreal.ca>, Gregory Heytings <ghe@sdf.org>,
> >  Eli Zaretskii <eliz@gnu.org>, Drew Adams <drew.adams@oracle.com>
> > 
> > > BTW, using `affix-function' also makes it possible to group completions,
> > > then the prefix of the first completion in the group could contain
> > > a group header with group name.
> > 
> > 'mule--ucs-names-annotation' has this comment:
> > 
> >   ;; FIXME: It would be much better to add this annotation before rather than
> >   ;; after the char name, so the annotations are aligned.
> > 
> > With `affix-function' it's possible to add the char before the char name.
> > Adding groups is also simple (I recall that Richard asked to group by blocks).
> 
> FWIW, I like the old behavior better, since it shows completions in
> the "traditional" way: first the stuff I typed completed, then
> whatever else is relevant.  So maybe this should be an optional
> feature.

Btw, just switching the order doesn't yet guarantee the names will
align, because characters are not guaranteed to have the same width on
display.  To really align them we need to use :align-to properties, or
at least TABs.  Of course, using those techniques we could also align
the display without switching the order...

Re: select yank via completion

[Stefan Monnier]

> So now a new patch let-binds (minibuffer-completing-file-name t)
> around completing-read to allow inserting SPC, problem solved:

Please don't, since we're not actually completing file names.

I agree with Drew that we should probably just get rid of the SPC
binding in minibuffer completions, but until we do that you'll need

    (minibuffer-with-setup-hook
        (lambda ()
          (use-local-map
           (let ((map (make-sparse-keymap)))
             (set-keymap-parent map (current-local-map))
             (define-key map " " nil)
             map)))
      ...)

or something similar.  We could/should arguably introduce some easier
way to do that.  Maybe something like the patch below so you can just do

    (minibuffer-with-setup-hook
        (lambda () (setq-local minibuffer-SPC-completes nil))
      ...)

??


        Stefan


diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 9d57a817b2..20df9665a2 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -2342,6 +2342,8 @@ completion-help-at-point
   (define-key map "\r" 'exit-minibuffer)
   (define-key map "\n" 'exit-minibuffer))
 
+(defvar minibuffer-SPC-completes t)
+
 (defvar minibuffer-local-completion-map
   (let ((map (make-sparse-keymap)))
     (set-keymap-parent map minibuffer-local-map)
@@ -2349,7 +2351,9 @@ minibuffer-local-completion-map
     ;; M-TAB is already abused for many other purposes, so we should find
     ;; another binding for it.
     ;; (define-key map "\e\t" 'minibuffer-force-complete)
-    (define-key map " " 'minibuffer-complete-word)
+    (define-key map " "
+      `(menu-item "" minibuffer-complete-word
+                  :filter ,(lambda (cmd) (if minibuffer-SPC-completes cmd))))
     (define-key map "?" 'minibuffer-completion-help)
     (define-key map [prior] 'switch-to-completions)
     (define-key map "\M-v"  'switch-to-completions)

Re: on helm substantial differences

[Stefan Monnier]

> Btw, just switching the order doesn't yet guarantee the names will
> align, because characters are not guaranteed to have the same width on
> display.  To really align them we need to use :align-to properties, or
> at least TABs.  Of course, using those techniques we could also align
> the display without switching the order...

Indeed, but:
- the width of the chars varies a lot less than the width of their
  names, so if you put the char column after the name column, it ends up
  pushed pretty far to the right, whereas with the other order, the chars
  stay close to their name.
- with the current `annotation-function` it's almost impossible to do
  this alignment because you don't have access to all the candidates,
  only to a single candidate at a time.
  You could arguably align based on the worst possible case (i.e. the
  longest name existing in the ucs-names database), but that would make
  the result even worse.
  So even if you want to keep the current ordering, you need to change
  the API, like Juri's patch does.


        Stefan

Re: select yank via completion

[Juri Linkov]

>> BTW, sometimes it's also handy to just browse the kill-ring
>> without immediately inserting an entry, to copy an edited entry
>> (that adds a new kill-ring entry) to paste later.  To allow
>> such browsing we need to remove "*" from the interactive spec of
>> 'yank-pop'.
>
> And then where do we bitch at the user if he/she does want to yank
> into a read-only buffer?

In the same place where 'C-x 8 RET' (insert-char) does this:
it has no "*" in the interactive spec, and allows using it
in read-only buffers to browse a list of character completions
with TAB, but exiting the minibuffer with RET signals the error
"Buffer is read-only" somewhere in the 'insert' functions.

Re: select yank via completion

[Juri Linkov]

>> So now a new patch let-binds (minibuffer-completing-file-name t)
>> around completing-read to allow inserting SPC, problem solved:
>
> Please don't, since we're not actually completing file names.
>
> I agree with Drew that we should probably just get rid of the SPC
> binding in minibuffer completions,

SPC can be an obstacle even in some completion styles
where words are separated by SPC.

> but until we do that you'll need
>
>     (minibuffer-with-setup-hook
>         (lambda ()
>           (use-local-map
>            (let ((map (make-sparse-keymap)))
>              (set-keymap-parent map (current-local-map))
>              (define-key map " " nil)
>              map)))
>       ...)

Yes, I thought so too that maybe better would be to use
minibuffer-with-setup-hook.  I tried this code, and it works well.

> or something similar.  We could/should arguably introduce some easier
> way to do that.  Maybe something like the patch below so you can just do
>
>     (minibuffer-with-setup-hook
>         (lambda () (setq-local minibuffer-SPC-completes nil))
>       ...)

The problem is that there is another self-inserting key that can be
used for text editing: '?' is bound to minibuffer-completion-help.
Maybe then the variable could be named e.g. minibuffer-self-insert?

Re: select yank via completion

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: drew.adams@oracle.com,  monnier@iro.umontreal.ca,  spacibba@aol.com,
>   bugs@gnu.support,  andreyk.mad@gmail.com,  emacs-devel@gnu.org,
>   rudalics@gmx.at,  ghe@sdf.org
> Date: Sat, 21 Nov 2020 21:38:16 +0200
> 
> >> BTW, sometimes it's also handy to just browse the kill-ring
> >> without immediately inserting an entry, to copy an edited entry
> >> (that adds a new kill-ring entry) to paste later.  To allow
> >> such browsing we need to remove "*" from the interactive spec of
> >> 'yank-pop'.
> >
> > And then where do we bitch at the user if he/she does want to yank
> > into a read-only buffer?
> 
> In the same place where 'C-x 8 RET' (insert-char) does this:
> it has no "*" in the interactive spec, and allows using it
> in read-only buffers to browse a list of character completions
> with TAB, but exiting the minibuffer with RET signals the error
> "Buffer is read-only" somewhere in the 'insert' functions.

But that the function will behave differently if called via
call-interactively with a non-nil 2nd argument, right?

Re: on helm substantial differences

[Juri Linkov]

>> FWIW, I like the old behavior better, since it shows completions in
>> the "traditional" way: first the stuff I typed completed, then
>> whatever else is relevant.  So maybe this should be an optional
>> feature.

With the latest patch it's optional, enabled by 'completions-detailed',
but an additional option could be added - specific to grouping only
in read-char-by-name.

> Btw, just switching the order doesn't yet guarantee the names will
> align, because characters are not guaranteed to have the same width on
> display.  To really align them we need to use :align-to properties, or
> at least TABs.  Of course, using those techniques we could also align
> the display without switching the order...

This is a problem.  I tried TAB with tab-width=3, but some characters
are still wider than 3.  Then I tried 'char-width' on these characters,
but 'char-width' returns 1 for wide characters.  This is strange,
I expected it to return 4 for characters wider than tab-width=3.

Re: select yank via completion

[Juri Linkov]

>> > And then where do we bitch at the user if he/she does want to yank
>> > into a read-only buffer?
>>
>> In the same place where 'C-x 8 RET' (insert-char) does this:
>> it has no "*" in the interactive spec, and allows using it
>> in read-only buffers to browse a list of character completions
>> with TAB, but exiting the minibuffer with RET signals the error
>> "Buffer is read-only" somewhere in the 'insert' functions.
>
> But that the function will behave differently if called via
> call-interactively with a non-nil 2nd argument, right?

(call-interactively FUNCTION &optional RECORD-FLAG KEYS)

  Optional second arg RECORD-FLAG non-nil
  means unconditionally put this command in the variable ‘command-history’.

So it's a good thing that this command will be added to ‘command-history’,
because then the user can switch to a non-read-only buffer and type 'C-x M-:'
to insert the previously selected character in a writable buffer.

Re: select yank via completion

[Stefan Monnier]

> The problem is that there is another self-inserting key that can be
> used for text editing: '?' is bound to minibuffer-completion-help.
> Maybe then the variable could be named e.g. minibuffer-self-insert?

But `minibuffer-completion-help` currently isn't bound to any other key,
so removing this binding is more problematic.  Also, I find the `?`
binding much less often problematic in practice (probably because
I type `?` much less often).


        Stefan

RE: select yank via completion

[Drew Adams]

> > It took decades to even get `SPC' to be self-inserting for
> > `read-file-name'.  I raised the same issue for `completing-read' at
> > that time (and before then).
> 
> So now a new patch let-binds (minibuffer-completing-file-name t)
> around completing-read to allow inserting SPC, problem solved:

FWIW, I can't believe you'd think that's the right
solution.  There's no file-name completion here.

The proper solution is to fix the non-file-name
completion keymaps, to let SPC self-insert like
any other ordinary char.

RE: select yank via completion

[Drew Adams]

(Going through unread mails chronologically...)

> > So now a new patch let-binds (minibuffer-completing-file-name t)
> > around completing-read to allow inserting SPC, problem solved:
> 
> Please don't, since we're not actually completing file names.

Indeed.

> I agree with Drew that we should probably just get rid of the SPC
> binding in minibuffer completions, 

So let's finally do it - now.

> but until we do that you'll need...

Ugh.  Why on earth would we consider this particular
thing (yanking) to be a one-off?  Let's just give SPC
first-class citizenship finally - let it self-insert.

What's the big deal?  Why go round and round Robinson's
Barn to avoid and work around a silly, obsolete design
decision from the Dark Ages?

Is there even one good reason (good meaning beyond just
habit/inertia) why completion - *in general* - should
not let SPC (and `?', for that matter) self-insert?

If ever it's inappropriate, for some specific use of
`completing-read', to let SPC self-insert, THEN we use
your hoop to jump through.  (Word-completion is the
only use case I know of.)

RE: select yank via completion

[Drew Adams]

> >> So now a new patch let-binds (minibuffer-completing-file-name t)
> >> around completing-read to allow inserting SPC, problem solved:
> >
> > Please don't, since we're not actually completing file names.
> >
> > I agree with Drew that we should probably just get rid of the SPC
> > binding in minibuffer completions,
> 
> SPC can be an obstacle even in some completion styles
> where words are separated by SPC.

Please provide an example/recipe.

Completion candidates can contain SPC chars.

How candidates are displayed has (should have)
no bearing on what chars the candidates contain.

Candidate separation in *Completions* (for
example) is simply for visual/display purposes.
There should be no dependence of the display
code or the completion code on what chars are
used in candidates.

> The problem is that there is another self-inserting key that can be
> used for text editing: '?' is bound to minibuffer-completion-help.
> Maybe then the variable could be named e.g. minibuffer-self-insert?

`?', like `SPC', should be self-inserting during
completion.  There's zero problem doing that.
I've been doing it for years.

I use `M-?', not `?', for completion help In my
case it's `icicle-minibuffer-help', but vanilla
Emacs could just bind `minibuffer-completion-help'
to `M-?'.

Making users use `C-q SPC' each time they want
to insert a `SPC' char is ridiculous.  Likewise,
for `C-q ?' to insert a `?' char.

Re: select yank via completion

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2020-11-22 01:22]:
> > >> So now a new patch let-binds (minibuffer-completing-file-name t)
> > >> around completing-read to allow inserting SPC, problem solved:
> > >
> > > Please don't, since we're not actually completing file names.
> > >
> > > I agree with Drew that we should probably just get rid of the SPC
> > > binding in minibuffer completions,
> > 
> > SPC can be an obstacle even in some completion styles
> > where words are separated by SPC.
> 
> Please provide an example/recipe.
> 
> Completion candidates can contain SPC chars.

Hundreds of thousands of my completion candidates in the database have
spaces and I use standard completion about just as fast. People's
names and organization names have spaces. So I complete all time
candidates with spaces. But SPACE is compatible with that type of
completion with `minibuffer-complete-word' as it will automatically
give me the right choice!

Yet I find the standard completion with SPC how it is now just fine to
complete those candidates with spaces that I usually have.

It is bound to (minibuffer-complete-word) and if there is any change
then is better that such change becomes optional as defcustom, so that
default is not changed. Otherwise it would be a game changer to me.

If you decide to change it to SPC as self-insert-command then it
breaks habits of many other users as well and it could be better to
just introduce customization for that new feature, and not change. I
say "new" because it is new to users who have different habit of
completing with space.

I have changed key bindings temporarily to self-insert and it becomes
weird and confusing as I am not familiar with it as my expectation was
to complete words with space.

Example for this candidate:

10010 Maro's Country Home [PDF by Page Nr.]

I have expected that this below would complete but then where do I go
back? (as thoughts) Which space or how many to delete here (as
thoughts). So confusion is taking place.

10010 Mar    Cou

Then I was at this point:

10010 Maro's Country Home         <--- spaces end here and I could not complete

So there would be various programming demands for changes when space
becomes suddenly out of blue self-insert instead
minibuffer-complete-word as usual.

Major problem would be changing what is default.

Re: select yank via completion

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2020-11-22 01:22]:
> I use `M-?', not `?', for completion help In my
> case it's `icicle-minibuffer-help', but vanilla
> Emacs could just bind `minibuffer-completion-help'
> to `M-?'.

I would rather like ? to stay where it is as it is also used in many
other similar functions in Emacs and it breaks habits of those who
seek for help at usual key.

(setq collection '("Did he say?"
                   "Did he say so?"))

(completing-read "Choice: " collection)

I understand that above completion is not easy. But those users who
need question mark or SPC they can always use C-q ? or C-q SPC as that
is built-in and exists and those users will (I guess) know it.

If that fact is missing in the Manual, then it should be added how to
use C-q to insert SPC or ?

If such option is added to self-insert ? or SPC then is better not to
add it as default, rather as customization. That way those users who
need self insert for those keys will have it ready.

Re: on helm substantial differences

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: spacibba@aol.com,  bugs@gnu.support,  andreyk.mad@gmail.com,
>   emacs-devel@gnu.org,  contovob@tcd.ie,  rudalics@gmx.at,
>   monnier@iro.umontreal.ca,  ghe@sdf.org,  drew.adams@oracle.com
> Date: Sat, 21 Nov 2020 22:18:08 +0200
> 
> >> FWIW, I like the old behavior better, since it shows completions in
> >> the "traditional" way: first the stuff I typed completed, then
> >> whatever else is relevant.  So maybe this should be an optional
> >> feature.
> 
> With the latest patch it's optional, enabled by 'completions-detailed',
> but an additional option could be added - specific to grouping only
> in read-char-by-name.
> 
> > Btw, just switching the order doesn't yet guarantee the names will
> > align, because characters are not guaranteed to have the same width on
> > display.  To really align them we need to use :align-to properties, or
> > at least TABs.  Of course, using those techniques we could also align
> > the display without switching the order...
> 
> This is a problem.  I tried TAB with tab-width=3, but some characters
> are still wider than 3.  Then I tried 'char-width' on these characters,
> but 'char-width' returns 1 for wide characters.  This is strange,
> I expected it to return 4 for characters wider than tab-width=3.

The result of char-width can only ever be 0, 1, or 2.  On GUI frames,
it doesn't always return the exact value, because characters can be
wider than 1, but narrower than 2.  But no character should be wider
than 2 columns.  Which characters did you see that required the width
of 3 or 4?

Re: on helm substantial differences

[Juri Linkov]

> The result of char-width can only ever be 0, 1, or 2.  On GUI frames,
> it doesn't always return the exact value, because characters can be
> wider than 1, but narrower than 2.  But no character should be wider
> than 2 columns.  Which characters did you see that required the width
> of 3 or 4?

I tried to set tab-width to 4, and the first character
that doesn't fit to the 4-column tab stop is:

ഐ	MALAYALAM LETTER AI

But (char-width ?ഐ) returns 1.  Some examples of more 4-column wide
characters where char-width returns 1:

ை	TAMIL VOWEL SIGN AI
ෛ	SINHALA VOWEL SIGN KOMBU DEKA
꧅	JAVANESE PADA LUHUR
𒐫	CUNEIFORM NUMERIC SIGN NINE SHAR2


Maybe there is another function that could return the real number of
tab columns a character takes on the screen?

Re: on helm substantial differences

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Date: Sun, 22 Nov 2020 10:36:02 +0200
> Cc: spacibba@aol.com, bugs@gnu.support, andreyk.mad@gmail.com,
>  emacs-devel@gnu.org, contovob@tcd.ie, rudalics@gmx.at,
>  monnier@iro.umontreal.ca, ghe@sdf.org, drew.adams@oracle.com
> 
> > The result of char-width can only ever be 0, 1, or 2.  On GUI frames,
> > it doesn't always return the exact value, because characters can be
> > wider than 1, but narrower than 2.  But no character should be wider
> > than 2 columns.  Which characters did you see that required the width
> > of 3 or 4?
> 
> I tried to set tab-width to 4, and the first character
> that doesn't fit to the 4-column tab stop is:
> 
> ഐ	MALAYALAM LETTER AI
> 
> But (char-width ?ഐ) returns 1.  Some examples of more 4-column wide
> characters where char-width returns 1:
> 
> ை	TAMIL VOWEL SIGN AI
> ෛ	SINHALA VOWEL SIGN KOMBU DEKA
> ꧅	JAVANESE PADA LUHUR
> 𒐫	CUNEIFORM NUMERIC SIGN NINE SHAR2

On GUI frames, this depends on the font in use.  You cannot really
rely on char-width to solve this.

> Maybe there is another function that could return the real number of
> tab columns a character takes on the screen?

There is: font-get-glyphs.  But I'd recommend to just use a TAB of a
suitable width instead of dealing with the complexity that
font-get-glyphs requires.

Re: on helm substantial differences

[Juri Linkov]

>> 𒐫	CUNEIFORM NUMERIC SIGN NINE SHAR2
>
> On GUI frames, this depends on the font in use.  You cannot really
> rely on char-width to solve this.
>
>> Maybe there is another function that could return the real number of
>> tab columns a character takes on the screen?
>
> There is: font-get-glyphs.  But I'd recommend to just use a TAB of a
> suitable width instead of dealing with the complexity that
> font-get-glyphs requires.

I can't find a suitable width for TAB: 5 makes a too wide gap between
a character and its name, and with 4 some characters are misaligned.

But despite the complexity of font-get-glyphs, is it reasonably fast?

I tried to rely on next-line that moves to the beginning of the line
on the character wider that a previous character on the previous line,
and keep increasing the current column while moving over all character lines:

(with-temp-buffer
  (dotimes (c (max-char))
    (insert c ?\n))
  (insert "1234567890\n")
  (goto-char (point-min))
  (dotimes (c (max-char))
    (when (bolp) (forward-char))
    (next-line))
  (current-column))

But it takes too much time on all characters.

Re: on helm substantial differences

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: spacibba@aol.com,  bugs@gnu.support,  andreyk.mad@gmail.com,
>   emacs-devel@gnu.org,  contovob@tcd.ie,  rudalics@gmx.at,
>   monnier@iro.umontreal.ca,  ghe@sdf.org,  drew.adams@oracle.com
> Date: Sun, 22 Nov 2020 22:11:01 +0200
> 
> > There is: font-get-glyphs.  But I'd recommend to just use a TAB of a
> > suitable width instead of dealing with the complexity that
> > font-get-glyphs requires.
> 
> I can't find a suitable width for TAB: 5 makes a too wide gap between
> a character and its name, and with 4 some characters are misaligned.

Then I think we should just use 5.

> But despite the complexity of font-get-glyphs, is it reasonably fast?

I cannot answer that, you will have to try and see.  Whether the speed
is reasonable depends on what the rest of the application does.

Re: select yank via completion

[Basil L. Contovounesios]

Thanks for this Juri, I can now ditch Ivy's counsel-yank-pop for vanilla
yank-pop again.

Juri Linkov <juri@linkov.net> writes:

> +                             (when (> (length s) (- 40 b))
> +                               (add-text-properties
> +                                (min (+ b 40) (length s)) (length s)
> +                                `(display ,ellipsis) s))

Any chance this threshold can be configurable?  I don't mind completions
as wide as or wider than the frame in Ivy's minibuffer.

-- 
Basil

Re: select yank via completion

[Juri Linkov]

>> +                             (when (> (length s) (- 40 b))
>> +                               (add-text-properties
>> +                                (min (+ b 40) (length s)) (length s)
>> +                                `(display ,ellipsis) s))
>
> Any chance this threshold can be configurable?  I don't mind completions
> as wide as or wider than the frame in Ivy's minibuffer.

The default value 40 was intended to be able to accommodate two columns of
completions in the *Completions* buffer in the default 80-column wide frame,
but indeed a hard-coded constant should be avoided.

Instead of adding a specific defcustom for kill-ring completions,
maybe some more general option is possible in minibuffer.el?
Something like how many columns the user expects for completions.

If a package like ivy is not column-oriented as the *Completions* buffer,
then ivy could set such an option to 1.  Then yank-pop could truncate
completions to the frame width.  Or such truncation should be moved
to minibuffer.el and extended with other features from its TODO item:

;;   - indicate how to display the completions in *Completions* (turn
;;     \n into something else, add special boundaries between
;;     completions).  E.g. when completing from the kill-ring.

Re: select yank via completion

[Andrii Kolomoiets]

Juri Linkov <juri@linkov.net> writes:

>>> +                             (when (> (length s) (- 40 b))
>>> +                               (add-text-properties
>>> +                                (min (+ b 40) (length s)) (length s)
>>> +                                `(display ,ellipsis) s))
>>
>> Any chance this threshold can be configurable?  I don't mind completions
>> as wide as or wider than the frame in Ivy's minibuffer.
>
> Instead of adding a specific defcustom for kill-ring completions,
> maybe some more general option is possible in minibuffer.el?
> Something like how many columns the user expects for completions.

Would be nice.

I'm using custom function in display-buffer-alist to show *Completions*
buffer in child frame.  To make completions occupy single column, frame
width is set to 1 initially and later resized to fit the buffer.

Such an option will make it easier.

Thanks!

Re: on helm substantial differences

[Juri Linkov]

>> > There is: font-get-glyphs.  But I'd recommend to just use a TAB of a
>> > suitable width instead of dealing with the complexity that
>> > font-get-glyphs requires.
>>
>> I can't find a suitable width for TAB: 5 makes a too wide gap between
>> a character and its name, and with 4 some characters are misaligned.
>
> Then I think we should just use 5.

tab-width=5 doesn't help to align characters in the second column
when completions are displayed in two columns.

To align the second column, 'completion--insert-strings'
inserts " \t" with `(display (space :align-to ,column)).

So when completions use \t to separate a character from its name,
then the two-column layout schematically looks like this:

character1 \t name1   \t with :align-to   character2 \t name2

And tab-width doesn't align properly \t between character2 and name2.

>> But despite the complexity of font-get-glyphs, is it reasonably fast?
>
> I cannot answer that, you will have to try and see.  Whether the speed
> is reasonable depends on what the rest of the application does.

It seems font-get-glyphs can't be used because on wide characters it
returns nil:

(font-get-glyphs (font-at (point)) 0 1 "𒐫")
=> [nil]

Re: on helm substantial differences

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: spacibba@aol.com,  bugs@gnu.support,  andreyk.mad@gmail.com,
>   emacs-devel@gnu.org,  contovob@tcd.ie,  rudalics@gmx.at,
>   monnier@iro.umontreal.ca,  ghe@sdf.org,  drew.adams@oracle.com
> Date: Wed, 25 Nov 2020 11:10:05 +0200
> 
> > Then I think we should just use 5.
> 
> tab-width=5 doesn't help to align characters in the second column
> when completions are displayed in two columns.
> 
> To align the second column, 'completion--insert-strings'
> inserts " \t" with `(display (space :align-to ,column)).
> 
> So when completions use \t to separate a character from its name,
> then the two-column layout schematically looks like this:
> 
> character1 \t name1   \t with :align-to   character2 \t name2
> 
> And tab-width doesn't align properly \t between character2 and name2.

I don't think I understand why.  Maybe the value of :align-to needs
tuning?  If not, what is the problem here?

> It seems font-get-glyphs can't be used because on wide characters it
> returns nil:
> 
> (font-get-glyphs (font-at (point)) 0 1 "𒐫")
> => [nil]

Is this with point on that wide character?

Re: on helm substantial differences

[Juri Linkov]

>> To align the second column, 'completion--insert-strings'
>> inserts " \t" with `(display (space :align-to ,column)).
>>
>> So when completions use \t to separate a character from its name,
>> then the two-column layout schematically looks like this:
>>
>> character1 \t name1   \t with :align-to   character2 \t name2
>>
>> And tab-width doesn't align properly \t between character2 and name2.
>
> I don't think I understand why.  Maybe the value of :align-to needs
> tuning?  If not, what is the problem here?

Tuning means adding :align-to to all '\t'?  E.g.:

character1 (\t :align-to) name1 (\t :align-to) character2 (\t :align-to) name2

But the caller doesn't know the value for the third :align-to.
So instead of this, the caller should calculate :width, e.g.:

character1 (\t :width) name1 (\t :align-to) character2 (\t :width) name2

>> It seems font-get-glyphs can't be used because on wide characters it
>> returns nil:
>>
>> (font-get-glyphs (font-at (point)) 0 1 "𒐫")
>> => [nil]
>
> Is this with point on that wide character?

Got it.  When point is on the character, everything works fine, and this code:

(let ((max-width 0))
    (dotimes (c #x1FFFF)
      (insert c)
      (let* ((font (font-at (1- (point))))
	     (glyphs (when font (font-get-glyphs
				 font 0 1
				 (string c)))))
	(when (and glyphs (aref glyphs 0))
	  (setq max-width (max max-width (aref (aref glyphs 0) 4)))))
      (delete-char -1))
    (ceiling max-width (frame-char-width)))

correctly returns 8, i.e. tab-width for the widest character.

The problem is that this code requires to be run in the selected buffer.
When enclosed in 'with-temp-buffer', it raises an error:

Debugger entered--Lisp error: (error "Specified window is not displaying the current buffer")
  font-at(1)

This is a pretty bad limitation.  Maybe better to pre-calculate widths
of all characters and generate a mapping from char to its width.
This will also improve performance - the above code takes 15 seconds.

Re: select yank via completion

[Juri Linkov]

>> The problem is that there is another self-inserting key that can be
>> used for text editing: '?' is bound to minibuffer-completion-help.
>> Maybe then the variable could be named e.g. minibuffer-self-insert?
>
> `?', like `SPC', should be self-inserting during
> completion.  There's zero problem doing that.
> I've been doing it for years.
>
> I use `M-?', not `?', for completion help In my
> case it's `icicle-minibuffer-help', but vanilla
> Emacs could just bind `minibuffer-completion-help'
> to `M-?'.

completion-in-region already uses `M-?', so maybe
completion-in-minibuffer could use the same `M-?'?

RE: select yank via completion

[Drew Adams]

> >> The problem is that there is another self-inserting key that can be
> >> used for text editing: '?' is bound to minibuffer-completion-help.
> >> Maybe then the variable could be named e.g. minibuffer-self-insert?
> >
> > `?', like `SPC', should be self-inserting during
> > completion.  There's zero problem doing that.
> > I've been doing it for years.
> >
> > I use `M-?', not `?', for completion help In my
> > case it's `icicle-minibuffer-help', but vanilla
> > Emacs could just bind `minibuffer-completion-help'
> > to `M-?'.
> 
> completion-in-region already uses `M-?', so maybe
> completion-in-minibuffer could use the same `M-?'?

Maybe.  I don't know what `completion-in-region' or
`completion-in-minibuffer' is.

Re: on helm substantial differences

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: spacibba@aol.com,  bugs@gnu.support,  andreyk.mad@gmail.com,
>   emacs-devel@gnu.org,  contovob@tcd.ie,  rudalics@gmx.at,
>   monnier@iro.umontreal.ca,  ghe@sdf.org,  drew.adams@oracle.com
> Date: Wed, 25 Nov 2020 21:16:10 +0200
> 
> >> character1 \t name1   \t with :align-to   character2 \t name2
> >>
> >> And tab-width doesn't align properly \t between character2 and name2.
> >
> > I don't think I understand why.  Maybe the value of :align-to needs
> > tuning?  If not, what is the problem here?
> 
> Tuning means adding :align-to to all '\t'?

No, just use :align-to VALUE and tune VALUE.

Why doesn't name2 get aligned? I still don't think I understand that.

Re: select yank via completion

[Juri Linkov]

>>>> +                             (when (> (length s) (- 40 b))
>>>> +                               (add-text-properties
>>>> +                                (min (+ b 40) (length s)) (length s)
>>>> +                                `(display ,ellipsis) s))
>>>
>>> Any chance this threshold can be configurable?  I don't mind completions
>>> as wide as or wider than the frame in Ivy's minibuffer.

As an intermediate fix, now code at least doesn't contain an arbitrary
constant, but defaults to the frame width.

>> Instead of adding a specific defcustom for kill-ring completions,
>> maybe some more general option is possible in minibuffer.el?
>> Something like how many columns the user expects for completions.
>
> Would be nice.
>
> I'm using custom function in display-buffer-alist to show *Completions*
> buffer in child frame.  To make completions occupy single column, frame
> width is set to 1 initially and later resized to fit the buffer.
>
> Such an option will make it easier.

Now 'completions-format' supports a new value 'one-column'.

Re: on helm substantial differences

[Juri Linkov]

>> >> character1 \t name1   \t with :align-to   character2 \t name2
>> >>
>> >> And tab-width doesn't align properly \t between character2 and name2.
>> >
>> > I don't think I understand why.  Maybe the value of :align-to needs
>> > tuning?  If not, what is the problem here?
>> 
>> Tuning means adding :align-to to all '\t'?
>
> No, just use :align-to VALUE and tune VALUE.
>
> Why doesn't name2 get aligned? I still don't think I understand that.

The caller is 'read-char-by-name', and it creates two completion strings
"character1 (\t with :align-to) name1" and "character2 (\t with :align-to) name2",
and sends them to 'completion--insert-strings' that adds "\t with :align-to"
as a separator between two columns.

How "character2 (\t with :align-to) name2" could predict the right value
of this 'align-to' if it doesn't know what value of 'align-to' will use
'completion--insert-strings' in its separator?

Re: select yank via completion

[Basil L. Contovounesios]

Juri Linkov <juri@linkov.net> writes:

>>>>> +                             (when (> (length s) (- 40 b))
>>>>> +                               (add-text-properties
>>>>> +                                (min (+ b 40) (length s)) (length s)
>>>>> +                                `(display ,ellipsis) s))
>>>>
>>>> Any chance this threshold can be configurable?  I don't mind completions
>>>> as wide as or wider than the frame in Ivy's minibuffer.
>
> As an intermediate fix, now code at least doesn't contain an arbitrary
> constant, but defaults to the frame width.

Thanks, but don't scroll bars, fringes, etc. have to be subtracted from
the frame width?

-- 
Basil

Re: select yank via completion

[Juri Linkov]

>>>>>> +                             (when (> (length s) (- 40 b))
>>>>>> +                               (add-text-properties
>>>>>> +                                (min (+ b 40) (length s)) (length s)
>>>>>> +                                `(display ,ellipsis) s))
>>>>>
>>>>> Any chance this threshold can be configurable?  I don't mind completions
>>>>> as wide as or wider than the frame in Ivy's minibuffer.
>>
>> As an intermediate fix, now code at least doesn't contain an arbitrary
>> constant, but defaults to the frame width.
>
> Thanks, but don't scroll bars, fringes, etc. have to be subtracted from
> the frame width?

Is this the correct formula?

(floor (- (frame-inner-width)
          (frame-internal-border-width)
          (frame-fringe-width)
          (frame-scroll-bar-width))
       (frame-char-width))

Re: select yank via completion

[Eli Zaretskii]

On November 26, 2020 11:26:13 AM GMT+02:00, Juri Linkov <juri@linkov.net> wrote:
> >>>>>> +                             (when (> (length s) (- 40 b))
> >>>>>> +                               (add-text-properties
> >>>>>> +                                (min (+ b 40) (length s))
> (length s)
> >>>>>> +                                `(display ,ellipsis) s))
> >>>>>
> >>>>> Any chance this threshold can be configurable?  I don't mind
> completions
> >>>>> as wide as or wider than the frame in Ivy's minibuffer.
> >>
> >> As an intermediate fix, now code at least doesn't contain an
> arbitrary
> >> constant, but defaults to the frame width.
> >
> > Thanks, but don't scroll bars, fringes, etc. have to be subtracted
> from
> > the frame width?
> 
> Is this the correct formula?
> 
> (floor (- (frame-inner-width)
>           (frame-internal-border-width)
>           (frame-fringe-width)
>           (frame-scroll-bar-width))
>        (frame-char-width))

I think you want simply to call frame-text-cols.

Re: on helm substantial differences

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: spacibba@aol.com,  bugs@gnu.support,  andreyk.mad@gmail.com,
>   emacs-devel@gnu.org,  contovob@tcd.ie,  rudalics@gmx.at,
>   monnier@iro.umontreal.ca,  ghe@sdf.org,  drew.adams@oracle.com
> Date: Wed, 25 Nov 2020 22:35:48 +0200
> 
> > Why doesn't name2 get aligned? I still don't think I understand that.
> 
> The caller is 'read-char-by-name', and it creates two completion strings
> "character1 (\t with :align-to) name1" and "character2 (\t with :align-to) name2",
> and sends them to 'completion--insert-strings' that adds "\t with :align-to"
> as a separator between two columns.
> 
> How "character2 (\t with :align-to) name2" could predict the right value
> of this 'align-to' if it doesn't know what value of 'align-to' will use
> 'completion--insert-strings' in its separator?

Ah, okay.  Would it work to change read-char-by-name to generate
strings "character1 TAB name1" and "character2 TAB name2", and then
make completion--insert-strings to produce a suitably calculated
(SPC with :align-to) separator between them?  Then all you'd need to
do is set tab-width in the completions buffer.

Re: select yank via completion

[Basil L. Contovounesios]

Eli Zaretskii <eliz@gnu.org> writes:

> On November 26, 2020 11:26:13 AM GMT+02:00, Juri Linkov <juri@linkov.net> wrote:
>
>> Is this the correct formula?
>> 
>> (floor (- (frame-inner-width)
>>           (frame-internal-border-width)
>>           (frame-fringe-width)
>>           (frame-scroll-bar-width))
>>        (frame-char-width))
>
> I think you want simply to call frame-text-cols.

Does that take line-number-display-width into account as well?
Or does that not come into play here?

Thanks,

-- 
Basil

Re: select yank via completion

[Eli Zaretskii]

> From: "Basil L. Contovounesios" <contovob@tcd.ie>
> Cc: emacs-devel@gnu.org, Juri Linkov <juri@linkov.net>,  spacibba@aol.com,
>  Andrii Kolomoiets <andreyk.mad@gmail.com>, Jean Louis <bugs@gnu.support>,
>  rudalics@gmx.at, Stefan Monnier <monnier@iro.umontreal.ca>, Gregory
>  Heytings <ghe@sdf.org>, Drew Adams <drew.adams@oracle.com>
> Date: Thu, 26 Nov 2020 21:17:17 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I think you want simply to call frame-text-cols.
> 
> Does that take line-number-display-width into account as well?
> Or does that not come into play here?

The latter, I think.  display-line-numbers-mode is a buffer-local
feature, so if you want to look at frame dimensions, you shouldn't
take it into account.

Re: on helm substantial differences

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 706 bytes --]

>> How "character2 (\t with :align-to) name2" could predict the right value
>> of this 'align-to' if it doesn't know what value of 'align-to' will use
>> 'completion--insert-strings' in its separator?
>
> Ah, okay.  Would it work to change read-char-by-name to generate
> strings "character1 TAB name1" and "character2 TAB name2", and then
> make completion--insert-strings to produce a suitably calculated
> (SPC with :align-to) separator between them?  Then all you'd need to
> do is set tab-width in the completions buffer.

Thanks, the suggested solution works well.  This screenshot demonstrates
that spacing between a character and its name is the same in the first column
and in the second column:


[-- Attachment #2: tabwidth.png --]
[-- Type: image/png, Size: 54720 bytes --]

[-- Attachment #3: Type: text/plain, Size: 38 bytes --]


And the patch that implements this:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: tabwidth.patch --]
[-- Type: text/x-diff, Size: 3617 bytes --]

diff --git a/lisp/international/mule-cmds.el b/lisp/international/mule-cmds.el
index d361971a1f..d59f2c0ebf 100644
--- a/lisp/international/mule-cmds.el
+++ b/lisp/international/mule-cmds.el
@@ -3084,13 +3084,11 @@ ucs-names
         (puthash "BELL (BEL)" ?\a names)
         (setq ucs-names names))))
 
-(defun mule--ucs-names-annotation (name)
-  ;; FIXME: It would be much better to add this annotation before rather than
-  ;; after the char name, so the annotations are aligned.
-  ;; FIXME: The default behavior of displaying annotations in italics
-  ;; doesn't work well here.
-  (let ((char (gethash name ucs-names)))
-    (when char (format " (%c)" char))))
+(defun mule--ucs-names-affixation (names)
+  (mapcar (lambda (name)
+            (let ((char (gethash name ucs-names)))
+              (list name (concat (if char (format "%c" char) " ") "\t") "")))
+          names))
 
 (defun char-from-name (string &optional ignore-case)
   "Return a character as a number from its Unicode name STRING.
@@ -3133,13 +3131,14 @@ read-char-by-name
 as names, not numbers."
   (let* ((enable-recursive-minibuffers t)
 	 (completion-ignore-case t)
+	 (completion-tab-width 4)
 	 (input
 	  (completing-read
 	   prompt
 	   (lambda (string pred action)
 	     (if (eq action 'metadata)
 		 '(metadata
-		   (annotation-function . mule--ucs-names-annotation)
+		   (affixation-function . mule--ucs-names-affixation)
 		   (category . unicode-name))
 	       (complete-with-action action (ucs-names) string pred)))))
 	 (char
diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 87bf3d36fa..64498b3729 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -1134,6 +1134,7 @@ completion--cycle-threshold
 (defvar-local completion-all-sorted-completions nil)
 (defvar-local completion--all-sorted-completions-location nil)
 (defvar completion-cycling nil)      ;Function that takes down the cycling map.
+(defvar completion-tab-width 8)
 
 (defvar completion-fail-discreetly nil
   "If non-nil, stay quiet when there  is no match.")
@@ -1718,6 +1719,10 @@ completion--insert-strings
 	   (row 0)
            (first t)
 	   (laststring nil))
+      (unless (or tab-stop-list (zerop (mod colwidth completion-tab-width)))
+        ;; Align to tab positions for the case
+        ;; when the caller uses tabs inside prefix.
+        (setq colwidth (- colwidth (mod colwidth completion-tab-width))))
       ;; The insertion should be "sensible" no matter what choices were made
       ;; for the parameters above.
       (dolist (str strings)
@@ -1758,9 +1763,10 @@ completion--insert-strings
 		  ;; already past the goal column, there is still
 		  ;; a space displayed.
 		  (set-text-properties (1- (point)) (point)
-				       ;; We can't just set tab-width, because
-				       ;; completion-setup-function will kill
-				       ;; all local variables :-(
+				       ;; We can set tab-width using
+				       ;; completion-tab-width, but
+				       ;; the caller can prefer using
+				       ;; \t to align prefixes.
 				       `(display (space :align-to ,column)))
 		  nil))))
             (setq first nil)
diff --git a/lisp/simple.el b/lisp/simple.el
index c9f4f2bb44..8cd5de20d7 100644
--- a/lisp/simple.el
+++ b/lisp/simple.el
@@ -8822,6 +8822,8 @@ completion-setup-function
 	     insert-fun))
       (set (make-local-variable 'completion-reference-buffer) mainbuf)
       (if base-dir (setq default-directory base-dir))
+      (when completion-tab-width
+        (setq tab-width completion-tab-width))
       ;; Maybe insert help string.
       (when completion-show-help
 	(goto-char (point-min))

Re: on helm substantial differences

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: spacibba@aol.com,  bugs@gnu.support,  andreyk.mad@gmail.com,
>   emacs-devel@gnu.org,  contovob@tcd.ie,  rudalics@gmx.at,
>   monnier@iro.umontreal.ca,  ghe@sdf.org,  drew.adams@oracle.com
> Date: Fri, 27 Nov 2020 10:45:10 +0200
> 
> > Ah, okay.  Would it work to change read-char-by-name to generate
> > strings "character1 TAB name1" and "character2 TAB name2", and then
> > make completion--insert-strings to produce a suitably calculated
> > (SPC with :align-to) separator between them?  Then all you'd need to
> > do is set tab-width in the completions buffer.
> 
> Thanks, the suggested solution works well.  This screenshot demonstrates
> that spacing between a character and its name is the same in the first column
> and in the second column:

LGTM, thanks.

Re: select yank via completion

[Juri Linkov]

>> > I think you want simply to call frame-text-cols.
>> 
>> Does that take line-number-display-width into account as well?
>> Or does that not come into play here?
>
> The latter, I think.  display-line-numbers-mode is a buffer-local
> feature, so if you want to look at frame dimensions, you shouldn't
> take it into account.

So now it uses frame-text-cols.

Re: on helm substantial differences

[Juri Linkov]

>> > Ah, okay.  Would it work to change read-char-by-name to generate
>> > strings "character1 TAB name1" and "character2 TAB name2", and then
>> > make completion--insert-strings to produce a suitably calculated
>> > (SPC with :align-to) separator between them?  Then all you'd need to
>> > do is set tab-width in the completions buffer.
>> 
>> Thanks, the suggested solution works well.  This screenshot demonstrates
>> that spacing between a character and its name is the same in the first column
>> and in the second column:
>
> LGTM, thanks.

Pushed to master.