bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

[Daniel Mendler]

In some scenarios it is useful to disable the completion style
infrastructure. This includes Eglot or my Asynchronous Fuzzy Finder for
Emacs (https://github.com/minad/affe/).

The addition of a passthrough completion style helps these use cases.

(defun passthrough-all-completions (_str table pred _point)
  "Passthrough completion function.
See `completion-all-completions' for the arguments STR, TABLE, PRED and
POINT."
  (let ((completion-regexp-list))
    (all-completions "" table pred)))

(defun passthrough-try-completion (str table pred _point)
  "Passthrough completion function.
See `completion-try-completion' for the arguments STR, TABLE, PRED and
POINT."
  (let ((completion-regexp-list)
        (completion (try-completion str table pred)))
    (if (stringp completion)
        (cons completion (length completion))
      completion)))

(add-to-list 'completion-styles-alist
             (list 'passthrough
                   #'passthrough-try-completion
                   #'passthrough-all-completions
                   "Passthrough completion style."))

bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> (defun passthrough-all-completions (_str table pred _point)
>   "Passthrough completion function.
> See `completion-all-completions' for the arguments STR, TABLE, PRED and
> POINT."
>   (let ((completion-regexp-list))
>     (all-completions "" table pred)))

Clearly, this is not right: passthrough completion should pass the `str`
and `point` info to the completion table.  The completion table may opt
to ignore that information, but we shouldn't prevent them from using it.
I expect most passthrough uses will want to use `str`.

> (defun passthrough-try-completion (str table pred _point)
>   "Passthrough completion function.
> See `completion-try-completion' for the arguments STR, TABLE, PRED and
> POINT."
>   (let ((completion-regexp-list)
>         (completion (try-completion str table pred)))
>     (if (stringp completion)
>         (cons completion (length completion))
>       completion)))

Same here: I think it's better to pass `point` to the completion table
and let the completion table return the `cons` cell.

Furthermore, `all-completions` and `try-completion` on a completion
table is defined as returning only the prefix completion, so the above
code would force `table` to be a "broken" completion table that doesn't
obey the normal completion table API.


        Stefan

bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

[Daniel Mendler]

On 5/23/21 1:20 AM, Stefan Monnier wrote:
>> (defun passthrough-all-completions (_str table pred _point)
>>   "Passthrough completion function.
>> See `completion-all-completions' for the arguments STR, TABLE, PRED and
>> POINT."
>>   (let ((completion-regexp-list))
>>     (all-completions "" table pred)))
> 
> Clearly, this is not right: passthrough completion should pass the `str`
> and `point` info to the completion table.  The completion table may opt
> to ignore that information, but we shouldn't prevent them from using it.
> I expect most passthrough uses will want to use `str`.

I see what you mean. In my use case I didn't use an improperly
implemented completion table which does not ignore . However the
question is then if this "passthrough" style is really needed since if
you don't ignore the input, it is mostly equivalent to the emacs21 style.

It is at least good to discuss this. I think all use cases of the
passthrough table are a bit of an edge case (or broken tables in the
strict definition), e.g., my fuzzy finder or eglot.

> Furthermore, `all-completions` and `try-completion` on a completion
> table is defined as returning only the prefix completion, so the above
> code would force `table` to be a "broken" completion table that doesn't
> obey the normal completion table API.

Well, it forces the table to return all candidates.

Overall it seems that the passthrough you have in mind is already
covered by the emacs21 style. I had considered something different which
effectively disables the full completion infrastructure. Maybe these use
cases are too special to be catered for by minibuffer.el?

I have to think a bit more about this. Probably I can achieve the same
in my fuzzy finder using the emacs21 style.

Daniel

bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

[Daniel Mendler]

On 5/23/21 12:21 PM, Daniel Mendler wrote:
> I have to think a bit more about this. Probably I can achieve the same
> in my fuzzy finder using the emacs21 style.

Okay, I cannot achieve this, since I want to still forego the
highlighting. But besides that ignoring the string inside the
passthrough style itself is not necessary.

So a style which does not apply highlighting is all that is needed?

Daniel

bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Clearly, this is not right: passthrough completion should pass the `str`
>> and `point` info to the completion table.  The completion table may opt
>> to ignore that information, but we shouldn't prevent them from using it.
>> I expect most passthrough uses will want to use `str`.
>
> I see what you mean. In my use case I didn't use an improperly
> implemented completion table which does not ignore . However the
> question is then if this "passthrough" style is really needed since if
> you don't ignore the input, it is mostly equivalent to the emacs21 style.

The "passthrough" (elsewhere called "backend") completion-style is
definitely something we need to add, yes.  But what it should do is pass
all the args from `completion-all/try-completions` to the table and let
the table do *all* the work, i.e. let the completion table implement the
completion-style methods.  But we shouldn't call `all-completions` or
`try-completions` for that.  Instead we should call the completion-table
directly, as is done for the `completion-boundaries` method.


        Stefan

bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]

If it helps, I've a working copy of Stefan's idea over at
github.com/joaotavora/sly.

Indeed it's called backend there, and it's been working fine for some years
to let Sly's completion providing backend do the flex or prefix heavy
lifting, while still appearing as a normal completion table to Emacs
frontends, such as icomplete, vanilla, company-capf and even Helm.

A first approach could be just to copy that over to Emacs minubuffer.el.

João

On Sun, May 23, 2021, 16:51 Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> >> Clearly, this is not right: passthrough completion should pass the `str`
> >> and `point` info to the completion table.  The completion table may opt
> >> to ignore that information, but we shouldn't prevent them from using it.
> >> I expect most passthrough uses will want to use `str`.
> >
> > I see what you mean. In my use case I didn't use an improperly
> > implemented completion table which does not ignore . However the
> > question is then if this "passthrough" style is really needed since if
> > you don't ignore the input, it is mostly equivalent to the emacs21 style.
>
> The "passthrough" (elsewhere called "backend") completion-style is
> definitely something we need to add, yes.  But what it should do is pass
> all the args from `completion-all/try-completions` to the table and let
> the table do *all* the work, i.e. let the completion table implement the
> completion-style methods.  But we shouldn't call `all-completions` or
> `try-completions` for that.  Instead we should call the completion-table
> directly, as is done for the `completion-boundaries` method.
>
>
>         Stefan
>
>

[-- Attachment #2: Type: text/html, Size: 2273 bytes --]

bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

[Daniel Mendler]

On 5/23/21 5:50 PM, Stefan Monnier via Bug reports for GNU Emacs, the
Swiss army knife of text editors wrote:
>>> Clearly, this is not right: passthrough completion should pass the `str`
>>> and `point` info to the completion table.  The completion table may opt
>>> to ignore that information, but we shouldn't prevent them from using it.
>>> I expect most passthrough uses will want to use `str`.
>>
>> I see what you mean. In my use case I didn't use an improperly
>> implemented completion table which does not ignore . However the
>> question is then if this "passthrough" style is really needed since if
>> you don't ignore the input, it is mostly equivalent to the emacs21 style.
> 
> The "passthrough" (elsewhere called "backend") completion-style is
> definitely something we need to add, yes.  But what it should do is pass
> all the args from `completion-all/try-completions` to the table and let
> the table do *all* the work, i.e. let the completion table implement the
> completion-style methods.  But we shouldn't call `all-completions` or
> `try-completions` for that.  Instead we should call the completion-table
> directly, as is done for the `completion-boundaries` method.

But `all-completions` and `try-completion` already calls the table
directly if the table is indeed a table.

Basically `all-completions/try-completion` themselves are already
"passthrough" modulo some calling convention.

Am I missing something?

bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> The "passthrough" (elsewhere called "backend") completion-style is
>> definitely something we need to add, yes.  But what it should do is pass
>> all the args from `completion-all/try-completions` to the table and let
>> the table do *all* the work, i.e. let the completion table implement the
>> completion-style methods.  But we shouldn't call `all-completions` or
>> `try-completions` for that.  Instead we should call the completion-table
>> directly, as is done for the `completion-boundaries` method.
> But `all-completions` and `try-completion` already calls the table
> directly if the table is indeed a table.

Yes, they do, but if you call them, then *you* don't: they call the table in
their own way for their own purpose (e.g. for `all-completions` they
call it to get all the completions matching a given prefix and that's
it, whereas the whole point of `passthrough` is to be able to fetch
matches based on other criteria (defined by the completion table)).

> Basically `all-completions/try-completion` themselves are already
> "passthrough" modulo some calling convention.
> Am I missing something?

Passthrough is about passing the completion-style API directly to
the backend.  `all-completions` is part of the backend API, not the
completion-style API.


        Stefan

bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

[Daniel Mendler]

On 5/26/21 11:28 PM, Stefan Monnier via Bug reports for GNU Emacs, the
Swiss army knife of text editors wrote:
>>> The "passthrough" (elsewhere called "backend") completion-style is
>>> definitely something we need to add, yes.  But what it should do is pass
>>> all the args from `completion-all/try-completions` to the table and let
>>> the table do *all* the work, i.e. let the completion table implement the
>>> completion-style methods.  But we shouldn't call `all-completions` or
>>> `try-completions` for that.  Instead we should call the completion-table
>>> directly, as is done for the `completion-boundaries` method.
>> But `all-completions` and `try-completion` already calls the table
>> directly if the table is indeed a table.
> 
> Yes, they do, but if you call them, then *you* don't: they call the table in
> their own way for their own purpose (e.g. for `all-completions` they
> call it to get all the completions matching a given prefix and that's
> it, whereas the whole point of `passthrough` is to be able to fetch
> matches based on other criteria (defined by the completion table)).

Sorry, I don't get it. What is the difference in calling the backend
between the two cases:

(1) passthrough-all-completions -> all-completions -> backend

versus

(2) passthrough-all-completions -> backend?

In case (2) you call the backend directly but then you also have to
handle completion tables which are not functions and you end up
reimplementing parts of `all-completions` in elisp. `all-completions`
convers non-function completion tables to lists.

>> Basically `all-completions/try-completion` themselves are already
>> "passthrough" modulo some calling convention.
>> Am I missing something?
> 
> Passthrough is about passing the completion-style API directly to
> the backend.  `all-completions` is part of the backend API, not the
> completion-style API.

Yes, I am aware that `all-completions` is the backend API. Is anything
lost when going through `all-completions`? From what I've seen the
arguments are passed on directly to the backend table.

Daniel

bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Sorry, I don't get it. What is the difference in calling the backend
> between the two cases:
>
> (1) passthrough-all-completions -> all-completions -> backend
>
> versus
>
> (2) passthrough-all-completions -> backend?

In case 2 you get to choose the arguments passed to the backend function.

> In case (2) you call the backend directly but then you also have to
> handle completion tables which are not functions and you end up

`passthrough` doesn't make any sense for non-function backends.

> Yes, I am aware that `all-completions` is the backend API. Is anything
> lost when going through `all-completions`?

The possibility to return completion candidates that aren't "prefix
completions" (which is the whole purpose of `passthrough`)?


        Stefan

bug#48572: 28.0.50; Add `passthrough` completion style to minibuffer.el

[Dmitry Gutov]

On 21.05.2021 21:48, Daniel Mendler wrote:
> The addition of a passthrough completion style helps these use cases.

+1

I think we also need some documented protocol for adding highlightings 
to such returned completions. Maybe even a lazy one.