bug#47711: bug#48841: [PATCH] Add new `completion-filter-completions` API and deferred highlighting

[Eli Zaretskii <eliz@gnu.org>]

> Cc: joaotavora@gmail.com, dgutov@yandex.ru, monnier@iro.umontreal.ca,
>  48841@debbugs.gnu.org, 47711@debbugs.gnu.org
> From: Daniel Mendler <mail@daniel-mendler.de>
> Date: Mon, 16 Aug 2021 11:42:22 +0200
> 
> >> To address your technical comment - this variable is precisely what one
> >> of the technical difficulties mentioned in my other mail is about.  The
> >> question is how we can retain backward compatibility of the completion
> >> style 'all' functions, e.g., 'completion-basic-all-completions', while
> >> still allowing the function to return the newly introduced alist format
> >> with more data, which enables 'completion-filter-completions' to perform
> >> the efficient deferred highlighting.
> > 
> > I understand, but given that we provide this for other packages, it
> > shouldn't be an internal variable.
> 
> No, we explicitly don't provide this variable for other packages. It is
> explicitly only meant to be used for the existing completion styles
> emacs21, emacs22, basic, substring, partial-completion, initials and
> flex to opt-in in a backward-compatible/calling convention preserving
> way to the alist return format. The idea is to keep the existing APIs
> fully backward compatible.
> 
> Other packages should select the format returned from the completion
> styles differently. They should return the alist format on Emacs 28 or
> if the API 'completion-filter-completions' API is present. In the not so
> near future external packages which support only Emacs 28 and upwards
> will then only return the alist format and don't have to perform any
> detection anymore.

What if some package outside minibuffer.el will want to control the
format of the returned value, for some reason, like support for old
Emacs versions? are we going to disallow that?

> >> The new API 'completion-filter-completions' will substitute the existing
> >> API 'completion-all-completions'.
> > 
> > That's your hope, and I understand.  But we as a project didn't yet
> > decide to deprecate the original APIs, so talking about superseding is
> > premature.
> 
> It is not the hope - it is the explicit goal. The API has been designed
> to replace the existing API 'completion-all-completions'.

A goal is not a fact.  Until that goal is reached, we cannot in good
faith tell people an API is superseded.

> We can of
> course tone this down. However I, as a package author, would appreciate
> if Emacs tells me when a newer API aims to replace another API and when
> the documentation is explicit about it.

That's understood, and when we make that decision, we will of course
announce it.  But we didn't do so yet, and this discussion is not even
about that decision.  It could be, for example, that both APIs will
live side by side until we decide whether to deprecate the old one.

> Of course if you decide to have
> the doc strings written in a different tone, I will adapt my patch
> accordingly. Here I am just explaining why I chose the word "superseded"
> instead of a more neutral word.

I understand your motivation, I'm just saying that we cannot announce
deprecation before we actually decide to deprecate.

> > But the name says "filter completions".  Which would mean you take a
> > list of completions and filter out some of them.  A completion table
> > is much more general object than a list of strings.  Thus, I think
> > using "filter" here is sub-optimal.
> 
> Okay, you are right about this. But I think even if the name
> 'completion-filter-completions' is not 100% precise it still conveys
> what the API is about. 'completion-completions-alist' or
> 'completion-all-completions-as-alist' are valid names of course, but I
> dislike them for their verbosity. Already 'completion-all-completions'
> is quite verbose. A strong argument to use this long name is that the
> completion style functions are still called
> 'completion-basic-all-completions' etc. But if we accept that the new
> API 'completion-filter-completions' will actually supersede the existing
> API 'completion-all-completions' it makes sense to use a name which will
> not hurt our eyes in the long run. However this is of course just a
> personal preference of mine. I don't want to spent much time with name
> bikeshedding discussions. If you decide on a name, I will adapt my patch
> accordingly.

Emacs is frequently accused in having names that are hard to
discover.  The only time where we can improve that is when a symbol is
introduced, because later it's impossible for compatibility reasons.
So I'd like to come up with a good name before we install the changes.

That said, I'll let others chime in and agree or disagree with the
name you've chosen.

Thanks.