Re: Emacs completion matches selection UI

[Toby Cubitt <tsc25@cantab.net>]

Thanks for the great summary, Ted!

On Fri, Jan 03, 2014 at 12:49:35PM -0500, Ted Zlatanov wrote:
> On Thu, 02 Jan 2014 13:10:11 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 
> 
> SM> I'd love to see company-mode and completion-ui consolidated into Emacs's
> SM> core code.  The "completion sources" would be provided via
> SM> completion-at-point-functions, so we already have a place for
> SM> that part.  We don't yet have an API for the "completion UI
> SM> frontend", OTOH, so if we can agree on a good API for that part, we
> SM> could restructure the minibuffer.el code around that and then add
> SM> more UIs.
> 
> Based on the discussion so far, it seems that:
> 
> - there are good use cases for both completion-ui and company-mode, we
>   can't claim either is a superset of the other.  The two are not
>   fundamentally incompatible.

Agreed. My feeling is once people decide what it is they want included in
Emacs, and what the API should look like, re-purposing (parts of) the
Company and/or Completion-UI code bases (maybe others too?) will be a big
head start (assuming no copyright assignment roadblocks).

> - I personally used company-mode heavily in the last few weeks and it
>   works great, but I don't like that after a timeout, the up/down keys
>   get taken over.  I think displaying the completion candidates could
>   happen after a timeout, but selecting from them should happen only
>   when the user explicitly requests it with M-TAB in a buffer or TAB in
>   the minibuffer.

This is a minor detail, which I'm sure would be trivial to change in
Company if so desired. (Wouldn't surprise me if it can already be
disabled by rebinding keys in company-mode keymaps.)

If you enable a `completion-auto-show' interface and
`completion-auto-show-delay' (both enabled by default), Completion-UI
behaves in exactly the same way as Company. People seem to expect the
up/down keys to cycling through completions when they're displayed in a
tooltip - I got bug reports before I implemented it.

> - it should be possible to implement a generic frontend for completion
>   that both can use, and other tools like yasnippet can use IIUC.  The
>   eventual goal is to have "something" standard in Emacs that all such
>   packages (except perhaps helm, which is completely apart in its UI)
>   would use.  We have to ask Dmitry and Toby, among others, to guide us
>   with their experience of writing it once already, and we have to make
>   sure we don't end up with a solution no one wants to use :)

Indeed. It would be nice to get input from the other completion package
authors too (anything.el, auto-complete, icicles...)

> - the backend data should be able to carry more than plain strings,
>   things like docstrings and such, and IMO we should try to standardize
>   the possible extras at least somewhat.

Agreed. 

Since it's the completion UI that will have to interpret and make use of
the extras, one place to start is to think about what additional features
the UI might want to provide, beyond selecting from a list of possible
completions.

Luckily, we have many existing completion frameworks to draw on for
inspiration. At the moment, location data and various forms of
documentation (e.g. type information in code completion, docstrings,
etc.) are the only ones that spring to mind.

> We may need a deeper backend abstraction than
> `completion-at-point-functions' because of this.

It might be sufficient to extend the list of supported properties in the
`completion-at-point-functions' PROPS plist.

The keyword arguments to `completion-ui-register-source' are one starting
point for seeing what additional properties might be needed, as are the
corresponding APIs in Company, auto-complete, etc.

Also, if we want to allow users to optionally customize the completion UI
per-backend (a la Completion-UI), there has to be some way of identifying
backends in Customize. This seems necessary to me if the new API is to
become the standard way of implementing a completion UI in Emacs. Think
minibuffer completion, versus filename completion, versus predictive
completion, versus elisp code completion etc, all of which might want to
use slightly different UI options. Again, it might be enough to support a
:name property (or some such) in the `completion-at-point-functions'
PROPS plist.

> - the frontend choices made by the user should be able to feed back into
>   the data backend (the "predictive" use case)

A few optional hooks/call-backs would very likely be sufficient. Again,
the `completion-at-point-functions' PROPS plist would be the obvious
place to specify them.

> - we have enough interest and support in this work that it's worth
>   undertaking in earnest after the code freeze.  The discussion should
>   remain on emacs-devel (based on the wide interest so far).

At the moment, it sounds to me like the existing
`completion-at-point-functions' and `completion-at-point' /
`completion-in-region' APIs could be extended to provide a more "modern",
richer, and customizable generic Emacs completion UI. Then minibuffer
completion, Company, Predictive, anything, etc. could all eventually be
converted to use this.

> I hope everyone agrees.

Sounds good to me.

(Note that I'm unlikely to have time to code anything myself until
March/April, although the feature freeze will likely delay things until
about then anyway. I'm happy to help in a more limited way before then,
as time allows.)

Toby
-- 
Dr T. S. Cubitt
Royal Society University Research Fellow
and Fellow of Churchill College, Cambridge
Centre for Quantum Information
DAMTP, University of Cambridge

email: tsc25@cantab.net
web:   www.dr-qubit.org