Re: Emacs completion matches selection UI

[Dmitry Gutov <dgutov@yandex.ru>]

On 03.01.2014 02:58, Toby Cubitt wrote:
> No, definitely not a wrapper. I was proposing using it as an Elisp
> utility library. If anything, it's Company et al. that would build on top
> of and wrap around a generic Emacs completion UI, rather than the other
> way around.

Ah, that sounds good.

>> although I'd have to look into fitting any of them inside the Company's
>> interaction with the command loop,
>
> ...which BTW is why I suspect the full Company framework isn't suitable
> as a generic Emacs completion UI.

Yes, you could say that. Company is flexible, but could be not "general" 
enough.

> Sure, but as far as I understand (I could well be wrong - I've never used
> it in anger) you use Company by enabling company-mode in your buffer as a
> user, and this minor-mode changes how you interact with Emacs. Whereas
> you use Completion-UI by calling its functions from Elisp when coding a
> new Emacs package. It's really aimed at a "lower" level.

That's normally how it works, yes.

> If Company backend definitions should live in the completion package
> rather than Company, I could add it to Predictive instead.

They can, and they often do. A backend definition is just a function 
that can respond to a variable number of arguments with somewhat 
predetermined values.

> Well, I believe Completion-UI predates at least Company, anything, and
> auto-complete. So "yet-another-completion-package" is misleading. Had
> those packages existed when I coded Predictive, I doubt I'd have bothered
> coding Completion-UI.

I didn't know that. Indeed, it seems to predate Company by a year or so.

> It's a shame Company et al. didn't build on and contribute to
> Completion-UI, and instead chose to (each) reinvent the wheel. But that's
> the usual messy history of code (which maybe we're finally going to do
> something to tidy up?).

Maybe the widget APIs weren't particularly fitting, though. See my 
response to your proposal below.

> Do you want fast predictive completion of plain text, or
> text markup languages like LaTeX?

It could be a decent replacement for dabbrev (I mostly use it to quickly 
complete to some symbol already present before point in the same 
buffer), but the problem with "smart" algorithms that keep history is 
that often enough you can't predict the first option it will offer you. 
That can be annoying, so maybe I don't.

>>> I'd be very happy to see the UI parts of Company stripped out and made
>>> into a simple, generic completion UI package and added to Emacs.
>>
>> Would you like to propose an API for it? Same as popup.el?
>
> As a first attempt, I'd propose something like
>
>    (complete-in-buffer COMPLETION-SOURCE &optional PREFIX-FUNCTION)
>
> where COMPLETION-SOURCE is a function to call to obtain completions, and
> PREFIX-FUNCTION is an optional function to call to determine what to
> complete.

But what if we have a list of completion sources (see the description 
down below<*>)? Some of them can have a different notion of the prefix. 
In Company, we try to remember which backend we're working with at the 
moment, so that we only ask it for completions, candidate metadata, etc.

Though maybe it could work if PREFIX-FUNCTION iterated though backends, 
saved which one responded, and then we could use that value in 
COMPLETION-SOURCE, as well as when implementing the commands providing 
the additional info. In this case, we'll at least need to have a way to 
know when the widget was dismissed or refreshed to erase the saved 
backend value.

In `company-capf', we iterate though the completion-at-point-functions 
each time. This works, and works well enough, since the "does it fit" 
code is usually fast, but it looks fairly bizarre to me, logically.

Ideally, though, the widget interface would be a drawing API of sorts: 
"display these candidates at this point with this current index", 
"update index", "hide", since we have everything else implemented already.

> Calling this would invoke whatever completion widgets the user had
> enabled in customize (with sensible defaults - maybe the popup.el
> interface or similar, or perhaps the default should be the usual
> *Completions* buffer).

I'd still like to a natural way to extend such widgets with new 
commands. Passing a keymap as argument, or rebinding a dynamic variable 
that points to keymap works, I guess, but it's not very nice.

>> A mouse-only menu won't fit Company because it won't work for idle
>> completion, as I mentioned elsewhere in this thread, and we generally
>> want to provide a consistent keyboard interface. There'll also be issues
>> with implementation related to that.
>
> When you register a new UI with Completion-UI, you tell it whether the UI
> is suitable for idle completion. If it is, it becomes one of the
> customization options you can select for the idle completion UI.

Ok, it's reasonable approach, but that means that the UIs can't be 
treated uniformly, which is sort of a bummer. Ideally, we'd have a 
widget that's as useful as a menu when interacted with with a mouse, and 
works with keyboard, and doesn't interrupt the user's typing when 
displayed during idle completion.

If I were to implement something like this for Company, it would mean 
users who like this kind of menu would either have to forgo idle 
completion, or see different kind of UIs whether completion is idle or 
triggered manually. Which sounds weird, at least.

> Sounds like adding it to `company-backends' is more analogous registering
> it with `completion-ui-register-source'.

<*>

Not really. `company-backends' corresponds directly to 
`auto-completion-source', only instead of one function we have a list of 
them (which can also be nested one level deep, but that's an extra feature).

It's similar to `completion-at-point-functions' this way. First, each 
backend knows whether it will be applicable at point (whether the 
buffer's major mode fits, whether the syntax status at point fits, e.g. 
some backends work in strings, others only work outside, etc). So 
instead of calling an analog of `auto-completion-source', Company polls 
all configured backends until it gets a response, then saves the name of 
the backend thus found for the current completion invocation, and calls 
it again for the list of completions. After that, it can call the 
backend for calltips, candidate docs, et cetera.

> Customizing
> `auto-completion-source' is something the user would do, not the Elisp
> package coder.

Same with `company-backends', although we provide a reasonable default. 
But the user can change the global list, of change its local values, for 
example set it to a single-item value in some major mode hook.

> In fact, it sounds like the two APIs are rather similar:

> 2. Pass it to `completion-ui-register-source' (Completion-UI) or add it
>     to `company-backends' (Company) to let them know about it.

Guess the main difference is that, since we definine new backends with 
`defun', there's no way to update the list of available values, visible 
to the `defcustom' interface.

It makes matters a bit worse for third-party backends, but not by much, 
I think.

> At some point in the distant past, you used to just set a variable to
> tell Completion-UI about a new completion function. It became a macro
> call when I made Completion-UI more generic, and there needed to be a way
> of optionally supplying additional information about what the completion
> function does (e.g. if it does something other than standard prefix
> completion), and of defining optional call-back functions, etc.

We solve this problem by requiring backend functions to take the type of 
question it's going to answer as the first argument. Function calls are 
fast enough for our purpose, and this makes definitions quite succinct, 
especially for simple backends.

This pattern is used in many places in Emacs core, too.