Split off some backends from Company?

Split off some backends from Company?

[Dmitry Gutov]

Hi Stefan,

It's been suggested [0] that we split the backends that require 
third-party Emacs packages to be installed, from Company, to separate 
packages.

Tentatively, the list would look like:

company-bbdb, company-oddmuse, company-ropemacs.

Maybe also company-yasnippet. On the one hand it's distributed in GNU 
ELPA, so it's not really third-party, on the other hand, it still has to 
be installed for the respective backend to work.

Taking it further, we could do the same that depend on specific external 
programs to be available, but not Elisp code: company-clang, 
company-eclim, company-gtags, company-xcode.

There are two main benefits are, I think:

- More visibility for Company; users see the backends for specific 
completion sources they're interested in, in `M-x list-packages'.

- Each new package's description can include the list of its 
dependencies and how to install them, especially if the user has to 
install something outside of Emacs (ropemacs is the prime example). 
Before installing each new backend, the user would be able to read those 
instructions.

Drawbacks:

- If the backend doesn't depend on third-party Elisp code, or if that 
package is likely to be installed anyway; if the external program 
dependency or easy to install or likewise is likely to be already 
present on the user's system, then we lose out on user convenience, 
because otherwise the backend could "just work", if it was installed and 
present by default.

- The list of packages in GNU ELPA is fairly short, and adding 3-7 new 
packages, all named "company-...", might look like hijacking. There are 
many "ac-..." packages in third-party repositories, for example, but no 
such pattern hare, and I hesitate to change that.

- The setup proceduce for each such backend becomes more complicated: 
you have to install the backend, its dependencies (happens 
automatically), external dependencies, and add 1-2 new lines to your 
init file.

WDYT?

[0] 
https://github.com/company-mode/company-inf-ruby/issues/2#issuecomment-51649939

Re: Split off some backends from Company?

[Dmitry Gutov]

On 08/12/2014 07:50 PM, Stefan Monnier wrote:

> Rather than push them to separate packages, it would be preferable to
> turn them into completion-at-point-functions and push them to their
> corresponding parent package (i.e. to BBDB, oddmuse, and ropemacs).

Good point. We've discussed this before, but thanks for the reminder.

Doing that with company-bbdb would be hard, but looks like it'll be 
obsolete as soon as `message-completion-function' starts behaving 
correctly: `message-expand-name-databases' includes `bbdb' by default.

>> Maybe also company-yasnippet.
>
> Same here.

Probably not: using yasnippet as a completion source is a questionable 
decision, which each user should make themselves. It replaces the 
advertised way to interact with yasnippet (type the snippet key, press 
TAB), and the recommended way to use this backend is to group it with 
others, not by itself (think `completion-table-merge' instead of 
`completion-table-in-turn'), so putting it in 
`completion-at-point-functions' would probably be misguided.

>> Taking it further, we could do the same that depend on specific external
>> programs to be available, but not Elisp code: ...
>
> There's no such obvious "parent package" to hold those, so I guess
> separate ELPA packages would be OK.

Okay, it's good to have your approval. They're pretty unobtrusive, 
though, so I'll come back to them later.

>> - The setup proceduce for each such backend becomes more complicated: you
>> have to install the backend, its dependencies (happens automatically),
>> external dependencies, and add 1-2 new lines to your init file.
>
> Usually, enough ;;;###autoload trickery can do the setup transparently
> for the user.

Hmm, yes. I guess we could do that if the user explicitly installed a 
backend package.

Re: Split off some backends from Company?

[Stefan Monnier]

> Doing that with company-bbdb would be hard, but looks like it'll be obsolete
> as soon as `message-completion-function' starts behaving correctly:
> `message-expand-name-databases' includes `bbdb' by default.

Oh, right, I had forgotten about that one.

>>> Maybe also company-yasnippet.
>> Same here.
> Probably not: using yasnippet as a completion source is a questionable
> decision, which each user should make themselves. It replaces the advertised
> way to interact with yasnippet (type the snippet key, press TAB), and the
> recommended way to use this backend is to group it with others, not by
> itself (think `completion-table-merge' instead of
> `completion-table-in-turn'), so putting it in
> `completion-at-point-functions' would probably be misguided.

I see what you mean, but why does this need another package?
It seems the code could/should be in yasnippet, controlled by
something like a minor-mode.

> Hmm, yes. I guess we could do that if the user explicitly
> installed a backend package.

I think it's "should" rather than "could", because it's important for
the installation to be as straightforward as possible.


        Stefan

Re: Split off some backends from Company?

[Dmitry Gutov]

On 08/13/2014 04:38 PM, Stefan Monnier wrote:

>> Probably not: using yasnippet as a completion source is a questionable
>> decision, which each user should make themselves. It replaces the advertised
>> way to interact with yasnippet (type the snippet key, press TAB), and the
>> recommended way to use this backend is to group it with others, not by
>> itself (think `completion-table-merge' instead of
>> `completion-table-in-turn'), so putting it in
>> `completion-at-point-functions' would probably be misguided.
>
> I see what you mean, but why does this need another package?
> It seems the code could/should be in yasnippet, controlled by
> something like a minor-mode.

It could be in yasnippet, but then it would implicitly depend on Company 
anyway. A minor mode would, again, be something like an on/off switch, 
and company-yasnippet requires manual configuration to be useful, and 
users probably have to pick a good combination for each major mode they 
intend to use it in.

And some users just bind a key sequence to this command (which all 
Company backend double as), to invoke instead of pressing TAB for 
yasnippet expansion. I guess this specific usage could be facilitated 
via a minor mode, but I don't know how popular it is.

>> Hmm, yes. I guess we could do that if the user explicitly
>> installed a backend package.
>
> I think it's "should" rather than "could", because it's important for
> the installation to be as straightforward as possible.

On the one hand, the simplest setup is desirable. On the other hand, 
`company-backends' is designed to be customized, and to allow users to 
mix backends, to there's no single way to use a specific backend (see 
https://github.com/iquiw/company-ghc#4note as an example of alternative 
setup). So the simplicity has to be balanced against the odds of users 
being forced to undo the automatic setup.

Further, there is a convention in the third-party developer community of 
not doing too much in autoloads ("installing the package shouldn't turn 
it on", or something along these lines). I don't entirely agree with it 
(here's one discussion on the subject: 
https://github.com/skeeto/skewer-mode/issues/22#issuecomment-18454897), 
but I prefer to do less, rather than more, when the choice is not obvious.

Re: Split off some backends from Company?

[Stefan Monnier]

>> I see what you mean, but why does this need another package?
>> It seems the code could/should be in yasnippet, controlled by
>> something like a minor-mode.
> It could be in yasnippet, but then it would implicitly depend on Company
> anyway.

You mean it can't be a completion-at-point-function?

> And some users just bind a key sequence to this command (which all Company
> backend double as), to invoke instead of pressing TAB for yasnippet
> expansion. I guess this specific usage could be facilitated via a minor
> mode, but I don't know how popular it is.

Indeed the completion-at-point-function don't double as commands the way
company backends do.  And while it's easy to create a wrapper for it,
that wrapper will inevitably end up choosing a particular UI, so if we
want to use Company for it, we're stuck with a dependency on Company.

> setup). So the simplicity has to be balanced against the odds of users being
> forced to undo the automatic setup.

That can be a delicate balance, indeed.

> Further, there is a convention in the third-party developer community of not
> doing too much in autoloads ("installing the package shouldn't turn it on",
> or something along these lines).

That's true as well.  Of course "turn it on" is not always as clear-cut
as it sounds.  Adding oneself to company-backend might be seen as "turn
it on", but if you consider that it's only actually used if you do
enable company and if you use the appropriate major mode (or compiler
backend, or snippet library, ...), then you may decide that it's not
really "turned on".

To the extent that ELPA packages are normally installed by the end user
(my install.el package distinguished "system install" from "user
install" but package.el doesn't really provide much support for
system-wide installs, thjo it doesn't actively prevent them), I think
it's not that bad if installing a package also turns it on.
E.g. AUCTeX does just that (with a custom var that lets you "opt out"
if you prefer).
Of course, for people like me who install GNU ELPA via "git clone elpa;
make", we get all packages whether we want them or not, so "turns it on"
shouldn't be too much in your face.


        Stefan

Re: Split off some backends from Company?

[Dmitry Gutov]

On 08/14/2014 07:07 AM, Stefan Monnier wrote:
>>> I see what you mean, but why does this need another package?
>>> It seems the code could/should be in yasnippet, controlled by
>>> something like a minor-mode.
>> It could be in yasnippet, but then it would implicitly depend on Company
>> anyway.
>
> You mean it can't be a completion-at-point-function?

Not a useful one, no. Like mentioned, company-yasnippet's candidates 
list not exhaustive (in terms of things, in general, that the user would 
usually want to see completed), so it's won't be a good completion 
function by itself.

>> Further, there is a convention in the third-party developer community of not
>> doing too much in autoloads ("installing the package shouldn't turn it on",
>> or something along these lines).
>
> That's true as well.  Of course "turn it on" is not always as clear-cut
> as it sounds.  Adding oneself to company-backend might be seen as "turn
> it on", but if you consider that it's only actually used if you do
> enable company and if you use the appropriate major mode (or compiler
> backend, or snippet library, ...), then you may decide that it's not
> really "turned on".

Ok, that makes sense. But anyway, the above convention is only a 
secondary consideration, as far as I'm concerned.

> To the extent that ELPA packages are normally installed by the end user
> (my install.el package distinguished "system install" from "user
> install" but package.el doesn't really provide much support for
> system-wide installs, thjo it doesn't actively prevent them), I think
> it's not that bad if installing a package also turns it on.

Agreed.