Inline completion preview

Inline completion preview

[Eshel Yaron]

[-- Attachment #1: Type: text/plain, Size: 1883 bytes --]

Hi,

I want to propose adding an inline completion preview mechanism to Emacs.

By inline completion preview, I mean an overlay that pop ups after point
showing one of the candidates for at-point completion.  You can view
this mechanism as giving you a hint or suggesting a certain completion.

Of course, there are a few Emacs packages that provide such a facility
already.  Namely, I found and played around with the following packages:

- `company` includes three variants of such a preview frontend, one of
  them enabled by default, that show a completion candidate provided by
  `company`'s backends.  See [0].
- `fancy-dabbrev` does something similar for `dabbrev` abbreviations.
  See [1].
- `mono-complete` provides such a completion preview frontend with
  several pluggable backends, including `completion-at-point` and
  `dabbrev`.  See [2].
- `corfu-candidate-overlay` implements such a preview for
  `completion-at-point` candidates that relies on the `corfu` in-buffer
  completion interface.  See [3].

IIUC similar features exist in other editors and go by names such as
"suggestion preview" and "ghost text".

I like this completion preview concept and I think it may be interesting
for Emacs to provide it.  ISTM that the best approach is a simple
library that uses only `completion-at-point` as a backend, because
`completion-at-point` is already extensible enough to support a variety
of completion backends.

I'm attaching below a draft for a simple `completion-preview.el` for
your consideration, I've been testing it in several major modes with
different `completion-at-point` backends, including LSP-based.

WDYT?


Eshel

[0] http://company-mode.github.io/manual/Frontends.html#Preview-Frontends
[1] https://github.com/jrosdahl/fancy-dabbrev
[2] https://codeberg.org/ideasman42/emacs-mono-complete
[3] https://code.bsdgeek.org/adam/corfu-candidate-overlay


[-- Attachment #2: completion-preview.el --]
[-- Type: application/emacs-lisp, Size: 7360 bytes --]

Re: Inline completion preview

[Eli Zaretskii]

> From: Eshel Yaron <me@eshelyaron.com>
> Date: Thu, 26 Oct 2023 20:11:17 +0200
> 
> I want to propose adding an inline completion preview mechanism to Emacs.
> 
> By inline completion preview, I mean an overlay that pop ups after point
> showing one of the candidates for at-point completion.  You can view
> this mechanism as giving you a hint or suggesting a certain completion.

I wish people would work on adding to Emacs some a GUI infrastructure
for showing such completion candidates like other IDEs do, instead of
once again using overlays with after-string properties.  The result of
using overlay strings is simply not visually appealing, and on top of
that it causes quite a few complications for the display code.

> Of course, there are a few Emacs packages that provide such a facility
> already.  Namely, I found and played around with the following packages:
> 
> - `company` includes three variants of such a preview frontend, one of
>   them enabled by default, that show a completion candidate provided by
>   `company`'s backends.  See [0].
> - `fancy-dabbrev` does something similar for `dabbrev` abbreviations.
>   See [1].
> - `mono-complete` provides such a completion preview frontend with
>   several pluggable backends, including `completion-at-point` and
>   `dabbrev`.  See [2].
> - `corfu-candidate-overlay` implements such a preview for
>   `completion-at-point` candidates that relies on the `corfu` in-buffer
>   completion interface.  See [3].

You list those, but don't say why you think we should have yet another
such facility, and in core on top of that.  If we want something like
that in core, I'd rather include Company, not invent something new
(and partial).

Am I missing something?

Re: Inline completion preview

[Eshel Yaron]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Eshel Yaron <me@eshelyaron.com>
>> Date: Thu, 26 Oct 2023 20:11:17 +0200
>>
>> I want to propose adding an inline completion preview mechanism to Emacs.
>>
>> By inline completion preview, I mean an overlay that pop ups after point
>> showing one of the candidates for at-point completion.  You can view
>> this mechanism as giving you a hint or suggesting a certain completion.
>
> I wish people would work on adding to Emacs some a GUI infrastructure
> for showing such completion candidates like other IDEs do, instead of
> once again using overlays with after-string properties.

I'm not sure I know enough about the benefits of other approaches, what
do you envision?

> The result of using overlay strings is simply not visually appealing,

It doesn't seem to me very different from what I see in other editors,
but I guess that's a matter of taste.

> and on top of that it causes quite a few complications for the display
> code.
>

>> Of course, there are a few Emacs packages that provide such a facility
>> already.  Namely, I found and played around with the following packages:
>>
>> - `company` includes three variants of such a preview frontend, one of
>>   them enabled by default, that show a completion candidate provided by
>>   `company`'s backends.  See [0].
>> - `fancy-dabbrev` does something similar for `dabbrev` abbreviations.
>>   See [1].
>> - `mono-complete` provides such a completion preview frontend with
>>   several pluggable backends, including `completion-at-point` and
>>   `dabbrev`.  See [2].
>> - `corfu-candidate-overlay` implements such a preview for
>>   `completion-at-point` candidates that relies on the `corfu` in-buffer
>>   completion interface.  See [3].
>
> You list those, but don't say why you think we should have yet another
> such facility, and in core on top of that.

Sorry, that might have not been clear enough, I wrote:

    ISTM that the best approach is a simple library that uses only
    `completion-at-point` as a backend, because `completion-at-point` is
    already extensible enough to support a variety of completion
    backends.

None of the aforementioned packages take that approach, so I wrote
`completion-preview.el` do demonstrate and test this approach.  As to
the question of adding this feature to core, since it's quite a simple
and useful addition, I just think it could be nice to have it OOTB.

> If we want something like that in core, I'd rather include Company,
> not invent something new (and partial).

That works too, of course :)
FWIW My proposal is not really novel, it uses standard techniques that
you find in all the packages I mentioned.  It just does without an
extra level of frontend/backend indirection and targets
`completion-at-point` directly, which makes it a bit simpler.

Re: Inline completion preview

[Eli Zaretskii]

> From: Eshel Yaron <me@eshelyaron.com>
> Cc: emacs-devel@gnu.org
> Date: Thu, 26 Oct 2023 21:39:16 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I wish people would work on adding to Emacs some a GUI infrastructure
> > for showing such completion candidates like other IDEs do, instead of
> > once again using overlays with after-string properties.
> 
> I'm not sure I know enough about the benefits of other approaches, what
> do you envision?

Did you see how other IDEs implement this display?  That's what I
mean.

> > The result of using overlay strings is simply not visually appealing,
> 
> It doesn't seem to me very different from what I see in other editors,
> but I guess that's a matter of taste.

Then maybe you should describe the use case in more details, because
this could be a misunderstanding of sorts.  What are the use case(s)
where the feature you propose would be useful?

> Sorry, that might have not been clear enough, I wrote:
> 
>     ISTM that the best approach is a simple library that uses only
>     `completion-at-point` as a backend, because `completion-at-point` is
>     already extensible enough to support a variety of completion
>     backends.

This is an argument from the implementation POV, it basically says
"let's do it because we can".  The question I asked is different: why
do we need to invent something new when we seem to already have
similar functionalities that are widely used?  A response I expected
would explain why those existing packages cannot support the desired
feature, or are too heavy for such a simple feature, or have some
other issues when used for this functionality.

Re: Inline completion preview

[Eshel Yaron]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Eshel Yaron <me@eshelyaron.com>
>> Cc: emacs-devel@gnu.org
>> Date: Thu, 26 Oct 2023 21:39:16 +0200
>>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>> > I wish people would work on adding to Emacs some a GUI infrastructure
>> > for showing such completion candidates like other IDEs do, instead of
>> > once again using overlays with after-string properties.
>>
>> I'm not sure I know enough about the benefits of other approaches, what
>> do you envision?
>
> Did you see how other IDEs implement this display?  That's what I
> mean.

I've used this feature in other editors (e.g. VS Code), but I didn't
dive into the implementation, if that's what you mean.  Of course some
IDEs are not open source so gaining such insights into their
implementation is non-trivial :(

>> > The result of using overlay strings is simply not visually appealing,
>>
>> It doesn't seem to me very different from what I see in other editors,
>> but I guess that's a matter of taste.
>
> Then maybe you should describe the use case in more details, because
> this could be a misunderstanding of sorts.  What are the use case(s)
> where the feature you propose would be useful?

The use case is this: you're editing some code, maybe it's a language
you're not extremely proficient with, and you want your editor (Emacs)
to give you suggestions for code that it would make some sense to write.
Completion preview does that by showing you a completion candidate you
can insert, and displaying it as if you've already inserted it, so you
know exactly what it'll look like (you get a preview).  As you are
typing, Emacs updates the suggestion based on your keystrokes.  If you
couple that with a good completion backend, you get useful suggestions
with a very low friction interface.

>> Sorry, that might have not been clear enough, I wrote:
>>
>>     ISTM that the best approach is a simple library that uses only
>>     `completion-at-point` as a backend, because `completion-at-point` is
>>     already extensible enough to support a variety of completion
>>     backends.
>
> This is an argument from the implementation POV, it basically says
> "let's do it because we can".  The question I asked is different: why
> do we need to invent something new when we seem to already have
> similar functionalities that are widely used?

To be clear, I think adding Company to core Emacs instead of my
completion-preview.el or some such is a perfectly viable solution.  I do
think there are some considerations in favor of my proposal (see below),
and some against, naturally.

> A response I expected would explain why those existing packages cannot
> support the desired feature, or are too heavy for such a simple
> feature, or have some other issues when used for this functionality.

Alright, let me review the alternatives:

- `fancy-dabbrev` is probably insufficient because it only works with
  `dabbrev`.  It's also only available on MELPA which might make it
  harder to incorporate in core.

- `corfu-candidate-overlay` relies on `corfu`, and I've had a small
  issue while testing it that sometimes the overlay lingered after point
  moved, but otherwise I think that if `corfu` would be in core this
  package would be a fine solution.  MELPA-only as well.

- `mono-complete` is quite nice IMO, but it's a larger package that
  comes with multiple backends besides `completion-at-point`, including
  a bespoke python program that generates word predictions, and I think
  that something simpler may be more appropriate.  It is also only on MELPA.

- `company` is great, battle tested, fully featured, and on GNU ELPA.
  It's an interesting option to explore IMO.  It does add an extra level
  of complexity with its multiple frontends and backends scheme, that
  I'm not sure is needed in core.

- `completion-preview` only provides a completion preview frontend, and
  only targets `completion-at-point` as a backend, so it benefits from
  tight integration.  Obviously, it is does not provide all of the
  features that `company` provides, and it isn't very well tested.  OTOH
  it's less than 150 lines of code, so we could actually even add it to
  completion.el instead of having a separate library.



Best,

Eshel

Re: Inline completion preview

[Eli Zaretskii]

> From: Eshel Yaron <me@eshelyaron.com>
> Cc: emacs-devel@gnu.org
> Date: Fri, 27 Oct 2023 17:43:17 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Did you see how other IDEs implement this display?  That's what I
> > mean.
> 
> I've used this feature in other editors (e.g. VS Code), but I didn't
> dive into the implementation, if that's what you mean.

I meant how they show it visually.

> > Then maybe you should describe the use case in more details, because
> > this could be a misunderstanding of sorts.  What are the use case(s)
> > where the feature you propose would be useful?
> 
> The use case is this: you're editing some code, maybe it's a language
> you're not extremely proficient with, and you want your editor (Emacs)
> to give you suggestions for code that it would make some sense to write.
> Completion preview does that by showing you a completion candidate you
> can insert, and displaying it as if you've already inserted it, so you
> know exactly what it'll look like (you get a preview).  As you are
> typing, Emacs updates the suggestion based on your keystrokes.  If you
> couple that with a good completion backend, you get useful suggestions
> with a very low friction interface.

If you are talking about code completion, showing a single candidate
is rarely TRT, especially in programming languages that support
polymorphism.

Re: Inline completion preview

[Eshel Yaron]

[-- Attachment #1: Type: text/plain, Size: 1887 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Then maybe you should describe the use case in more details, because
>> > this could be a misunderstanding of sorts.  What are the use case(s)
>> > where the feature you propose would be useful?
>> 
>> The use case is this: you're editing some code, maybe it's a language
>> you're not extremely proficient with, and you want your editor (Emacs)
>> to give you suggestions for code that it would make some sense to write.
>> Completion preview does that by showing you a completion candidate you
>> can insert, and displaying it as if you've already inserted it, so you
>> know exactly what it'll look like (you get a preview).  As you are
>> typing, Emacs updates the suggestion based on your keystrokes.  If you
>> couple that with a good completion backend, you get useful suggestions
>> with a very low friction interface.
>
> If you are talking about code completion, showing a single candidate
> is rarely TRT, especially in programming languages that support
> polymorphism.

Yes, I'm talking about showing a single code completion candidate, in a
special way, that happens automatically under certain conditions, and
that is independent of how you otherwise view your list of completion
candidates (be it the *Completions* buffer or in some other popup) when
you trigger in-buffer completion.  For clarity, I'm attaching below a
screenshot showing how such completion preview in an Elisp buffer.
Sometimes, previewing just one candidate that way can be quite useful,
for example when there's only one matching completion candidate for
whatever you're typing.  In completion-preview.el there's a user option
`completion-preview-exact-match-only` that lets you require exactly this
condition for showing the completion preview.  In Company, that's the
default behavior (showing such a preview when there exactly one matching
candidate).


[-- Attachment #2: Screenshot 2023-10-27 at 8.05.12 PM.png --]
[-- Type: image/png, Size: 21664 bytes --]

Re: Inline completion preview

[chad]

[-- Attachment #1: Type: text/plain, Size: 1715 bytes --]

On Fri, Oct 27, 2023 at 3:34 PM Eshel Yaron <me@eshelyaron.com> wrote:

> [..]  For clarity, I'm attaching below a
> screenshot showing how such completion preview in an Elisp buffer.
>

I've seen a few systems try to add this sort of completion preview to
editing, and it seems to mostly be based on (ultra-common) mobile device
autocomplete UI. The problem with bringing that model into an editor is
that mobile device autocomplete is about helping users *append* text to the
*end* of a very small chunk of editable text. Those systems also have
various ways to "move the cursor", and they're uniformly considered worse,
because the interaction model is a bad fit for the user affordances.

I've seen a few systems that add this sort of in-line expansion suggestion
to actual text editors, where most of the editing happens in the middle,
not the end, and the jumpiness that results is, well, it's usually not
considered good. (The flashing/updating you get just from adapting the code
highlighting to such systems is jarring enough that most editors add extra
layers to mitigate it.) I don't know if it's accidental or intentional, but
I believe this sort of problem is why you almost always see examples
demonstrate mid-text suggestions via pop-ups, and in-line suggestions
nearly always demonstrated (as in this screenshot), at the end.

Put another way: there are strong reasons to consider Eli's suggestion that
Emacs grow/evolve infrastructure to display such suggestions using UI
methods beyond "just insert some specially-marked text", even if some users
will be happy to have the text quickly/easily plopped directly into the
buffer.

As usual, I hope this helps,
~Chad

[-- Attachment #2: Type: text/html, Size: 2165 bytes --]

Re: Inline completion preview

[Juri Linkov]

> Alright, let me review the alternatives:

What about icomplete-in-buffer?

Re: Inline completion preview

[Eshel Yaron]

Hi Juri,

Juri Linkov <juri@linkov.net> writes:

>> Alright, let me review the alternatives:
>
> What about icomplete-in-buffer?

I don't think that `icomplete-in-buffer` is an alternative to
`completion-preview`, they are two independent features, and you can
even use them together.  (I've posted the code for `completion-preview`
upthread, and you can also find it here
https://git.sr.ht/~eshel/completion-preview if you want to try it out.)
Crucially, the preview that `completion-preview` provides appears
automatically as you type, before you actively trigger completion.


Best,

Eshel

Re: Inline completion preview

[Alexander Adolf]

[-- Attachment #1: Type: text/plain, Size: 1171 bytes --]

Company provides an ecosystem of its own, and that can make one’s setup more complicated. 

I have moved to corfu, which fully builds on Emacs’s ecosystem, and have never looked back. 

That said, if anything of that sort would be considered for being added to core, corfu would be my only candidate. 

  --alex

-- 
www.condition-alpha.com / @c_alpha
Sent from my iPhone; apologies for brevity and autocorrect weirdness. 

> On 28. Oct 2023, at 21:02, Eshel Yaron <me@eshelyaron.com> wrote:
> 
> Hi Juri,
> 
> Juri Linkov <juri@linkov.net> writes:
> 
>>> Alright, let me review the alternatives:
>> 
>> What about icomplete-in-buffer?
> 
> I don't think that `icomplete-in-buffer` is an alternative to
> `completion-preview`, they are two independent features, and you can
> even use them together.  (I've posted the code for `completion-preview`
> upthread, and you can also find it here
> https://git.sr.ht/~eshel/completion-preview if you want to try it out.)
> Crucially, the preview that `completion-preview` provides appears
> automatically as you type, before you actively trigger completion.
> 
> 
> Best,
> 
> Eshel
> 

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 1944 bytes --]

Re: Inline completion preview

[João Távora]

On Sun, Oct 29, 2023 at 4:25 PM Alexander Adolf
<alexander.adolf@condition-alpha.com> wrote:
>
> Company provides an ecosystem of its own, and that can make one’s setup more complicated.

I agree it provides more abstractions then are essential, but you don't
have to use them.  Eglot-managed buffers prefer the company-capf backend
by default and it works fine just fine.

So the "complicated setup" you're looking for might just be one line:

   (setq company-backends '(company-capf))

> I have moved to corfu, which fully builds on Emacs’s ecosystem, and
> have never looked back.

company-capf is perfectly compatible with Emacs's completion functions
as well.

Corfu has known problems with Eglot, for example.  As far I understand,
the way to fix them (presumably) is based on an additional thing
called "cape" which works by monkey patching Eglot's functions via
advice, which I don't think is really elegant.

Also company works on TTY Emacs out of the box, and I think Corfu
doesn't.  More stuff to add in.

So, as I see it, with Company you get hassle free completion by
customizing a way a bit of its functionality, whereas with Corfu you
have to add stuff to your .emacs.  As I see it, it's really Corfu
that has the most complicated setup.

João

Re: Inline completion preview

[Juergen Fenn]

Am 30.10.23 um 00:29 Uhr schrieb João Távora:
> Also company works on TTY Emacs out of the box, and I think Corfu
> doesn't.  More stuff to add in.

Just one package corfu-terminal, it seems, see
https://github.com/minad/corfu :

| NOTE: Corfu uses child frames to show the popup and falls back to the
| default setting of the completion-in-region-function on non-graphical
| displays. If you want to use Corfu in the terminal, install the
| package corfu-terminal, which provides an alternative overlay-based
| display.

I haven't tested this so far as I don't use Emacs in terminal …

Regards,
Jürgen.

Re: Inline completion preview

[João Távora]

On Mon, Oct 30, 2023 at 9:26 AM Juergen Fenn <jfenn@gmx.net> wrote:

> Am 30.10.23 um 00:29 Uhr schrieb João Távora:
> > Also company works on TTY Emacs out of the box, and I think Corfu
> > doesn't.  More stuff to add in.
>
> Just one package corfu-terminal, it seems, see
> https://github.com/minad/corfu :

Actually two, you have to bring in the extra dependency "popon".

But I can confirm it works, I've used this for testing Corfu.
It's just that the post I was replying to criticized Company's
"ecosystem" and "complicated setup" and praised Corfu for not
needing these things, and I wanted to point out that IMO and
IME it's much the other way around.

João

Re: Inline completion preview

[Dmitry Gutov]

On 30/10/2023 11:36, João Távora wrote:
> On Mon, Oct 30, 2023 at 9:26 AM Juergen Fenn<jfenn@gmx.net>  wrote:
> 
>> Am 30.10.23 um 00:29 Uhr schrieb João Távora:
>>> Also company works on TTY Emacs out of the box, and I think Corfu
>>> doesn't.  More stuff to add in.
>> Just one package corfu-terminal, it seems, see
>> https://github.com/minad/corfu  :
> Actually two, you have to bring in the extra dependency "popon".

And popon isn't in GNU ELPA and does not have copyright assignment.

Re: Inline completion preview

[Alexander Adolf]

João Távora <joaotavora@gmail.com> writes:

> On Sun, Oct 29, 2023 at 4:25 PM Alexander Adolf
> <alexander.adolf@condition-alpha.com> wrote:
>>
>> Company provides an ecosystem of its own, and that can make one’s setup more complicated.
>
> I agree it provides more abstractions then are essential, but you don't
> have to use them.  Eglot-managed buffers prefer the company-capf backend
> by default and it works fine just fine.
>
> So the "complicated setup" you're looking for might just be one line:
>
>    (setq company-backends '(company-capf))

Indeed. But the complexity I was intending to refer to came with
integrating company with other 3rd party packages. Because of comapny's
age (it predates some of Emacs's current completion infrastructure),
many 3rd party packages have specific company adaptation code. As
packages and Emacs evolve, an adaptation or two tend to break with each
update. This kept my company config growing and growing.

My corfu config is still as small as it was on day one.

>> I have moved to corfu, which fully builds on Emacs’s ecosystem, and
>> have never looked back.
>
> company-capf is perfectly compatible with Emacs's completion functions
> as well.

It does. But in such a configuration you're only using a rather small
subset of company's code. What's the point of having such a big codebase
then? Lot sof unneeded complexity, IMHO.

> Corfu has known problems with Eglot, for example.

And it used to have with LSP. To my experience, in 99% of the cases the
problem and the fix lies with the other package, which was not using
Emacs's completion system properly (as was for instance teh case with
LSP).

> As far I understand, the way to fix them (presumably) is based on an
> additional thing called "cape" which works by monkey patching Eglot's
> functions via advice, which I don't think is really elegant.

Cape can wrap completion code, which is not compliant with Emacs's
completion framework, into wrapper functions which make it compliant
with the interfaces Emacs provides for, and expects from completion
functions. Thus, cape is a migration aid. You can switch from whatever
you used before to corfu, and keep using all your legacy completion
sources. This buys you time to find other completion sources which are
good `completion-at-point` citizens to replace your legacy ones.

> Also company works on TTY Emacs out of the box, and I think Corfu
> doesn't.  More stuff to add in.

Corfu works with TTY Emacs of of the box, too.

Under a graphical environment, corfu uses a frame provided by the
graphical environment (for instance the window manager) to display the
candidates. That way the display of the "popover" is independent of the
fonts used in the buffer. Consider as an example an Org buffer, where
the headings are rendered in a bigger font. Company will render popvover
text in that area using the bigger font.

Result with company:
----------------------------------------------------------------------

       The quick brown fox jum
                              +-------------+
                              | candidate 1 |
       * BIG FONT HEADING      | CANDIDATE 2    |
         BIG FONT HEADING      | CANDIDATE 2    |
                              | candidate 3 |
                              | candidate 4 |
                              +-------------+
----------------------------------------------------------------------

Duh! Company certainly showing its age here.


Result with corfu:
----------------------------------------------------------------------

       The quick brown fox jum
                              +-------------+
                              | candidate 1 |
       * BIG FONT HEADING     | candidate 2 |
         BIG FONT HEADING     | candidate 3 |               
                              | candidate 4 |
                              +-------------+
----------------------------------------------------------------------

> So, as I see it, with Company you get hassle free completion by
> customizing a way a bit of its functionality, whereas with Corfu you
> have to add stuff to your .emacs.  As I see it, it's really Corfu
> that has the most complicated setup.

My experience is the exact opposite.

As mentioned, I used company for many years. I kept compiling
complexities over complexities, and hacks for this and that in my
dot-emacs.

Making the switch to corfu was like a liberation. Tiny config, zero
hassle ever since.

Just my two cents anyway.

  --alexander

Re: Inline completion preview

[João Távora]

On Mon, Oct 30, 2023 at 12:42 PM Alexander Adolf
<alexander.adolf@condition-alpha.com> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
>
> > On Sun, Oct 29, 2023 at 4:25 PM Alexander Adolf
> > <alexander.adolf@condition-alpha.com> wrote:
> >>
> >> Company provides an ecosystem of its own, and that can make one’s setup more complicated.
> >
> > I agree it provides more abstractions then are essential, but you don't
> > have to use them.  Eglot-managed buffers prefer the company-capf backend
> > by default and it works fine just fine.
> >
> > So the "complicated setup" you're looking for might just be one line:
> >
> >    (setq company-backends '(company-capf))
>
> Indeed. But the complexity I was intending to refer to came with
> integrating company with other 3rd party packages. Because of comapny's
> age (it predates some of Emacs's current completion infrastructure),
> many 3rd party packages have specific company adaptation code. As
> packages and Emacs evolve, an adaptation or two tend to break with each
> update. This kept my company config growing and growing.

But you don't _have_ to use that integrations and third party packages,
do you?  You used it because you wanted to.

As far as I understand if you replaced all your complicated
setup with that one line I gave you, Company works with exactly
the same things Corfu does (CAPF completion backends, acronym for
completion-at-point-functions).

We should not compare apples to oranges and make decisions
based on that.

João

Re: Inline completion preview

[Alexander Adolf]

João Távora <joaotavora@gmail.com> writes:

> [...]
>> Indeed. But the complexity I was intending to refer to came with
>> integrating company with other 3rd party packages. Because of comapny's
>> age (it predates some of Emacs's current completion infrastructure),
>> many 3rd party packages have specific company adaptation code. As
>> packages and Emacs evolve, an adaptation or two tend to break with each
>> update. This kept my company config growing and growing.
>
> But you don't _have_ to use that integrations and third party packages,
> do you?  You used it because you wanted to.

I wanted to use those other 3rd party packages with company, so I had to
accept the complexity of my dot-emacs code. Hence, for me the only way
to avoid the complexity in my dot-emacs would have been to not use the
other 3rd party packages, or to not use company, or both.

> [...]
> We should not compare apples to oranges and make decisions
> based on that.

I'm not sure I follow you here. Could you perhaps be so kind to shed
some more light on what you were referring to here?


Many thanks and cheers,

  --alexander

Re: Inline completion preview

[João Távora]

On Mon, Oct 30, 2023 at 5:32 PM Alexander Adolf
<alexander.adolf@condition-alpha.com> wrote:

> I wanted to use those other 3rd party packages with company, so I had to
> accept the complexity of my dot-emacs code. Hence, for me the only way
> to avoid the complexity in my dot-emacs would have been to not use the
> other 3rd party packages, or to not use company, or both.

No.  I don't use any third party packages and I get to use Company
in exactly the same contexts you use Corfu.  Read below.

> > [...]
> > We should not compare apples to oranges and make decisions
> > based on that.
>
> I'm not sure I follow you here. Could you perhaps be so kind to shed
> some more light on what you were referring to here?

Well, I thought I was pretty clear, but OK, here goes spelled out again:

In your first post to this thread you mentioned that Company had an
"ecosystem" and needed "complicated setup" to configure it.  Then
you stated that Corfu solved this problem.

I can agree to the first part (that Company does have an "ecosystem"
of various third-party plugin extensions) and I may even agree
to the fact that those extensions are complicated to setup
(who knows, they might be, I don't know which ones you mean and
I've never tried them).

But comparing _that_ to Corfu is unfair and "apples to oranges",
since Corfu only sets out to do what the built-in, 1st party,
"company-capf" backend does.  Company can do quite a bit more,
which you may or not appreciate -- and you clearly don't.

That is just fine, so I suggested that if what you appreciate
is Corfu's purported simplicity of setup, then in Company, you
need only add that one line to your setup:

  (setq company-backends '(company-capf))

Furthermore, I mentioned two confirmed known problems with Corfu
that are known to necessitate third-party extensions based on
advice to work with core Emacs facilities such as Eglot.  I also
mentioned Corfu's known inability to work with TTY Emacsen out
of the box, and someone pointed out that that needs an extra
package and its dependency.  For a fair comparison, these things
should be added to Corfu's "simple" setup.

It's only once you get two Emacsen side by side, running each
Company and Corfu with their respective "simple" setups, that
you'll be comparing apples to apples and make a strong point.
Else, you'll be comparing apples to oranges and your point
will be very weak.

Hoping that is enough light shed,
João

Re: Inline completion preview

[Dmitry Gutov]

On 30/10/2023 14:42, Alexander Adolf wrote:
>> Also company works on TTY Emacs out of the box, and I think Corfu
>> doesn't.  More stuff to add in.
> Corfu works with TTY Emacs of of the box, too.

Are you sure you don't have corfu-terminal installed?

It's in NonGNU ELPA because it uses the 'popon' package which doesn't 
have its copyright assigned to FSF, and thus couldn't be included with 
Emacs until that happens.

> Under a graphical environment, corfu uses a frame provided by the
> graphical environment (for instance the window manager) to display the
> candidates. That way the display of the "popover" is independent of the
> fonts used in the buffer. Consider as an example an Org buffer, where
> the headings are rendered in a bigger font. Company will render popvover
> text in that area using the bigger font.
> 
> Result with company:
> ----------------------------------------------------------------------
> 
>         The quick brown fox jum
>                                +-------------+
>                                | candidate 1 |
>         * BIG FONT HEADING      | CANDIDATE 2    |
>           BIG FONT HEADING      | CANDIDATE 2    |
>                                | candidate 3 |
>                                | candidate 4 |
>                                +-------------+
> ----------------------------------------------------------------------
> 
> Duh! Company certainly showing its age here.
> 
> 
> Result with corfu:
> ----------------------------------------------------------------------
> 
>         The quick brown fox jum
>                                +-------------+
>                                | candidate 1 |
>         * BIG FONT HEADING     | candidate 2 |
>           BIG FONT HEADING     | candidate 3 |
>                                | candidate 4 |
>                                +-------------+
> ----------------------------------------------------------------------

You can try out this branch: 
https://github.com/company-mode/company-mode/pull/1394

It should deal with big fonts, as well as multiple width characters, 
fine. The only thing that seems impossible to implement is compatibility 
with line numbering. Perhaps I'll someday switch to popon/popup.el's 
approach of one-overlay-per-line when I have more time.

If 'popon' were in GNU ELPA, however, Company could also use it. In 
fact, writing a frontend using it for your own purpose should already be 
easy enough.

Re: Inline completion preview

[Alexander Adolf]

Dmitry Gutov <dmitry@gutov.dev> writes:

> On 30/10/2023 14:42, Alexander Adolf wrote:
>>> Also company works on TTY Emacs out of the box, and I think Corfu
>>> doesn't.  More stuff to add in.
>> Corfu works with TTY Emacs of of the box, too.
>
> Are you sure you don't have corfu-terminal installed?
> [...]

Quoting from the corfu readme [1]:
---------------------------- Begin Quote -----------------------------
NOTE: Corfu uses child frames to show the popup and falls back to the
default setting of the completion-in-region-function on non-graphical
displays. If you want to use Corfu in the terminal, install the package
corfu-terminal, which provides an alternative overlay-based display.
----------------------------- End Quote ------------------------------

Thus, there is a fallback behaviour of corfu on TTYs, and you ar enot
loosing any completion _semantics_ in such a case.

If you want a similar "eye candy" on a TTY, then one would have to use
corfu-terminal as you point out.

[1] https://github.com/minad/corfu/blob/main/README.org

Looking at the "Alternatives" section in [1] may also be informative.

>> [...]
>> Result with company:
>> ----------------------------------------------------------------------
>> 
>>         The quick brown fox jum
>>                                +-------------+
>>                                | candidate 1 |
>>         * BIG FONT HEADING      | CANDIDATE 2    |
>>           BIG FONT HEADING      | CANDIDATE 2    |
>>                                | candidate 3 |
>>                                | candidate 4 |
>>                                +-------------+
>> ----------------------------------------------------------------------
>> [...]
>
> You can try out this branch: 
> https://github.com/company-mode/company-mode/pull/1394
> [...]

Or you could adopt the child frame technique from corfu to avoid the
problem occurring in the first place?


Cheers,

  --alexander

Re: Inline completion preview

[Dmitry Gutov]

On 30/10/2023 19:43, Alexander Adolf wrote:
> Dmitry Gutov <dmitry@gutov.dev> writes:
> 
>> On 30/10/2023 14:42, Alexander Adolf wrote:
>>>> Also company works on TTY Emacs out of the box, and I think Corfu
>>>> doesn't.  More stuff to add in.
>>> Corfu works with TTY Emacs of of the box, too.
>>
>> Are you sure you don't have corfu-terminal installed?
>> [...]
> 
> Quoting from the corfu readme [1]:
> ---------------------------- Begin Quote -----------------------------
> NOTE: Corfu uses child frames to show the popup and falls back to the
> default setting of the completion-in-region-function on non-graphical
> displays. If you want to use Corfu in the terminal, install the package
> corfu-terminal, which provides an alternative overlay-based display.
> ----------------------------- End Quote ------------------------------
> 
> Thus, there is a fallback behaviour of corfu on TTYs, and you ar enot
> loosing any completion _semantics_ in such a case.

This is clearly not comparable behavior for terminal users.

>>> [...]
>>> Result with company:
>>> ----------------------------------------------------------------------
>>>
>>>          The quick brown fox jum
>>>                                 +-------------+
>>>                                 | candidate 1 |
>>>          * BIG FONT HEADING      | CANDIDATE 2    |
>>>            BIG FONT HEADING      | CANDIDATE 2    |
>>>                                 | candidate 3 |
>>>                                 | candidate 4 |
>>>                                 +-------------+
>>> ----------------------------------------------------------------------
>>> [...]
>>
>> You can try out this branch:
>> https://github.com/company-mode/company-mode/pull/1394
>> [...]
> 
> Or you could adopt the child frame technique from corfu to avoid the
> problem occurring in the first place?

The GUI-only users are free to install company-posframe, with similar 
benefits. It's not in GNU ELPA, though: still collecting the last 
copyright assignment signature.

But the aforementioned branch (PR) should also improve behavior for 
terminal users, though, where the characters can also be more than 1 
column wide (usually 2, in such case).

Re: Inline completion preview

[Eshel Yaron]

Hi Dmitry, João, Alex,

I find this company/corfo comparison quite compelling, but I'd like to
steer the discussion back to the topic at hand: adding inline completion
*preview* to Emacs.  Could you check out my proposed implementation[0]
and weigh in on adding it to core, possibly as part of completion.el?

Thanks,

Eshel


[0] https://git.sr.ht/~eshel/completion-preview/tree/master/item/completion-preview.el

Re: Inline completion preview

[Alexander Adolf]

Eshel Yaron <me@eshelyaron.com> writes:

> [...]
> I find this company/corfo comparison quite compelling, but I'd like to
> steer the discussion back to the topic at hand: adding inline completion
> *preview* to Emacs.  Could you check out my proposed implementation[0]
> and weigh in on adding it to core, possibly as part of completion.el?
> [...]

Thanks for the prod, Eshel. Good getting things back on track.

It seems to me that corfu's `corfu-preview-current` option does just
that when set to `insert`. I tried it, and to me at least it seemed to
do what you describe.

In that light, I'd mostly concur with what Eli commented last Thursday:
> > [...]
> > You list those, but don't say why you think we should have yet another
> > such facility, and in core on top of that. If we want something like
> > that in core, I'd rather include Company, not invent something new (and
> > partial).
> > 
> > Am I missing something?
> > [...]

With the difference that I would advocate for corfu being included.


Cheers,

  --alexander

Re: Inline completion preview

[Eshel Yaron]

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

> Eshel Yaron <me@eshelyaron.com> writes:
>
>> [...]
>> I find this company/corfo comparison quite compelling, but I'd like to
>> steer the discussion back to the topic at hand: adding inline completion
>> *preview* to Emacs.  Could you check out my proposed implementation[0]
>> and weigh in on adding it to core, possibly as part of completion.el?
>> [...]
>
> Thanks for the prod, Eshel. Good getting things back on track.

Gladly :)

> It seems to me that corfu's `corfu-preview-current` option does just
> that when set to `insert`. I tried it, and to me at least it seemed to
> do what you describe.

Hmm no, it doesn't do quite the same thing.  I use `corfu` with those
settings myself, alongside with `completion-preview-mode`, which
operates as you type without invoking your in-buffer completion UI.  You
can enable `completion-preview-mode` and start typing in some
programming buffer to see the difference.


Cheers,

Eshel

Re: Inline completion preview

[Dmitry Gutov]

On 31/10/2023 08:50, Eshel Yaron wrote:
> I find this company/corfo comparison quite compelling, but I'd like to
> steer the discussion back to the topic at hand: adding inline completion
> *preview*  to Emacs.  Could you check out my proposed implementation[0]
> and weigh in on adding it to core, possibly as part of completion.el?

Ok, I tried your code. This is probably an issue: C-M-i is rebound.

With completion-preview-mode on, this key combination does something 
different than the usual (it inserts the previewed completion).

I can imagine that this behavior can be useful for certain personal 
habits/used languages/etc, but it's a change from the default.

Otherwise, I can't really comment on the inclusion, since it's not 
something I would have used, and I don't know how many users would 
really benefit.

Re: Inline completion preview

[Eshel Yaron]

Dmitry Gutov <dmitry@gutov.dev> writes:

> On 31/10/2023 08:50, Eshel Yaron wrote:
>> I find this company/corfo comparison quite compelling, but I'd like to
>> steer the discussion back to the topic at hand: adding inline completion
>> *preview*  to Emacs.  Could you check out my proposed implementation[0]
>> and weigh in on adding it to core, possibly as part of completion.el?
>
> Ok, I tried your code.

Great, I appreciate it.

> This is probably an issue: C-M-i is rebound.
>
> With completion-preview-mode on, this key combination does something
> different than the usual (it inserts the previewed completion).

That's an interesting point.  Basically, we want to provide a convenient
way for users to "accept the suggestion" and insert the preview.  So in
general there are two choices AFAICT:

- (re)bind some convenient key when the preview is visible, and document
  that fact while explaining how users can remove/change this binding if
  they want to.
- provide no bindings (or maybe bind an unused but less convenient key)
  by default, and recommend binding a convenient key in the docs.

Either way, `C-M-i` seems like a natural choice since inserting the
previewed completion is strongly related to `completion-at-point`, so
the idea is that this key combination does something similar
(completion) whether the preview is visible or not.  WDYT?

> I can imagine that this behavior can be useful for certain personal
> habits/used languages/etc, but it's a change from the default.

I agree, although the user explicitly enables this minor mode, so it's
entirely unexpected for it to change an otherwise default behavior, no?

> Otherwise, I can't really comment on the inclusion, since it's not
> something I would have used, and I don't know how many users would
> really benefit.

Thanks,

Eshel

Re: Inline completion preview

[João Távora]

On Wed, Nov 1, 2023 at 8:30 AM Eshel Yaron <me@eshelyaron.com> wrote:

> > I can imagine that this behavior can be useful for certain personal
> > habits/used languages/etc, but it's a change from the default.
>
> I agree, although the user explicitly enables this minor mode, so it's
> entirely unexpected for it to change an otherwise default behavior, no?

Yes, this makes some sense, especially since as far as I understand it,
this minor mode is really about "changing the default behaviour of
C-M-i".

But OTOH there are other better ways to do that, i.e. via
completion-at-point-functions.   Are you sure your completion frontend
can't work via those?  The function you add to that variable could
just endeavour to check if there's any preview showing as an overlay,
and it if is insert it.  Then you would need no rebinding.

Also are you 100% sure Company doesn't do this?  I'm pretty sure
it does this when there is only one completion, at least.

João

Re: Inline completion preview

[Eshel Yaron]

João Távora <joaotavora@gmail.com> writes:

> On Wed, Nov 1, 2023 at 8:30 AM Eshel Yaron <me@eshelyaron.com> wrote:
>
>> > I can imagine that this behavior can be useful for certain personal
>> > habits/used languages/etc, but it's a change from the default.
>>
>> I agree, although the user explicitly enables this minor mode, so it's [not]
>> entirely unexpected for it to change an otherwise default behavior, no?
>
> Yes, this makes some sense, especially since as far as I understand it,
> this minor mode is really about "changing the default behaviour of
> C-M-i".
>
> But OTOH there are other better ways to do that, i.e. via
> completion-at-point-functions.   Are you sure your completion frontend
> can't work via those?  The function you add to that variable could
> just endeavour to check if there's any preview showing as an overlay,
> and it if is insert it.  Then you would need no rebinding.

Yes, that could be quite a nice solution.  I'll try working it in and
see how it goes.  Thanks for this suggestion.

> Also are you 100% sure Company doesn't do this?  I'm pretty sure
> it does this when there is only one completion, at least.

No, I'm sure Company does do more or less this.  As you say, and as I
mentioned upthread, Company provides a preview frontend that kicks in
when there's just one completion candidate by default.  (This is in
contrast with `corfu`, that provides no such preview.)  There are some
differences between my implementation and what you get with Company's
preview frontend, but if Company would make it to core Emacs that would
indeed mostly obviate the need for my `completion-preview`, at least as
far as I'm concerned.  Is there a viable path for upstreaming Company?

Re: Inline completion preview

[João Távora]

On Wed, Nov 1, 2023 at 10:44 AM Eshel Yaron <me@eshelyaron.com> wrote:

> > But OTOH there are other better ways to do that, i.e. via
> > completion-at-point-functions.   Are you sure your completion frontend
> > can't work via those?  The function you add to that variable could
> > just endeavour to check if there's any preview showing as an overlay,
> > and it if is insert it.  Then you would need no rebinding.
>
> Yes, that could be quite a nice solution.  I'll try working it in and
> see how it goes.  Thanks for this suggestion.

Glad it helps.  If you do this and it works out, then I think it's a good
addition to core Emacs.  Not as great as a fully fledged completion tooltip
but nice nonetheless.

> > Also are you 100% sure Company doesn't do this?  I'm pretty sure
> > it does this when there is only one completion, at least.
>
> No, I'm sure Company does do more or less this.  As you say, and as I
> mentioned upthread, Company provides a preview frontend that kicks in
> when there's just one completion candidate by default.  (This is in
> contrast with `corfu`, that provides no such preview.)  There are some
> differences between my implementation and what you get with Company's
> preview frontend, but if Company would make it to core Emacs that would
> indeed mostly obviate the need for my `completion-preview`, at least as
> far as I'm concerned.  Is there a viable path for upstreaming Company?

I've suggested this in the past, and it's mostly for Dmitry to answer,
but IMO that path should definitely contain a scaling back of Company's
company-backends infrastructure, basically importing only company-capf,
which is (IME of course) the only real backend we should focus on. This
would help sanitize some of the incorrect perceptions of complexity in
Company setup (the kind that Alex's argument suffers from).

João

Re: Inline completion preview

[Eshel Yaron]

João Távora <joaotavora@gmail.com> writes:

> On Wed, Nov 1, 2023 at 10:44 AM Eshel Yaron <me@eshelyaron.com> wrote:
>
>> > But OTOH there are other better ways to do that, i.e. via
>> > completion-at-point-functions.   Are you sure your completion frontend
>> > can't work via those?  The function you add to that variable could
>> > just endeavour to check if there's any preview showing as an overlay,
>> > and it if is insert it.  Then you would need no rebinding.
>>
>> Yes, that could be quite a nice solution.  I'll try working it in and
>> see how it goes.  Thanks for this suggestion.
>
> Glad it helps.  If you do this and it works out, then I think it's a good
> addition to core Emacs.  Not as great as a fully fledged completion tooltip
> but nice nonetheless.

Thanks, I've implemented your suggestion and it seems to work nicely.
See the updated code at
https://git.sr.ht/~eshel/completion-preview/tree/master/item/completion-preview.el

>>  ...Is there a viable path for upstreaming Company?
>
> I've suggested this in the past, and it's mostly for Dmitry to answer,
> but IMO that path should definitely contain a scaling back of Company's
> company-backends infrastructure, basically importing only company-capf,
> which is (IME of course) the only real backend we should focus on. This
> would help sanitize some of the incorrect perceptions of complexity in
> Company setup (the kind that Alex's argument suffers from).

I fully agree we should focus on the capf backend.

I guess I'll wait for further opinions regarding adding either (some of)
Company or `completion-preview` to core, otherwise I can start preparing
a patch for adding `completion-preview` that'd include some additional
documentation etc.


Cheers,

Eshel

Re: Inline completion preview

[Dmitry Gutov]

On 01/11/2023 13:07, João Távora wrote:
>>> Also are you 100% sure Company doesn't do this?  I'm pretty sure
>>> it does this when there is only one completion, at least.
>> No, I'm sure Company does do more or less this.  As you say, and as I
>> mentioned upthread, Company provides a preview frontend that kicks in
>> when there's just one completion candidate by default.  (This is in
>> contrast with `corfu`, that provides no such preview.)  There are some
>> differences between my implementation and what you get with Company's
>> preview frontend, but if Company would make it to core Emacs that would
>> indeed mostly obviate the need for my `completion-preview`, at least as
>> far as I'm concerned.  Is there a viable path for upstreaming Company?
> I've suggested this in the past, and it's mostly for Dmitry to answer,
> but IMO that path should definitely contain a scaling back of Company's
> company-backends infrastructure, basically importing only company-capf,
> which is (IME of course) the only real backend we should focus on. This
> would help sanitize some of the incorrect perceptions of complexity in
> Company setup (the kind that Alex's argument suffers from).

If we simply do that, I'll end up with bug reports from users with two 
different configurations and sets of capabilities (under the same name).

And a lot of users wondering why many snippets they found on the 
Internet (or even in the current manual) dosn't work because the Emacs's 
built-in distribution of Company doesn't include all other backends.

If/when we migrate to a new and better completion API, we'll discuss 
migrating all of Company's completion sources to it.

Alternatively, we keep migrating the backends to the current CAPF 
format. But then, to support "merging" completions, the core would need 
to include something like the capf combinators from Daniel's cape.el. 
But that is the scenario where we drop the idea of CAPFv2 and try 
evolving the current one gradually.

Re: Inline completion preview

[Juergen Fenn]

Am 01.11.23 um 13:23 Uhr schrieb Dmitry Gutov:
>> I've suggested this in the past, and it's mostly for Dmitry to answer,
>> but IMO that path should definitely contain a scaling back of Company's
>> company-backends infrastructure, basically importing only company-capf,
>> which is (IME of course) the only real backend we should focus on. This
>> would help sanitize some of the incorrect perceptions of complexity in
>> Company setup (the kind that Alex's argument suffers from).
>
> If we simply do that, I'll end up with bug reports from users with two
> different configurations and sets of capabilities (under the same name).
>
> And a lot of users wondering why many snippets they found on the
> Internet (or even in the current manual) dosn't work because the Emacs's
> built-in distribution of Company doesn't include all other backends.
>
> If/when we migrate to a new and better completion API, we'll discuss
> migrating all of Company's completion sources to it.

I'd like to second this suggestion as a user writing more prose than
code, although I also write a lot of prose in LaTeX which seems to be
somewhere in between. ;)

When playing with different "completion frameworks" for a while lately,
in the end I stuck with company. But the only backend in Company I could
actually use was company-dabbrev because with dabbrev it is possible to
set company-dabbrev-ignore-case to nil. Without this, whatever kind of
completion you implement will be mostly useless if you use it in a
language environment where case needs to be preserved like, e.g.,
German. So could you please consider this use-case. Thank you.

Regards,
Jürgen.

Re: Inline completion preview

[Dmitry Gutov]

On 01/11/2023 16:07, Juergen Fenn wrote:
> 
> Am 01.11.23 um 13:23 Uhr schrieb Dmitry Gutov:
>>> I've suggested this in the past, and it's mostly for Dmitry to answer,
>>> but IMO that path should definitely contain a scaling back of Company's
>>> company-backends infrastructure, basically importing only company-capf,
>>> which is (IME of course) the only real backend we should focus on. This
>>> would help sanitize some of the incorrect perceptions of complexity in
>>> Company setup (the kind that Alex's argument suffers from).
>> If we simply do that, I'll end up with bug reports from users with two
>> different configurations and sets of capabilities (under the same name).
>>
>> And a lot of users wondering why many snippets they found on the
>> Internet (or even in the current manual) dosn't work because the Emacs's
>> built-in distribution of Company doesn't include all other backends.
>>
>> If/when we migrate to a new and better completion API, we'll discuss
>> migrating all of Company's completion sources to it.
> I'd like to second this suggestion as a user writing more prose than
> code, although I also write a lot of prose in LaTeX which seems to be
> somewhere in between. 😉
> 
> When playing with different "completion frameworks" for a while lately,
> in the end I stuck with company. But the only backend in Company I could
> actually use was company-dabbrev because with dabbrev it is possible to
> set company-dabbrev-ignore-case to nil. Without this, whatever kind of
> completion you implement will be mostly useless if you use it in a
> language environment where case needs to be preserved like, e.g.,
> German. So could you please consider this use-case. Thank you.

You might find `company-ispell` also somewhat useful for prose.

Also check out the master branch, if you can: it has some performance 
improvements for this backend, among other things.

Re: Inline completion preview

[Dmitry Gutov]

On 01/11/2023 12:44, Eshel Yaron wrote:
> There are some
> differences between my implementation and what you get with Company's
> preview frontend, but if Company would make it to core Emacs that would
> indeed mostly obviate the need for my `completion-preview`, at least as
> far as I'm concerned.  Is there a viable path for upstreaming Company?

There is a subproject worked on, called "ELPA bundling", where some 
packages from ELPA get distributed together with Emacs's releases.

It seems to have stalled at the moment, but anyone interested can try to 
find out its current state and contribute.

Re: Inline completion preview

[Dmitry Gutov]

On 01/11/2023 10:30, Eshel Yaron wrote:
> Either way, `C-M-i` seems like a natural choice since inserting the
> previewed completion is strongly related to `completion-at-point`, so
> the idea is that this key combination does something similar
> (completion) whether the preview is visible or not.  WDYT?

It removes certain power from the user, namely, being able to press 
C-M-i to see all available completions, not just the previewed one.

I think I'd simply use RET as the key. Since the preview only appears 
after the user had pressed C-M-i already, this should conflict with the 
regular usage of RET.

Re: Inline completion preview

[Dmitry Gutov]

Sorry,

On 01/11/2023 14:14, Dmitry Gutov wrote:
> On 01/11/2023 10:30, Eshel Yaron wrote:
>> Either way, `C-M-i` seems like a natural choice since inserting the
>> previewed completion is strongly related to `completion-at-point`, so
>> the idea is that this key combination does something similar
>> (completion) whether the preview is visible or not.  WDYT?
> 
> It removes certain power from the user, namely, being able to press 
> C-M-i to see all available completions, not just the previewed one.
> 
> I think I'd simply use RET as the key. Since the preview only appears 
> after the user had pressed C-M-i already, this should conflict with the 
> regular usage of RET.
                                                  ^ shouldn't

Re: Inline completion preview

[Eshel Yaron]

Dmitry Gutov <dmitry@gutov.dev> writes:

> On 01/11/2023 10:30, Eshel Yaron wrote:
>> Either way, `C-M-i` seems like a natural choice since inserting the
>> previewed completion is strongly related to `completion-at-point`, so
>> the idea is that this key combination does something similar
>> (completion) whether the preview is visible or not.  WDYT?
>
> It removes certain power from the user, namely, being able to press
> C-M-i to see all available completions, not just the previewed one.

Yes, that's a certain downside, although you can regain that power by
typing C-g before C-M-i (or any command other than self-insert-command
for that matter).

> I think I'd simply use RET as the key. Since the preview only appears
> after the user had pressed C-M-i already, this should conflict with
> the regular usage of RET.

I'm not sure I completely follow, but I do agree that RET is another
good choice for "accepting the suggestion".  OTOH note that the
completion preview appears automatically, without you pressing C-M-i, so
I feel that it would probably be better to leave RET alone for inserting
a newline.  Another good candidate, IMO, is TAB.  But I do like how with
João's suggestion we don't rebind any key, which makes things somewhat
simpler.  If a user does want to use RET or TAB in order to insert the
suggestion when the preview is visible, they can bind
`completion-at-point` to RET in `completion-preview-active-mode-map`.

Re: Inline completion preview

[Dmitry Gutov]

On 01/11/2023 14:29, Eshel Yaron wrote:
>> I think I'd simply use RET as the key. Since the preview only appears
>> after the user had pressed C-M-i already, this should conflict with
>> the regular usage of RET.
> I'm not sure I completely follow, but I do agree that RET is another
> good choice for "accepting the suggestion".  OTOH note that the
> completion preview appears automatically, without you pressing C-M-i, so
> I feel that it would probably be better to leave RET alone for inserting
> a newline.  Another good candidate, IMO, is TAB.  But I do like how with
> João's suggestion we don't rebind any key, which makes things somewhat
> simpler.  If a user does want to use RET or TAB in order to insert the
> suggestion when the preview is visible, they can bind
> `completion-at-point` to RET in `completion-preview-active-mode-map`.

Fair points.

Though I do imagine a user would normally expect to have /some/ default 
binding to act on the shown preview.

Re: Inline completion preview

[Eli Zaretskii]

> Date: Wed, 1 Nov 2023 14:14:43 +0200
> Cc: Alexander Adolf <alexander.adolf@condition-alpha.com>,
>  João Távora <joaotavora@gmail.com>,
>  Juri Linkov <juri@linkov.net>, Eli Zaretskii <eliz@gnu.org>,
>  emacs-devel@gnu.org
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> On 01/11/2023 10:30, Eshel Yaron wrote:
> > Either way, `C-M-i` seems like a natural choice since inserting the
> > previewed completion is strongly related to `completion-at-point`, so
> > the idea is that this key combination does something similar
> > (completion) whether the preview is visible or not.  WDYT?
> 
> It removes certain power from the user, namely, being able to press 
> C-M-i to see all available completions, not just the previewed one.
> 
> I think I'd simply use RET as the key. Since the preview only appears 
> after the user had pressed C-M-i already, this should conflict with the 
> regular usage of RET.

Other applications that provide this feature use TAB, not RET.

Re: Inline completion preview

[Dmitry Gutov]

On 01/11/2023 14:34, Eli Zaretskii wrote:
>> Date: Wed, 1 Nov 2023 14:14:43 +0200
>> Cc: Alexander Adolf<alexander.adolf@condition-alpha.com>,
>>   João Távora<joaotavora@gmail.com>,
>>   Juri Linkov<juri@linkov.net>, Eli Zaretskii<eliz@gnu.org>,
>>   emacs-devel@gnu.org
>> From: Dmitry Gutov<dmitry@gutov.dev>
>>
>> On 01/11/2023 10:30, Eshel Yaron wrote:
>>> Either way, `C-M-i` seems like a natural choice since inserting the
>>> previewed completion is strongly related to `completion-at-point`, so
>>> the idea is that this key combination does something similar
>>> (completion) whether the preview is visible or not.  WDYT?
>> It removes certain power from the user, namely, being able to press
>> C-M-i to see all available completions, not just the previewed one.
>>
>> I think I'd simply use RET as the key. Since the preview only appears
>> after the user had pressed C-M-i already, this should conflict with the
>> regular usage of RET.
> Other applications that provide this feature use TAB, not RET.

Works for me.

Re: Inline completion preview

[Po Lu]

Dmitry Gutov <dmitry@gutov.dev> writes:

> The GUI-only users are free to install company-posframe, with similar
> benefits. It's not in GNU ELPA, though: still collecting the last
> copyright assignment signature.

I hope I'm not interrupting, but the free time to follow these
discussions with hasn't come by me as of late.

There are several shortcomings to child frames as they stand: when a
child frame is displayed, the manner in which focus is transferred
between frames becomes erratic; each frame occupies plenty of memory,
reconfiguring such frames is slow, and particular operations possible
with regular frames cannot be performed on child frames under specific
window systems (such as PGTK, where restacking child frames after their
creation is forbidden).

It's better if Emacs didn't incorporate any code or documentation
encouraging users to make use of child frames until such time as the
aforementioned deficiencies are resolved.

Re: Inline completion preview

[Dmitry Gutov]

On 01/11/2023 15:28, Po Lu wrote:
> Dmitry Gutov <dmitry@gutov.dev> writes:
> 
>> The GUI-only users are free to install company-posframe, with similar
>> benefits. It's not in GNU ELPA, though: still collecting the last
>> copyright assignment signature.
> 
> I hope I'm not interrupting, but the free time to follow these
> discussions with hasn't come by me as of late.
> 
> There are several shortcomings to child frames as they stand: when a
> child frame is displayed, the manner in which focus is transferred
> between frames becomes erratic; each frame occupies plenty of memory,
> reconfiguring such frames is slow, and particular operations possible
> with regular frames cannot be performed on child frames under specific
> window systems (such as PGTK, where restacking child frames after their
> creation is forbidden).

These are the same problems that I've seen in the bug reports, and when 
testing posframe-company: some platforms complained about focus issues 
(which, to be fair to the author, he worked on resolving), and with 
general responsiveness somewhat worse than the overlay-based popup.

It would really help if someone with experience in GUI programming on 
several platforms did some work on the feature. Or created a more 
focused popup primitive which would use native solutions on each 
platform (while supporting all the same text formatting, though).

Re: Inline completion preview

[Ihor Radchenko]

Eshel Yaron <me@eshelyaron.com> writes:

> Sorry, that might have not been clear enough, I wrote:
>
>     ISTM that the best approach is a simple library that uses only
>     `completion-at-point` as a backend, because `completion-at-point` is
>     already extensible enough to support a variety of completion
>     backends.
>
> None of the aforementioned packages take that approach, so I wrote
> `completion-preview.el` do demonstrate and test this approach.  As to
> the question of adding this feature to core, since it's quite a simple
> and useful addition, I just think it could be nice to have it OOTB.
>

Don't corfu do it already?
From https://github.com/minad/corfu:

    Corfu is a small package, which relies on the Emacs completion
    facilities and concentrates on providing a polished completion UI.
    In-buffer completion UIs in Emacs can hook into
    completion-in-region, which implements the interaction with the
    user. Completions at point are either provided by commands like
    dabbrev-completion or by pluggable backends
    (completion-at-point-functions, Capfs) and are then passed to
    completion-in-region. Many programming, text and shell major modes
    implement a Capf. Corfu does not include its own completion
    backends. The Emacs built-in Capfs and the Capfs provided by
    third-party programming language packages are often sufficient.
    Additional Capfs and completion utilities are provided by the Cape
    package.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Inline completion preview

[Eshel Yaron]

Ihor Radchenko <yantar92@posteo.net> writes:

> Eshel Yaron <me@eshelyaron.com> writes:
>
>> Sorry, that might have not been clear enough, I wrote:
>>
>>     ISTM that the best approach is a simple library that uses only
>>     `completion-at-point` as a backend, because `completion-at-point` is
>>     already extensible enough to support a variety of completion
>>     backends.
>>
>> None of the aforementioned packages take that approach, so I wrote
>> `completion-preview.el` do demonstrate and test this approach.  As to
>> the question of adding this feature to core, since it's quite a simple
>> and useful addition, I just think it could be nice to have it OOTB.
>>
>
> Don't corfu do it already?

No, this is a somewhat different feature,

> From https://github.com/minad/corfu:
>
>     Corfu is a small package, which relies on the Emacs completion
>     facilities and concentrates on providing a polished completion UI.

Indeed `corfu` provides a completion UI, but not an inline completion preview.
`corfu`'s completion UI replaces the *Completions* buffer for
in-buffer completions.  Completion preview is something related but
independent from the completion UI.  Namely a suggestion for a single
completion candidate that automatically shows up after point as you
type.  It appears inline, as if you've already inserted that candidate,
in that sense it gives you a preview for the completion operation.

The package `corfu-candidate-overlay` is another project that does
provide completion preview and builds on top of `corfu` instead of
`completion-at-point` directly.

Re: Inline completion preview

[João Távora]

On Thu, Oct 26, 2023 at 7:38 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Eshel Yaron <me@eshelyaron.com>
> > Date: Thu, 26 Oct 2023 20:11:17 +0200
> >
> > I want to propose adding an inline completion preview mechanism to Emacs.
> >
> > By inline completion preview, I mean an overlay that pop ups after point
> > showing one of the candidates for at-point completion.  You can view
> > this mechanism as giving you a hint or suggesting a certain completion.
>
> I wish people would work on adding to Emacs some a GUI infrastructure
> for showing such completion candidates like other IDEs do, instead of
> once again using overlays with after-string properties.  The result of
> using overlay strings is simply not visually appealing, and on top of
> that it causes quite a few complications for the display code.

I think agree with this, but it should work with Non-GUI Emacs, too right?
In a TUI Emacs, there have been these nice pop-up menus for a while.  At first
sight the display infrastructure being used there seems suitable for showing
completions, so I wonder if that infrastructure couldn't be refactored
then reused
for a completion popup in core.

João

Re: Inline completion preview

[João Távora]

On Fri, Oct 27, 2023 at 12:49 PM João Távora <joaotavora@gmail.com> wrote:
>
> On Thu, Oct 26, 2023 at 7:38 PM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > From: Eshel Yaron <me@eshelyaron.com>
> > > Date: Thu, 26 Oct 2023 20:11:17 +0200
> > >
> > > I want to propose adding an inline completion preview mechanism to Emacs.
> > >
> > > By inline completion preview, I mean an overlay that pop ups after point
> > > showing one of the candidates for at-point completion.  You can view
> > > this mechanism as giving you a hint or suggesting a certain completion.
> >
> > I wish people would work on adding to Emacs some a GUI infrastructure
> > for showing such completion candidates like other IDEs do, instead of
> > once again using overlays with after-string properties.  The result of
> > using overlay strings is simply not visually appealing, and on top of
> > that it causes quite a few complications for the display code.
>
> I think agree with this, but it should work with Non-GUI Emacs, too right?
> In a TUI Emacs, there have been these nice pop-up menus for a while.  At first
> sight the display infrastructure being used there seems suitable for showing
> completions, so I wonder if that infrastructure couldn't be refactored
> then reused for a completion popup in core.

Just to clarify what I mean: my gripe with overlays+after-string props
is not that they are visually unappealing, but that they often clash with
other extensions that also use overlays and other things.  Company has
problems with line numbers for example, I seem to recall.

Re: Inline completion preview

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 27 Oct 2023 12:49:40 +0100
> Cc: Eshel Yaron <me@eshelyaron.com>, emacs-devel@gnu.org
> 
> On Thu, Oct 26, 2023 at 7:38 PM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > From: Eshel Yaron <me@eshelyaron.com>
> > > Date: Thu, 26 Oct 2023 20:11:17 +0200
> > >
> > > I want to propose adding an inline completion preview mechanism to Emacs.
> > >
> > > By inline completion preview, I mean an overlay that pop ups after point
> > > showing one of the candidates for at-point completion.  You can view
> > > this mechanism as giving you a hint or suggesting a certain completion.
> >
> > I wish people would work on adding to Emacs some a GUI infrastructure
> > for showing such completion candidates like other IDEs do, instead of
> > once again using overlays with after-string properties.  The result of
> > using overlay strings is simply not visually appealing, and on top of
> > that it causes quite a few complications for the display code.
> 
> I think agree with this, but it should work with Non-GUI Emacs, too right?

Ideally, yes.

> In a TUI Emacs, there have been these nice pop-up menus for a while.  At first
> sight the display infrastructure being used there seems suitable for showing
> completions, so I wonder if that infrastructure couldn't be refactored
> then reused
> for a completion popup in core.

The text-mode frames should preferably also support this, but we
should first think how to do that on GUI frames, and only afterwards
how to express the same feature on TTY frames.

I also don't see any catastrophe in having features that can only work
on GUI frames, although we should try to avoid that.  For example,
popup menus and dialog boxes were not supported on TTY frames for 2
decades, and we didn't consider that to be such a bad thing.  So we
could develop the feature discussed here on GUI, and then take the
next 2 decades to find a way of doing that on TTY.

Re: Inline completion preview

[João Távora]

On Fri, Oct 27, 2023 at 1:18 PM Eli Zaretskii <eliz@gnu.org> wrote:
 don't see any catastrophe in having features that can only work
> on GUI frames, although we should try to avoid that.  For example,
> popup menus and dialog boxes were not supported on TTY frames for 2
> decades, and we didn't consider that to be such a bad thing.

Because popup menus and dialog boxes aren't nearly as useful as
a completion popup.  Who even uses the mouse?

> So we
> could develop the feature discussed here on GUI, and then take the
> next 2 decades to find a way of doing that on TTY.

Ha.

Well, my opinion of fancy-shmancy GUI completion in other IDEs is that
they don't really add anything in terms of usablity.  Company-mode
looks pretty good to me when you look past the odd bug, and the only
difference between GUI and TUI versions is some fancy symbol characters
that could probably also be implemented in TUI.

Let's do this for both TUI and GUI emacs.  Then possibly upgrade
the fancyness on GUI Emacs.

João

Re: Inline completion preview

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Fri, 27 Oct 2023 13:55:47 +0100
> Cc: me@eshelyaron.com, emacs-devel@gnu.org
> 
> On Fri, Oct 27, 2023 at 1:18 PM Eli Zaretskii <eliz@gnu.org> wrote:
>  don't see any catastrophe in having features that can only work
> > on GUI frames, although we should try to avoid that.  For example,
> > popup menus and dialog boxes were not supported on TTY frames for 2
> > decades, and we didn't consider that to be such a bad thing.
> 
> Because popup menus and dialog boxes aren't nearly as useful as
> a completion popup.  Who even uses the mouse?

TTY menus are supported even without the mouse.

> Let's do this for both TUI and GUI emacs.

Like I said: ideally, that's what we should do.

Re: Inline completion preview

[Dmitry Gutov]

On 27/10/2023 14:49, João Távora wrote:
> On Thu, Oct 26, 2023 at 7:38 PM Eli Zaretskii<eliz@gnu.org>  wrote:
>>> From: Eshel Yaron<me@eshelyaron.com>
>>> Date: Thu, 26 Oct 2023 20:11:17 +0200
>>>
>>> I want to propose adding an inline completion preview mechanism to Emacs.
>>>
>>> By inline completion preview, I mean an overlay that pop ups after point
>>> showing one of the candidates for at-point completion.  You can view
>>> this mechanism as giving you a hint or suggesting a certain completion.
>> I wish people would work on adding to Emacs some a GUI infrastructure
>> for showing such completion candidates like other IDEs do, instead of
>> once again using overlays with after-string properties.  The result of
>> using overlay strings is simply not visually appealing, and on top of
>> that it causes quite a few complications for the display code.
> I think agree with this, but it should work with Non-GUI Emacs, too right?
> In a TUI Emacs, there have been these nice pop-up menus for a while.  At first
> sight the display infrastructure being used there seems suitable for showing
> completions, so I wonder if that infrastructure couldn't be refactored
> then reused
> for a completion popup in core.

I very much agree with this too.

In TUI, the menu popups implementation could be partially reused.

In GUI, someone could look into bringing posframe (a GNU ELPA packages 
which is based on child frames) into the core in some shape or form. But 
the hard part is debugging how the child frames interact with various 
platforms and window managers -- this is where the majority of its bugs 
lie, and we seem to lack that qualification (or at least the qualified 
people don't seem to have enough free time). Or the popups could use 
something else in GUI, IDK.