Add completion to compilation-read-command

Add completion to compilation-read-command

[Spyros Roum]

[-- Attachment #1: Type: text/plain, Size: 983 bytes --]

Hello all,

Recently, I started using `M-x compile` more but as I was used to my 
shell suggesting past commands as I type (and tools like atuin), I was 
missing auto-complete a lot.
I managed to add this functionality by writing a simple function based 
on compilation-read-command that uses completing-read instead of 
read-shell-command.
Then I used advice-add to overwrite the 
original compilation-read-command with mine.

So far this works well, and as far as I can tell there is no good reason 
not to make compile auto-completing, it already has a history that you 
can navigate anyway.

With that said, this is the first time I write here and the first time 
I'm trying to contribute to emacs, so I'm not sure what the best way to 
go from here would be.
I think some decisions would need to be taken, for once I am not sure if 
it's acceptable to change the default and make it completing or if there 
should be an option for it.

Looking forward to your feedback, thanks

[-- Attachment #2: Type: text/html, Size: 1359 bytes --]

Re: Add completion to compilation-read-command

[Philip Kaludercic]

Spyros Roum <spyros.roum@posteo.net> writes:

> Hello all,
>
> Recently, I started using `M-x compile` more but as I was used to my
> shell suggesting past commands as I type (and tools like atuin)

In case anyone else hasn't heard of this, it describes itself as

  Atuin replaces your existing shell history with a SQLite database, and
  records additional context for your commands. Additionally, it
  provides optional and fully encrypted synchronisation of your history
  between machines, via an Atuin server.

(https://github.com/atuinsh/atuin)

>                                                                , I was
> missing auto-complete a lot.
> I managed to add this functionality by writing a simple function based
> on compilation-read-command that uses completing-read instead of
> read-shell-command.

Do you know about the `bash-completion' package?  It enhances
`read-shell-command' completion with completion data provided by bash.
It is very easy to set up,

  (use-package bash-completion :ensure t
    :init (bash-completion-setup))

should do it.

> Then I used advice-add to overwrite the
> original compilation-read-command with mine.
>
> So far this works well, and as far as I can tell there is no good
> reason not to make compile auto-completing, it already has a history
> that you can navigate anyway.
>
> With that said, this is the first time I write here and the first time
> I'm trying to contribute to emacs, so I'm not sure what the best way
> to go from here would be.
> I think some decisions would need to be taken, for once I am not sure
> if it's acceptable to change the default and make it completing or if
> there should be an option for it.

I am not sure if you meant to attach any code, but that would probably
be the best place to start.

> Looking forward to your feedback, thanks

Re: Add completion to compilation-read-command

[Spyros Roum]

[-- Attachment #1: Type: text/plain, Size: 2811 bytes --]


Philip Kaludercic wrote:
> Spyros Roum<spyros.roum@posteo.net> writes:
>
>> Hello all,
>>
>> Recently, I started using `M-x compile` more but as I was used to my
>> shell suggesting past commands as I type (and tools like atuin)
> In case anyone else hasn't heard of this, it describes itself as
>
>    Atuin replaces your existing shell history with a SQLite database, and
>    records additional context for your commands. Additionally, it
>    provides optional and fully encrypted synchronisation of your history
>    between machines, via an Atuin server.
>
> (https://github.com/atuinsh/atuin)

Thanks for adding the description, I should have done that myself.


>>                                                                 , I was
>> missing auto-complete a lot.
>> I managed to add this functionality by writing a simple function based
>> on compilation-read-command that uses completing-read instead of
>> read-shell-command.
> Do you know about the `bash-completion' package?  It enhances
> `read-shell-command' completion with completion data provided by bash.
> It is very easy to set up,
>
>    (use-package bash-completion :ensure t
>      :init (bash-completion-setup))
>
> should do it.

I was not aware of this, but it doesn't seem to do what I'm looking for.
For once, I am not using bash, but even ignoring that it doesn't seem to 
have the effect I'm looking for.

I'm trying to get the compile prompt to suggest completion based on past 
commands I've run.


>> Then I used advice-add to overwrite the
>> original compilation-read-command with mine.
>>
>> So far this works well, and as far as I can tell there is no good
>> reason not to make compile auto-completing, it already has a history
>> that you can navigate anyway.
>>
>> With that said, this is the first time I write here and the first time
>> I'm trying to contribute to emacs, so I'm not sure what the best way
>> to go from here would be.
>> I think some decisions would need to be taken, for once I am not sure
>> if it's acceptable to change the default and make it completing or if
>> there should be an option for it.
> I am not sure if you meant to attach any code, but that would probably
> be the best place to start.
>
>> Looking forward to your feedback, thanks

You are right I should have posted code, so here is what I have:

   (defun compilation-read-command-with-autocomplete (command)
     "Use `completing-read` to add autocomplete powers to compilation read"
     (completing-read "Compile command: " compile-history
       nil nil command
       (if (equal (car compile-history) command)
         '(compile-history . 1)
         'compile-history)))

   (advice-add
     #'compilation-read-command
     :override #'compilation-read-command-with-autocomplete)

[-- Attachment #2: Type: text/html, Size: 4332 bytes --]

Re: Add completion to compilation-read-command

[Philip Kaludercic]

Spyros Roum <spyros.roum@posteo.net> writes:

> Philip Kaludercic wrote:
>> Spyros Roum<spyros.roum@posteo.net> writes:
>>
>>> Hello all,
>>>
>>> Recently, I started using `M-x compile` more but as I was used to my
>>> shell suggesting past commands as I type (and tools like atuin)
>> In case anyone else hasn't heard of this, it describes itself as
>>
>>    Atuin replaces your existing shell history with a SQLite database, and
>>    records additional context for your commands. Additionally, it
>>    provides optional and fully encrypted synchronisation of your history
>>    between machines, via an Atuin server.
>>
>> (https://github.com/atuinsh/atuin)
>
> Thanks for adding the description, I should have done that myself.

No worries.

>>>                                                                 , I was
>>> missing auto-complete a lot.
>>> I managed to add this functionality by writing a simple function based
>>> on compilation-read-command that uses completing-read instead of
>>> read-shell-command.
>> Do you know about the `bash-completion' package?  It enhances
>> `read-shell-command' completion with completion data provided by bash.
>> It is very easy to set up,
>>
>>    (use-package bash-completion :ensure t
>>      :init (bash-completion-setup))
>>
>> should do it.
>
> I was not aware of this, but it doesn't seem to do what I'm looking for.
> For once, I am not using bash, but even ignoring that it doesn't seem
> to have the effect I'm looking for.
>
> I'm trying to get the compile prompt to suggest completion based on
> past commands I've run.

My bad.  In that case, why not use C-r to search `compile-history'?

>
>>> Then I used advice-add to overwrite the
>>> original compilation-read-command with mine.
>>>
>>> So far this works well, and as far as I can tell there is no good
>>> reason not to make compile auto-completing, it already has a history
>>> that you can navigate anyway.
>>>
>>> With that said, this is the first time I write here and the first time
>>> I'm trying to contribute to emacs, so I'm not sure what the best way
>>> to go from here would be.
>>> I think some decisions would need to be taken, for once I am not sure
>>> if it's acceptable to change the default and make it completing or if
>>> there should be an option for it.
>> I am not sure if you meant to attach any code, but that would probably
>> be the best place to start.
>>
>>> Looking forward to your feedback, thanks
>
> You are right I should have posted code, so here is what I have:
>
>   (defun compilation-read-command-with-autocomplete (command)
>     "Use `completing-read` to add autocomplete powers to compilation read"
>     (completing-read "Compile command: " compile-history
>       nil nil command
>       (if (equal (car compile-history) command)
>         '(compile-history . 1)
>         'compile-history)))
>
>   (advice-add
>     #'compilation-read-command
>     :override #'compilation-read-command-with-autocomplete)

If this were added to Emacs, we probably wouldn't use advice.  I think
the best strategy for you would be to prepare a patch against compile.el
and add a user option that allows prompting the user with
completing-read.  What do you think?

Re: Add completion to compilation-read-command

[Spyros Roum]

[-- Attachment #1: Type: text/plain, Size: 3479 bytes --]

Philip Kaludercic wrote:
>>>>                                                                  , I was
>>>> missing auto-complete a lot.
>>>> I managed to add this functionality by writing a simple function based
>>>> on compilation-read-command that uses completing-read instead of
>>>> read-shell-command.
>>> Do you know about the `bash-completion' package?  It enhances
>>> `read-shell-command' completion with completion data provided by bash.
>>> It is very easy to set up,
>>>
>>>     (use-package bash-completion :ensure t
>>>       :init (bash-completion-setup))
>>>
>>> should do it.
>> I was not aware of this, but it doesn't seem to do what I'm looking for.
>> For once, I am not using bash, but even ignoring that it doesn't seem
>> to have the effect I'm looking for.
>>
>> I'm trying to get the compile prompt to suggest completion based on
>> past commands I've run.
> My bad.  In that case, why not use C-r to search `compile-history'?

I'm not sure if you are suggesting that I should be able to use C-r in 
the compile prompt, if so there might be a keybind issue as I'm using 
evil and C-r does not do that for me.
If that's the case, could you name the function that does that? It could 
be what I'm looking for.

>>>> Then I used advice-add to overwrite the
>>>> original compilation-read-command with mine.
>>>>
>>>> So far this works well, and as far as I can tell there is no good
>>>> reason not to make compile auto-completing, it already has a history
>>>> that you can navigate anyway.
>>>>
>>>> With that said, this is the first time I write here and the first time
>>>> I'm trying to contribute to emacs, so I'm not sure what the best way
>>>> to go from here would be.
>>>> I think some decisions would need to be taken, for once I am not sure
>>>> if it's acceptable to change the default and make it completing or if
>>>> there should be an option for it.
>>> I am not sure if you meant to attach any code, but that would probably
>>> be the best place to start.
>>>
>>>> Looking forward to your feedback, thanks
>> You are right I should have posted code, so here is what I have:
>>
>>    (defun compilation-read-command-with-autocomplete (command)
>>      "Use `completing-read` to add autocomplete powers to compilation read"
>>      (completing-read "Compile command: " compile-history
>>        nil nil command
>>        (if (equal (car compile-history) command)
>>          '(compile-history . 1)
>>          'compile-history)))
>>
>>    (advice-add
>>      #'compilation-read-command
>>      :override #'compilation-read-command-with-autocomplete)
> If this were added to Emacs, we probably wouldn't use advice.  I think
> the best strategy for you would be to prepare a patch against compile.el
> and add a user option that allows prompting the user with
> completing-read.  What do you think?

I agree advice is not a good solution if this gets merged, this is just 
what I have in my config to make it work.

I believe an option defaulting to the current behavior is fine, assuming 
we don't want to just change it to the completing version all together.
However, I am somewhat lost on what that option might look like. 
Probably some customizable option based on which 
compilation-read-command's internals change to either the current or my 
version?

If so, I will need to figure out how the customizable system works 
internally in order to use it. Any pointers on that would be appreciated.

[-- Attachment #2: Type: text/html, Size: 4734 bytes --]

Re: Add completion to compilation-read-command

[Philip Kaludercic]

Spyros Roum <spyros.roum@posteo.net> writes:

> Philip Kaludercic wrote:
>>>>>                                                                  , I was
>>>>> missing auto-complete a lot.
>>>>> I managed to add this functionality by writing a simple function based
>>>>> on compilation-read-command that uses completing-read instead of
>>>>> read-shell-command.
>>>> Do you know about the `bash-completion' package?  It enhances
>>>> `read-shell-command' completion with completion data provided by bash.
>>>> It is very easy to set up,
>>>>
>>>>     (use-package bash-completion :ensure t
>>>>       :init (bash-completion-setup))
>>>>
>>>> should do it.
>>> I was not aware of this, but it doesn't seem to do what I'm looking for.
>>> For once, I am not using bash, but even ignoring that it doesn't seem
>>> to have the effect I'm looking for.
>>>
>>> I'm trying to get the compile prompt to suggest completion based on
>>> past commands I've run.
>> My bad.  In that case, why not use C-r to search `compile-history'?
>
> I'm not sure if you are suggesting that I should be able to use C-r in
> the compile prompt, if so there might be a keybind issue as I'm using
> evil and C-r does not do that for me.
> If that's the case, could you name the function that does that? It
> could be what I'm looking for.

Oh, sorry about that.  C-r is just the regular `isearch-backward'
binding.

>>>>> Then I used advice-add to overwrite the
>>>>> original compilation-read-command with mine.
>>>>>
>>>>> So far this works well, and as far as I can tell there is no good
>>>>> reason not to make compile auto-completing, it already has a history
>>>>> that you can navigate anyway.
>>>>>
>>>>> With that said, this is the first time I write here and the first time
>>>>> I'm trying to contribute to emacs, so I'm not sure what the best way
>>>>> to go from here would be.
>>>>> I think some decisions would need to be taken, for once I am not sure
>>>>> if it's acceptable to change the default and make it completing or if
>>>>> there should be an option for it.
>>>> I am not sure if you meant to attach any code, but that would probably
>>>> be the best place to start.
>>>>
>>>>> Looking forward to your feedback, thanks
>>> You are right I should have posted code, so here is what I have:
>>>
>>>    (defun compilation-read-command-with-autocomplete (command)
>>>      "Use `completing-read` to add autocomplete powers to compilation read"
>>>      (completing-read "Compile command: " compile-history
>>>        nil nil command
>>>        (if (equal (car compile-history) command)
>>>          '(compile-history . 1)
>>>          'compile-history)))
>>>
>>>    (advice-add
>>>      #'compilation-read-command
>>>      :override #'compilation-read-command-with-autocomplete)
>> If this were added to Emacs, we probably wouldn't use advice.  I think
>> the best strategy for you would be to prepare a patch against compile.el
>> and add a user option that allows prompting the user with
>> completing-read.  What do you think?
>
> I agree advice is not a good solution if this gets merged, this is
> just what I have in my config to make it work.
>
> I believe an option defaulting to the current behavior is fine,
> assuming we don't want to just change it to the completing version all
> together.

I think that would be to invasive.

> However, I am somewhat lost on what that option might look
> like. Probably some customizable option based on which
> compilation-read-command's internals change to either the current or
> my version?

The easiest choice is just a user option that holds function, presumably
without taking any argument The default value would use regular
shell-read-command, and a suggested alternative could be your
suggestion.

> If so, I will need to figure out how the customizable system works
> internally in order to use it. Any pointers on that would be
> appreciated.

It will probably look something like this

        (defcustom compilation-prompt-function 
          #'complation-prompt-using-read-shell-command
          "[document the user option here]."
          :version "31.0"
          :type 'function)

Re: Add completion to compilation-read-command

[Juri Linkov]

> I'm trying to get the compile prompt to suggest completion based on past
> commands I've run.

You can use 'C-x <up>' in the compile prompt to complete on past commands.

Re: Add completion to compilation-read-command

[Spyros Roum]

[-- Attachment #1: Type: text/plain, Size: 591 bytes --]

Juri Linkov wrote:
>> I'm trying to get the compile prompt to suggest completion based on past
>> commands I've run.
> You can use 'C-x <up>' in the compile prompt to complete on past commands.

This is indeed a lot closer to what I want, however it still lacks a lot 
of things compared to my solution.

Unless there is some package/mode I don't know about, It's a lot less 
dynamic, for instance I can't keep typing to reduce
possible items.
My solution can leverage all of vertico and friend, and in general 
provides an experience that's the same as other minibuffer prompts, like 
M-x.

[-- Attachment #2: Type: text/html, Size: 1174 bytes --]

Re: Add completion to compilation-read-command

[Juri Linkov]

>> You can use 'C-x <up>' in the compile prompt to complete on past commands.
>
> This is indeed a lot closer to what I want, however it still lacks a lot of
> things compared to my solution. 
>
> Unless there is some package/mode I don't know about, It's a lot less
> dynamic, for instance I can't keep typing to reduce possible items. 

There were many proposals how to implement `eager-update' to update
*Completions* as you type.  We just need to put together all parts.

Re: Add completion to compilation-read-command

[Spyros Roum]

[-- Attachment #1: Type: text/plain, Size: 941 bytes --]

Juri Linkov wrote:
>>> You can use 'C-x <up>' in the compile prompt to complete on past commands.
>> This is indeed a lot closer to what I want, however it still lacks a lot of
>> things compared to my solution.
>>
>> Unless there is some package/mode I don't know about, It's a lot less
>> dynamic, for instance I can't keep typing to reduce possible items.
> There were many proposals how to implement `eager-update' to update
> *Completions* as you type.  We just need to put together all parts.

I'm probably out of my depth here, as I'm not very familiar with 
*Completions*.
Is there a good reason to stick to it when `completing-read` works well 
and as far as I can tell
does a very similar/the same job?

I have a patch almost ready based on Philip's feedback that I will be 
sharing today or tomorrow.
If, however, the consensus is that it's not a good fit and that `C-x 
<up>` and *Completions* is good enough,
that's fine by me.

[-- Attachment #2: Type: text/html, Size: 1645 bytes --]

Re: Add completion to compilation-read-command

[Philip Kaludercic]

Spyros Roum <spyros.roum@posteo.net> writes:

> Juri Linkov wrote:
>>>> You can use 'C-x <up>' in the compile prompt to complete on past commands.
>>> This is indeed a lot closer to what I want, however it still lacks a lot of
>>> things compared to my solution.
>>>
>>> Unless there is some package/mode I don't know about, It's a lot less
>>> dynamic, for instance I can't keep typing to reduce possible items.
>> There were many proposals how to implement `eager-update' to update
>> *Completions* as you type.  We just need to put together all parts.
>
> I'm probably out of my depth here, as I'm not very familiar with
> *Completions*.
> Is there a good reason to stick to it when `completing-read` works
> well and as far as I can tell
> does a very similar/the same job?

*Completions* is just the buffer that pops up by default when
completing-read cannot expand the input unambiguously anymore?

> I have a patch almost ready based on Philip's feedback that I will be
> sharing today or tomorrow.
> If, however, the consensus is that it's not a good fit and that `C-x
> <up>` and *Completions* is good enough,
> that's fine by me.

I don't see a harm in having an additional user option.  Why, it might
even be a popular thing to allow all shell commands prompts to fall back
onto completing-read, considering how evil users apparently cannot make
use of C-r?

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Juri Linkov <juri@linkov.net>
>> Cc: philipk@posteo.net,  emacs-devel@gnu.org
>> Date: Tue, 24 Dec 2024 19:03:50 +0200
>> 
>> > I'm trying to get the compile prompt to suggest completion based on past
>> > commands I've run.
>> 
>> You can use 'C-x <up>' in the compile prompt to complete on past commands.
>
> Why the heck is this useful command completely undocumented??

I second this confusion, and want to mention that the binding is not
that convenient + might be shadowed for windmove users.

Re: Add completion to compilation-read-command

[Juri Linkov]

> I don't see a harm in having an additional user option.

Maybe with the name 'compilation-read-command-function'?

>>> You can use 'C-x <up>' in the compile prompt to complete on past commands.
>>
>> Why the heck is this useful command completely undocumented??
>
> I second this confusion, and want to mention that the binding is not
> that convenient + might be shadowed for windmove users.

Agreed, the current keybinding is not great.  This is why it was undocumented.
Probably a better key sequence should end with the TAB key since it's
a completion command.

RE: [External] : Re: Add completion to compilation-read-command

[Drew Adams]

> >> You can use 'C-x <up>' in the compile prompt to complete on past
> commands.
> >
> > This is indeed a lot closer to what I want, however it still lacks a
> > lot of things compared to my solution.
> > Unless there is some package/mode I don't know about, It's a lot less
> > dynamic, for instance I can't keep typing to reduce possible items.
> 
> There were many proposals how to implement `eager-update' to update
> *Completions* as you type.  We just need to put together all parts.

Taking me back...

Icicles has had all of that since 2006.

https://www.emacswiki.org/emacs/Icicles_-_History_Enhancements

https://www.emacswiki.org/emacs/Icicles_-_Nutshell_View#ProgressiveCompletion

https://www.emacswiki.org/emacs/Icicles_-_Nutshell_View#ChippingAway

https://www.emacswiki.org/emacs/Icicles_-_Icompletion#IncrementalCompletion

https://www.emacswiki.org/emacs/Icicles_-_Progressive_Completion

Re: [External] : Re: Add completion to compilation-read-command

[Juri Linkov]

>> There were many proposals how to implement `eager-update' to update
>> *Completions* as you type.  We just need to put together all parts.
>
> Taking me back...
>
> Icicles has had all of that since 2006.
>
> https://www.emacswiki.org/emacs/Icicles_-_History_Enhancements
>
> https://www.emacswiki.org/emacs/Icicles_-_Nutshell_View#ProgressiveCompletion
>
> https://www.emacswiki.org/emacs/Icicles_-_Nutshell_View#ChippingAway
>
> https://www.emacswiki.org/emacs/Icicles_-_Icompletion#IncrementalCompletion
>
> https://www.emacswiki.org/emacs/Icicles_-_Progressive_Completion

Thanks for the references.  Incremental and Progressive Completions
look similar.

Re: Add completion to compilation-read-command

[Philip Kaludercic]

Spyros Roum <spyros.roum@posteo.net> writes:

> Juri Linkov wrote:
>>> I'm trying to get the compile prompt to suggest completion based on past
>>> commands I've run.
>> You can use 'C-x <up>' in the compile prompt to complete on past commands.
>
> This is indeed a lot closer to what I want, however it still lacks a
> lot of things compared to my solution.
>
> Unless there is some package/mode I don't know about, It's a lot less
> dynamic, for instance I can't keep typing to reduce
> possible items.

One should keep in mind that completion is not the same as narrowing.
The former expands substrings, while the latter is more like a system
for selecting an option from a list, that usually comes with some query
system.
              
Confusing these two can lead to uncomfortable edge-cases for both sides,
and sadly a lot of packages assume that completing-read is the same as a
hypothetical selecting-read.

> My solution can leverage all of vertico and friend, and in general
> provides an experience that's the same as other minibuffer prompts,
> like M-x.

But for someone like me who doesn't use a selecting-narrowing framework
like vertico, it suddenly means that SPC is rebound to
`minibuffer-complete-word' and entering new commands becomes *a lot* more
cumbersome.

Re: Add completion to compilation-read-command

[Spyros Roum]

[-- Attachment #1: Type: text/plain, Size: 1791 bytes --]

Philip Kaludercic wrote:
> Spyros Roum<spyros.roum@posteo.net> writes:
>
>> Juri Linkov wrote:
>>>> I'm trying to get the compile prompt to suggest completion based on past
>>>> commands I've run.
>>> You can use 'C-x <up>' in the compile prompt to complete on past commands.
>> This is indeed a lot closer to what I want, however it still lacks a
>> lot of things compared to my solution.
>>
>> Unless there is some package/mode I don't know about, It's a lot less
>> dynamic, for instance I can't keep typing to reduce
>> possible items.
> One should keep in mind that completion is not the same as narrowing.
> The former expands substrings, while the latter is more like a system
> for selecting an option from a list, that usually comes with some query
> system.
>                
> Confusing these two can lead to uncomfortable edge-cases for both sides,
> and sadly a lot of packages assume that completing-read is the same as a
> hypothetical selecting-read.
I was 100% confusing these two, thanks for clearing it up.
To make sure I understand correctly, when using `completing-read` to 
read user input, that is narrowing,
while the `C-x <up>` and *Completions* buffer is completion. And from 
this thread I learned that Icicles can be used to enhance completion
similarly to how narrowing frameworks (like vertico) enhance narrowing.
>> My solution can leverage all of vertico and friend, and in general
>> provides an experience that's the same as other minibuffer prompts,
>> like M-x.
> But for someone like me who doesn't use a selecting-narrowing framework
> like vertico, it suddenly means that SPC is rebound to
> `minibuffer-complete-word' and entering new commands becomes *a lot* more
> cumbersome.
And this is where the user option for the function would come in useful.

[-- Attachment #2: Type: text/html, Size: 3019 bytes --]

Re: Add completion to compilation-read-command

[Philip Kaludercic]

Spyros Roum <spyros.roum@posteo.net> writes:

> Philip Kaludercic wrote:
>> Spyros Roum<spyros.roum@posteo.net> writes:
>>
>>> Juri Linkov wrote:
>>>>> I'm trying to get the compile prompt to suggest completion based on past
>>>>> commands I've run.
>>>> You can use 'C-x <up>' in the compile prompt to complete on past commands.
>>> This is indeed a lot closer to what I want, however it still lacks a
>>> lot of things compared to my solution.
>>>
>>> Unless there is some package/mode I don't know about, It's a lot less
>>> dynamic, for instance I can't keep typing to reduce
>>> possible items.
>> One should keep in mind that completion is not the same as narrowing.
>> The former expands substrings, while the latter is more like a system
>> for selecting an option from a list, that usually comes with some query
>> system.
>>                Confusing these two can lead to uncomfortable
>> edge-cases for both sides,
>> and sadly a lot of packages assume that completing-read is the same as a
>> hypothetical selecting-read.
> I was 100% confusing these two, thanks for clearing it up.

np.

> To make sure I understand correctly, when using `completing-read` to
> read user input, that is narrowing,
> while the `C-x <up>` and *Completions* buffer is completion. 

No, `completing-read' just invokes an interface, that frontends can
implement a UI for.  The default UI provides a completing/expanding
interface, while Vertico, Helm, Ivy, etc. do selecting/narrowing.  When
using `completing-read' you cannot really assume one or the other, so in
effect one has to find a middle ground.  It is best you try your code in
emacs -Q without any changes and see how it behaves.  Things like SPC
doing something else than you would expect is just one pitfall, others
I can recall are providing text that is difficult to input (I wrote a
package a few years back called "insert-kaomoji" that used
`completing-read' to prompt the user eastern-style emoticons; it is easy
to use with a selecting framework, but more inconvenient if the user is
first made to complete a string that is difficult to write, as most of
the characters are not easy to type).

>                                                              And from
> this thread I learned that Icicles can be used to enhance completion
> similarly to how narrowing frameworks (like vertico) enhance narrowing.

I have never used it, because it is not available as a package and it
redefines built-in functions, but my understanding it that it is
something similar.

>>> My solution can leverage all of vertico and friend, and in general
>>> provides an experience that's the same as other minibuffer prompts,
>>> like M-x.
>> But for someone like me who doesn't use a selecting-narrowing framework
>> like vertico, it suddenly means that SPC is rebound to
>> `minibuffer-complete-word' and entering new commands becomes *a lot* more
>> cumbersome.
> And this is where the user option for the function would come in useful.

Right, and I just wanted to second Juri's suggestion for the name.

Re: Add completion to compilation-read-command

[Spyros Roum]

[-- Attachment #1.1: Type: text/plain, Size: 1784 bytes --]

Philip Kaludercic wrote:
>> To make sure I understand correctly, when using `completing-read` to
>> read user input, that is narrowing,
>> while the `C-x <up>` and *Completions* buffer is completion.
> No, `completing-read' just invokes an interface, that frontends can
> implement a UI for.  The default UI provides a completing/expanding
> interface, while Vertico, Helm, Ivy, etc. do selecting/narrowing.  When
> using `completing-read' you cannot really assume one or the other, so in
> effect one has to find a middle ground.  It is best you try your code in
> emacs -Q without any changes and see how it behaves.  Things like SPC
> doing something else than you would expect is just one pitfall, others
> I can recall are providing text that is difficult to input (I wrote a
> package a few years back called "insert-kaomoji" that used
> `completing-read' to prompt the user eastern-style emoticons; it is easy
> to use with a selecting framework, but more inconvenient if the user is
> first made to complete a string that is difficult to write, as most of
> the characters are not easy to type).

I tried the attached patch with `emacs -q` and now I understand how this 
would be very annoying without something like vertico installed haha. 
Nevertheless, it seems to work!

I'm attaching a patch based on the conversation so far to get feedback on.

I believe I should also write something to the NEWS file.

Also, I'm wondering if this counts as non-trivial enough that I would 
need to sign the FSF copyright.

Regarding testing, as far as I can tell, there are currently no tests 
for `compilation-read-command`.
Should I add anything? If yes, I'll probably need someone to point to 
existing tests for similar things that I can copy from.

Am I forgetting anything else?

[-- Attachment #1.2: Type: text/html, Size: 2502 bytes --]

[-- Attachment #2: 0001-Add-option-for-making-compilation-read-command-use-c.patch --]
[-- Type: text/x-patch, Size: 2018 bytes --]

From e1068206662913978d541f924205a0615f8d2d95 Mon Sep 17 00:00:00 2001
From: Spyros Roum <spyros.roum@posteo.net>
Date: Wed, 25 Dec 2024 17:32:31 +0200
Subject: [PATCH] Add option for making compilation-read-command use
 completing-read

---
 lisp/progmodes/compile.el | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index 6784a12fd63..a318937e96d 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -1797,12 +1797,34 @@ compilation-mode-font-lock-keywords
    '((compilation--ensure-parse))
    compilation-mode-font-lock-keywords))
 
-(defun compilation-read-command (command)
+(defun compilation-prompt-read-shell-command (command)
   (read-shell-command "Compile command: " command
                       (if (equal (car compile-history) command)
                           '(compile-history . 1)
                         'compile-history)))
 
+(defun compilation-prompt-read-command-with-completion (command)
+  (completing-read "Compile command: " compile-history
+                   nil nil command
+                   (if (equal (car compile-history) command)
+                       '(compile-history . 1)
+                     'compile-history)))
+
+(defcustom compilation-read-command-function
+  'compilation-prompt-read-shell-command
+  "Function used by `compilation-read-command' to get user's input.
+Defaults to `compilation-prompt-read-shell-command',
+but `compilation-prompt-read-command-with-completion' can be used instead for
+a completing version based on past runs."
+  :version "31.0"
+  :type 'function
+  :options '(compilation-prompt-read-command-with-completion))
+
+(defun compilation-read-command (command)
+  "Prompt user for command to run.
+`compilation-read-command-function' controls the way input is read from the minibuffer."
+  (funcall compilation-read-command-function command))
+
 \f
 ;;;###autoload
 (defun compile (command &optional comint)
-- 
2.47.1

Re: Add completion to compilation-read-command

[Philip Kaludercic]

Spyros Roum <spyros.roum@posteo.net> writes:

> Philip Kaludercic wrote:
>>> To make sure I understand correctly, when using `completing-read` to
>>> read user input, that is narrowing,
>>> while the `C-x <up>` and *Completions* buffer is completion.
>> No, `completing-read' just invokes an interface, that frontends can
>> implement a UI for.  The default UI provides a completing/expanding
>> interface, while Vertico, Helm, Ivy, etc. do selecting/narrowing.  When
>> using `completing-read' you cannot really assume one or the other, so in
>> effect one has to find a middle ground.  It is best you try your code in
>> emacs -Q without any changes and see how it behaves.  Things like SPC
>> doing something else than you would expect is just one pitfall, others
>> I can recall are providing text that is difficult to input (I wrote a
>> package a few years back called "insert-kaomoji" that used
>> `completing-read' to prompt the user eastern-style emoticons; it is easy
>> to use with a selecting framework, but more inconvenient if the user is
>> first made to complete a string that is difficult to write, as most of
>> the characters are not easy to type).
>
> I tried the attached patch with `emacs -q` and now I understand how
> this would be very annoying without something like vertico installed
> haha. Nevertheless, it seems to work!

Of course it will work, after all it is just text (input) with some
convenient bindings for easy completion.  Even if SPC is bound to
something else, you can always use C-q SPC.

> I'm attaching a patch based on the conversation so far to get feedback on.
>
> I believe I should also write something to the NEWS file.

Right, you can always check the CONTRIBUTE file in the emacs.git root.

> Also, I'm wondering if this counts as non-trivial enough that I would
> need to sign the FSF copyright.

It looks like it to me, and either way it is not bad to do that so that
you have it behind you for future contributions either to Emacs or ELPA.

> Regarding testing, as far as I can tell, there are currently no tests
> for `compilation-read-command`.
> Should I add anything? If yes, I'll probably need someone to point to
> existing tests for similar things that I can copy from.

You could take a look at test/lisp/progmodes/compile-tests.el, but I
don't think this breaks the patch, especially when dealing with
something as interactive as what you are proposing.

>
> Am I forgetting anything else?
> From e1068206662913978d541f924205a0615f8d2d95 Mon Sep 17 00:00:00 2001
> From: Spyros Roum <spyros.roum@posteo.net>
> Date: Wed, 25 Dec 2024 17:32:31 +0200
> Subject: [PATCH] Add option for making compilation-read-command use
>  completing-read

A ChageLog style commit message would be expected here as well.  You can
inject the structure in vc-mode using C-c C-w.

>
> ---
>  lisp/progmodes/compile.el | 24 +++++++++++++++++++++++-
>  1 file changed, 23 insertions(+), 1 deletion(-)
>
> diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
> index 6784a12fd63..a318937e96d 100644
> --- a/lisp/progmodes/compile.el
> +++ b/lisp/progmodes/compile.el
> @@ -1797,12 +1797,34 @@ compilation-mode-font-lock-keywords
>     '((compilation--ensure-parse))
>     compilation-mode-font-lock-keywords))
>  
> -(defun compilation-read-command (command)
> +(defun compilation-prompt-read-shell-command (command)
>    (read-shell-command "Compile command: " command
>                        (if (equal (car compile-history) command)
>                            '(compile-history . 1)
>                          'compile-history)))
>  
> +(defun compilation-prompt-read-command-with-completion (command)
> +  (completing-read "Compile command: " compile-history
> +                   nil nil command
> +                   (if (equal (car compile-history) command)
> +                       '(compile-history . 1)
> +                     'compile-history)))
> +
> +(defcustom compilation-read-command-function
> +  'compilation-prompt-read-shell-command

Please sharp-quote (#').

> +  "Function used by `compilation-read-command' to get user's input.
> +Defaults to `compilation-prompt-read-shell-command',
> +but `compilation-prompt-read-command-with-completion' can be used instead for

If possible, avoid the passive phrase here.

> +a completing version based on past runs."
> +  :version "31.0"
> +  :type 'function
> +  :options '(compilation-prompt-read-command-with-completion))

It would also be good to add the default value here.  I prefer using
(list ...) and sharp-quoting all the function names, so that the
byte-compiler verifies that these are all known.

> +
> +(defun compilation-read-command (command)
> +  "Prompt user for command to run.
> +`compilation-read-command-function' controls the way input is read from the minibuffer."
> +  (funcall compilation-read-command-function command))
> +
>  \f
>  ;;;###autoload
>  (defun compile (command &optional comint)

Re: Add completion to compilation-read-command

[Juri Linkov]

> -(defun compilation-read-command (command)
> +(defun compilation-prompt-read-shell-command (command)
>    (read-shell-command "Compile command: " command
>                        (if (equal (car compile-history) command)
>                            '(compile-history . 1)
>                          'compile-history)))
>  
> +(defun compilation-prompt-read-command-with-completion (command)
> +  (completing-read "Compile command: " compile-history
> +                   nil nil command
> +                   (if (equal (car compile-history) command)
> +                       '(compile-history . 1)
> +                     'compile-history)))

Thanks.  The only problem I see is that the function name
doesn't indicate that it's completing on compile-history.
Probably a better name would be:

  compilation-read-command-with-history-completion

Re: Add completion to compilation-read-command

[Spyros Roum]

[-- Attachment #1: Type: text/plain, Size: 1410 bytes --]

Juri Linkov wrote:
>> -(defun compilation-read-command (command)
>> +(defun compilation-prompt-read-shell-command (command)
>>     (read-shell-command "Compile command: " command
>>                         (if (equal (car compile-history) command)
>>                             '(compile-history . 1)
>>                           'compile-history)))
>>   
>> +(defun compilation-prompt-read-command-with-completion (command)
>> +  (completing-read "Compile command: " compile-history
>> +                   nil nil command
>> +                   (if (equal (car compile-history) command)
>> +                       '(compile-history . 1)
>> +                     'compile-history)))
> Thanks.  The only problem I see is that the function name
> doesn't indicate that it's completing on compile-history.
> Probably a better name would be:
>
>    compilation-read-command-with-history-completion

I see what you are saying, however, I do like the current 
`compilation-prompt` prefix on
these commands, as it distinguishes them from `compilation-read-command`.

If I were to keep both the `-prompt` prefix, and the 
`history-completion` suffix,
I worry that the command name would be too long at 55 chars.

Instead, I suggest the name 
`compilation-prompt-read-with-history-completion`

Effectively dropping the word `command`. I think `compilation-prompt` is 
enough and `command` doesn't add
any information.

[-- Attachment #2: Type: text/html, Size: 1986 bytes --]

RE: [External] : Re: Add completion to compilation-read-command

[Drew Adams]

1. I wasn't going to reply to the statements made 
about "completion" versus "narrowing", "selection", 
etc.  I think such a characterization isn't very 
useful, and it can be misleading.

Since I'm now replying (below), about Icicles, I'll
first say this about completion/narrowing/selection:

Emacs "completion" has always meant matching a (user)
pattern against some set of things, with the pattern
typically provided as text in the minibuffer or some 
other buffer.

IOW, completion is about _matching_, first & foremost.

Completion has also typically involved expanding or
replacing the pattern, based on the matches.  E.g.,
you type "forw", hit TAB, and your input pattern
might change to "forward-".

(In Icicles, such pattern expansion is optional - you
can toggle it on/off anytime during completion.)

The words "narrowing" and "selection" are less about
a mechanism, which "completion" speaks to, and more
about possible _uses_ of a completion mechanism.

I'm guessing that those who distinguished such things
from completion had a limited view of its possible
uses: limited to just obtaining a (single) completion
of a pattern - much as if the only use of `grep' were
to show all matches for a pattern and then let you
pick/access/use only one of them.

If that's how "completion" is viewed, then yes, 
"narrowing" and "selection" seem like very different 
things from completion.  That's not the way I view 
completion.

Of course, we use `grep' in many other ways, including
(1) seeing what all the matches are, and where they're
located (discovery etc.), and (2) piping `grep' output
to further `grep' patterns or other transformation
functions.

The uses of completion are really as general as those
of pattern matching.  Completion isn't at all limited
to obtaining a single completion.

The idea behind "narrowing" (it's really _progressive_
narrowing, since just a single act of completion
already narrows the original domain of candidates to
only those that match the initial pattern) is that
you can _chain together_ acts of completion/matching,
as you do when you pipe `grep' output through another
`grep' (or another kind of filter or a transformer),
etc. 

IOW, you can match one pattern, getting a set of
matches, and then filter those matches (as a new
domain of candidates) with another pattern, and so on.

This is different from just changing the input
pattern and recompleting, which just starts over again
with the original domain of candidates.

Of course, even such plain recompletion isn't limited
to a get-me-a-completion use case.  Like progressive
completion, it acts on a given _set_ of elements to
produce a new _set_.  Completion, in general, is an 
operation on sets: given a pattern and a set of
candidates, it returns a new set of candidates, which
match the pattern.

[ More generally, you can think "map"/"reduce": Besides
pattern-matching with filter functions, _transforming_
functions can be mapped across a set of candidates.
And an aggregation ("reduce") function can be applied
to a set of candidates together.  Dunno whether other
so-called "completion frameworks" offer this, but
Icicles does. ]

> > from this thread I learned that Icicles can be used 
> > to enhance completion similarly to how narrowing 
> > frameworks (like vertico) enhance narrowing.

It's the other way around.  Icicles did that for
decades before there were any other "narrowing
frameworks" or "completion frameworks".  And it's
not about "enhancing narrowing".  It's simply about 
ways of using (taking advantage of) completion
/ matching.

> I have never used it, because it is not available as 
> a package and it redefines built-in functions, but
> my understanding it that it is something similar.

2, I don't care that you've never used it, or why.
But for the record:

* It is a package.  It was on MELPA, till MELPA stopped
  pulling code from Emacs Wiki.  My other libraries 
  were also on MELPA.

* Icicles only redefines some predefined functions, and
  only while `icy-mode' is enabled (on).  When off, the
  original definitions are restored.

3. I'm no expert on the many "completion frameworks"
that've been added since Icicles.  But I have a hunch 
that none of them have something like the progressive 
completion of Icicles:

When you initiate a new completion / matching
/ filtering with a new pattern, you can do so in a
recursive minibuffer.  The previous set of matches
(completion candidates) then becomes the domain of
candidates for the current act of completing.

Using recursive minibuffers means, among other things, 
that you can return to any higher level and continue 
from there, branching down another trail of levels,
and so on.  In effect, there's a tree of possibilities.  
(You can also pop back to the top level directly 
anytime.)

> >> But for someone like me who doesn't use a
> >> selecting-narrowing framework like vertico, it
> >> suddenly means that SPC is rebound to
> >> `minibuffer-complete-word' and entering new
> >> commands becomes *a lot* more cumbersome.

4. On this I'll just say that I argued long ago for
Emacs to _not_ bind SPC in minibuffer keymaps to word 
completion.

To me, that's just an artifact / legacy left over from 
the fact that Emacs used completion pretty much only
for `M-x' and reading file names, and Emacs commands 
and file names rarely, if ever, contained SPC chars 
(esp. back then).

Completion being potentially usable for matching
against _any_ set of candidates, it makes sense in the 
vast majority of contexts (including potential) for
SPC to be self-inserting.

(Of course, in some contexts you might want using SPC 
in the minibuffer to do something besides adding a SPC 
char to your pattern to be matched.)

I argued the same for `?' and `C-j' (newline): by 
default they, like SPC, should self-insert in the 
minibuffer.  Eventually, after decades, SPC finally 
became self-inserting, but only for file-name
completion.

(I'd be willing to bet that other  "completion 
frameworks", like Icicles, let you insert such chars 
without using `C-q' etc.  I'm not betting on vanilla 
Emacs doing so anytime soon.)

Re: Add completion to compilation-read-command

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: philipk@posteo.net,  emacs-devel@gnu.org
> Date: Tue, 24 Dec 2024 19:03:50 +0200
> 
> > I'm trying to get the compile prompt to suggest completion based on past
> > commands I've run.
> 
> You can use 'C-x <up>' in the compile prompt to complete on past commands.

Why the heck is this useful command completely undocumented??