Re: Updating *Completions* as you type

[Juri Linkov <juri@linkov.net>]

>>>>> - Again, the user is still able to configure the display-sort-function
>>>>>   by configuring the individual completion table.
>>>>
>>>> Does this mean that every individual completion table should have
>>>> a separate user option?
>>>
>>> No: only the completion tables which specify a display-sort-function in
>>> their metadata.  All such completion tables should have a user option to
>>> configure that display-sort-function.
>>
>> How then users could change the sorting order for individual tables
>> that don't specify a display-sort-function to use an order different
>> from completions-sort?
>
> They can use the category if the table specifies one.
>
> If the table neither specifies a category nor provides a table-specific
> option, the display sort function for that table isn't currently
> configurable.  Which I think we're both fine with?

I think we should gradually add a category to most completion tables
to make them completely configurable, not just with display-sort-function,
but with all possible metadata.

Adding a category resembles a long-lasting and still ongoing process of
adding specific minibuffer history as a symbol to the HIST argument
of different calls of read-from-minibuffer.

> So: we already accept that for some completion tables, it won't be
> possible to customize their display sort function out of the box.
>
>>> Well, yes.  So then we agree that a user option for an individual
>>> completion table, if it exists, should take precedence over
>>> completion-category-overrides?
>>
>> The problem is that we can't distinguish two cases:
>>
>> 1. when display-sort-function is hard-coded in metadata
>>    by the author of the completion table;
>> 2. when display-sort-function in metadata
>>    gets the value from the user option.
>
> I think we should just eliminate any instances of case 1.

I don't think this is realistic to add an individual option in all cases.

> Case 1 just means the completion table's display sort function isn't
> currently configurable.  Which I think we've already accepted will be
> the case for some tables, until we go and make them configurable either
> by adding a table-specific option, adding a category, or both.
>
>> Since we can't distinguish these cases, then it makes more sense
>> when completion-category-overrides overrides everything:
>>
>>   (alist-get 'display-sort-function (alist-get category completion-category-overrides))
>>   (alist-get 'display-sort-function metadata) ;; metadata with/out individual options
>>   (alist-get 'display-sort-function (alist-get category completion-category-defaults))
>>
>> There is no problem with this because completion-category-overrides
>> is a user option as well, so everything still is under user control.
>
> Only if the completion table specifies a category.  Which most do not.
>
> So we'd need to change it to specify a category.

Agreed.

> And if we're doing that, we could also change it to not hard-code
> a display-sort-function at the same time.

Disagreed when this means to add an option in all cases.

> That is, for any tables where the display-sort-function is currently
> hardcoded, we can add a category, and remove the hardcoded
> display-sort-function from the table metadata, and add the
> display-sort-function to completion-category-defaults.
>
>>> So then we're only disagreeing over whether such options should exist?
>>
>> Yes, I think we should add individual options only in exceptional cases.
>>
>>> These individual options would also provide a natural place to document
>>> behavior like "if you configure the display-sort-function for buffer
>>> completion to 'identity, then the buffer sort order will match
>>> (buffer-list)".  But the user could still make use of that information
>>> by configuring the category.
>>
>> I agree that an option with documentation could help in such cases
>> when a non-trivial sorting function is provided for a completion table.
>>
>>>> I see no need to add individual options as all.  Every completion table
>>>> that significantly differs from other tables so that it needs a separate
>>>> display-sort-function, could provide a separate category.  For example,
>>>> there is a category 'buffer'.  If 'switch-to-buffer' needs another
>>>> display-sort-function it could provide a category 'buffer-for-switching'.
>>>
>>> That won't work with the scenario I described before with sorting
>>> file-name completion by mtime, where changing the sorting requires also
>>> changing the completion table.
>>
>> I agree that individual options are required in such rare cases when
>> their values affect the completion table and its formatting.
>
> OK, I think I can agree with that, if we agree that in those rare cases,
> the individual options should take precedence over the category-based
> configuration.

The individual options can't take precedence until all tables don't
hardcode metadata that is hardly achievable.

>>> Also, this would require adding a category for essentially every
>>> completion table.  For example, I see that read-from-kill-ring specifies
>>> a display-sort-function, currently set to 'identity.
>>
>> It's much simpler to add an extra line with a category.
>>
>>> If we wanted to make that configurable, it seems much easier to just do
>>>
>>>           (if (eq action 'metadata)
>>>               ;; Keep sorted by recency
>>> -             '(metadata (display-sort-function . identity))
>>> +             `(metadata (display-sort-function . ,read-from-kill-ring-sort))
>>>             (complete-with-action action completions string pred)))
>>
>> This is an incomplete patch, there should be also a dozen of lines
>> with defcustom, its docstring, the version number and a list
>> of choices, etc.  And all this for a very small percent of users
>> who would like to change this order.  This is too wasteful.
>> It would be much more efficient to allow doing the same
>> by customizing completion-category-overrides.
>
> The docstring and list of choices for read-from-kill-ring-sort are
> something we want anyway - we would like to document that 'identity for
> read-for-kill-ring keeps the kill ring sorted by recency, for example.
> I see no better place to document that.
>
> The version number is also something we want anyway: if we just add a
> category to read-from-kill-ring in Emacs 30, this will work only in
> Emacs 30 and not in Emacs 29, and there's no way for a user to know that
> other than by reading NEWS.

I don't think such unnecessary defcustoms should be added lightly,
even documentation is of no help for such obvious things as 'identity'
that intuitively is understandable as keeping the original order.

> For such tables, I see three good possibilities (in order of my own
> preference):
>
> A.
> - Add read-from-kill-ring-sort defaulting to identity (with docstring)
> (diff is 1 line + defcustom)
>
> B.
> - Add read-from-kill-ring-sort defaulting to nil (with docstring)
> - add the 'read-kill category to the metadata
> - add 'read-kill to completion-category-defaults
> (diff is 3 lines + defcustom)
>
> C.
> - Remove display-sort-function from the metadata
> - add the 'read-kill category to the metadata
> - add 'read-kill to completion-category-defaults
> (diff is 3 lines)
>
> If you really don't want the defcustom and associated documentation, I'm
> OK with C.
>
> The option which I think is not good is:
>
> D.
> - add the 'read-kill category to the metadata
> - make completion-category-overrides take precedence over what is
>   specified in the table metadata
>
> (diff is 1 line)
>
> This is a slightly smaller diff than option C, but I think it's a
> fundamentally worse approach than C, because in the rare cases where we
> do want an individual option for the table, we won't have a way for that
> option to take precedence over the broader category-based configuration.

I still don't understand why do you worry about this precedence when
the user option completion-category-overrides is nil by default.

Could you describe a use cases when such precedence might become a problem?