bug#47711: bug#48841: bug#47711: [PATCH VERSION 2] Add new `completion-filter-completions` API and deferred highlighting

[Dmitry Gutov <dgutov@yandex.ru>]

On 14.08.2021 10:12, Eli Zaretskii wrote:
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Sat, 14 Aug 2021 05:47:43 +0300
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, 48841@debbugs.gnu.org,
>>   47711@debbugs.gnu.org
>>
>> I thought I explained the problem with this previously.
>>
>> It's basically this: we cannot mutate what we don't own. Across all of
>> completion functions out there, there will be such that return "shared"
>> strings (meaning, not copied or newly allocated) from their completion
>> tables. And modifying them is bad, with consequences which can present
>> themselves in unexpected, often subtle ways.
>>
>> Since up until now completion-pcm--hilit-commonality copied all strings
>> before modifying, completion tables such as described (with "shared"
>> strings) have all been "legal". Suddenly deciding to stop supporting
>> them would be a major API breakage with consequences that are hard to
>> predict. And while I perhaps agree that it's an inconvenience, I don't
>> think it's a choice we can simply make as this stage in c-a-p-f's
>> development.
> 
> This sounds like an argument against Daniel's approach as well, no?
> Because if a completion API returns strings it "doesn't own", there
> will be restrictions on Lisp programs that use those strings, because
> those Lisp programs previously could do anything they liked with those
> strings, and now they cannot.  Or am I missing something?

Good question. It is not.

The completion tables described above have all been doing "legal" 
things, in our general understanding.

But any callers of completion-all-completions were never really allowed 
to modify the returned strings (those still were strings that code 
"doesn't own").

Of course, some of those callers (I don't know any, though) might have 
taken advantage of being able to modify the strings with impunity 
because of completion-all-completions' implementation detail, but 
they'll have a chance to clean up their act when switching to 
completion-filter-completions.

>>     1. (setq s (symbol-name 'car))
>>
>>     2. (put-text-property 1 3 'face 'error s)
>>
>>     3. Switch to a buffer in fundamental mode
>>
>>     4. (insert (symbol-name 'car)) --> see the error face in the buffer
>>
>> Now imagine that some completion table collects symbol names by passing
>> obarray through #'symbol-name rather than #'all-completions, and voila,
>> if the completion machinery adds properties (any properties, not just
>> face) to those strings, you have just modified a bunch of global values.
>> That's not good.
> 
> How is this different from Daniel's proposal of returning the original
> strings?  AFAIU, he just shifts the responsibility from the completion
> code to the caller of the completion code, but basically leaves the
> problem still very much real and pretty much into our face.

This is a shift of responsibility in the right direction. The callers 
might as well do the string copying when needed, but the fact of the 
matter is, they usually only need to "copy" 20-100 strings (or however 
many is displayed), if they need to modify them at all. That's where we 
win: copying 100 strings is better than copying 10000.

Gotta run now, will reply to other email later.