bug#72826: 30.0.90; icomplete-in-buffer becomes unusably slow in large Eshell

[Sean Whitton <spwhitton@spwhitton.name>]

[CCing Juri and João who have worked on this.]

Hello,

On Tue 27 Aug 2024 at 03:43pm +03, Eli Zaretskii wrote:

>> From: Sean Whitton <spwhitton@spwhitton.name>
>> Date: Tue, 27 Aug 2024 15:26:03 +0800
>>
>> Hello,
>>
>> 1. emacs -q
>> 2. (setopt icomplete-in-buffer t)
>> 3. M-x icomplete-mode
>> 4. M-x eshell
>> 5. cat very_large_file.txt # I'm using an 80k line log file
>> 6. ls <TAB>
>> 7. Try to use C-, and C-. to cycle through the completions.
>>   => Emacs basically locks up.
>>
>> The buffer being large should surely not affect icomplete-in-buffer in
>> this way.  It shouldn't need to care about all the rest of the buffer.
>
> Strangely enough, I can only reproduce this on GNU/Linux, but not on
> MS-Windows.
>
> Profiling seems to indicate most of the time is spent in
> completion-all-sorted-completions and in GC.

I just tried profiling it myself:

       38958  93% - icomplete-post-command-hook
       38958  93%  - icomplete-exhibit
       38819  93%   - icomplete-completions
       38749  92%      string-width
          43   0%    + icomplete--sorted-completions
          27   0%      buffer-string
         139   0%   + sit-for

I think the culprit is probably the evaulation of
(string-width (buffer-string)) in icomplete-completions.
That'll always be fast for Icomplete in the minibuffer, but for
icomplete-in-buffer it depends on the size of the buffer.

I think this code is trying to determine how much space is taken up by
the minibuffer prompt and the user's input typed so far.  So, we can
probably find a cheaper way to compute the corresponding value for
icomplete-in-buffer.

-- 
Sean Whitton