bug#48581: 27.2; Default value of lazy-highlight-buffer-max-at-a-time is too low

bug#48581: 27.2; Default value of lazy-highlight-buffer-max-at-a-time is too low

[Augusto Stoffel]

The value of lazy-highlight-buffer-max-at-a-time determines how long
it takes to finish computing the isearch lazy count.  The current
default value of 20 seems suboptimal.

I made a simple experiment measuring the (real) time to count the
~15000 matches of the string "e" in the file isearch.el, with the
following results:

lazy-highlight-buffer-max-at-a-time | time to finish counting
20 (current setting)                | 1.5 s
50                                  | 0.8 s
100                                 | 0.6 s
200                                 | 0.5 s
nil (do it all at once)             | 0.4 s

Based on this, I would like to suggest changing the default to 200, or
something in that order of magnitude.

The downside of this change would be an increase in the time Emacs is
unresponsive doing lazy counting/highlighting.  However, this time
remains below a few milliseconds in a typical case, and on the other
hand a sufficiently complex regexp on a sufficiently large buffer can
hang isearch even with the current default settings.

bug#48581: 27.2; Default value of lazy-highlight-buffer-max-at-a-time is too low

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Date: Sat, 22 May 2021 11:25:44 +0200
> 
> The value of lazy-highlight-buffer-max-at-a-time determines how long
> it takes to finish computing the isearch lazy count.  The current
> default value of 20 seems suboptimal.
> 
> I made a simple experiment measuring the (real) time to count the
> ~15000 matches of the string "e" in the file isearch.el, with the
> following results:
> 
> lazy-highlight-buffer-max-at-a-time | time to finish counting
> 20 (current setting)                | 1.5 s
> 50                                  | 0.8 s
> 100                                 | 0.6 s
> 200                                 | 0.5 s
> nil (do it all at once)             | 0.4 s
> 
> Based on this, I would like to suggest changing the default to 200, or
> something in that order of magnitude.

You assume that (a) the main purpose of Isearch is to count the
matches, and (b) that a case with 15,000 matches is the common one?

> The downside of this change would be an increase in the time Emacs is
> unresponsive doing lazy counting/highlighting.  However, this time
> remains below a few milliseconds in a typical case

What kind of CPU do you have there, and how many milliseconds does it
take Emacs to highlight 20 vs 200 matches?

bug#48581: 27.2; Default value of lazy-highlight-buffer-max-at-a-time is too low

[Augusto Stoffel]

On Sat, 22 May 2021 at 12:44, Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Augusto Stoffel <arstoffel@gmail.com>
>> Date: Sat, 22 May 2021 11:25:44 +0200
>> 
>> The value of lazy-highlight-buffer-max-at-a-time determines how long
>> it takes to finish computing the isearch lazy count.  The current
>> default value of 20 seems suboptimal.
>> 
>> I made a simple experiment measuring the (real) time to count the
>> ~15000 matches of the string "e" in the file isearch.el, with the
>> following results:
>> 
>> lazy-highlight-buffer-max-at-a-time | time to finish counting
>> 20 (current setting)                | 1.5 s
>> 50                                  | 0.8 s
>> 100                                 | 0.6 s
>> 200                                 | 0.5 s
>> nil (do it all at once)             | 0.4 s
>> 
>> Based on this, I would like to suggest changing the default to 200, or
>> something in that order of magnitude.
>
> You assume that (a) the main purpose of Isearch is to count the
> matches,

No, the main purpose of Isearch is to search for a string :-).
`isearch-lazy-highlight-buffer-update' has more than one purpose, but I
assume that finishing faster is always better than taking longer than
necessary.

> and (b) that a case with 15,000 matches is the common one?

No, that was just so I that the measured time is not too small.

However, the delay before the lazy count appears in the echo area during
day-to-day Isearch operation is noticeable to the naked eye.  If you
keep looking, you will notice it.

Note that the precise value of this parameter is irrelevant, only the
order of magnitude plays a significant role.  I don't think there's a
need to do a more formal benchmark to conclude 20 is to small and 2000
is probably too big.

>
>> The downside of this change would be an increase in the time Emacs is
>> unresponsive doing lazy counting/highlighting.  However, this time
>> remains below a few milliseconds in a typical case
>
> What kind of CPU do you have there,

A decent but not particularly powerful laptop from 2019.

> and how many milliseconds does it take Emacs to highlight 20 vs 200
> matches?

I didn't specifically compute that, but you can get an upper bound based
on the provided data: 2 ms for 20 matches and 6.7 ms for 200 matches.

bug#48581: 27.2; Default value of lazy-highlight-buffer-max-at-a-time is too low

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: 48581@debbugs.gnu.org
> Date: Sat, 22 May 2021 12:49:16 +0200
> 
> > You assume that (a) the main purpose of Isearch is to count the
> > matches,
> 
> No, the main purpose of Isearch is to search for a string :-).
> `isearch-lazy-highlight-buffer-update' has more than one purpose, but I
> assume that finishing faster is always better than taking longer than
> necessary.

In most real-life cases, at least in mine, you never need to look at
all the matches, only at the first few.

> However, the delay before the lazy count appears in the echo area during
> day-to-day Isearch operation is noticeable to the naked eye.  If you
> keep looking, you will notice it.

I know it's noticeable, and that directly contradicts your time
estimations, because human eyes cannot notice such short delays.  In
general, anything shorter than 50 ms will look instantaneous to us, at
least IME.

bug#48581: 27.2; Default value of lazy-highlight-buffer-max-at-a-time is too low

[Augusto Stoffel]

On Sat, 22 May 2021 at 14:08, Eli Zaretskii <eliz@gnu.org> wrote:

> In most real-life cases, at least in mine, you never need to look at
> all the matches, only at the first few.

That's right.  However, despite the name,
`lazy-highlight-buffer-max-at-a-time' also influences the time needed to
compute the number of matches in the buffer.  So, when you set
`isearch-lazy-count' to t, those numbers in the table of my first
message indicate the delay between typing a search string and seeing the
number of matches in the buffer (as well as the relative position of the
current match) in the echo area.

Perhaps you don't use `isearch-lazy-count', but if you do, you must have
noticed it's often a bit slow to update.

bug#48581: 27.2; Default value of lazy-highlight-buffer-max-at-a-time is too low

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: 48581@debbugs.gnu.org
> Date: Sat, 22 May 2021 14:17:25 +0200
> 
> On Sat, 22 May 2021 at 14:08, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > In most real-life cases, at least in mine, you never need to look at
> > all the matches, only at the first few.
> 
> That's right.  However, despite the name,
> `lazy-highlight-buffer-max-at-a-time' also influences the time needed to
> compute the number of matches in the buffer.  So, when you set
> `isearch-lazy-count' to t, those numbers in the table of my first
> message indicate the delay between typing a search string and seeing the
> number of matches in the buffer (as well as the relative position of the
> current match) in the echo area.

That was exactly my point: the variable you suggest changing affects
more than just one use case, and it affects them in ways that in a
sense contradict one another.

If we want to change the default value, we should find a way of doing
that without hurting "the other" use case.

bug#48581: 27.2; Default value of lazy-highlight-buffer-max-at-a-time is too low

[Augusto Stoffel]

On Sat, 22 May 2021 at 15:30, Eli Zaretskii <eliz@gnu.org> wrote:

> That was exactly my point: the variable you suggest changing affects
> more than just one use case, and it affects them in ways that in a
> sense contradict one another.
>
> If we want to change the default value, we should find a way of doing
> that without hurting "the other" use case.

What are the other uses cases, and in which way could they be hurt by
`isearch-lazy-highlight-buffer-update' being faster to finish?

Earlier you mentioned highlighting the visible matches.  Note that this
is controlled by a different variable, namely
`lazy-highlight-max-at-a-time' (which, not long ago, was also changed
from its old default of 20, albeit for a different reason).

bug#48581: 27.2; Default value of lazy-highlight-buffer-max-at-a-time is too low

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: 48581@debbugs.gnu.org
> Date: Sat, 22 May 2021 15:14:16 +0200
> 
> On Sat, 22 May 2021 at 15:30, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > That was exactly my point: the variable you suggest changing affects
> > more than just one use case, and it affects them in ways that in a
> > sense contradict one another.
> >
> > If we want to change the default value, we should find a way of doing
> > that without hurting "the other" use case.
> 
> What are the other uses cases, and in which way could they be hurt by
> `isearch-lazy-highlight-buffer-update' being faster to finish?

The most obvious case is to find the first match and act on that.
Highlighting too many other matches will delay the response to the
next command.

bug#48581: 27.2; Default value of lazy-highlight-buffer-max-at-a-time is too low

[Juri Linkov]

> lazy-highlight-buffer-max-at-a-time | time to finish counting
> 20 (current setting)                | 1.5 s
> 50                                  | 0.8 s
> 100                                 | 0.6 s
> 200                                 | 0.5 s
> nil (do it all at once)             | 0.4 s
>
> Based on this, I would like to suggest changing the default to 200, or
> something in that order of magnitude.

Thanks for the suggestion.  I tried to change it to 200, I got such times:

lazy-highlight-buffer-max-at-a-time | time to finish counting
20 (current setting)                | 7 s
200                                 | 2 s

I hope this change will be a big improvement for everyone,
because users will lose less time waiting it to finish,
while keeping isearch still responsive.

bug#48581: 27.2; Default value of lazy-highlight-buffer-max-at-a-time is too low

[Juri Linkov]

tags 48581 fixed
close 48581 28.0.50
thanks

>> lazy-highlight-buffer-max-at-a-time | time to finish counting
>> 20 (current setting)                | 1.5 s
>> 50                                  | 0.8 s
>> 100                                 | 0.6 s
>> 200                                 | 0.5 s
>> nil (do it all at once)             | 0.4 s
>>
>> Based on this, I would like to suggest changing the default to 200, or
>> something in that order of magnitude.
>
> Thanks for the suggestion.  I tried to change it to 200, I got such times:
>
> lazy-highlight-buffer-max-at-a-time | time to finish counting
> 20 (current setting)                | 7 s
> 200                                 | 2 s
>
> I hope this change will be a big improvement for everyone,
> because users will lose less time waiting it to finish,
> while keeping isearch still responsive.

Now tenfold increase was pushed to master.