bug#56682: Fix the long lines font locking related slowdowns

[Dmitry Gutov <dgutov@yandex.ru>]

On 07.08.2022 03:11, Gregory Heytings wrote:
> 
>>>> I'm not seeing any particular sluggishness in these operations when 
>>>> visiting dictionary.json.
>>>
>>> Numbers, please.  You have a very fast machine, so what doesn't look 
>>> sluggish on your system could very well be so on others.
>>
>> How do you measure these operations including the redisplay lag?
>>
>> Anyway, all of these look instant.
>>
> 
> They aren't, no.  Of course it all depends on where you are in the file, 
> on your CPU, and so forth.  For me (with locked narrowing disabled, in 
> dictionary.json, with emacs -Q) M-> takes about 3 seconds.

And it's about 1.5s here. Once.

> Likewise, if 
> I do for example C-s aan, and then repeat C-s, whenever the next match 
> is far enough in the buffer I see a delay of about 2 seconds.

Sounds like a performance problem inside isearch. I might take a look.

It shouldn't force font-lock over a big region, and if syntax-ppss cache 
is already filled, there is no other obvious place for that delay to 
come from.

> Another 
> test you can do is to lean on C-v after opening the buffer, you'll see 
> that Emacs becomes very sluggish, sometimes I have to wait more than 5 
> seconds to avance from one screenfull.

First of all, these aren't the tests Eli asked for or I performed. Why 
don't you compare apples to oranges?

Leaning on C-v isn't a scientific test anyway: it compares the speed of 
two processes internal to Emacs (event handling and rendering), rather 
than something objective. And waiting for X seconds means you wait for 
however many screens you managed to page through, for all of them to get 
syntax-highlighted. That is indeed a way to break the "jit" in 
"jit-lock", but how is that a useful test?

Anyway, could you please try the combined patch I posted recently 
together with dictionary.json?

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=56682#1084

>> Correct syntax highlighting requires parsing the buffer from the 
>> beginning. Otherwise we get random results, essentially.
>>
> 
> It all depends how you define "correct".  You may remind that I 
> suggested to introduce some heuristics in the algorithm.

Stefan mentioned such possibility as well.

> With 
> appropriate heuristics, you could probably get reasonably good results 
> even if you only have access to a small portion of the buffer.  For 
> example, instead of counting the number of '"' characters from BOB to 
> know for sure if you are in a string, you mark those that are likely at 
> the beginning of a string and those that are likely at the end of a 
> string, and you select the most likely possibility among all 
> combinations to highlight the buffer portion as well as possible.

Any such heuristics are possible, but major modes are the best place to 
make that decision. And until the major mode has installed something 
like this, syntax-ppss should default to the "correct" behavior. Which 
means operating outside of narrowing.