bug#56682: Fix the long lines font locking related slowdowns

[Gregory Heytings <gregory@heytings.org>]

>>> I'm not seeing any particular sluggishness in these operations when 
>>> visiting dictionary.json.
>> 
>> Numbers, please.  You have a very fast machine, so what doesn't look 
>> sluggish on your system could very well be so on others.
>
> How do you measure these operations including the redisplay lag?
>
> Anyway, all of these look instant.
>

They aren't, no.  Of course it all depends on where you are in the file, 
on your CPU, and so forth.  For me (with locked narrowing disabled, in 
dictionary.json, with emacs -Q) M-> takes about 3 seconds.  Likewise, if I 
do for example C-s aan, and then repeat C-s, whenever the next match is 
far enough in the buffer I see a delay of about 2 seconds.  Another test 
you can do is to lean on C-v after opening the buffer, you'll see that 
Emacs becomes very sluggish, sometimes I have to wait more than 5 seconds 
to avance from one screenfull.

>
> Correct syntax highlighting requires parsing the buffer from the 
> beginning. Otherwise we get random results, essentially.
>

It all depends how you define "correct".  You may remind that I suggested 
to introduce some heuristics in the algorithm.  With appropriate 
heuristics, you could probably get reasonably good results even if you 
only have access to a small portion of the buffer.  For example, instead 
of counting the number of '"' characters from BOB to know for sure if you 
are in a string, you mark those that are likely at the beginning of a 
string and those that are likely at the end of a string, and you select 
the most likely possibility among all combinations to highlight the buffer 
portion as well as possible.