bug#56682: Fix the long lines font locking related slowdowns

[Eli Zaretskii <eliz@gnu.org>]

> Date: Tue, 2 Aug 2022 17:29:47 +0300
> Cc: monnier@iro.umontreal.ca, 56682@debbugs.gnu.org, gregory@heytings.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> > IOW, in this case the Emacs developers, due to long-standing bug
> > reports about this situation, recognized that it_can_  be improved,
> > albeit in slightly unorthodox ways, and have taken the measures to
> > optimize Emacs for the users.
> 
> It would be a shame to have the better-behaving (faster) major modes 
> exhibit worse behavior that they could have because of the approach we 
> choose to solve the long-lines problem.

Like I said: I'm okay with extending 'widen' to allow it to optionally
break the lock on the narrowing.  Then those modes which will indeed
show reasonable performance in these cases can use that optional
behavior.  Otherwise, I think we've waited for such improvements in
the modes long enough.

> Regarding the long-standing bug reports, we did solve a bunch of issues 
> already. One major one, IIUC, was redisplay of already fontified text on 
> long lines. Another piece of the puzzle was added by Stefan in 
> 15b2138719b340.

I invite you and Stefan to show that the improvement is performant
enough in the cases we are using as examples of long lines.

> So perhaps we should re-evaluate the testing scenario to see where the 
> current bottlenecks are. If we current main issue is the 55s spent in 
> syntax-ppss, a more constructive approach would be to look into 
> optimizing parse-partial-sexp. Or even give up on certain scenarios, 
> admitting that waiting 55s once to visit the end of a 1 GB buffer is not 
> so bad (and that could part could also be sped up by setting 
> syntax-propertize-function to nil and using a very simple syntax table, 
> for instance).

I reported the problem with syntax-propertize to Stefan 1.5 month ago,
see https://debbugs.gnu.org/cgi/bugreport.cgi?bug=45898#92.  There was
an improvement in that department, but AFAICT the net result is still
unsatisfactory if you disable the recent long-line optimizations
(e.g., by setting long-line-threshold to nil).

So, unless I'm mistaken, the same scenario described there is still
relevant, and doesn't yet need to be reevaluated.  But more data
points are always welcome, so feel free (you and everyone else) to
test the performance with and without this feature, on different files
with different major modes, and report back what you have found.  At
least I don't think the work on this is completed, we still have a lot
of turf to cover and probably deal with fallout we haven't yet heard
about.