bug#56682: Fix the long lines font locking related slowdowns

[Eli Zaretskii <eliz@gnu.org>]

> Date: Fri, 5 Aug 2022 21:02:35 +0300
> Cc: 56682@debbugs.gnu.org, gregory@heytings.org, monnier@iro.umontreal.ca
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> > How is (parse-partial-sexp 1 (point-max)) related to the issue at
> > hand?
> 
> Or perhaps I should answer this way:
> 
> We move to near EOB.
> fontification-functions are called.
> 
> jit-lock
> calls
> (font-lock-fontify-region point-near-buffer-end (point-max))
> which calls
> font-lock-fontify-syntactically-region
> which calls both
>    (syntax-propertize (point-max))
>    and
>    (syntax-ppss point-near-buffer-end) -> and it calls parse-partial-sexp
> 
> syntax-propertize will also likely call syntax-ppss itself, probably 
> through the major mode's syntax-propertize-function. But if 
> syntax-propertize-function is nil, parse-partial-sexp gets called 
> anyway, over the whole buffer, which makes it the main workload in 
> fontifying near EOB.
> 
> Now, if syntax-propertize-function is non-nil, parse-partial-sexp will 
> also call it, and it adds its overhead (sometimes a multiple of p-p-s), 
> which also scales linearly with the length of the buffer.
> 
> So if one can demonstrate that (parse-partial-sexp (point-min) 
> (point-max)) takes about the same time as it takes to fontify the last 
> screen-ful of a buffer, then that says that everything else that 
> jit-lock does to fontify, is negligible, time-wise.

So you have demonstrated that, if visiting a file and moving inside it
calls parse-partial-sexp to scan the entire buffer, then this could be
some, perhaps a large, part of the slowdown.

First, we need to establish that indeed parse-partial-sexp is called
in that manner in the relevant major modes (not just one of them), or
by font-lock itself regardless of the mode.

Second, we need to establish that indeed this takes a large portion of
the time in the slow operations.  Not just one particular operation,
but most or all of them.

And after that, we may have some food for thought.