bug#56682: Fix the long lines font locking related slowdowns

[Eli Zaretskii <eliz@gnu.org>]

> Date: Sun, 14 Aug 2022 20:47:40 +0300
> Cc: 56682@debbugs.gnu.org, gregory@heytings.org, monnier@iro.umontreal.ca
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> > The better way is to acknowledge that some inaccuracies are acceptable
> > in those cases.  With that in mind, one can design a syntax analyzer
> > that looks back only a short ways, until it finds some place that
> > could reasonably serve as an anchor point for heuristic decisions
> > about whether we are inside or outside a string or comment, and then
> > verifying that guess with some telltale syntactic elements that follow
> > (like semi-colons or comment-end delimiters in C).  While this kind of
> > heuristics can sometimes fail, if they only fail rarely, the result is
> > a huge win.
> 
> You cannot design a language-agnostic syntax analyzer like that.

_I_ cannot, but hopefully someone else will.

> > In any case, the way to speed up these cases is to look at the profile
> > and identify the code that is slowing us down; then attempt to make it
> > faster.  (20 sec is actually long enough for us to interrupt Emacs
> > under a debugger and look at the backtrace to find the culprit.)
> 
> I've profiled and benchmarked this scenario already: all of the delay 
> (17 seconds, to be precise) come from parse-partial-sexp. 1 GB is a lot.

Before we get to 1GB files, there are 20MB files and 250MB files.  I
found quite a few low-hanging fruit in those that are worth plucking,
while we wait for parse-partial-sexp to get its act together.

> > If that solves the problems in a reasonable way for very long lines,
> > maybe we will eventually have such an option.
> 
> Can I merge the branch, then?

Please wait until I have time to review it.

> I was hoping for a stylistic review, perhaps. Like, whether you like the 
> name of the variable, and should it be split in two.
> 
> A change of the default value(s) is on the table too.

Will definitely do, I'm just busy with "other things" right now, most
of them related to other aspects of long lines.

> > One such major mode and one such file was presented long ago : a
> > single-line XML file.
> 
> XMl is indeed slower. It takes almost 3 seconds for me to scroll to the 
> end of a 20 MB XML file.
> 
> Most of it comes from sgml--syntax-propertize-ppss, which is probably 
> justified: XML is a more complex language.

Did you wait till nxml-mode did its initial scan and displayed "Valid"
in the mode line?  The performance is quite different before and after
that.

> But other than the initial delay, scrolling, and isearch, and local 
> editing, all work fast, unlike the original situation with JSON.

With which branch?