bug#56682: Fix the long lines font locking related slowdowns

[Eli Zaretskii <eliz@gnu.org>]

> Date: Thu, 4 Aug 2022 03:49:43 +0300
> Cc: gerd.moellmann@gmail.com, 56682@debbugs.gnu.org,
>  Eli Zaretskii <eliz@gnu.org>, monnier@iro.umontreal.ca
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> > Now 
> > compare that with the feature branch, with which the end of the buffer 
> > is displayed instantaneously.  That five seconds delay is caused by 
> > fontification-functions.
> 
> At some point we should accept that visiting a huge file might take some 
> time (5 seconds doesn't sound terrible, depending on the context). 
> Because the alternative is mis-fontifications and broken display.
> 
> >> Could you clarify what you mean by "access ... to the place where ... 
> >> is defined"? "new Downloadify.Container" is highlighted by a regular 
> >> regexp matcher, not some custom elisp code which has to visit the 
> >> position where the identifier is defined.
> >>
> > 
> > Sorry, I cannot be more precise, I don't have the "downloadify.js" file 
> > here.  It was just a guess, based on what I saw on the screenshot, that 
> > one function called by fontification-functions collects all class 
> > definitions and highlights their identifiers elsewhere in the buffer 
> > with a specific face.  When the buffer is narrowed, that function may 
> > not see the Downloadify.Container definition (which is, I guess, placed 
> > near the beginning of the file) anymore.
> 
> Here I'm attaching a version of downloadify.js we can use for comparison 
> (please rename the extension from .sj to .js locally; Gmail was not 
> letting it through otherwise). It's not a huge file, just about 88K.
> 
> As long as I keep my Emacs window/frame width half of the desktop, I can 
> reliably reproduce the problem with the lack of highlighting for 
> "Downloadify.Container" while other tokens are still highlighted.
> 
> I'm also attaching a screenshot of another problem: suddenly the bottom 
> several screens of the buffer are mis-highlighted as if starting inside 
> a string. That very much look like a result of breaking syntax-ppss's 
> visibility of the buffer.
> 
> So the buffer scrolls quickly but looks bad.

I don't understand the point you are trying to make.

On the one hand, you say "At some point we should accept that visiting
a huge file might take some time", which seems to imply that you agree
in general with some (hopefully graceful) degradation when editing
files with such long lines.  But OTOH you object to have that
degradation in the fontification?  IOW, you prefer Emacs to become
much slower, but still fontify correctly?  If so, just enlarge the
value of long-line-threshold, with the effect that Emacs will become
more sluggish before the long-line optimizations kick in.  If this is
your point, then maybe lobby for enlarging the default value of that
variable.

But if you are saying that Emacs should behave as it does with that
variable being nil, then I don't understand your position.  With that
variable nil, Emacs becomes _unusable_, not just slow, with files that
are not even too large by any modern measure, just because the lines
are very long.  And in those cases, why is it wrong to decide that
occasional glitches in fontifications are a lesser evil than a
complete lockup of the Emacs UI, which usually results in users
killing the session?  We decided that some imperfection in
fontifications are the "graceful degradation" we are willing to endure
in order to make Emacs reasonably performant in those cases.  What is
wrong with that tradeoff?  And what alternative tradeoff would you
suggest instead?

(You mention "broken display" in addition to inaccurate
fontifications, but I don't understand what does that allude to.
Which instances of broken display did you see, and how to reproduce
them?)