bug#45898: 27.1; wedged in redisplay again

[Eli Zaretskii <eliz@gnu.org>]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: larsi@gnus.org,  psainty@orcon.net.nz,  Emacs-hacker2018@jovi.net,
>   45898@debbugs.gnu.org
> Date: Wed, 22 Jun 2022 19:39:14 -0400
> 
> > Isn't there a way to limit what font-lock considers a "line" such that
> > it doesn't consider more than some number N of characters?  What could
> > potentially happen if we set N to, like, 10,000 characters?
> 
> Misfontification around that boundary.
> 
> > Are you saying that many regular expressions in font-lock-keywords are
> > anchored at beginning or end of a line?
> 
> No but for example if the 10000 char boundary falls in the middle of the
> word "function", then all the highlighting rules which rely on matching
> the keyword "function" will fail.

So we value accurate fontification more than we value a usable Emacs
session?  That sounds strange to me, since fontification is basically
a nicety, in the sense that it is not necessary for the text being
readable by humans.

To put this in perspective, I'm about to land a feature which, when
activated, will sometimes produce semi-updated windows (like incorrect
position of cursor) or even completely outdated windows (where what's
on the glass doesn't necessarily reflect where we are in the text,
like if you are at EOB, but the window still shows the last portion of
buffer text).  All that is done as a means to make Emacs usable even
when you for some reason visited a file with such pathologically long
lines.  If the weird results of redisplay under that feature are
acceptable in these extreme situations, I wonder why sacrificing
accuracy of font-lock wouldn't be.

> > And even if the regexp-based font-lock needs to do it line-by-line,
> > does it really _have_ to invoke parse-partial-sexp for the entire
> > line, when doing syntactical fontifications? why is that?
> 
> AFAICT this is a call that does
> 
>     (parse-partial-sexp (point) end nil nil state 'syntax-table)
> 
> where `end` happens to be point-max (because of font-lock's rounding up
> to whole lines) which will not itself parse all the way to `end` but
> will instead stop at the next string/comment boundary.  This is used
> (inside a loop which will indeed go all the way to `end`,
> i.e. `point-max`) to highlight strings and comments (as you can see
> from the stack trace, this is `font-lock-fontify-syntactically-region`).

Yes, that's what I see.

> For such huge files, it's arguably more important to be more-or-less
> usable than it is to get the highlighting right, so there's a good case
> for turning off font-lock or breaking it somewhat by using arbitrary
> limits in the handling of `font-lock-extend-region-functions` in
> `font-lock-default-fontify-region`.

I'm talking about cases where the user visits such files without
knowing in advance that he/she has better turned off font-lock there.
I'm asking whether one of the things we could do when we discover that
parse-partial-sexp is called for such a large chunk of text could be
automatic introduction of such a limit only for that buffer, so that
font-lock won't be such a significant part of the reason for redisplay
slowness in these cases.

Also, can you tell why visiting the same file in Text mode doesn't
produce the same effect?  I guess this is related to font-lock
settings of js.el, but my question is: would any mode that uses
syntactic fontification be similarly affected, or is there anything
special with js.el specifically?  IOW, is the above tendency to scan
everything to the end of line inherent in syntactic fontifications?

Thanks.