bug#45898: 27.1; wedged in redisplay again

[Eli Zaretskii <eliz@gnu.org>]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Date: Fri, 1 Jul 2022 13:44:26 +0200
> Cc: monnier@iro.umontreal.ca,
>  larsi@gnus.org,
>  psainty@orcon.net.nz,
>  Emacs-hacker2018@jovi.net,
>  45898@debbugs.gnu.org
> 
> I have medium_line.json now, and I see slowness (with truncate-lines nil and font-lock), after finding the file and doing random things.

What "random things" did you try?

> Is it sufficient to do just that for some time while tracing?

Yes, I think so.

> Do we have some Lisp code to automated this, and make it more reproducible?

Possible, but we need to discuss first which commands such Lisp code
will invoke.  Each command that involves the display code has
different CPU processing requirements, and thus reacts differently to
such files.

> Or, even, should be do truncate-line t first, and without font-lock?

That's the next step, I think.  In general, you can look in so-long.el
for the list of features we found to be expensive in files with long
lines; font-lock is just one of them.  As for truncate-line, IME it
sometimes makes redisplay faster and sometimes slower.  I guess the
slower cases are where font-lock is the main culprit, since it doesn't
stop at the window edge.

> (Did someone else observe it being fast at first, and then later slowing down?)

That is what normally happens, and the effect on different commands is
different, unsurprisingly.  For example, with C-n I see consistent
slowdown as I go deeper and deeper into the file, and around 20% I
start getting "too long to redisplay" errors if I set
max-redisplay-ticks to 400,000.  By contrast, C-v's performance seems
to be much less affected by the position where I invoke it.  Still, it
becomes slower as I move forward: about 3 times as slow at 30% into
the file as it is at BOB.

The difference is almost certainly due to the fact that these commands
use different methods provided by the display engine code to do their
jobs.  In particular, C-n calls vertical-motion, which begins by going
back to the previous newline, and then comes back by moving the
display iteration.  Which in this case means it traverses all the text
from BOB to where you invoked C-n.