bug#69385: 30.0.50; Long lines with bidi text slow down Emacs

[Eli Zaretskii <eliz@gnu.org>]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: 69385@debbugs.gnu.org
> Date: Mon, 26 Feb 2024 22:47:21 +0100
> 
> I created the test buffer now by first typing "Hello" and the inserting
> " السّلام عليكم " on the right, so the base paragraph direction was LTR.
> Then I ran the benchmarks, and there is indeed a big difference, but one
> that I think will surprise you.

It doesn't, see below.

> First, the timing for the LTR run with bidi-display-reordering set
> to nil was:
> 
> (0.035104007 0 0.0)
> 
> This is, unsurprisingly, almost the same as the corresponding test run
> with RTL base paragraph direction, which was (0.034058467 0 0.0).
> 
> In contrast, recall that with bidi-display-reordering at the default
> value t, the RTL test run was (5.249231941 1 0.014300497000000023), but
> now the LTR test run gave this:
> 
> (10.613699099 1 0.012965359999999981)

Yes, that's the result of the fix I installed yesterday: it makes the
display correct, but it costs more CPU, sometimes much more.

To see why this is so costly, disable auto-composition-mode in the
buffer, and compare the times with bidi-display-reordering nil and
non-nil.  With auto-composition-mode disabled, you see the effect of
the reordering alone (because characters are still reordered for
display, just not shaped correctly, as C-f will show you), and it
should be quite small; at least, that's what I see here.  The
(mandatory) shaping of Arabic text is what takes most of the redisplay
time in these cases.  In particular, M-> does a recenter to show EOB
nicely, and that is very expensive in this case; disable that
recentering by using "(goto-char (point-max))" instead to see the
difference.

> The buffer size here is considerably larger than the buffers for my test
> runs; I couldn't remember it exactly, so I typed `M-: (point)' in the
> buffer, with the result 43901.  It took a number of seconds for the
> result to be displayed, during which Emacs was unresponsive, so I then
> ran (benchmark-run nil (point)) in that buffer, with this result:
> 
> (9.030000000000001e-07 0 0.0)
> 
> I don't know what this result means, but the reported execution time
> bears no relation to actual elapsed time until the value of point was
> displayed.  So I reran the benchmark and also manually timed it with a
> stopwatch.  This is the result of the second benchmark:
> 
> (3.1120000000000004e-06 0 0.0)
> 
> However, the stopwatch showed fully 15 seconds.  Maybe benchmark-run
> doesn't work for point for some reason.

A Lisp program that just calls '(point)' doesn't include the redisplay
caused by "M-:", which enters the minibuffer, and therefore triggers a
fairly thorough redisplay of the window, and by display of the result.
That's Emacs's MVC design in action for you.

> This sounds plausible, but my results seem to indicate there something
> else (or more) going on.

Yes: that something is character composition, which in the case of
Arabic text is quite expensive, because every chunk of Arabic text
between SPACEs goes through the shaping engine.

(This explains well why Emacs tries so hard to call the shaper only
when absolutely necessary, contrary to HarfBuzz folks' opinions that
we should pass all the text through the shaping engine, to get correct
display with ligatures, kerning, and other niceties.  They are right,
of course, but with the way character composition was designed in
Emacs, doing so would be prohibitive from the performance POV.)

> > I will see if we can do better in this matter.
> 
> I appreciate whatever you can do.

From what I've seen so far, it won't be easy, because it requires
changes to some key logic in the display code.  Hmm...