bug#56682: locked narrowing

[Dmitry Gutov <dgutov@yandex.ru>]

On 29/11/2022 14:47, Eli Zaretskii wrote:
>> Date: Tue, 29 Nov 2022 05:20:30 +0200
>> Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>,
>>   56682@debbugs.gnu.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>> Another thing that bothered me when testing: going from eob to bob in
>> dictionary.json took ~4 seconds every time, independent of buffer being
>> edited or not. And that delay didn't show up anywhere in the profiler.
>> (setq bidi-inhibit-bpa nil) didn't help at all, but (setq
>> bidi-display-reordering nil) eliminated that delay. So I'm still kind of
>> puzzled why we bother to restrict font-lock, but didn't do that with
>> this significantly more costly computation.
> 
> Why do you think we didn't restrict them?  Limitations on costly
> bidi-related stuff is in Emacs since the very beginning, in v24, AFAIR.  And
> no code in bidi.c or xdisp.c ever goes outside the current narrowing.

Just because it was really slow. Apologies, I seem to be unable to 
reproduce this effect now. I'll get back to this if I see it again.

>> Even with with common setup (and emacs -Q):
>>
>> 1. (setq long-line-locked-narrowing-region-size 50000)
>> 2. (setq bidi-display-reordering nil)
>> 3. Toggle show-paren-mode off, just in case.
>> 4. electric-pair-mode off, same.
>>
>> typing in the latter file exhibits random pauses in redisplay up to 0.5s
>> (and sometimes 1s+). I haven't managed to catch the exact source of
>> those pauses (and they're longer with my personal config), but even
>> regular editing is slower: evaluating
>>
>>     (benchmark 1 '(progn (insert " ") (redisplay t)))
>>
>> in dictionary.json reports something like 0.038906s, but in
>> dictionary-pp.json it prints ~0.136387s.
> 
> It is wrong to discuss here issues that are evidently unrelated to long
> lines and the measures to make Emacs more efficient with long lines.  Please
> file a separate bug report about the delays you see, and please post there
> the file dictionary-pp.json you used in these tests.  Bonus points for
> including in the report buffer positions where "typing exhibits random
> pauses in redisplay", to make search for those more efficient.

My theory was that it's likely related, since there haven't been any 
other major changes to the display engine later. Of course I could be 
mistaken.

If you don't see the problem right away, sure I can file this separately.

dictionary-pp.json is produced from dictionary.json using 'M-x 
json-pretty-print-buffer'. It's 27 MB long, do you want me to upload it 
to some file sharing platform?

Positions: any near bob.

The profiler output looks like this:

          526  87% - command-execute
          516  85%  - call-interactively
          378  62%   - funcall-interactively
          370  61%    - eval-expression
          370  61%     - eval
          370  61%      - benchmark
          370  61%       - benchmark-call
          194  32%        - #<lambda 0x91bc787b0d851>
          194  32%         - progn
          194  32%            redisplay
          160  26%        - #<lambda 0x91bc787b0d851>
          160  26%         - progn
          160  26%            redisplay
           16   2%        - #<lambda 0x91bc787b0d851>
           16   2%         - progn
           16   2%            redisplay

> Last, but not least: the "optimizations" (I prefer to call them "shortcuts")
> which we installed for making Emacs more responsive with long lines are just
> that: shortcuts.  They deliberately ignore certain possible complications in
> buffer text and properties/overlays, and bypass the potentially costly code
> which handles them, in order to make the display code significantly faster,
> at a price of being less accurate and correct.  So I'm not surprised that
> you sometimes see faster redisplay with long lines: it could well be due to
> these simplifications -- and the price is that in some situations the
> display will be incorrect.  So it could turn out that there's nothing wrong
> with the "unoptimized" redisplay: it simply sometimes takes time to do all
> that it has to do to produce correct display on the screen.  The resulting
> loss of accuracy and/or correctness is IMO justified when lines are so long
> as to make Emacs unusable without these shortcuts, but not when Emacs can
> cope with the complete and accurate display perfectly well, albeit somewhat
> slower.  So you might be basically comparing apples with oranges here.
> 
> But that is theory: we need specific recipes to investigate the causes for
> the delay.
> 
>> Commit 89a10ffc (before the last merge) behaves the same, but going
>> further back, this commit from July behaves differently: b283e36cf19.
>>
>> Emacs built from that version reports ~0.033022s in dictionary-pp.json
>> and 0.114781s at bob in dictionary.json (scrolling to the end
>> predictably freezes that Emacs build).
> 
> I don't understand what exactly you timed here (since it cannot be scrolling
> to eob).  Is it the "insert one SPC" or is it something else?

"insert one SPC and redisplay".

The 'insert' call by itself is very fast.

> In any case, 0.114781s just 19% (and 22 msec) slower than 0.136387s, so is
> this something serious to talk about?  There could be any number of reasons
> for this, including some changes to the JSON mode, perhaps?

It's 0.114 vs 0.033 (in dictionary-pp.json, file without long lines).