so-long-mode and line-move-visual

so-long-mode and line-move-visual

[Eli Zaretskii]

Phil,

so-long.el has this passage:

 (defcustom so-long-variable-overrides
   '((bidi-inhibit-bpa . t)
     (bidi-paragraph-direction . left-to-right)
     (buffer-read-only . t)
     (global-hl-line-mode . nil)
     (line-move-visual . t)
     (show-paren-mode . nil)
     (truncate-lines . nil)
     (which-func-mode . nil))
   "Variables to override, and the values to override them with.

 The variables are given buffer-local values.  By default this happens if
 `so-long-action' is set to either `so-long-mode' or `so-long-minor-mode'.

 If `so-long-revert' is subsequently invoked, then the variables are restored
 to their original states.

 The combination of `line-move-visual' (enabled) and `truncate-lines' (disabled)
 is important for maximising responsiveness when moving vertically within an
 extremely long line, as otherwise the full length of the line may need to be
 scanned to find the next position.

It looks to me that the last paragraph (and the overrides to go with
it) has it backwards: line-move-visual set to nil produces _better_
performance than a non-nil value, and if truncate-lines is turned on,
the boost is even larger.  the explanation why is that so is correct,
but it describes the situation that's the opposite of the settings we
recommend.  I wonder why and how we came to the conclusion recorded in
the current code and the doc string.

Or what am I missing?

Re: so-long-mode and line-move-visual

[Phil Sainty]

Hi Eli,

The settings were arrived at by experimentation, and observing
what caused the fewest problems in severe cases.

With the option combo currently set by so-long, Emacs is only
displaying a small portion of a long line, and basic cursor
movements are only moving around within that visible area, so
it's less likely that the user will cause themselves problems
by moving point.  I'm also mostly concerned about the initial
opening of a file (to ensure Emacs doesn't freeze up simply by
visiting something), in which case that small portion is going
to be (typically) the beginning of the line, where performance
is good.

If the settings were otherwise, it can be extremely problematic
to simply move up and down in a buffer.  A nil line-move-visual
risks the user accidentally skipping to the end of an enormous
line (which may be very bad for performance), whereas a non-nil
value protects them from that.

Truncation meanwhile allows multiple enormous lines to be
visible at the same time, which can be terrible.  I've always
used the "one_line.json" example file from the question at
https://emacs.stackexchange.com/q/598 as a test case, and if I
create a multi-line file where each line is a copy of that (plus
a newline), then I can happily open that with default so-long
settings; but if I then M-x toggle-truncate-lines, Emacs freezes
so badly that I have to kill it.

As such, I don't think these defaults should be changed.


-Phil

Re: so-long-mode and line-move-visual

[Eli Zaretskii]

> Date: Sun, 08 May 2022 22:02:34 +1200
> From: Phil Sainty <psainty@orcon.net.nz>
> Cc: emacs-devel@gnu.org
> 
> With the option combo currently set by so-long, Emacs is only
> displaying a small portion of a long line, and basic cursor
> movements are only moving around within that visible area, so
> it's less likely that the user will cause themselves problems
> by moving point.

How is this consistent with this:

  (defcustom so-long-variable-overrides
    '((bidi-inhibit-bpa . t)
      (bidi-paragraph-direction . left-to-right)
      (buffer-read-only . t)
      (global-hl-line-mode . nil)
      (line-move-visual . t)
      (show-paren-mode . nil)
      (truncate-lines . nil)  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
      (which-func-mode . nil))

When truncate-lines is nil, lines are NOT truncated, so long lines
wrap around, and we display whole lines (at least their parts that fit
in a window, and windows nowadays tend to be very tall), not their
small portions.  What am I missing?

> A nil line-move-visual risks the user accidentally skipping to the
> end of an enormous line (which may be very bad for performance),
> whereas a non-nil value protects them from that.

That's not what happens in reality, though.  When line-move-visual is
nil, C-n moves to the next _physical_ line by looking for the next
newline in the buffer (via 'memchr'), which is very fast.  If lines
are truncated, this move to the next physical line skips large
portions of the buffer that are not visible on display, and skips them
very quickly.  By contrast, with line-move-visual non-nil, we use
simulation of the display code to find what is the character directly
below the cursor on the next visual line, and that requires us to
traverse all the characters in-between, loading fonts and colors,
computing the metrics of the font glyphs, etc.

> As such, I don't think these defaults should be changed.

Well, I found the defaults sub-optimal in the case someone presented
on help-gnu-emacs yesterday, thus my question.  If you are still
unconvinced, so be it.

Re: so-long-mode and line-move-visual

[Phil Sainty]

On 2022-05-08 23:20, Eli Zaretskii wrote:
>> With the option combo currently set by so-long, Emacs is only
>> displaying a small portion of a long line
> 
> When truncate-lines is nil, lines are NOT truncated, so long
> lines wrap around, and we display whole lines (at least their
> parts that fit in a window, and windows nowadays tend to be very
> tall), not their small portions.  What am I missing?

I think you're only missing the sheer scale of the line lengths I
am referring to when I say that -- even if tall, an entire window
can *easily* be completely filled by only "a small portion" of the
kind of line I'm trying to protect against (for the example I gave
before, the line is 18 megabytes of JSON).


>> As such, I don't think these defaults should be changed.
> 
> Well, I found the defaults sub-optimal in the case someone
> presented on help-gnu-emacs yesterday, thus my question.

I'm not subscribed to help-gnu-emacs but I can see the thread in
the archives.  At first glance so-long does what it's supposed to
do with that symbol.ts file.  The description of the problem is a
bit vague, and I don't know why they'd see different performance
from the minor mode vs the major mode (the major mode for *.ts
files would presumably need to be causing the slowness directly,
but that would be unusual).  Alternatively, perhaps they're using
the "Doom Emacs" config which may still be enabling `font-lock-mode'
in `so-long-minor-mode' (which I told them was a bad idea when I
first became aware, but I don't think they changed it).

I can see that the user in question has had success with using
`longlines-mode', which does seem like a good solution for them.


> If you are still unconvinced, so be it.

I am.  I do appreciate that the defaults aren't ideal for all
cases, but I nevertheless believe they're the right defaults, as I
don't believe there's any particular collection of settings which
is optimal in all situations.  My overriding concern is preventing
Emacs from ever locking up due to merely visiting a file, so that
goal has always informed the default settings.


-Phil