bug#38407: 27.0.50; infinite loop with display of large file without newlines

[Phil Sainty <psainty@orcon.net.nz>]

On 6/12/19 9:04 PM, Eli Zaretskii wrote:
> Did you try a different heuristic for detecting long lines, one that
> isn't based on searching some portion of the buffer?  E.g., what
> about going to a couple of random places and looking at the value
> returned by current-column?

No, the library has essentially used the current approach from the
beginning.  However one of the changes I made for the new version was
to allow users to set a custom function for detecting long lines via
the `so-long-predicate' option, so it's easy to plug in alternatives.

I haven't planned to implement any alternatives myself, but if other
people do so I'd be interested to see how they compare in practice.


> In general, searching for a newline is very fast in Emacs, so if you
> use the right primitives, you should be able to scan quite a lot of
> text before the time begins affecting responsiveness.

`so-long-detected-long-line-p' is the provided predicate, and it
relies on `forward-line' (which I've assumed is as quick as I'm going
to get for finding newlines), along with narrowing to limit the amount
of text scanned to the maximum value that we care about (i.e. the
minimum length of a too-long line).

I've just done a bit of testing with much larger files and values, and
suspect that we could indeed make these values quite dramatically
bigger without any noticeable impact.

Files with lots of lines, yet no excessively-long ones, will ensure
that so-long scans the (customizable) maximum number of lines that
it's going to try before giving up.  I expect that will be a useful
test case for testing responsiveness on slow systems.


-Phil