bug#38407: 27.0.50; infinite loop with display of large file without newlines

[Phil Sainty <psainty@orcon.net.nz>]

On 6/12/19 4:01 AM, Eli Zaretskii wrote:
> If we want [so-long] to be enabled by default, it "needs work", IMO.
> For starters, the default line length beyond which it kicks in, 250
> characters, is waaay too low.  IME, problems start with much longer
> lines, perhaps more than 10000 characters.

Agreed.  There is a whole discussion pending (I intended to raise it
soon), asking people to test the library more widely, and to help to
figure out what the default settings should actually be.  The current
settings are almost entirely based on modes and features I have tried
personally, after all.

That 250 value is, as you note, incredibly small.  As with many things
in so-long it's something of a heuristic and trade-off.  The reasoning
was that not all files with extremely long lines are just one line --
newline chars may still occur in the text for reasons other than the
formatting of the file -- and so the question was: "given the context
of a buffer in some prog-mode derivative, what length is "long enough"
to indicate that we're not looking at a normal file, but something
which probably contains minified code?

In conjunction is the "how many lines do we test" variable.  It would
be nice to arrive at a decision without doing a lot of work, so we
don't want to inspect a really large number of lines; but we also
don't necessarily want to *require* that we've seen a line which will
definitely cause problems -- a shorter line which strongly suggests
that we're looking at minified code/data may be sufficient.

With the current conservative (*very* conservative) values we very
quickly reach a decision, but we're more likely to trigger in false-
positive situations (this has happened to me, but only very rarely;
and by default `so-long-revert' is a C-c C-c away).  With larger
values we may more effort to reach a decision.  With *much* larger
values, that extra effort might be noticeable (on slower systems
even if not on faster ones), so I think we want to be a bit careful
of that.

(Not that I've been testing large values for performance -- I've been
leaving that for the later discussion, and in the meantime I've been
content to have the small default values, in part because it seemed
more likely that people would discover potential problems with the
library if it was a bit more trigger-happy.)

As with most things in so-long, there's no single right solution;
but I'm keen to arrive at a set of defaults that people can agree
are a reasonable starting point for most users; and I'm certainly
not suggesting that I've done that already ("what should the default
settings be?" has been an open question from the outset, really.)


-Phil