bug#46859: 28.0.50; [PATCH]: Add option to truncate long lines in xref.el

[Dmitry Gutov <dgutov@yandex.ru>]

Hi again,

On 07.03.2021 22:03, Theodor Thornhill via Bug reports for GNU Emacs, 
the Swiss army knife of text editors wrote:

>> Please try out the attached preparation patch.
>>
>> It improves the performance of the "very long line" case drastically
>> over here, while not doing any truncation yet. Looks like we regressed
>> that case when we added rendering of multiple matches on the same line.
>>
> 
> Yes, this seems to help a lot. Now the search is down from 11 seconds to
> 1.09. It is comparable to the other good efforts.

Excellent!

I've pushed this change, along with some stuff it depends on, to master 
in commit 8e103ebef1.

>> We can add the truncation feature on top of it.
> 
> I think we should, since moving around in the xref-buffer is still very slow.

I see that too.

>> Probably also in xref--collect-matches-1 (truncating the value of
>> SUMMARY just before the xref-make-match call).
>>
>> Alternatively, we could experiment with hiding parts of the long line
>> using some display/visibility features (except the truncate-lines
>> variable, that one keeps things slow). That could be done in
>> xref--insert-xrefs or somewhere nearby. That is trickier, though, given
>> that we'll probably want to unhide it (wholly or partially) when
>> iterating over matches inside.
> 
> At this point I'm really thinking that truncating without bothering too
> much about losing information is worth it, and the added complexity by
> retaining information would only make regressions more feasible.

With the latest change, retaining that info should be particularly 
difficult: you adjust the SUMMARY values inside xref--collect-matches-1 
using the context information at hand, and that's almost it (mostly note 
to self: also need to update xref--outdated-p accordingly).

> I
> assume these files are _actually_ read once every blue moon. To maximize
> the speedup should be at a higher priority than retaining the matches,
> IMO. In any case, if there is a hit on one of these long lines, the
> current efforts will render them as results to the xref
> buffer. Searching or editing these files wouldn't be emacs' strength
> anyways :)

Depends on the performance improvement multiplier, I suppose.

But I'm inclined to believe that if the user did search those files, and 
did not include them in, say, project-vc-ignores value, they probably 
want to be able to see all matches. Sometimes losing valid hits can be a 
significant problem, and since Xref is implemented in an opaque way it 
is, we should make an effort not to omit information in the name of 
performance, at least while feasible.

> My proposal for the "best" fix would be:
> 
> - truncating long lines by default, both for grep and ripgrep
> - adding some variation of the "-M <n>" value for ripgrep by default
> 
> What do you think?

I think that would be a good non-default option for users who really 
know what they're doing. And it's already available for those who use 
ripgrep.

After all, there can be files out there with some long lines (not 
kilobytes long, probably, but >500 chars? why not) that aren't minified 
CSS or JS. If those were the only problem, we could as well recommend 
everybody add those to their project ignores and be done with it (which 
is what I usually do personally).