Re: State of the overlay tree branch?

[Sebastian Sturm <s.sturm@arkona-technologies.de>]

for the record, I just switched back to emacs master (no noverlay) and 
the time reported by (benchmark-run 1000 (line-number-at-pos (point)) 
increased by a factor of ~40, to 75-80s. At this level, editing is 
unbearably slow. With the semantic highlighter disabled, the same 
measurement yields ~2.5s (still painfully slow, but borderline usable), 
so about the same time reported by the noverlay branch.
To me, this suggests that noverlay indeed improves performance, though 
not necessarily to the level I had previously claimed. find_newline may 
solve this particular issue completely.
Since the time taken by line-number-at-pos seems to fluctuate wildly for 
(to me) unknown reasons, I'll try and see if I can set up a systematic 
way to collect reliable data.

On 03/19/2018 12:03 AM, Sebastian Sturm wrote:
> concerning the performance improvement with noverlay, it seems I spoke 
> to soon. I've now had the issue reappear, both with the noverlay branch, 
> and with the semantic highlighter set to use font-lock. Sorry for the 
> misinformation.
> Again, however, line-number-at-pos shows up as a large CPU time consumer 
> in the profiler report, and benchmark-run still reports several ms per 
> invocation (though this time it's usually around 2 to 4 ms instead of 
> the 20 to 25 I measured earlier), so I'd still be very much interested 
> in a faster line-number-at-pos implementation.
> 
> On 03/18/2018 10:04 PM, Sebastian Sturm wrote:
>> I also found it surprising that overlays would slow down line 
>> counting, but since I don't know anything about the architecture of 
>> the Emacs display engine, or its overlay implementation, I figured 
>> that overlays must be to blame because
>>
>> (i) the issue went away after switching to the feature/noverlay branch
>>
>> (ii) configuring the semantic highlighter to use its font-lock backend 
>> also resolved the performance issue (though with the font-lock 
>> backend, highlights are easily messed up by editing operations which 
>> makes the overlay variant far more appealing)
>>
>> I also found that some other heavy users of overlays such as 
>> avy-goto-word-0-{above,below} feel faster with the feature/noverlay 
>> branch, so I'd welcome a merge of the overlay branch even if there was 
>> a technically superior alternative to line-number-at-pos that didn't 
>> suffer from overlay-related performance issues.
>>
>> That being said, your suggestion sounds intriguing. What would be 
>> required to expose find_newline to Lisp? Would I simply have to wrap 
>> it in one of Emacs's DEFINE_<something> macros? Is there some 
>> documentation on the Emacs C backend?
>>
>> On 03/18/2018 09:39 PM, Eli Zaretskii wrote:
>>  >> From: Sebastian Sturm <s.sturm@arkona-technologies.de>
>>  >> Date: Sun, 18 Mar 2018 21:14:53 +0100
>>  >>
>>  >> [1] I'm using cquery for my C++ editing needs, which comes with an
>>  >> overlay-based semantic highlighting mechanism. With my emacs
>>  >> configuration, lsp-mode/lsp-ui emit 6 calls to line-number-at-pos per
>>  >> character insertion, which consume ~20 to 25 ms each when performing
>>  >> edits close to the bottom of a 66KB C++ file (measured using
>>  >> (benchmark-run 1000 (line-number-at-pos (point))) on a release 
>> build of
>>  >> emacs-27/git commit #9942734...). Using the noverlay branch, this 
>> figure
>>  >> drops to ~160us per call.
>>  >
>>  > If lsp-mode/lsp-ui needs a fast line counter, one can easily be
>>  > provided by exposing find_newline to Lisp.  IME, it's lightning-fast,
>>  > and should run circles around count-lines (used by 
>> line-number-at-pos).
>>  >
>>  > (I'm not sure I even understand how overlays come into play here,
>>  > btw.)
>>
>