Re: cc-mode fontification feels random

[Eli Zaretskii <eliz@gnu.org>]

> Date: Sat, 12 Jun 2021 03:08:44 +0200
> From: Ergus <spacibba@aol.com>
> Cc: emacs-devel@gnu.org
> 
> BTW: Eli was concerned about the extra copy of the buffer text to send
> it to tree-sitter. In this case the time to memcopy an array with all
> xdisp text is ~0.00085 seconds.

If the intent is to use buffer-(sub)string, then you forget the price
of consing.  That would trigger frequent GC cycles, which will all but
kill the otherwise fast performance.

> Any way if we don't want the copy we can use
> ts_parser_set_included_ranges to exclude the gap and pass the text
> pointer directly without any copy.

I hope someone will try that and report the results.

The other design issue with TS integration is that I'd like it to plug
into the JIT font-lock interface of the display engine, so that we
don't unnecessarily fontify parts of the buffer that won't be
displayed, and always do fontify the parts that will be.  I don't
really care if TS actually processes a much larger chunk of text, if
it does that quickly enough, but processing the resulting faces will
take time on the Emacs side, and that is better avoided.  More
importantly, integration into JIT font-lock machinery means we don't
need to use other hooks, which is a step back, since using such hooks
for fontification was already shown to have serious problems in pre-21
Emacs: they don't always catch all the changes which require
re-fontification.