Re: Tree-sitter integration on feature/tree-sitter

[Eli Zaretskii <eliz@gnu.org>]

> From: Yoav Marco <yoavm448@gmail.com>
> Cc: casouri@gmail.com, emacs-devel@gnu.org
> Date: Wed, 11 May 2022 14:16:03 +0300
> 
> Tree sitter parsers make a parse tree. To extract syntax highlighting
> from the tree you run a query that matches tree structures returns
> annotated nodes.
> 
> Queries need to be compiled to run them. Right now queries are kept as
> elisp strings and compiled each time we want to run them.
> 
> Compiling on each run is fine if we run the query extracting syntax
> highlighting information once, but to highlight a buffer in parts
> on-demand means running (thus compiling) the query every time on
> different ranges. When editing buffer the query is re-run on each change
> too.
> 
> I made a benchmark that tested whether reusing the compiled query would
> make anything faster, and turns out it's the biggest bottleneck in
> treesit-font-lock-fontify-region.

How much time does this bottleneck take us?  (I've seen the benchmarks
you posted, but I don't think I understood them, in terms of what
exactly was fontified and due to which command.)

Keeping a cache related to a buffer is a nuisance, because you need to
manage it: track its state and detect when it's invalid etc.  So if
the slowdown is not large enough in absolute (not relative!) terms, my
advice would be to just suck it up.

So let's start with the benchmarks, and please tell what exactly did
Emacs do to trigger fontifications in each benchmark.

Thanks.