Re: Tree-sitter api

[Yoav Marco <yoavm448@gmail.com>]

Hi, Yuan and I had a discussion on github
https://github.com/casouri/emacs/issues/5
and he suggested we move here. I'm quoting our comments for conveniece.

Yoav Marco <yoavm448@gmail.com> writes:

> Hi! My question is about the lines:
> https://github.com/casouri/emacs/blob/a4f90c5f95476914fb8789c67652af1025644af8/src/tree-sitter.c#L1375-L1380
>
>          /* TODO: We could cache the query object, so that repeatedly
>          querying with the same query can reuse the query object.  It also
>          saves us from expanding the sexp query into a string.  I don't
>          know how much time that could save though.  */
>       TSQuery *ts_query = ts_query_new (lang, source, strlen (source),
>                                        &error_offset, &error_type);
>
>
> Regarding error handling mostly.
>
> In this branch queries are saved as *strings* and compiled in the internals on
> each use. In elisp-tree-sitter, you call `tsc-make-query` and use the object
> it returns for calls to tsc-query-captures which is the analog for
> tree-sitter-query-capture.
>
> What happens if your query is deformed, or simply has a typo in a node name?
> We call `tree-sitter-query-capture` on each keystroke in
> `tree-sitter-font-lock-fontify-region`. With the compilation occurring
> ahead-of-time it would fail once, but here wouldn't it barrage you with
> errors?
>
> Especially with patterns that aren't set in stone and can be modified like
> font-lock keywords, I think compiling the query when the pattern is added is
> better than on each execution.
>
> One nice thing though about compiling queries only when queried is that you
> can call `ts_query_delete` straight away. With users compiling queries it
> would need to be up to garbage collection, I think.

Yuan Fu <notifications@github.com> writes:

>> What happens if your query is deformed, or simply has a typo in a node name?
>> We call tree-sitter-query-capture on each keystroke in
>> tree-sitter-font-lock-fontify-region. With the compilation occurring
>> ahead-of-time it would fail once, but here wouldn't it barrage you with
>> errors?
>
> Not quite barraging, jit-lock will just silently fail and leave a bunch of
> logs in Messages. I don't think error out when calling
> tree-sitter-query-capture is a grave problem, since 1) it doesn't barrage as
> you worried and 2) I don't expect queries in major modes to ship wrong code:
> it's not like a bug that could go undiscovered, if the query has a typo, the
> major mode writer will certainly find out when he/she tries to fontify a
> buffer.
>
> I can see some advantages to compile the query ahead of time. 1) It would be
> helpful to know there is an error before calling
> tree-sitter-font-lock-fontify-region and see an unfontified buffer, not
> knowing what went wrong. I can add a function, say, tree-sitter-compile-query
> that checks a query (as in query pattern) and passes it on if its correct. 2)
> It could potentially saves recompilation of the query. But computing the query
> most probably takes negligible time.
>
> On the other hand, compiling the query has downsides: I don't know what does
> tsc-make-query return, I assume an internal object? I try to minimize the
> number of new object types I introduce to Emacs, for hygiene. So far I've
> managed to add only parser object and node object. If there aren't good
> reasons I'm inclined to not add a query object. So far the advantages that I
> see aren't very convincing.
>
> If you want to continue the discussion, I suggest we continue at emacs-devel,
> that way others who are more knowledgable than I can join and offer their
> opinion.