Re: Update on tree-sitter structure navigation

[Dmitry Gutov <dmitry@gutov.dev>]

On 07/09/2023 06:18, Danny Freeman wrote:
> 
> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>>> clojure-ts-mode keeps a URL for the parser, but doesn't do anything
>>> about the git revision. It easily could but I don't feel the need (yet)
>>> since I am also a maintainer of the clojure grammar and know when we're
>>> about to break grammar consumers.
>>
>> Sure, that's easy enough to do when the package is only in ELPA: upgrade the grammar, upgrade the
>> package, all in lockstep.
> 
> Yeah, soon after I sent that email I realized there is no reason for me
> not to specify a version for the grammar so I pushed a change doing just
> that.

Nice.

>> Grammars distributed from distros are more of a problem, because it's not always a good idea to
>> abort with "wrong version". But perhaps we could do that and recommend installing from Git in such
>> cases anyway?
> 
> In some cases, distros might place the grammars in a strange location
> made accessible on `treesit-extra-load-path`, which takes precedence
> over the grammars that are installed from git in the user's Emacs
> directory. This is what nix does, but is probably an outlier. I would
> guess more conventional distributions might just make them accessible
> where dynamic libraries are normally located and the grammars installed
> from git would take precedence.

Perhaps the user's Emacs directory should take precendence over 
treesit-extra-load-path. Or treesit-install-language-grammar should pick 
a higher-priority place instead. It just makes sense that the 
user-installed grammar would be loaded first.

>> Another problem is that grammars don't have good versioning, and even if they did, we'd have to
>> sometimes update the "upper bound" (we'd need coarse ranges, right? rather that one fixed version
>> requirement) more frequently than Emacs is released. Less of a problem for modes in ELPA, though.
> 
> Yeah I think ranges would be right. It would be good to say, we tested
> this with versions N through M, anything else might not work. There
> would still need to be some checks and patches like what exists in
> js-ts-mode now. But that seems unavoidable, but could be cleaner if we
> had a good way to ID grammars. Not sure about how we'd keep up with
> grammars. Maybe we just can't and would need to have users install older
> versions. That seems okay?

Basically, yes: if the current available grammar is outside of the 
compatibility range (and/or we get query errors, I'm not sure where to 
put the balance: I suppose sometimes the query will succeed but it 
wouldn't match some elements which it matched before), we issue a 
warning to the user that they're recommended to use 
treesit-install-language-grammar - installing the last-known good hash, 
which might as well be older than the current installed grammar.

>>> I'm not so sure we can have a great way to do this without a change to
>>> the tree-sitter libraries. I would love to see some kind of increasing
>>> version number generated in the grammar's C source that we could then
>>> access. It could be used to make decisions about what queries to use, or
>>> to warn the user they need to use a different grammar (maybe offering to
>>> install a compatible version).
>>
>> Yes, that would be an improvement, worth being up on the issue tracker maybe.
> 
> Yeah, I think this is a good move. I opened up one here
> https://github.com/tree-sitter/tree-sitter/issues/2611
> Of course, anyone feel free to chime in.

Thanks! I left a note too.