Re: Update on tree-sitter structure navigation

[Danny Freeman <danny@dfreeman.email>]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> clojure-ts-mode keeps a URL for the parser, but doesn't do anything
>> about the git revision. It easily could but I don't feel the need (yet)
>> since I am also a maintainer of the clojure grammar and know when we're
>> about to break grammar consumers.
>
> Sure, that's easy enough to do when the package is only in ELPA: upgrade the grammar, upgrade the
> package, all in lockstep.

Yeah, soon after I sent that email I realized there is no reason for me
not to specify a version for the grammar so I pushed a change doing just
that.

> Unless nixos or other distros are going to start distributing it as well, and you'll need to care
> about having the recent clojure-ts-mode being loaded with old versions of the grammar.

Luckily the grammar has not changed since my package was released.
But you are right it will eventually become an issue.

>> With the way Emacs can load a grammar provided by the user's
>> distribution, keeping information about the version of the grammar in
>> the major mode doesn't help all that much. Even if we did it we have no
>> idea what version might be have been built used the user's
>> .emacs.d/tree-sitter folder. That would require something like putting a
>> version number in the file name, or maybe applying a patch to the
>> grammar's C source that allowed us to get a version, SHA, something at
>> runtime.
>
> Well, it would at least allow the user to rebuild the grammar to the version best known to work.
> Also, perhaps if the mode tracks the changes in the hash over time, it could see whether the grammar
> needs to be rebuilt. Finally, treesit-install-language-grammar could track which revision was last
> compiled.
>
> So there is *something* we could do for the users who upgrade their grammars from Git.
>
> Grammars distributed from distros are more of a problem, because it's not always a good idea to
> abort with "wrong version". But perhaps we could do that and recommend installing from Git in such
> cases anyway?

In some cases, distros might place the grammars in a strange location
made accessible on `treesit-extra-load-path`, which takes precedence
over the grammars that are installed from git in the user's Emacs
directory. This is what nix does, but is probably an outlier. I would
guess more conventional distributions might just make them accessible
where dynamic libraries are normally located and the grammars installed
from git would take precedence.

> Another problem is that grammars don't have good versioning, and even if they did, we'd have to
> sometimes update the "upper bound" (we'd need coarse ranges, right? rather that one fixed version
> requirement) more frequently than Emacs is released. Less of a problem for modes in ELPA, though.

Yeah I think ranges would be right. It would be good to say, we tested
this with versions N through M, anything else might not work. There
would still need to be some checks and patches like what exists in
js-ts-mode now. But that seems unavoidable, but could be cleaner if we
had a good way to ID grammars. Not sure about how we'd keep up with
grammars. Maybe we just can't and would need to have users install older
versions. That seems okay?

>> I'm not so sure we can have a great way to do this without a change to
>> the tree-sitter libraries. I would love to see some kind of increasing
>> version number generated in the grammar's C source that we could then
>> access. It could be used to make decisions about what queries to use, or
>> to warn the user they need to use a different grammar (maybe offering to
>> install a compatible version).
>
> Yes, that would be an improvement, worth being up on the issue tracker maybe.

Yeah, I think this is a good move. I opened up one here
https://github.com/tree-sitter/tree-sitter/issues/2611
Of course, anyone feel free to chime in.

Thank you,
-- 
Danny Freeman