Re: Tree-sitter maturity

[Philip Kaludercic <philipk@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Xiyue Deng <manphiz@gmail.com>,  emacs-devel@gnu.org
>> Date: Fri, 27 Dec 2024 10:54:29 +0000
>> 
>> Richard Stallman <rms@gnu.org> writes:
>> 
>> > If we add something like this to Emacs, there is an issue we need to
>> > take care about: to make carefully sure that it does not install
>> > any nonfree grammars.  I don't know how those grammars are released,
>> > ir by whom, or how much they care about free software.  We can't
>> > take for granted that they do.
>> >
>> > Perhaps we could check automatically that the grammar found is properly
>> > licenses, and disregard any grammars that are not free.
>> >
>> > By contrast, if grammars are going to be packaged and released for
>> > distros, and chosen for installation by users, then it is the user's
>> > responsibility, not Emacs's responsibility, to reject the nonfree ones
>> > (and the GNU/Linux distro might insist on that).
>> 
>> It might take a while for that to happen, which is why I still believe
>> it would be better if tree-sitter major modes would populate
>> `treesit-language-source-alist' on their own, and point to the specific
>> checkouts that the major mode developer tested their implementation
>> against.
>
> We could have done that, but there's no way we could keep the value of
> treesit-language-source-alist up-to-date, because the grammar
> libraries put out new versions much more frequently than Emacs
> releases, especially if you consider libraries that have no official
> versions at all (in which case we can only point to some revision in
> their repository).

Is there a reason we need to keep them *that* up to date?  What happens
when someone is on an older version of Emacs, and with Tree Sitter
changes in the meantime the current development tip is not compatible
with the library that Emacs binds against?  It seems more reliable to me
to pin what commit the grammar was tested against, if only as a hint.

And the release issue can be circumvented if we either list the
specific revisions to use  in `treesit-language-source-alist' or just
download the tarball directly.  Each -ts-mode.el file can just hint at
the commit to use with a 

  (add-to-list
   'treesit-language-source-alist
   '(python "https://github.com/tree-sitter/py-tree-sitter/tarball/d65416ba825a99054f7b64649c6b93c6fa7f5e04"))

expression.

> The question that bothers me is how useful is it to have
> treesit-language-source-alist that is outdated?  What do we expect the
> users to do with such an outdated value?

One idea is to first download the pinned version, and if that can't be
built to download the tip.  We could also prompt the user to check with
them.

If this is a serious issue (which I cannot estimate), we can provide an
official package similar to the on MELPA with updated references.