Re: Tree-sitter maturity

[Philip Kaludercic <philipk@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: rms@gnu.org,  manphiz@gmail.com,  emacs-devel@gnu.org
>> Date: Fri, 27 Dec 2024 14:11:01 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> It might take a while for that to happen, which is why I still believe
>> >> it would be better if tree-sitter major modes would populate
>> >> `treesit-language-source-alist' on their own, and point to the specific
>> >> checkouts that the major mode developer tested their implementation
>> >> against.
>> >
>> > We could have done that, but there's no way we could keep the value of
>> > treesit-language-source-alist up-to-date, because the grammar
>> > libraries put out new versions much more frequently than Emacs
>> > releases, especially if you consider libraries that have no official
>> > versions at all (in which case we can only point to some revision in
>> > their repository).
>> 
>> Is there a reason we need to keep them *that* up to date?
>
> There is: later versions generally improve on older ones and fix some
> bugs.

I would hope that most releases are stable most of the time setting
aside future changes to the language.

Related to this, it might not be bad to have some way to keep an
overview over updates to the grammars, and not just jump from one
development tip to another.

>> What happens when someone is on an older version of Emacs, and with
>> Tree Sitter changes in the meantime the current development tip is
>> not compatible with the library that Emacs binds against?  It seems
>> more reliable to me to pin what commit the grammar was tested
>> against, if only as a hint.
>
> It's more reliable, but it could have the effect of keeping people
> from upgrading when they can.  What is better?  I don't know.
>
>> > The question that bothers me is how useful is it to have
>> > treesit-language-source-alist that is outdated?  What do we expect the
>> > users to do with such an outdated value?
>> 
>> One idea is to first download the pinned version, and if that can't be
>> built to download the tip.  We could also prompt the user to check with
>> them.
>
> If we do that, it is basically what we have today: the responsibility
> for deciding which version to install is on the user.

It should make it easier to install a specific release.

>> If this is a serious issue (which I cannot estimate), we can provide an
>> official package similar to the on MELPA with updated references.
>
> Yes, if we have volunteers to keep these up-to-date (which involves
> testing the new versions when they come out).

Which is why I think it would be best to tie this up with the -ts-modes
by explicitly listing where and what grammar to fetch.

But part of my premise is that outdated (but stable) is better than
unstable (but newer).

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ihor Radchenko <yantar92@posteo.net>
>> Cc: Philip Kaludercic <philipk@posteo.net>, rms@gnu.org, manphiz@gmail.com,
>>  emacs-devel@gnu.org
>> Date: Fri, 27 Dec 2024 18:29:47 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > We could have done that, but there's no way we could keep the value of
>> > treesit-language-source-alist up-to-date, because the grammar
>> > libraries put out new versions much more frequently than Emacs
>> > releases, especially if you consider libraries that have no official
>> > versions at all (in which case we can only point to some revision in
>> > their repository).
>> 
>> May this list be updated from ELPA? Via a special package or maybe via compat.el.
>
> Everything is possible, if someone volunteers to do the job.  (There's
> still the question I asked whether such a package will be useful and
> used by enough users, but that is secondary.)
>
> In general, people are suggesting all kinds of semi-solutions for a
> problem that has no solution, and isn't ours to solve to begin with.
> I don't mind providing such semi-solutions, if someone volunteers, I'm
> just pointing out their (almost obvious) downsides, so people would
> not make a mistake of thinking we can solve this like we are used to
> solve such problems when they are under our complete control.

I also think that ELPA could help here.  To make the semi-suggestion
more concrete, if we went ahead and would specify the exact sources to
use in each -ts-mode.el file, it would be easy to write a package that
scraped emacs.git for these declarations and bundled them into a
package.

If there is no principle objection to these suggestions, I can write up
a prototype.