Re: Update on tree-sitter structure navigation

[Eli Zaretskii <eliz@gnu.org>]

> Date: Sat, 9 Sep 2023 13:24:46 +0300
> Cc: casouri@gmail.com, emacs-devel@gnu.org, danny@dfreeman.email,
>  theo@thornhill.no, jostein@secure.kjonigsen.net, dev@rjt.dev,
>  wkirschbaum@gmail.com, pedz@easesoftware.com
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 09/09/2023 09:32, Eli Zaretskii wrote:
> >> Date: Fri, 8 Sep 2023 23:52:48 +0300
> >> Cc: emacs-devel@gnu.org, danny@dfreeman.email, theo@thornhill.no,
> >>   jostein@secure.kjonigsen.net, dev@rjt.dev, wkirschbaum@gmail.com,
> >>   pedz@easesoftware.com
> >> From: Dmitry Gutov <dgutov@yandex.ru>
> >>
> >>> FTR, I have nothing against this technique, I just said that it will
> >>> need volunteers to assume this non-trivial job for each major mode,
> >>> and therefore I personally don't believe this to be a reliable
> >>> solution in practice.  But if volunteers step forward to do this, I
> >>> don't object.
> >>
> >> I don't see a way around it, if the grammars continue to add breaking
> >> changes.
> > 
> > If the only way we see is impractical, this doesn't help, does it?
> 
> Is it definitely impractical if it's known to work for NeoVim?

NeoVim is a different editor, with (potentially) different user
audience, different development and release schedules, different
distribution practices, different downstream distros and upgrade
procedures, etc.  Without considering all of these aspects and
comparing them to ours, I don't think it's meaningful to say "works"
when we discuss what we should do in our case.

> > No, that'd be worse than what we have now: those commit hashes will
> > quickly become outdated (most grammar libraries are very actively
> > developed), and create the false impression that any later version will
> > not work.
> 
> But it's not the first thing the user sees, just internal information: 
> we tested with this version last, it's known to work, so if you want to 
> have a known well-working configuration, you will install this one. 
> Might as well install the latest and try their luck, though.

How is it useful to ask users to use, say, 2-year old versions of
grammar libraries, especially for languages where either the language
or the library (or both) change quickly?

> Further, most important grammars seem to be in a reasonably complete 
> state by now.

They add features and fix problems all the time.  So I disagree with
the "reasonably complete" part, and so are the developers of those
libraries, evidently.

> > The job is to track all the commits of the corresponding libraries and
> > keep the last commit known to work constantly up-to-date, with delays
> > that are at most days, not weeks or months.
> 
> Consider that js-ts-mode is "broken" in Emacs 29.1 now with the latest 
> grammar. If there was the last-known-working hash, we could offer the 
> users a friendlier way to install it.

How is it friendlier to downgrade to an older version (which would
require fetching it, building it with a C compiler, and installing it)
than to patch a single Lisp file?  Actually, people don't even need to
patch their Emacs installations, they could instead have a fixed
version of the Lisp file in their home directories or in site-lisp.

> >> What I'm saying is in this case not doing this job well (e.g. updating
> >> the commit hashes and font-lock/indent rules very rarely) might still be
> >> better than not doing it at all.
> > 
> > I strongly disagree.  I think that doing this job not well is _worse_
> > than not doing it.  It takes just a few days, sometimes a couple of
> > weeks, from submission of the report about a breakage till a fix is
> > available, so people could install it soon enough and be done (since
> > these modes are not preloaded).
> 
> But the ts modes in an Emacs release (now in 29.1, later in 29.2, etc) 
> remain incompatible with any future grammar changes, right?

That depends on the change: not every change breaks our modes.  Only
changes that remove or rename the syntactic elements on which we rely
are breaking changes, from our POV.

> > And we always strive to fix these
> > breakages in a way that makes the code more immune to further changes,
> > so there's hope that with time the frequency of these problems will
> > become lower.
> 
> We can continue with this approach too (it's not incompatible with 
> saving the last-known-hash anyway), and it could also be of benefit 
> later if grammars grow proper versions, but it's also a bit of 
> maintenance headache on its own.

I agree it's a maintenance headache, but this issue will cause us
headaches no matter what we do.