Re: Tree-sitter maturity

[Eli Zaretskii <eliz@gnu.org>]

> Date: Sat, 04 Jan 2025 16:18:21 -0500
> From: Daniel Colascione <dancol@dancol.org>
> CC: owinebar@gmail.com, bjorn.bidar@thaodan.de, philipk@posteo.net,
>  emacs-devel@gnu.org, rms@gnu.org, manphiz@gmail.com
> 
> >> >It's _our_ win.  The problem is shifted to the distros, where I think
> >> >it belongs.
> >> 
> >> No, it isn't, because distros *can't* do a good job of this.
> >
> >Then they need to get their act together and improve.
> 
> They cannot and will not do major software engineering to work around compatibility problems that should not exist. Do you expect a Fedora packager to notice or care when the distro version of some grammar is subtly incompatible with foo-ts-mode.el in Emacs? And patch the grammar or Emacs or both? And for the Debian and Arch packagers and so to do that too? Probably with subtly different bugs? We ought to have learned after the openssl security debacle not to let distros make nontrivial code changes.

Yes, I expect them to do that.  AFAIK, they actually do this for
optional libraries they package and provide.

> >> Besides: plenty of people (I'd guess a majority going by how the industry is set up) were using Emacs outside the context of a distro. If we weren't allergic to telemetry, we'd know.
> >
> >People who do that (I'm one of them, btw) can build their grammars.  I
> >do.
> 
> Yeah it's totally normal to download a macOS or Windows package and then have to set up a development tool chain to make the program they just installed actually work. Nothing user hostile about that.

People who don't use distros build their own Emacs, yes?  So they
should already have this set up.  Compiling a grammar library requires
a C compiler and Binutils, plus Make, that's all.

> It's really hard to see how this "rely on the distro" model is supposed to work in the real world. Besides, some of the antique platforms you insist we support, like MS-DOS and Windows 98, have nothing resembling a package manager, packages, or any way to automatically get some compatible version of a grammar. Not including the batteries means the toy doesn't work.

I publish my 32-bit Windows builds of more than 70 grammar libraries
on the ezwinports site, so the users of those antique Windows 9X
platforms can download and install them.

> >> You said that we would need to get written permission from grammar projects to include their code in Emacs. When I asked where this requirement comes from, you said it had always been this way and that RMS might have more information. He appears not to.
> >
> >RMS explicitly asked me to do that for every package we admit into
> >Emacs.
> 
> Is RMS a judge? The assertion is that there is a *legal requirement* to get permission to include something in Emacs. I don't think any such requirement exists. RMS may impose that requirement, but he's not the law, and unlike the law, his policy can change.

I don't own this project, so when RMS asks me to do something, I do
that.

> >Again, you are missing the point: we need to find an appropriate
> >version for each branch, and we need to track the versions of the
> >grammar so ass to be able to know, for each of our branches, the
> >compatible version of the grammar.  The maintenance burden is
> >multiplied by the branches.
> 
> The way it should work is that you have a checked in foo-ts-mode.el and right alongside it a foo.js file that describes the tree sitter grammar that foo-ts-mode uses. When you cut a branch, you snapshot both files. No extra overhead.
> 
> Want to update foo.js? Just sync it from upstream and check it in, just like you'd check in any modified foo-ts-mode.el. You only have to do that when you want to update the grammar, and that happens whenever the mode author feels like doing it --- *exactly* the way it works today when you have a standalone foo-mode.el.

This all is work someone has to do, and do that separately for each of
our two branches.

> >Responsibility has nothing to do with engineering or moving parts.
> 
> I have no idea what you're talking about. There's responsibility to make a worse engineering choice and make a custom downloader? What's the nature of this responsibility?

I think this is a clear sign that this discussion should be ended.

I stand by my opinions: it is not our job to distribute grammar
libraries, and bringing them into Emacs is too much trouble for us.