Re: Tree-sitter introduction documentation

[Eli Zaretskii <eliz@gnu.org>]

> Date: Tue, 27 Dec 2022 16:11:22 +0200
> Cc: monnier@iro.umontreal.ca, theophilusx@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> > Frankly, I don't see why we, the upstream project, need to worry about
> > anyone else.  It isn't our job.  That's what distros are there for.
> 
> Previously all one needed for a language support mode is to download and 
> load an .el file. As we drift to the idea of using externally-maintained 
> grammars, and the "native" modes become less useful (possibly deprecated 
> in 5-10), it seems like it will become more of our responsibility to 
> streamline.

Please hold your horses, and let's keep things in their proper
perspective.  We've just introduced this feature, and have a
relatively small number of very "young" new modes using them.  We
haven't yet released even a single Emacs version with that feature,
and don't know whether migration to it will be slow and take years, or
fast as a bush-fire.  We don't know whether the fact we depend on
grammar libraries is a good or a bad thing for Emacs (we are not like
other projects, so it matters).  We don't know whether users will like
these new modes.

In this situation, rushing to some decisions that will have
long-standing maintenance consequences is premature at best.

> >> Do we expect all (most?) distros to compile all the popular grammars?
> > 
> > I honestly don't know.  On the one hand, there aren't many Emacs modes
> > which use tree-sitter, but OTOH they could start growing like
> > mushrooms once Emacs 29 hits the streets.  I do expect them to offer
> > the ones they consider useful/needed, for some value of that.  I
> > really don't see any significant difference in this regard between
> > grammar libraries and, say, librsvg.  Both are used in Emacs, and the
> > lack of either disables useful Emacs features.  So it's a no-brainer
> > for me.  But then I'm not a distro maintainer, and never have been.
> 
> On the flip side, the third-party modes can provide their own 
> download-compile-install instructions, which will make it easier to end 
> users.
> 
> The barrier to creating such a mode, though, is now higher.

Not to creating, to using it.  People who develop such modes are a few
and far in-between, and generally won't have any trouble finding and
building a grammar library.

The barrier is also somewhat higher when we introduce support for a
new image format, but we don't hesitate then, do we?

> > What's the solution?  All the "solutions" I saw until now require a
> > working and well-configured C/C++ compiler (sometimes both C and C++),
> > linker, and C/C++ runtimes.  A user who has them already installed can
> > easily build a grammar library with two simple commands.  A user who
> > doesn't have a C/C++ development environment will not find those
> > "solutions" useful at all.  And asking us to distribute binaries for
> > half a dozen popular systems is IMNSHO unreasonable.
> 
> I think it's common enough for a user to have build tools installed, but 
> not know well enough how to set up a C project. Think junior-middle 
> developers in a number of languages which are not C.

It doesn't need any project, it is literally two command lines.
Here's an example:

  gcc -O2 -I.   -c -o parser.o parser.c
  gcc  -shared parser.o scanner.o  -ltree-sitter -o libtree-sitter-c-sharp.dll

> > I cannot disagree more.  Look at this from my POV: once the list
> > becomes even semi-official, people will expect it to be of the same
> > high quality as all the rest of Emacs, and they _will_ complain and
> > report inaccuracies.  It's a nuisance, especially for such a "hot"
> > feature set.
> 
> They will report inaccuracies, which will be helpful to fixing them. 
> That is certainly a workload, but still small compared to the current 
> flow of bug reports, I think. Or the many hours one would spend fixing a 
> font- or redisplay-related problem.

Small, but not small enough.  Wading through sites, comparing
information, looking up licenses, tracking new grammar libraries --
this doesn't take hours, but it doesn't take seconds, either.  And
it's something you must do frequently enough to stay on top of
things.  It's a significant job.

> Anyway, my point was not to put this burden on you specifically. If you 
> might recall, I've always advocated toward "smaller core with many 
> plugins" as a model of Emacs development.

That's a different disagreement between us.

> The Neovim repo will likely be a good resource for this in the near 
> future.

What about "far future"?  Today it's neovim, tomorrow it's something
else.  Again, it's a burden we are better without.  Unless Someone
steps forward, of course.

> The AST for one version of a grammar might be incompatible enough with a 
> newer one, making the TS queries, font-lock and indentation rules 
> obsolete or at least slightly broken. nvim-treesitter works around this 
> by locking the repository version of a grammar corresponding to the 
> current language support code.
> 
> How much this will be a problem in practice for us? I'm not sure. 

Neither am I.  We'll have to wait and see.

> > (Of course, if such a motivated volunteer steps forward, he or she
> > will be most welcome.)
> 
> My guess is we have a few people here already who might be interested.

They will be welcome.