Re: feature/tree-sitter: Where to Put C/C++ Stuff

[Eli Zaretskii <eliz@gnu.org>]

> From: Theodor Thornhill <theo@thornhill.no>
> Cc: emacs-devel@gnu.org, dev@rjt.dev, emacs-devel@gnu.org
> Date: Tue, 01 Nov 2022 08:55:44 +0100
> 
> Yes, well, partially.  I think that we are too likely to create unwanted
> issues by merging the two too closely.

Then we should merge them "not too closely", I guess.  The challenge
is to merge them so that we gain the most and lose the least.

> 1: Use CC mode for one thing and tree-sitter for the rest
> While first implementing tree-sitter in c-sharp mode we tried just
> applying font-locking, and use cc mode for indentation and the rest.
> What happened was that we immediately inherited the performance issues
> from cc mode straight into our code.

If those same performance issues exist today, then we don't lose
anything, do we?  We just gain less than we could.  But the amount of
work required for rewriting the other parts of CC Mode is huge, and we
don't want to leave users of CC Mode in a dilemma whether to switch to
a new mode and lose everything else for a significant amount of time,
or give up tree-sitter and stay with CC Mode.  Not something I'd agree
to.

I also have hard time believing that you can reimplement those slow
parts of CC Mode to be much faster, but if you have code to show which
does that, I'm sure I'd be interested to look at it and consider
improving CC Mode using that code.

> Specifically, when typing in a
> file with too many (from cc mode's perspective) strings, typing lag rose
> to several seconds per press.  I filed several bug reports on this both
> here and to Alan.  After some time and much heroics we got some
> improvement on this from Alan, but c-sharp already had moved on.

I don't know what c-sharp mode does besides fontification and
indentation, but CC Mode does a lot more, see below.  If you
disregarded a significant part of that, or if it is not relevant for
editing C# code, then your particular experience is not very
educational for the purposes of this discussion, and could lead us to
wrong conclusions.

It is trivially correct that a new mode can move much faster and make
breaking changes, but this is unacceptable for a mode that comes with
Emacs.  We respect our users much more than 3rd-party packages out
there do, and we do that for good reasons.

> 2: Using separate names for modes.
> The great advantage here is easy to understand.  You have no inheritance
> issues, and are free to develop features without regards to legacy.  A
> disadvantage is that some users depend on that major mode name for other
> stuff.

That's a _huge_ disadvantage, in my book.

> 3: Confusion with where to file bugs

Not relevant in our case: the bugs should be filed with Emacs.

> 4: How do we know what to disable?
> If there's a problem somewhere in the tree-sitter variant of the cc mode
> derived new mode, and we see some issue - who makes the fix?

Also not relevant: the answer is "we the Emacs project make the fix".

> 5: While tree-sitter is only an engine, it provides a lot more goodies
> We have a huge opportunity to create real new frameworks for emacs now,
> but limiting us to merge the features/modes suggests that we cannot
> reliably do overarching advancements such as we see now in the
> feature/tree-sitter branch.

Yes.  And trying to make breaking changes in important Emacs features
such as CC Mode is really a non-starter.  It isn't going to happen.

> 6: What are the goodies that we really need from CC mode?
> CC mode provides indentation and font locking.  What else does it
> provide that isn't replaceable pretty quickly?  I mean this not as a
> contrarian, but out of real curiosity.

CC Mode has a full-blown manual, where this question is answered.
Here's a partial list of features outside of the fontification and
indentation area, which I collected just by looking at the top-level
menus of that manual:

 . filling and breaking text in comments and strings
 . automatic insertion of newlines after braces, colons, commas, semi-colons
 . whitespace cleanups
 . minor modes: electric, hungry-delete, comment-style
 . c-offsets-alist and interactive indentation customization (related
   to indentation, but still extremely important, and not directly in
   tree-sitter)

> My guess is that we can get to feature parity and well beyond that
> in a very short amount of time, if we're not hindered by merging
> everything.

As they say, "show me the code".  If you can write up a C/C++ mode
from scratch which supports most everything in the CC Mode manual, do
it better/cleaner than CC Mode does, and do it before the emacs-29
branch is cut, in a month or so, I might change my mind.

> Sorry for the long mail, but I think we are missing the point by viewing
> tree-sitter simply as an engine to plop in aside cc mode for
> convenience, and not the real infrastructure change it is.

Who said we view tree-sitter that way?

What actually happens is that we gradually introduce tree-sitter as an
engine for replacing the implementation of Emacs features where it is
faster and/or better.  That is the plan.  There's no limit to these
replacements, except what tree-sitter can do and how we can use that.
But one thing we will NOT do is throw away existing important features
before we have equivalent replacements and before users tell us the
replacements are indeed better.

> There is no need to sunset cc mode, but equally there is no need to
> limit tree-sitter.

There's no limits.  The fact that we use tree-sitter for what we use
it now is just because _we_ decided to do that initially, in order to
have it in Emacs 29 as a useful infrastructure that users can take
advantage of.  I don't believe in releasing Emacs with infrastructure
that has no user-level features built on it.

> > Tree-sitter doesn't (and cannot) replace everything a major mode does
> > for a programming language.  So a completely new mode means we through
> > the baby with the bathwater.
> 
> I don't agree, but I'm very curious to what else would take a
> significant effort _apart_ from indentation feature parity with cc mode is.

See above: just read the CC Mode manual, and see for yourself.