Re: Suggesting that feature/tree-sitter be merged (was Re: Tree-sitter and major mode inheritance)

[Philip Kaludercic <philipk@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Yuan Fu <casouri@gmail.com>,  emacs-devel <emacs-devel@gnu.org>,
>>   Theodor Thornhill <theo@thornhill.no>,  Eli Zaretskii <eliz@gnu.org>,
>>   jostein@kjonigsen.net
>> Date: Fri, 18 Nov 2022 22:34:13 +0000
>> 
>> Jostein Kjønigsen <jostein@secure.kjonigsen.net> writes:
>> 
>> > Instead of waiting for "every" major-mode to be re-implemented into a
>> > tree-sitter derivative in the feature/tree-sitter branch before we
>> > merge... How about we just accept the current "core" tree-sitter
>> > implementation as good enough, and consider merging that to git master
>> > as is.
>> 
>> I think this sounds like a good idea -- as someone who has mostly just
>> been following the discussions.  The core bindings and major modes that
>> are based on these are separate issues, with a clear dependency linked
>> them.
>
> From where I stand, it makes very little sense to release Emacs 29
> with tree-sitter support that is limited to primitives and some
> minimal Lisp glue on top of that.  Tree-sitter was added to Emacs to
> allow major modes provide better support for editing program source
> code, so having tree-sitter "support" in Emacs 29 that didn't include
> at least several major modes using it would be disappointing at best.
> It would mean we ourselves have no idea how to make major modes use
> the feature.  Moreover, adding those few major modes on the branch
> exposed several deficiencies in the original design and
> implementation, and required changes to make the integration better;
> releasing Emacs 29 with those issues unresolved (and unknown) would
> require significant, sometimes incompatible changes in the future,
> which is another reason why it would be wrong.
>
> Basically, my firm belief is that adding to Emacs infrastructure
> without user-level applications built on that infrastructure is wrong
> and runs the risk of producing features that are not used or need deep
> surgery before they become useful.  We should avoid doing that as much
> as possible.

My question is, do these user-level applications have to be distributed
along with Emacs, or could they be made to be "explicitly" opt-in by
installing them from ELPA.  In-core appears to usually bring a
commitment to maintain a library, and deprecating can take years.  If
Emacs 29 lays the technical foundations, the low-level API for
treesitter to work, we can have packages on ELPA experiment with the
higher-level abstractions.  Whatever is the most successful approach,
can be added to Emacs later on.

>> As an aside: This might also be a good opportunity to clean up some of
>> the current major mode implementations and make them more consistent.
>> The issue with custom options to enable tree-sitter for every major mode
>> has revealed an inherent duplication of features.  There are other
>> inconsistencies, especially regarding bindings for equivalent operations
>> (e.g. in interpreted language with a repl, how to load function into the
>> current session: Lisp, Prolog, Python all differ in minor details).
>
> Cleaning up major modes is a Good Thing that needs no opportunities.
> We should do that whenever we know and agree how.

Fair enough, but just as above I think these kinds of experiments are
better made outside of the core, in ELPA, to avoid committing to
mistakes.  If it works out, it can be added.

>> The current branch has major modes, should these be deleted before
>> merging?
>
> Definitely not!  These modes are there because we want Emacs 29 to
> have them, and we want users to use them and report back.

IIUC these modes aren't ripe yet, or at least aren't satisfying
replacements for the existing modes.  If tree-sitter were not to be
merged for that reason, that would delay the ability to use tree-sitter
on a widespread basis for at least another release.  My proposal above
would make it possible, and encourage users to report on their
experience, while allowing for the flexibility to make the right
decisions in the long term.