From: "Björn Bidar" <bjorn.bidar@thaodan.de>
To: Yuan Fu <casouri@gmail.com>
Cc: Eli Zaretskii <eliz@gnu.org>,
Peter Oliver <p.d.oliver@mavit.org.uk>,
Stefan Kangas <stefankangas@gmail.com>,
emacs-devel@gnu.org
Subject: Re: Tree-sitter maturity
Date: Mon, 23 Dec 2024 02:43:09 +0200 [thread overview]
Message-ID: <30474.9437769473$1734914665@news.gmane.org> (raw)
In-Reply-To: <00554790-CACA-4233-8846-9E091CF1F7AA@gmail.com> (Yuan Fu's message of "Fri, 20 Dec 2024 01:29:14 -0800")
Yuan Fu <casouri@gmail.com> writes:
>> On Dec 20, 2024, at 1:13 AM, Björn Bidar <bjorn.bidar@thaodan.de> wrote:
>>
>> Yuan Fu <casouri@gmail.com> writes:
>>
>>>> On Dec 18, 2024, at 5:34 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>>>>
>>>>> From: Yuan Fu <casouri@gmail.com>
>>>>> Date: Tue, 17 Dec 2024 14:11:51 -0800
>>>>> Cc: Peter Oliver <p.d.oliver@mavit.org.uk>,
>>>>> Stefan Kangas <stefankangas@gmail.com>,
>>>>> Emacs Devel <emacs-devel@gnu.org>,
>>>>> Eli Zaretskii <eliz@gnu.org>
>>>>>
>>>>>>> It’s also worth noting that Tree-sitter itself is somewhat
>>>>>> immature; the developers say that until it reaches version 1.0, we
>>>>>> should be wary of potentially unannounced incompatible changes
>>>>>> (although they are trying harder to avoid this, over time).
>>>>>>
>>>>>>
>>>>>> [1] https://build.opensuse.org/package/show/editors/tree-sitter
>>>>>
>>>>> I wonder if we can formalize a way for tree-sitter major modes to
>>>>> state the compatible version of language grammar it uses. Maybe a
>>>>> package.el cookies, or a variable that set, or even just comments
>>>>> in the beginning of the file.
>>>>>
>>>>> Many major modes already adds entries to treesit-language-source-alist, that could be a good option too.
>>>>>
>>>>> I especially want built-in major modes to give a version, so that
>>>>> packagers can package Emacs with the right version of tree-sitter
>>>>> grammar. I know Eli has problems with pinning a grammar version for
>>>>> builtin modes before, but I wonder what’s he’s stance now?
>>>>
>>>> What's changed?
>>>
>>> People are starting to package tree-sitter and tree-sitter
>>> grammars. If Emacs can be packaged with the right grammars, then
>>> tree-sitter modes will work out-of-the-box.
>>
>> Please don't. That would require nodejs to build Emacs bundled with
>> these grammars. These grammar packages are also not just used with
>> Emacs.
>>
>> Grammars are very easy to package once the infrastructure to reuse the
>> packaging automation in the package manager is there. Don't try to
>> reinvent that IMHO. If you must generated and build the parser implement
>> a bindings.gyp parser so you can automate the compilation process
>> independently of the grammar.
>
> There might be some misunderstanding. We don’t want to build the
> grammars as part of building Emacs. Ideally building the grammars are
> the package managers job. We just want to list the versions of
> grammars that are known to work with the major modes, so packagers
> have an easier time to package Emacs with the right version of
> grammars.
Ah ok now I understand. I don't think that would work.
>>
>> For reference here's my implementation of it in python:
>> https://build.opensuse.org/projects/editors:tree-sitter/packages/tree-sitter/files/tree-sitter-target.py?expand=1
>>
>>>>
>>>> Many language grammars don't make official releases and thus don't
>>>> have versions. Moreover, AFAIK there's no API to determine the
>>>> version of the grammar library we load. So how can we manage such
>>>> version-pinning in a way that (a) is up-to-date, and (b) doesn't
>>>> preclude people from using a grammar library due to false negatives?
>>>
>>> I’m talking about a softer pin. We’re basically providing a “known to
>>> work” version. This way packagers can package Emacs with a
>>> known-to-work version of grammar, so the builtin modes work
>>> out-of-the-box. This doesn’t prevent people from using a newer version
>>> and sending us a bug report, and we still try our best to make the
>>> major modes work with the newest grammar.
>>>
>>> If the grammar doesn’t have an explicit version, then we can just use a commit hash. I believe all the packaging systems support that?
>>
>> That doesn't make sense as the versions numbers are arbitrary, e.g. not
>> always does the version number relate the changes to grammar but also to
>> the in-tree dependencies in the repository packaging the
>> language-grammar bindings which have nothing todo with the parser.
>
> Sure, let’s call it snapshot then. I just want to make sure when
> packagers package Emacs with tree-sitter grammars, the grammar works
> with Emacs’s major mode.
The point was that now matter what you call the development of grammars
is more or less fluent. Maybe there are some more mature grammar but
those should be the minority.
But lets just assume for a second it would be possible to freeze or
recommend the supported grammar versions. The development of grammars is
to fast for that, especially for builtin modes.
>>
>> What matters much more is the tree-sitter version which is more related
>> to Emacs itself rather than the particular version of the grammar.
>
> The tree-sitter library version is up to the packagers right? As long as it satisfies Emacs’ requirements and is compatible with the bundled grammars.
Do mean bundled or recommended grammars? Grammars bundled would be again
grammars included within the Emacs sources which is a different thing
from what I you were saying further above.
Yes the tree-sitter version is up to the package or respectively the
distribution.
The only issue that existed regarding was that tree-sitter once broke
the ABI without bumping the sover but that's fixed now or was fixed when
Emacs correctly rebuilt once a dependency of it changed.
next prev parent reply other threads:[~2024-12-23 0:43 UTC|newest]
Thread overview: 127+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-11-20 15:13 My resignation from Emacs development Alan Mackenzie
2024-11-20 15:34 ` Eli Zaretskii
2024-11-20 16:23 ` Christopher Dimech
2024-11-21 6:22 ` Gerd Möllmann
2024-11-21 10:05 ` Christopher Dimech
2024-11-21 11:23 ` Gerd Möllmann
2024-11-21 11:40 ` Eli Zaretskii
2024-11-21 10:29 ` Alan Mackenzie
2024-11-21 12:26 ` Christopher Dimech
2024-11-20 16:42 ` Alfred M. Szmidt
2024-11-20 17:04 ` tomas
2024-11-20 21:56 ` Dmitry Gutov
2024-11-21 2:28 ` Stefan Kangas
2024-11-21 12:34 ` Tree-sitter maturity (was: My resignation from Emacs development) Peter Oliver
2024-11-23 13:41 ` Stefan Kangas
2024-11-24 2:10 ` Tree-sitter maturity Björn Bidar
[not found] ` <67428b3d.c80a0220.2f3036.adbdSMTPIN_ADDED_BROKEN@mx.google.com>
2024-12-17 22:11 ` Yuan Fu
2024-12-18 13:34 ` Eli Zaretskii
2024-12-19 1:40 ` Yuan Fu
2024-12-19 8:17 ` Eli Zaretskii
2024-12-20 9:13 ` Björn Bidar
[not found] ` <6765355b.c80a0220.1a6b24.3117SMTPIN_ADDED_BROKEN@mx.google.com>
2024-12-20 9:29 ` Yuan Fu
2024-12-23 0:43 ` Björn Bidar [this message]
[not found] ` <6768b256.c80a0220.222b1b.64e6SMTPIN_ADDED_BROKEN@mx.google.com>
2024-12-24 1:20 ` Yuan Fu
[not found] ` <87frmfxm8y.fsf@>
2024-12-24 4:52 ` Richard Stallman
2024-12-24 12:32 ` Eli Zaretskii
2024-12-24 21:31 ` Xiyue Deng
2024-12-19 12:23 ` Peter Oliver
2024-12-19 12:42 ` Eli Zaretskii
2024-12-19 13:15 ` Vincenzo Pupillo
2024-12-20 8:59 ` Björn Bidar
2024-11-21 13:01 ` My resignation from Emacs development Alan Mackenzie
2024-11-21 13:48 ` Eli Zaretskii
2024-11-21 14:29 ` Alfred M. Szmidt
2024-11-22 0:01 ` Po Lu
2024-11-22 7:03 ` Eli Zaretskii
2024-11-22 8:14 ` Robert Pluim
2024-11-22 8:32 ` Eli Zaretskii
2024-11-22 23:59 ` Po Lu
2024-11-23 6:39 ` Eli Zaretskii
2024-11-21 16:29 ` Alan Mackenzie
2024-11-22 5:35 ` Adam Porter
2024-11-22 7:24 ` Madhu
2024-11-22 8:11 ` Eli Zaretskii
2024-11-22 9:26 ` Madhu
2024-11-22 12:07 ` Eli Zaretskii
2024-11-22 12:40 ` Stefan Kangas
2024-11-22 13:06 ` Alan Mackenzie
2024-11-22 13:39 ` Stefan Kangas
2024-11-22 14:25 ` Eli Zaretskii
2024-11-25 4:28 ` Richard Stallman
2024-11-26 17:37 ` Alan Mackenzie
2024-12-13 4:35 ` Richard Stallman
2024-12-15 15:27 ` Alan Mackenzie
2024-12-15 15:48 ` Eli Zaretskii
2024-12-15 20:43 ` Alan Mackenzie
2024-12-19 4:22 ` Richard Stallman
2024-12-19 8:26 ` Eli Zaretskii
2024-11-23 22:18 ` Andrea Corallo
2024-11-22 10:57 ` Alan Mackenzie
2024-11-22 23:19 ` Adam Porter
2024-11-26 19:01 ` Daniel Radetsky
2024-11-26 19:51 ` Christopher Dimech
2024-11-27 2:18 ` Adam Porter
2024-11-27 9:36 ` Daniel Radetsky
2024-11-27 9:59 ` Christopher Dimech
2024-11-30 3:52 ` Richard Stallman
2024-11-30 7:53 ` Eli Zaretskii
2024-11-30 16:22 ` Discuss new features/enhancements or large changes for users in emacs-devel [was: My resignation from Emacs development] Drew Adams
2024-11-30 16:56 ` Eli Zaretskii
2024-11-30 21:06 ` [External] : " Drew Adams
2024-12-01 6:00 ` Eli Zaretskii
2024-12-03 7:26 ` My resignation from Emacs development Richard Stallman
2024-12-03 13:33 ` Eli Zaretskii
2024-11-30 16:21 ` Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development] Drew Adams
2024-11-30 17:05 ` Eli Zaretskii
2024-11-30 21:09 ` [External] : " Drew Adams
2024-12-01 6:12 ` Eli Zaretskii
2024-12-01 19:23 ` Drew Adams
2024-12-03 7:25 ` Richard Stallman
2024-12-03 13:32 ` Eli Zaretskii
2024-12-06 4:48 ` Richard Stallman
2024-12-02 4:09 ` Richard Stallman
2024-12-02 13:04 ` Discuss new features/enhancements or large changes for users in emacs-devel Eli Zaretskii
2024-12-02 15:32 ` [External] : " Drew Adams
2024-12-05 5:08 ` Richard Stallman
2024-12-05 6:33 ` Eli Zaretskii
2024-12-02 15:29 ` [External] : Re: Discuss new features/enhancements or large changes for users in emacs-devel [was My resignation from Emacs development] Drew Adams
2024-11-27 2:06 ` My resignation from Emacs development Adam Porter
2024-11-27 9:17 ` Daniel Radetsky
2024-11-22 15:36 ` Stefan Kangas
2024-11-22 17:48 ` Alan Mackenzie
2024-11-23 23:43 ` Stefan Monnier via Emacs development discussions.
2024-11-23 6:10 ` Richard Stallman
2024-11-23 7:48 ` Eli Zaretskii
2024-11-23 11:06 ` Christopher Dimech
2024-11-23 11:54 ` Eli Zaretskii
2024-11-23 12:48 ` Christopher Dimech
2024-11-23 23:59 ` Adam Porter
2024-12-01 3:50 ` Sean Whitton
2024-12-01 6:19 ` tomas
2024-11-24 18:12 ` Suhail Singh
2024-11-26 4:56 ` Richard Stallman
2024-11-26 7:38 ` Suhail Singh
2024-11-21 5:59 ` Gerd Möllmann
2024-11-22 11:36 ` Alan Mackenzie
2024-11-22 11:52 ` Eli Zaretskii
2024-11-23 10:36 ` Alan Mackenzie
2024-11-23 11:31 ` Eli Zaretskii
2024-11-21 13:39 ` Andrea Corallo
2024-11-21 19:01 ` Alfred M. Szmidt
2024-11-21 19:19 ` Christopher Dimech
2024-11-21 19:47 ` Eli Zaretskii
2024-11-21 19:40 ` Jim Porter
2024-11-24 4:35 ` Richard Stallman
2024-11-21 23:57 ` Po Lu
2024-11-22 17:26 ` On committing significant and/or controversial changes (was: My resignation from Emacs development) Ihor Radchenko
2024-11-22 17:47 ` Ship Mints
2024-11-22 19:04 ` Eli Zaretskii
2024-11-24 2:35 ` On committing significant and/or controversial changes Björn Bidar
2024-11-24 4:41 ` Adam Porter
2024-11-30 2:16 ` Björn Bidar
[not found] ` <87ttbx73zu.fsf@>
2024-11-24 8:26 ` Eli Zaretskii
2024-11-22 19:01 ` Eli Zaretskii
2024-11-23 6:10 ` My resignation from Emacs development Richard Stallman
2024-11-23 8:50 ` Eli Zaretskii
2024-11-23 6:10 ` Richard Stallman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='30474.9437769473$1734914665@news.gmane.org' \
--to=bjorn.bidar@thaodan.de \
--cc=casouri@gmail.com \
--cc=eliz@gnu.org \
--cc=emacs-devel@gnu.org \
--cc=p.d.oliver@mavit.org.uk \
--cc=stefankangas@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.