From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Bj=C3=B6rn?= Bidar Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter maturity Date: Fri, 20 Dec 2024 11:13:22 +0200 Message-ID: <24089.0422648349$1734686065@news.gmane.org> References: <1ed88fca-788a-fe9f-b6c8-edb2f49751c9@mavit.org.uk> <67428b3d.c80a0220.2f3036.adbdSMTPIN_ADDED_BROKEN@mx.google.com> <86ldwdm7xg.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="8125"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Eli Zaretskii , Peter Oliver , Stefan Kangas , emacs-devel@gnu.org To: Yuan Fu Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Dec 20 10:14:17 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tOZ5M-0001zN-Po for ged-emacs-devel@m.gmane-mx.org; Fri, 20 Dec 2024 10:14:17 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tOZ4b-0004E8-VE; Fri, 20 Dec 2024 04:13:29 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tOZ4Z-0004Df-NQ for emacs-devel@gnu.org; Fri, 20 Dec 2024 04:13:27 -0500 Original-Received: from thaodan.de ([185.216.177.71]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tOZ4X-0008Kc-E6; Fri, 20 Dec 2024 04:13:27 -0500 Original-Received: from odin (dsl-trebng12-50dc7b-49.dhcp.inet.fi [80.220.123.49]) by thaodan.de (Postfix) with ESMTPSA id 1ADA3D00072; Fri, 20 Dec 2024 11:13:23 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=thaodan.de; s=mail; t=1734686003; bh=a/I6qS0rpbyCCCjpEnC8whuaIno+RVBUpLgfoESkj54=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=qmfQOgiYNcUsJlT6blCNqkMpCqCYkYpTZOt9GENeGma8nbBwz6oQO2thG0ZatZTQz Kx/QU4urVjjKy7zFIG93///H6g5WMjysXKVSBEMFYR4O4cjuh3UVeJAkOsrdZqdJHe MdGnQvLqGbg4v+2aEAln9i7SUW15VfMXiIrup/2V3SntFXzJN7sLxrudWtGeU/wyem VNZL1lC/Zwh1sCCuyZvjGIlFF4qhnVdQ+HJaln+Ekn2D0ZgzuEf53srP58Zi6moPGW HPHOkMJBKxSrax+0Uhz/TNPxB7S+E4GblP1HdEWXLGCVdd5znA9L6v2pmT3h/AYUr8 5h9ig2uok+FSr2AIKk9Xvyi0VABk7Bdvctnu+EyiGlBQOiMkDVy4fPUOn6xzy9aJQz vxuLbEyFSHnVBEWB158XgWfzwSgkrhGacPpqL6p9grda0hMtVR8t2f6wdkDmQeliFg wG6oOevrJu0pjHB5qzlwJks1FuPcBuTN0Kzj7fmqXYxse9HiFe2gjwAdgpdrDGPYzq WqBz1iDVDu9+E8NSM0Tp4181R1omV/CZknTvFhPx6oDFVyyOo1OCGajsbFTFnzoC/R rr4dfSw1tuZDHre04hUhYZG3BlyUQicyZGj5Q/9+EYs4MaFv4fYByjcznvWns5AHpQ 3UcF0dtbvuW4xU2yndD3cQbQ= In-Reply-To: (Yuan Fu's message of "Wed, 18 Dec 2024 17:40:39 -0800") Autocrypt: addr=bjorn.bidar@thaodan.de; prefer-encrypt=nopreference; keydata= mDMEZNfpPhYJKwYBBAHaRw8BAQdACBEmr+0xwIIHZfIDlZmm7sa+lHHSb0g9FZrN6qE6ru60JUJq w7ZybiBCaWRhciA8Ympvcm4uYmlkYXJAdGhhb2Rhbi5kZT6IlgQTFgoAPgIbAwULCQgHAgIiAgYV CgkICwIEFgIDAQIeBwIXgBYhBFHxdut1RzAepymoq1wbdKFlHF9oBQJk1/YmAhkBAAoJEFwbdKFl HF9oB9cBAJoIIGQKXm4cpap+Flxc/EGnYl0123lcEyzuduqvlDT0AQC3OlFKm/OiqJ8IMTrzJRZ8 phFssTkSrrFXnM2jm5PYDoiTBBMWCgA7FiEEUfF263VHMB6nKairXBt0oWUcX2gFAmTX6T4CGwMF CwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQXBt0oWUcX2hbCQEAtru7kvM8hi8zo6z9ux2h K+B5xViKuo7Z8K3IXuK5ugwA+wUfKzomzdBPhfxDsqLcEziGRxoyx0Q3ld9aermBUccHtBxCasO2 cm4gQmlkYXIgPG1lQHRoYW9kYW4uZGU+iJMEExYKADsCGwMFCwkIBwICIgIGFQoJCAsCBBYCAwEC HgcCF4AWIQRR8XbrdUcwHqcpqKtcG3ShZRxfaAUCZNf2FQAKCRBcG3ShZRxfaCzSAP4hZ7cSp0YN XYpcjHdsySh2MuBhhoPeLGXs+2kSiqBiOwD/TP8AgPEg/R+SI9GI9on7fBJJ0mp2IT8kZ2rhDOjg gA6IkwQTFgoAOxYhBFHxdut1RzAepymoq1wbdKFlH Received-SPF: pass client-ip=185.216.177.71; envelope-from=bjorn.bidar@thaodan.de; helo=thaodan.de X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, INVALID_MSGID=0.568, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:326790 Archived-At: Yuan Fu writes: >> On Dec 18, 2024, at 5:34=E2=80=AFAM, Eli Zaretskii wrote: >>=20 >>> From: Yuan Fu >>> Date: Tue, 17 Dec 2024 14:11:51 -0800 >>> Cc: Peter Oliver , >>> Stefan Kangas , >>> Emacs Devel , >>> Eli Zaretskii >>>=20 >>>>> It=E2=80=99s also worth noting that Tree-sitter itself is somewhat >>>> immature; the developers say that until it reaches version 1.0, we >>>> should be wary of potentially unannounced incompatible changes >>>> (although they are trying harder to avoid this, over time). >>>>=20 >>>>=20 >>>> [1] https://build.opensuse.org/package/show/editors/tree-sitter >>>=20 >>> I wonder if we can formalize a way for tree-sitter major modes to >>> state the compatible version of language grammar it uses. Maybe a >>> package.el cookies, or a variable that set, or even just comments >>> in the beginning of the file. >>>=20 >>> Many major modes already adds entries to treesit-language-source-alist,= that could be a good option too. >>>=20 >>> I especially want built-in major modes to give a version, so that >>> packagers can package Emacs with the right version of tree-sitter >>> grammar. I know Eli has problems with pinning a grammar version for >>> builtin modes before, but I wonder what=E2=80=99s he=E2=80=99s stance n= ow? >>=20 >> What's changed? > > People are starting to package tree-sitter and tree-sitter > grammars. If Emacs can be packaged with the right grammars, then > tree-sitter modes will work out-of-the-box. Please don't. That would require nodejs to build Emacs bundled with these grammars. These grammar packages are also not just used with Emacs. Grammars are very easy to package once the infrastructure to reuse the packaging automation in the package manager is there. Don't try to reinvent that IMHO. If you must generated and build the parser implement a bindings.gyp parser so you can automate the compilation process independently of the grammar. For reference here's my implementation of it in python: https://build.opensuse.org/projects/editors:tree-sitter/packages/tree-sitte= r/files/tree-sitter-target.py?expand=3D1 >>=20 >> Many language grammars don't make official releases and thus don't >> have versions. Moreover, AFAIK there's no API to determine the >> version of the grammar library we load. So how can we manage such >> version-pinning in a way that (a) is up-to-date, and (b) doesn't >> preclude people from using a grammar library due to false negatives? > > I=E2=80=99m talking about a softer pin. We=E2=80=99re basically providing= a =E2=80=9Cknown to > work=E2=80=9D version. This way packagers can package Emacs with a > known-to-work version of grammar, so the builtin modes work > out-of-the-box. This doesn=E2=80=99t prevent people from using a newer ve= rsion > and sending us a bug report, and we still try our best to make the > major modes work with the newest grammar. > > If the grammar doesn=E2=80=99t have an explicit version, then we can just= use a commit hash. I believe all the packaging systems support that? That doesn't make sense as the versions numbers are arbitrary, e.g. not always does the version number relate the changes to grammar but also to the in-tree dependencies in the repository packaging the language-grammar bindings which have nothing todo with the parser. What matters much more is the tree-sitter version which is more related to Emacs itself rather than the particular version of the grammar.