From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Bj=C3=B6rn?= Bidar Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter maturity Date: Tue, 31 Dec 2024 05:22:29 +0200 Message-ID: <39746.4914758288$1735615383@news.gmane.org> References: <1ed88fca-788a-fe9f-b6c8-edb2f49751c9@mavit.org.uk> <67428b3d.c80a0220.2f3036.adbdSMTPIN_ADDED_BROKEN@mx.google.com> <86ldwdm7xg.fsf@gnu.org> <6765355b.c80a0220.1a6b24.3117SMTPIN_ADDED_BROKEN@mx.google.com> <00554790-CACA-4233-8846-9E091CF1F7AA@gmail.com> <86msgl2red.fsf@gnu.org> <87o710sr7y.fsf@debian-hx90.lan> <8734i9tmze.fsf@posteo.net> <86plldwb7w.fsf@gnu.org> <87ttapryxr.fsf@posteo.net> <0883EB00-3BB2-4BC8-95D1-45F4497C0526@dancol.org> <87msge8bv8.fsf@dancol.org> <6771db94.050a0220.386e00.e451SMTPIN_ADDED_BROKEN@mx.google.com> <77FBB3FF-A0F5-416C-AE35-39C0D818FBA9@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="35881"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Daniel Colascione , Lynn Winebarger , Philip Kaludercic , emacs-devel , Eli Zaretskii , Richard Stallman , manphiz@gmail.com To: Yuan Fu Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Dec 31 04:22:55 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tSSqM-0009DB-JK for ged-emacs-devel@m.gmane-mx.org; Tue, 31 Dec 2024 04:22:54 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tSSqA-0006yH-GC; Mon, 30 Dec 2024 22:22:42 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tSSq9-0006xz-2T for emacs-devel@gnu.org; Mon, 30 Dec 2024 22:22:41 -0500 Original-Received: from thaodan.de ([2a03:4000:4f:f15::1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tSSq3-0005n9-7Z; Mon, 30 Dec 2024 22:22:40 -0500 Original-Received: from odin (dsl-trebng12-50dc7b-49.dhcp.inet.fi [80.220.123.49]) by thaodan.de (Postfix) with ESMTPSA id B2DE6D00049; Tue, 31 Dec 2024 05:22:30 +0200 (EET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=thaodan.de; s=mail; t=1735615351; bh=7s0ulfMkyKGP/zjduJ7yq8C0LM8Co0g2/oiCQKKsmOo=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=WNUWoHJLdH8qAKXnnQ3SEa4gA9o9ba5kPaNv5HX8KFstQDMbpdbPNgv8frsk2zIH7 qXQupyw0zDgvztB+XkG7XQjdYMqGgAiQ0G7dpsMJBoKq/zV9LL11vxcnEMHRGhVoqK h9+hZ7uY2H5EyffNwELI8PCAv81fDvXV58WjYh55QjrEq2APUCTQaAtlO2+Znh5bZQ CkIB8p1ANoA3VvrtzlSks0EMvHPW3HOgGMAJs0OC1ur8C1eFuX5rE9k0qPatFmdV2J p4mKWy1uDCKI6xFgKO4av0CE9nJQ0U2zortnYOwcf6pel98JsrnR2Vjaf9H399s04W EpSvDvf1F3gei2BRwN1PRtZK3tozUgt6YoBTS0bsy3gpvR8wttM7mcFqhRDw/7R5pa 9ZURsT1+0Wel8Ow4bbM/H0mWvOWUockR6euclXcN7ROv9Jfzxody1rd0pQk+buuGEx 2AVAwEmPvJhDq6jEjF2vs0q6AwYkr2vSw0/+S1PfOsjc3Kp7rj2Z3eB/cCrOks0WW3 +UcK5cte5pIdGrBmkiad6IR9K96PCqD9VD47aZtOUiu5e0FHbCwJzvMDOMh7X6nsBK 1L6wDkXkYk/zf+od3jYQ2XYmOyHhOfuTBP7UkEGh4d1+gco4auySNiw260D0tVrCgW mTHj5qGGg9a137Zpv5Aswnds= In-Reply-To: <77FBB3FF-A0F5-416C-AE35-39C0D818FBA9@gmail.com> (Yuan Fu's message of "Sun, 29 Dec 2024 16:30:52 -0800") Autocrypt: addr=bjorn.bidar@thaodan.de; prefer-encrypt=nopreference; keydata= mDMEZNfpPhYJKwYBBAHaRw8BAQdACBEmr+0xwIIHZfIDlZmm7sa+lHHSb0g9FZrN6qE6ru60JUJq w7ZybiBCaWRhciA8Ympvcm4uYmlkYXJAdGhhb2Rhbi5kZT6IlgQTFgoAPgIbAwULCQgHAgIiAgYV CgkICwIEFgIDAQIeBwIXgBYhBFHxdut1RzAepymoq1wbdKFlHF9oBQJk1/YmAhkBAAoJEFwbdKFl HF9oB9cBAJoIIGQKXm4cpap+Flxc/EGnYl0123lcEyzuduqvlDT0AQC3OlFKm/OiqJ8IMTrzJRZ8 phFssTkSrrFXnM2jm5PYDoiTBBMWCgA7FiEEUfF263VHMB6nKairXBt0oWUcX2gFAmTX6T4CGwMF CwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQXBt0oWUcX2hbCQEAtru7kvM8hi8zo6z9ux2h K+B5xViKuo7Z8K3IXuK5ugwA+wUfKzomzdBPhfxDsqLcEziGRxoyx0Q3ld9aermBUccHtBxCasO2 cm4gQmlkYXIgPG1lQHRoYW9kYW4uZGU+iJMEExYKADsCGwMFCwkIBwICIgIGFQoJCAsCBBYCAwEC HgcCF4AWIQRR8XbrdUcwHqcpqKtcG3ShZRxfaAUCZNf2FQAKCRBcG3ShZRxfaCzSAP4hZ7cSp0YN XYpcjHdsySh2MuBhhoPeLGXs+2kSiqBiOwD/TP8AgPEg/R+SI9GI9on7fBJJ0mp2IT8kZ2rhDOjg gA6IkwQTFgoAOxYhBFHxdut1RzAepymoq1wbdKFlH Received-SPF: pass client-ip=2a03:4000:4f:f15::1; envelope-from=bjorn.bidar@thaodan.de; helo=thaodan.de X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, INVALID_MSGID=0.568, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327473 Archived-At: Yuan Fu writes: >> On Dec 29, 2024, at 3:29=E2=80=AFPM, Bj=C3=B6rn Bidar wrote: >>=20 >> Daniel Colascione writes: >>=20 >>> Lynn Winebarger writes: >>>=20 >>>> On Fri, Dec 27, 2024, 9:25=E2=80=AFAM Daniel Colascione wrote: >>>>=20 >>>>>=20 >>>>>=20 >>>>> It's a shame there's no way to write TS grammars in plain elisp. I fi= gure >>>>> vendoring both the source and the generated code would be best, as it= 'd >>>>> allow building Emacs anywhere but still make it convenient on systems= with >>>>> needed tools (JS runtime, Rust, etc.) to update and modify the gramma= r. As >>>>> with any scheme involving checking in generated outputs, the source a= nd >>>>> output can get out of sync, but I think there are build time guardrai= ls we >>>>> can build to make sure it doesn't happen. >>>>>=20 >>>>=20 >>>> I looked into this last year. The tree-sitter library provides a pars= ing >>>> engine that references a fairly standard LR type parsing table in bina= ry >>>> form. I got stuck in adding a generic primitive functionality for rea= ding >>>> and writing arbitrary binary data structures based on a data descripti= on >>>> DSL, since I wouldn't want to tie the interpreter core to the data >>>> structures of an external, dynamically-loadable library. But, I wasn't >>>> sure such an extension would be accepted into emacs, as I am not an ex= pert >>>> on the possible security implications. >>>>=20 >>>> Other than that, emacs already has the code for calculating (LA)LR par= sing >>>> tables in the semantic packages. The tree-sitter grammar compiler may= have >>>> additional logic for providing multiple starting symbols, but the pars= ing >>>> engine should still function with a classic parsing table. >>>=20 >>> Thanks. Such an approach would let us treat tree-sitter grammars a lot >>> more like font-lock-keywords, and I think for some modes, that'd be a >>> good option. (Of course, SHTDI.) >>>=20 >>> Tree sitter, as wonderful as it is, strikes me as a bit of a Rube >>> Goldberg machine architecturally: JS *and* Rust *and* C? Really? :-) > >> I was wondering the same. How the hell? There had been some talks to >> support a more lightweight JavaScript interpreter as an alternative but >> it hasn't gone anyway. Somehow because compatibility reason. I don't how >> could node be dependency for these. Grammars are mostly without >> dependencies except some have dependencies to other grammars on the >> source level such as the C++ require the C grammar. > > I don=E2=80=99t think you need nodejs to build the grammar. You might nee= d it > to develop the grammar, but compiling grammar.js to parser.c only > requires the tree-sitter CLI which is written in Rust. If you want to build from source the you need to compile grammar.js. That the compiled grammar is bundled is practice for compatibility reasons.