From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter maturity Date: Sun, 29 Dec 2024 17:00:17 -0800 Message-ID: <328E287A-2FDF-42AC-A535-665F98F47C92@gmail.com> References: <1ed88fca-788a-fe9f-b6c8-edb2f49751c9@mavit.org.uk> <67428b3d.c80a0220.2f3036.adbdSMTPIN_ADDED_BROKEN@mx.google.com> <86ldwdm7xg.fsf@gnu.org> <6765355b.c80a0220.1a6b24.3117SMTPIN_ADDED_BROKEN@mx.google.com> <00554790-CACA-4233-8846-9E091CF1F7AA@gmail.com> <86msgl2red.fsf@gnu.org> <87o710sr7y.fsf@debian-hx90.lan> <8734i9tmze.fsf@posteo.net> <86plldwb7w.fsf@gnu.org> <87ttapryxr.fsf@posteo.net> <0883EB00-3BB2-4BC8-95D1-45F4497C0526@dancol.org> <87msge8bv8.fsf@dancol.org> <6771db94.050a0220.386e00.e451SMTPIN_ADDED_BROKEN@mx.google.com> <77FBB3FF-A0F5-416C-AE35-39C0D818FBA9@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18374"; mail-complaints-to="usenet@ciao.gmane.io" Cc: =?utf-8?Q?Bj=C3=B6rn_Bidar?= , Lynn Winebarger , Philip Kaludercic , emacs-devel , Eli Zaretskii , Richard Stallman , manphiz@gmail.com To: Daniel Colascione Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Dec 30 02:00:59 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tS49T-0004bA-6Q for ged-emacs-devel@m.gmane-mx.org; Mon, 30 Dec 2024 02:00:59 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tS496-0001OV-Hu; Sun, 29 Dec 2024 20:00:36 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tS494-0001Nt-DV for emacs-devel@gnu.org; Sun, 29 Dec 2024 20:00:34 -0500 Original-Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tS492-0000K5-Di; Sun, 29 Dec 2024 20:00:34 -0500 Original-Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-2161eb95317so115135525ad.1; Sun, 29 Dec 2024 17:00:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735520430; x=1736125230; darn=gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=3yQnbGBVxFDQ5UsJk1QaoiiHQHEsXV9qjYQY0YeGf4M=; b=JF8NhjtYg5d7EBwIV7GIkY8vy1AOS4xr5OwiFxauTCF7ZCwOvdvDyC4PhymMxjvUWD MuNnpn/jT4vPSYEweVWcCW+HbSO2gjxSZ9XEsj+nJO89hqbf2VQHqDLFnPdW5qy3Nnf4 C/XLAccPd4zm3hmluAlVjmecA0WprZ5RTwMZ7BMpRIU44WryfNntz3QPX0q5qVcekWUl j1y5R0dSoqIxy0RI3QdkotNPcjiVJaHAS1X5ygQm7TFLSCsDEtvY3GiQAhJwIssoLGYQ ECJJ4GJ0gSWczvwzYdgmjRq7o+uhRCoccEo2/+ZVEIVYwKrndb+F+RsVovjggdA/QYtV gBhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735520430; x=1736125230; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3yQnbGBVxFDQ5UsJk1QaoiiHQHEsXV9qjYQY0YeGf4M=; b=CaofY0b02TxJY2LmdPqkGVahlS0gRnCKjEvlppp5uB3LyFSaroU6yrD9AIWdBqxldf pg2aOLNb+r++QJOwDT0BdpOzCK/rLtB8UjhohxSNUGMwiSCMHb5/f+uHCrDijYOMkt54 0eSNL3Ot/+VA41WtI4++5Tbl7dDJkM+ZW7rnYAMv1IaswqEWX39TFOxMbSVyXrWM2cDB HhP0GtqOIBigpFUve9+fA75gMlCztyl+h1CO+4osqv5jg/oULA5u5yUYuWQU72lchBhi 2QvzSVDdSY1jbi2IyKmDkqequZNSoBLCSKwlIof2tpUoh2kWyL6xz7iimXDZDZaqK5K7 yOHw== X-Forwarded-Encrypted: i=1; AJvYcCUDbBOmIn7zE0/vt2qfO5WK9JeDRm7b58uox+kx7EI7PqUM2AkOxyraYBFIuqovnMSQaqfSq82g0OxmF9M=@gnu.org, AJvYcCXAAP+adrShg8aB8vA3mYGNQMVauxKPWg/JtthYHiS47XqqnzXst/cdFHfsnzr2DAGy5kuk@gnu.org, AJvYcCXbXCtl+v+pbR27/lDWfC2JacNzT/0uLrJSQDgl23jrsR2JvB+bYhv5JDscH7CAbsA8P5fS@gnu.org X-Gm-Message-State: AOJu0Ywzbzi/REhv1zuUruoGCb2wOV90luxeZdGCUa7r/n2BIYPiZsqL 5cqMBWvi7WDUogitkPPejg/705OtJiAkRUUpyXUJa3mxErlHdZmi X-Gm-Gg: ASbGncvWKX8e9LIZf5OXrGiygGjAVDnAdcSgRa9KiDxvYzGF8Hf6IrvcWp7/FRGP9+H ctOwzy7U4pqHkuk7gGtLepuoMpJV++7gZex54rjWMjiU2iS6GEQJ8VO0a+KC8/xfKDy1kjJTX+m o+bfx8ntROJ7ekAL1DNtgZ1ggSr90e6PhCdyEdY5O7cmr9Yc9PYyVgbcN30T/DKXbcbB/qu5ggl fdvBNOIOjeAvULfxlW58l/l8mIBmAA1JShQ5XntU2i8n+75XbnmEI3iSJGbyICq6bUZFlXMr0pV zKGN X-Google-Smtp-Source: AGHT+IFG9cctW1DRgw9DMrm+TvRDdIGMUsCypvFx3iyJQjYRdAxnnVApekag3y9ULyKywGKVqdNNpQ== X-Received: by 2002:a17:903:186:b0:216:7ee9:2235 with SMTP id d9443c01a7336-219e6f14499mr440504965ad.43.1735520430057; Sun, 29 Dec 2024 17:00:30 -0800 (PST) Original-Received: from smtpclient.apple ([2601:646:8f81:6120:f1c9:d034:5332:4d9a]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc96291bsm168551625ad.14.2024.12.29.17.00.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 29 Dec 2024 17:00:29 -0800 (PST) In-Reply-To: X-Mailer: Apple Mail (2.3776.700.51) Received-SPF: pass client-ip=2607:f8b0:4864:20::636; envelope-from=casouri@gmail.com; helo=mail-pl1-x636.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327381 Archived-At: > On Dec 29, 2024, at 4:36=E2=80=AFPM, Daniel Colascione = wrote: >=20 >=20 >=20 > On December 29, 2024 7:30:52 PM EST, Yuan Fu = wrote: >>=20 >>=20 >>> On Dec 29, 2024, at 3:29=E2=80=AFPM, Bj=C3=B6rn Bidar = wrote: >>>=20 >>> Daniel Colascione writes: >>>=20 >>>> Lynn Winebarger writes: >>>>=20 >>>>> On Fri, Dec 27, 2024, 9:25=E2=80=AFAM Daniel Colascione = wrote: >>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> It's a shame there's no way to write TS grammars in plain elisp. = I figure >>>>>> vendoring both the source and the generated code would be best, = as it'd >>>>>> allow building Emacs anywhere but still make it convenient on = systems with >>>>>> needed tools (JS runtime, Rust, etc.) to update and modify the = grammar. As >>>>>> with any scheme involving checking in generated outputs, the = source and >>>>>> output can get out of sync, but I think there are build time = guardrails we >>>>>> can build to make sure it doesn't happen. >>>>>>=20 >>>>>=20 >>>>> I looked into this last year. The tree-sitter library provides a = parsing >>>>> engine that references a fairly standard LR type parsing table in = binary >>>>> form. I got stuck in adding a generic primitive functionality for = reading >>>>> and writing arbitrary binary data structures based on a data = description >>>>> DSL, since I wouldn't want to tie the interpreter core to the data >>>>> structures of an external, dynamically-loadable library. But, I = wasn't >>>>> sure such an extension would be accepted into emacs, as I am not = an expert >>>>> on the possible security implications. >>>>>=20 >>>>> Other than that, emacs already has the code for calculating (LA)LR = parsing >>>>> tables in the semantic packages. The tree-sitter grammar compiler = may have >>>>> additional logic for providing multiple starting symbols, but the = parsing >>>>> engine should still function with a classic parsing table. >>>>=20 >>>> Thanks. Such an approach would let us treat tree-sitter grammars a = lot >>>> more like font-lock-keywords, and I think for some modes, that'd be = a >>>> good option. (Of course, SHTDI.) >>>>=20 >>>> Tree sitter, as wonderful as it is, strikes me as a bit of a Rube >>>> Goldberg machine architecturally: JS *and* Rust *and* C? Really? = :-) >>=20 >>> I was wondering the same. How the hell? There had been some talks to >>> support a more lightweight JavaScript interpreter as an alternative = but >>> it hasn't gone anyway. Somehow because compatibility reason. I don't = how >>> could node be dependency for these. Grammars are mostly without >>> dependencies except some have dependencies to other grammars on the >>> source level such as the C++ require the C grammar. >>=20 >> I don=E2=80=99t think you need nodejs to build the grammar. You might = need it to develop the grammar, but compiling grammar.js to parser.c = only requires the tree-sitter CLI which is written in Rust. >>=20 >> Yuan >=20 > Doesn't the CLI shell out to Node? No, the CLI is written in Rust. The nodejs package is just a shim (it = just downloads pre-built binary from GitHub releases). Yuan=