From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter maturity Date: Sun, 29 Dec 2024 02:21:21 -0800 Message-ID: <81F142F0-A0E8-472D-8AF0-6607E7ACF8D9@gmail.com> References: <67428b3d.c80a0220.2f3036.adbdSMTPIN_ADDED_BROKEN@mx.google.com> <86ldwdm7xg.fsf@gnu.org> <6765355b.c80a0220.1a6b24.3117SMTPIN_ADDED_BROKEN@mx.google.com> <00554790-CACA-4233-8846-9E091CF1F7AA@gmail.com> <86msgl2red.fsf@gnu.org> <87o710sr7y.fsf@debian-hx90.lan> <8734i9tmze.fsf@posteo.net> <86plldwb7w.fsf@gnu.org> <87ttapryxr.fsf@posteo.net> <0883EB00-3BB2-4BC8-95D1-45F4497C0526@dancol.org> <87plldrx6a.fsf@posteo.net> <87ikr5rwx0.fsf@posteo.net> <86ed1rq6gc.fsf@gnu.org> <73737665-984E-4C97-9183-7805C1BCB550@dancol.org> <86bjwuricz.fsf@gnu.org> <5D1AB17D-8847-42ED-B246-8DB4D11F7FB7@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="39407"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , emacs-devel@gnu.org To: Daniel Colascione Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Dec 29 11:22:14 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tRqR4-000A76-A6 for ged-emacs-devel@m.gmane-mx.org; Sun, 29 Dec 2024 11:22:14 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tRqQU-0001Ek-EE; Sun, 29 Dec 2024 05:21:38 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tRqQT-0001Eb-D7 for emacs-devel@gnu.org; Sun, 29 Dec 2024 05:21:37 -0500 Original-Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tRqQR-0001wo-83; Sun, 29 Dec 2024 05:21:36 -0500 Original-Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-21a1e6fd923so45799175ad.1; Sun, 29 Dec 2024 02:21:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735467693; x=1736072493; darn=gnu.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=1gnlQAxQonsgFI2M7cqjfoyiLwYP1RCMW901yUEH+5o=; b=kGUfRVTg/0/o6sEmvw/IeAuNc6QCNqJiXZJUryEbfyN5FC/SJb1xfAw+4X+eplqfyt Et8ZIpckBXoVdT65zTixxcfNyca3cueQ5rlMjJY5abbGjx6lQ830ru3DhmwNzeieegdz AyxnNY7Pv8r+4KJXQozGt5lLn7y0OdO9VDMjn8fE1L3hDNYoYyny3h13S9ELYrZaPiiq QcInH5E4n+walq6zdoanAor+n1MAUTdlW4KGcjV9u+i/XLXb2PtRdXlNyM7jfyTm0C40 ktWgrg+eBcp2FNCYfH9q6Fo9+5MiC0sqfC+dyvFI0jYyA9Lwy7usL40WgSbpa2ZVUHZi jSIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735467693; x=1736072493; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1gnlQAxQonsgFI2M7cqjfoyiLwYP1RCMW901yUEH+5o=; b=b8THFsAE3agbWHOxJHkiT+FQIBL8IE5byKyWuSnfAOn3W8ri+wTJUd7nJYRKXMuejD K7s2cAywrnFrHQkeytjegbGociz7VhKQsLortYIRPUYrq1GFzapLKWC1QoCcI2RQ6aEN EjsOgDatCr6LPhpTskNLokenjrKm4SGVXXLIwE3zXErljL/yv5Y/OAMTiHbs25A1LsS7 mE89gGt+sz0UrPueMnIzL7atEFWpOLTl73iFx2pKS3ybqFp/kqJEGOyQu6UkvS0+tC2y JXn5ZKgvlp+F55LxR7MTAO2qWsOrTbu3sZuRLbLUIwF0tdrj2V0plCnLkU57nIb4JUh9 BTfQ== X-Forwarded-Encrypted: i=1; AJvYcCUE9i8mtC9keabYPWm1dD3vXYhIxl2+eX2PKq5ig8tPDnC1mW5DWif9r1/3k4zXaGfPaBjpW9zzCkA5jQ==@gnu.org X-Gm-Message-State: AOJu0YwN6hF3yWBXrvo56LmbFJ2s98iSz61u8OE4aIsdaJeX9K8ouO3s juMfWYwkkcPjL4C67RM8CKFPTElIsY1QUjJq+dr29BPnwz3ytdTC X-Gm-Gg: ASbGncsTXYa7vK9MH48u5GWZWA8kItws0WoGDaXvSFtx2cvJgECmam907WJ8ESo8GNG oW+OCTimDZCwC0OeT04rRV7THC4QdvESCIVr0tXGxJXEUXDVUFmfOkLgPu1f15ifSeqVXiIZNNh Y28bE/UoHS17GueJ7gvsrVeTJwY7bgXirElk76hy90+W3jje4RCGhGJ0AkAkMlgOOQrXIkejwBo Jkxqpq2h+ZuJHBBe+5oNQoJHsab6GI4ToCl4PYmz1kQbZOxHp8THfXWBZiFJGCpLU/Ojmq+3ngU g9h5 X-Google-Smtp-Source: AGHT+IFcHzsrb6SYt4flLOURJjL+TCWqOXsiLMBy8OFaUQQ027e2Vf07FeoZMaJexeEd7XgMQNLfNg== X-Received: by 2002:a05:6a00:44cc:b0:725:df1a:275 with SMTP id d2e1a72fcca58-72abdeb7016mr51118701b3a.23.1735467693320; Sun, 29 Dec 2024 02:21:33 -0800 (PST) Original-Received: from smtpclient.apple ([2601:646:8f81:6120:f1c9:d034:5332:4d9a]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72aad816529sm17437859b3a.5.2024.12.29.02.21.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 29 Dec 2024 02:21:32 -0800 (PST) In-Reply-To: X-Mailer: Apple Mail (2.3776.700.51) Received-SPF: pass client-ip=2607:f8b0:4864:20::631; envelope-from=casouri@gmail.com; helo=mail-pl1-x631.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327329 Archived-At: > On Dec 29, 2024, at 1:14=E2=80=AFAM, Daniel Colascione = wrote: >=20 >=20 >=20 > On December 29, 2024 3:59:50 AM EST, Yuan Fu = wrote: >>=20 >>=20 >>> On Dec 29, 2024, at 12:41=E2=80=AFAM, Eli Zaretskii = wrote: >>>=20 >>>> Date: Sun, 29 Dec 2024 03:01:44 -0500 >>>> From: Daniel Colascione >>>>=20 >>>>>> Enforcing this policy will just mean that Emacs doesn't support = *at all* some languages out of the box and will put even more wind in = the sails of soft forks like Doom. Tree sitter language descriptions are = free software. There's no reason not to rely on them. >>>>>=20 >>>>> We started with this concept of adding tree-sitter based modes to >>>>> auto-mode-alist by default, but found that people who don't have = the >>>>> grammar installed didn't appreciate seeing the warnings about the >>>>> missing grammars. So Emacs 29 made these modes optional, = activated >>>>> only by an explicit user action. Emacs 30 still does that. >>>>>=20 >>>>> We are currently discussing how to improve this (see the thread = Re: >>>>> Turning on/off tree-sitter modes, which seems to have stalled = lately). >>>>> But until the grammar libraries are ubiquitous, and we can rely on >>>>> them being present on most systems, I think we will still need = some >>>>> user say-so before enabling tree-sitter based modes. >>>>=20 >>>> Wouldn't vendoring the grammars, and maybe even tree sitter itself, = silence the complaints about the warnings? Tree sitter is pure = algorithmic code. It doesn't have any particular platform dependencies. = Why not simplify the whole system and make it a mandatory (and = optionally bundled) dependency so that the show cognitive load of having = to consider non-TS environments is just deleted? >>>=20 >>> First, the tree-sitter library itself is optional, so Emacs could be >>> built without it. Or are you suggesting to import the library as = well >>> into Emacs? If we don't import the library, making it a mandatory >>> dependency is not TRT, IMO, because some users don't need the modes >>> supported by tree-sitter, so forcing them to install the library = that >>> is not really useful to them is not right. We never do anything = like >>> that with any other external libraries. GMP is special, but even = for >>> it we added our own "mini-gmp". >>>=20 >>> Next, importing the grammar libraries into Emacs is not a simple >>> matter, either. Their sources are in JavaScript, so if we want to = let >>> users produce modified grammars (as we do with everything we have in >>> the release tarballs), they will need to have Node.js etc. = installed, >>> which will become a prerequisite. And there are other = complications, >>> like the need to sync regularly with their upstream repositories. >>> Moreover, there's no precedent for doing this, if you exclude lwlib >>> and oldXMenu (which are different, since they are not developed >>> outside Emacs). >>>=20 >>> So I, for one, am not very happy to add this to our maintenance >>> burden. It might make things easier for some (but see below), but = it >>> doesn't come for free. >>>=20 >>> I also don't understand the fuss, really. Compiling a grammar = library >>> after cloning the repository takes seconds, so why do we have to do >>> all this on behalf of the users if the users can do it so easily, = even >>> if distros don't? E.g., I have on my system almost 70 grammar >>> libraries, which I regularly update and build with a small number of >>> simple Makefiles -- how hard can that be for anyone who is = interested >>> in these modes? Why does it have to be _our_ responsibility, any = more >>> than, say, Grep or Findutils -- which are also heavily used by = Emacs? >>> Or even the image libraries? Why shouldn't this be the job of the >>> distros? The upstream project doesn't have to think about = packaging, >>> it's the job of the distros. >>=20 >> Also, distros are picking up on packaging tree-sitter grammars, so = I=E2=80=99m hopeful of a future where Emacs is packaged with tree-sitter = grammars for the builtin major modes. And AFAIK packagers very much = dislike editors bundling tree-sitter library and grammars themselves. >>=20 >> Bundling grammars has another complication which is tree-sitter = library-grammar compatibility. If we were to bundle grammars, we must = also bundle tree-sitter library, lest we risk to encounter a tree-sitter = library provided by the system that=E2=80=99s incompatible with the = bundled grammars. And bundling the tree-sitter library is obviously = undesirable. >>=20 >> Yuan >=20 > The grammars don't make any backwards compatibility guarantees. There = have been multiple Emacs bugs arising from grammars unilaterally = changing terminal names and such. They don=E2=80=99t have to be backward compatible. Any change to the = grammar is probably backward-incompatible simply due to the nature of = grammars. The situation will be better once package managers start = packing grammars with Emacs. I=E2=80=99m also working on making Emacs = explicitly state what versions of grammars are compatible, so people can = choose the right grammar version to install. > ISTM the only way to guarantee compatibility is to vendor the whole = stack. Vendoring the whole stack is one way, yes, but why do you think letting = package managers to package Emacs with the right grammars wouldn=E2=80=99t= work?=20 Yuan=