From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter maturity Date: Sun, 29 Dec 2024 09:59:38 -0500 Message-ID: <7AE5D5EA-BE67-4B89-BB31-F6A032BA1A69@dancol.org> References: <86ldwdm7xg.fsf@gnu.org> <6765355b.c80a0220.1a6b24.3117SMTPIN_ADDED_BROKEN@mx.google.com> <00554790-CACA-4233-8846-9E091CF1F7AA@gmail.com> <86msgl2red.fsf@gnu.org> <87o710sr7y.fsf@debian-hx90.lan> <8734i9tmze.fsf@posteo.net> <86plldwb7w.fsf@gnu.org> <87ttapryxr.fsf@posteo.net> <0883EB00-3BB2-4BC8-95D1-45F4497C0526@dancol.org> <87plldrx6a.fsf@posteo.net> <87ikr5rwx0.fsf@posteo.net> <86ed1rq6gc.fsf@gnu.org> <73737665-984E-4C97-9183-7805C1BCB550@dancol.org> <86bjwuricz.fsf@gnu.org> <5D1AB17D-8847-42ED-B246-8DB4D11F7FB7@gmail.com> <81F142F0-A0E8-472D-8AF0-6607E7ACF8D9@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="39124"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: K-9 Mail for Android Cc: Eli Zaretskii , emacs-devel@gnu.org To: Yuan Fu Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Dec 29 16:00:29 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tRumK-0009ze-Dz for ged-emacs-devel@m.gmane-mx.org; Sun, 29 Dec 2024 16:00:28 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tRule-0004eL-JC; Sun, 29 Dec 2024 09:59:46 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tRulc-0004dq-LF for emacs-devel@gnu.org; Sun, 29 Dec 2024 09:59:44 -0500 Original-Received: from dancol.org ([2600:3c01:e000:3d8::1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tRulZ-00054O-E8; Sun, 29 Dec 2024 09:59:44 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID: References:In-Reply-To:Subject:CC:To:From:Date:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=11KqmHPQt11l9OhCL+mIHWfUHI5yWAYK3w4Ip2cNtmY=; b=lMSXZI7Ec+y+H/2gz4hG6aEpBv J8R/cwKuDsZsZWKmUVqQXy5eTNFvlww8PVp03X12jE4kVdttK0QvamD8lkCJDv4sBaxJ4+zlmTdxH ybFn06uWHvXBPFDOTOrKPu/GOAZ/X9oAFzgBma/wXEhCyMYxgtWxny2h7A6voBOAO9Y3YuvHRC6Xh UcUE4LpDc7Ru6ro0vjCV0NAmuA2MuBaB+NFw/X8tK5W6wH7RsTMft2qzqDyV9V0/YAXc5teI5CNL+ aopzbxC2bMy8a5cy+jDp1/EvyVdkgiX1esIWyk91+fhewf/KSdfrkZphieXhjxbC+dUIBKzp/4nqk xN7XuBqw==; Original-Received: from 2603-9001-4203-1ab2-8c81-c450-971b-d6ef.inf6.spectrum.com ([2603:9001:4203:1ab2:8c81:c450:971b:d6ef]:57046 helo=[IPv6:::1]) by dancol.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1tRulX-0007fj-02; Sun, 29 Dec 2024 09:59:39 -0500 In-Reply-To: <81F142F0-A0E8-472D-8AF0-6607E7ACF8D9@gmail.com> Received-SPF: pass client-ip=2600:3c01:e000:3d8::1; envelope-from=dancol@dancol.org; helo=dancol.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327342 Archived-At: On December 29, 2024 5:21:21 AM EST, Yuan Fu wrote: > > >> On Dec 29, 2024, at 1:14=E2=80=AFAM, Daniel Colascione wrote: >>=20 >>=20 >>=20 >> On December 29, 2024 3:59:50 AM EST, Yuan Fu wrot= e: >>>=20 >>>=20 >>>> On Dec 29, 2024, at 12:41=E2=80=AFAM, Eli Zaretskii = wrote: >>>>=20 >>>>> Date: Sun, 29 Dec 2024 03:01:44 -0500 >>>>> From: Daniel Colascione >>>>>=20 >>>>>>> Enforcing this policy will just mean that Emacs doesn't support *a= t all* some languages out of the box and will put even more wind in the sai= ls of soft forks like Doom=2E Tree sitter language descriptions are free so= ftware=2E There's no reason not to rely on them=2E >>>>>>=20 >>>>>> We started with this concept of adding tree-sitter based modes to >>>>>> auto-mode-alist by default, but found that people who don't have th= e >>>>>> grammar installed didn't appreciate seeing the warnings about the >>>>>> missing grammars=2E So Emacs 29 made these modes optional, activat= ed >>>>>> only by an explicit user action=2E Emacs 30 still does that=2E >>>>>>=20 >>>>>> We are currently discussing how to improve this (see the thread Re: >>>>>> Turning on/off tree-sitter modes, which seems to have stalled latel= y)=2E >>>>>> But until the grammar libraries are ubiquitous, and we can rely on >>>>>> them being present on most systems, I think we will still need some >>>>>> user say-so before enabling tree-sitter based modes=2E >>>>>=20 >>>>> Wouldn't vendoring the grammars, and maybe even tree sitter itself, = silence the complaints about the warnings? Tree sitter is pure algorithmic = code=2E It doesn't have any particular platform dependencies=2E Why not sim= plify the whole system and make it a mandatory (and optionally bundled) dep= endency so that the show cognitive load of having to consider non-TS enviro= nments is just deleted? >>>>=20 >>>> First, the tree-sitter library itself is optional, so Emacs could be >>>> built without it=2E Or are you suggesting to import the library as w= ell >>>> into Emacs? If we don't import the library, making it a mandatory >>>> dependency is not TRT, IMO, because some users don't need the modes >>>> supported by tree-sitter, so forcing them to install the library that >>>> is not really useful to them is not right=2E We never do anything li= ke >>>> that with any other external libraries=2E GMP is special, but even f= or >>>> it we added our own "mini-gmp"=2E >>>>=20 >>>> Next, importing the grammar libraries into Emacs is not a simple >>>> matter, either=2E Their sources are in JavaScript, so if we want to = let >>>> users produce modified grammars (as we do with everything we have in >>>> the release tarballs), they will need to have Node=2Ejs etc=2E instal= led, >>>> which will become a prerequisite=2E And there are other complication= s, >>>> like the need to sync regularly with their upstream repositories=2E >>>> Moreover, there's no precedent for doing this, if you exclude lwlib >>>> and oldXMenu (which are different, since they are not developed >>>> outside Emacs)=2E >>>>=20 >>>> So I, for one, am not very happy to add this to our maintenance >>>> burden=2E It might make things easier for some (but see below), but = it >>>> doesn't come for free=2E >>>>=20 >>>> I also don't understand the fuss, really=2E Compiling a grammar libr= ary >>>> after cloning the repository takes seconds, so why do we have to do >>>> all this on behalf of the users if the users can do it so easily, eve= n >>>> if distros don't? E=2Eg=2E, I have on my system almost 70 grammar >>>> libraries, which I regularly update and build with a small number of >>>> simple Makefiles -- how hard can that be for anyone who is interested >>>> in these modes? Why does it have to be _our_ responsibility, any mor= e >>>> than, say, Grep or Findutils -- which are also heavily used by Emacs? >>>> Or even the image libraries? Why shouldn't this be the job of the >>>> distros? The upstream project doesn't have to think about packaging, >>>> it's the job of the distros=2E >>>=20 >>> Also, distros are picking up on packaging tree-sitter grammars, so I= =E2=80=99m hopeful of a future where Emacs is packaged with tree-sitter gra= mmars for the builtin major modes=2E And AFAIK packagers very much dislike = editors bundling tree-sitter library and grammars themselves=2E >>>=20 >>> Bundling grammars has another complication which is tree-sitter librar= y-grammar compatibility=2E If we were to bundle grammars, we must also bund= le tree-sitter library, lest we risk to encounter a tree-sitter library pro= vided by the system that=E2=80=99s incompatible with the bundled grammars= =2E And bundling the tree-sitter library is obviously undesirable=2E >>>=20 >>> Yuan >>=20 >> The grammars don't make any backwards compatibility guarantees=2E There= have been multiple Emacs bugs arising from grammars unilaterally changing = terminal names and such=2E > >They don=E2=80=99t have to be backward compatible=2E Any change to the gr= ammar is probably backward-incompatible simply due to the nature of grammar= s=2E The situation will be better once package managers start packing gramm= ars with Emacs=2E I=E2=80=99m also working on making Emacs explicitly state= what versions of grammars are compatible, so people can choose the right g= rammar version to install=2E > >> ISTM the only way to guarantee compatibility is to vendor the whole sta= ck=2E > >Vendoring the whole stack is one way, yes, but why do you think letting p= ackage managers to package Emacs with the right grammars wouldn=E2=80=99t w= ork?=20 I'm thinking of single-versioning issues=2E If there's a Debian package fo= r, e=2Eg=2E the C++ grammar, and different grammar versions are incompatibl= e, and two programs want that grammar, each at a different version, there's= no package system that will make both programs happy=2E Just vendoring the stack keeps things simple, reproducible, and compatible= =2E The grammars are small enough that I don't see much of a downside in ve= ndoring them either=2E