From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Daniel Colascione Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter maturity Date: Sun, 29 Dec 2024 04:14:37 -0500 Message-ID: References: <67428b3d.c80a0220.2f3036.adbdSMTPIN_ADDED_BROKEN@mx.google.com> <86ldwdm7xg.fsf@gnu.org> <6765355b.c80a0220.1a6b24.3117SMTPIN_ADDED_BROKEN@mx.google.com> <00554790-CACA-4233-8846-9E091CF1F7AA@gmail.com> <86msgl2red.fsf@gnu.org> <87o710sr7y.fsf@debian-hx90.lan> <8734i9tmze.fsf@posteo.net> <86plldwb7w.fsf@gnu.org> <87ttapryxr.fsf@posteo.net> <0883EB00-3BB2-4BC8-95D1-45F4497C0526@dancol.org> <87plldrx6a.fsf@posteo.net> <87ikr5rwx0.fsf@posteo.net> <86ed1rq6gc.fsf@gnu.org> <73737665-984E-4C97-9183-7805C1BCB550@dancol.org> <86bjwuricz.fsf@gnu.org> <5D1AB17D-8847-42ED-B246-8DB4D11F7FB7@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="16327"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: K-9 Mail for Android Cc: emacs-devel@gnu.org To: Yuan Fu , Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Dec 29 10:15:44 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tRpOh-00046b-71 for ged-emacs-devel@m.gmane-mx.org; Sun, 29 Dec 2024 10:15:43 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tRpNr-0000u1-11; Sun, 29 Dec 2024 04:14:51 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tRpNj-0000tg-VE for emacs-devel@gnu.org; Sun, 29 Dec 2024 04:14:45 -0500 Original-Received: from dancol.org ([2600:3c01:e000:3d8::1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tRpNh-0006aV-Rr; Sun, 29 Dec 2024 04:14:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=dancol.org; s=x; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID: References:In-Reply-To:Subject:CC:To:From:Date:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=81t4Kw7JVRgeSLuAXX8Tei69v/wLVfhXIDc/Fa9yTFo=; b=DcgHO5NX2cQp9E7GLZjvyc/JoB eE+0jr3yu8JAWsA7PUjfb7c8xA4W1gjasVHSSGFDlbFIZcPT6Buu/WRGAmOLq6B2vyDJRJjRjsmSv Qvis4eB+2zxG9FJqce8Ym/gyCmEAqLrITP+mwNJccGebUh+iQgEyde1URcLaeyy0HrnPlZ9aMBb5M CO99lI541SVKDV14E6M9e+iprtLAFivzdGEZEmsABQO8289yJSfunvFK0qWKZYAuQXFHLhrok2rM7 LDBxK8kO/qDrBo/edaEYAv+9DEuCkQbxYLbmx4OahC+8LjhNDrVZIj41ucCQenrhKAgXJE3H/wysu dQK+WDiA==; Original-Received: from 2603-9001-4203-1ab2-8c81-c450-971b-d6ef.inf6.spectrum.com ([2603:9001:4203:1ab2:8c81:c450:971b:d6ef]:33758 helo=[IPv6:::1]) by dancol.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.96) (envelope-from ) id 1tRpNe-000612-0L; Sun, 29 Dec 2024 04:14:38 -0500 In-Reply-To: <5D1AB17D-8847-42ED-B246-8DB4D11F7FB7@gmail.com> Received-SPF: pass client-ip=2600:3c01:e000:3d8::1; envelope-from=dancol@dancol.org; helo=dancol.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327325 Archived-At: On December 29, 2024 3:59:50 AM EST, Yuan Fu wrote: > > >> On Dec 29, 2024, at 12:41=E2=80=AFAM, Eli Zaretskii wr= ote: >>=20 >>> Date: Sun, 29 Dec 2024 03:01:44 -0500 >>> From: Daniel Colascione >>>=20 >>>>> Enforcing this policy will just mean that Emacs doesn't support *at = all* some languages out of the box and will put even more wind in the sails= of soft forks like Doom=2E Tree sitter language descriptions are free soft= ware=2E There's no reason not to rely on them=2E >>>>=20 >>>> We started with this concept of adding tree-sitter based modes to >>>> auto-mode-alist by default, but found that people who don't have the >>>> grammar installed didn't appreciate seeing the warnings about the >>>> missing grammars=2E So Emacs 29 made these modes optional, activated >>>> only by an explicit user action=2E Emacs 30 still does that=2E >>>>=20 >>>> We are currently discussing how to improve this (see the thread Re: >>>> Turning on/off tree-sitter modes, which seems to have stalled lately)= =2E >>>> But until the grammar libraries are ubiquitous, and we can rely on >>>> them being present on most systems, I think we will still need some >>>> user say-so before enabling tree-sitter based modes=2E >>>=20 >>> Wouldn't vendoring the grammars, and maybe even tree sitter itself, si= lence the complaints about the warnings? Tree sitter is pure algorithmic co= de=2E It doesn't have any particular platform dependencies=2E Why not simpl= ify the whole system and make it a mandatory (and optionally bundled) depen= dency so that the show cognitive load of having to consider non-TS environm= ents is just deleted? >>=20 >> First, the tree-sitter library itself is optional, so Emacs could be >> built without it=2E Or are you suggesting to import the library as wel= l >> into Emacs? If we don't import the library, making it a mandatory >> dependency is not TRT, IMO, because some users don't need the modes >> supported by tree-sitter, so forcing them to install the library that >> is not really useful to them is not right=2E We never do anything like >> that with any other external libraries=2E GMP is special, but even for >> it we added our own "mini-gmp"=2E >>=20 >> Next, importing the grammar libraries into Emacs is not a simple >> matter, either=2E Their sources are in JavaScript, so if we want to le= t >> users produce modified grammars (as we do with everything we have in >> the release tarballs), they will need to have Node=2Ejs etc=2E installe= d, >> which will become a prerequisite=2E And there are other complications, >> like the need to sync regularly with their upstream repositories=2E >> Moreover, there's no precedent for doing this, if you exclude lwlib >> and oldXMenu (which are different, since they are not developed >> outside Emacs)=2E >>=20 >> So I, for one, am not very happy to add this to our maintenance >> burden=2E It might make things easier for some (but see below), but it >> doesn't come for free=2E >>=20 >> I also don't understand the fuss, really=2E Compiling a grammar librar= y >> after cloning the repository takes seconds, so why do we have to do >> all this on behalf of the users if the users can do it so easily, even >> if distros don't? E=2Eg=2E, I have on my system almost 70 grammar >> libraries, which I regularly update and build with a small number of >> simple Makefiles -- how hard can that be for anyone who is interested >> in these modes? Why does it have to be _our_ responsibility, any more >> than, say, Grep or Findutils -- which are also heavily used by Emacs? >> Or even the image libraries? Why shouldn't this be the job of the >> distros? The upstream project doesn't have to think about packaging, >> it's the job of the distros=2E > >Also, distros are picking up on packaging tree-sitter grammars, so I=E2= =80=99m hopeful of a future where Emacs is packaged with tree-sitter gramma= rs for the builtin major modes=2E And AFAIK packagers very much dislike edi= tors bundling tree-sitter library and grammars themselves=2E > >Bundling grammars has another complication which is tree-sitter library-g= rammar compatibility=2E If we were to bundle grammars, we must also bundle = tree-sitter library, lest we risk to encounter a tree-sitter library provid= ed by the system that=E2=80=99s incompatible with the bundled grammars=2E A= nd bundling the tree-sitter library is obviously undesirable=2E > >Yuan The grammars don't make any backwards compatibility guarantees=2E There ha= ve been multiple Emacs bugs arising from grammars unilaterally changing ter= minal names and such=2E ISTM the only way to guarantee compatibility is to = vendor the whole stack=2E