From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Jostein_Kj=C3=B8nigsen?= Newsgroups: gmane.emacs.devel Subject: Re: Call for volunteers: add tree-sitter support to major modes Date: Tue, 11 Oct 2022 10:43:57 +0200 Message-ID: <545D7C0D-6787-41C3-A9B8-824E8FE2B853@secure.kjonigsen.net> References: <874jwake9u.fsf@yahoo.com> Mime-Version: 1.0 (1.0) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="10753"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , theo@thornhill.no, acm@muc.de, emacs-devel@gnu.org, jostein@kjonigsen.net To: Po Lu Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Oct 11 10:53:36 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oiB15-0002f4-93 for ged-emacs-devel@m.gmane-mx.org; Tue, 11 Oct 2022 10:53:36 +0200 Original-Received: from localhost ([::1]:41430 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oiB14-00070T-5R for ged-emacs-devel@m.gmane-mx.org; Tue, 11 Oct 2022 04:53:34 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:52542) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiAs3-0007xd-P0 for emacs-devel@gnu.org; Tue, 11 Oct 2022 04:44:15 -0400 Original-Received: from wout5-smtp.messagingengine.com ([64.147.123.21]:39933) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oiAs1-0008R0-EQ; Tue, 11 Oct 2022 04:44:15 -0400 Original-Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 0B17732000F9; Tue, 11 Oct 2022 04:44:09 -0400 (EDT) Original-Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Tue, 11 Oct 2022 04:44:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= secure.kjonigsen.net; h=cc:cc:content-transfer-encoding :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to; s=fm2; t=1665477849; x=1665564249; bh=yK6qXvnOMl 5cH2ZAfIyQzd5BZ9oH+NKe+Zt26IseOw4=; b=E4uB20xRI6ZX+Mdyx/TD2Gx7Hq +JXQuNf5ISDCYPlPulmaLLSnF3Cw/pdzbyfOxQahM3xH0K7tfKI0lRe1y/Y14kzn 2O9eWwM5Nr8fEUJ9nbaGkimrjSzq4EuQf1uHxHUGx6HcjFv/vEH3OJXnEMT8jxz5 P0i+e6GS2cNS9HMho4pNF3mSd6TkqGHG8Zv0O20WBp7mQC0mxvbk3n069Z12l4Qt Y9BFrRAjTc8e2pFm2pHQhFYj1k2pwFgaRoTHn+qaw4HACHEFp8kysMl0PAP4v9x/ zFgIbUsfRvZXGeTtrnolYTd9/mVeLPzxnVN/zm2AI7BTNAK4rnuzo7tiwgqQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1665477849; x= 1665564249; bh=yK6qXvnOMl5cH2ZAfIyQzd5BZ9oH+NKe+Zt26IseOw4=; b=S UA3ntK2FwMojeOy3+APCxfwXFKZVvdymMqXwz+6OfA33zQjrVHCaLN3AjmkSNhDO trmVAfx0csLSrh8GU8qlRQsGSEGamtAjxBO7wyAAWOpcIk27sQvVX7/QfGQ8XsrZ 1GqAaIz2o/8jT6GhW8Vv3/K0IoNLlFpOHXD2DTSrFZNM7HkzEcTnt+MIhOxz3zQb ITMrBnCV95RslqFLWsw/Crhud938qMYgvTH7JiuNtGU93XeB9iDReNAD+R9CjtS0 i8I7zlQvRwkvUW7QjwMwBp8pEMH6jRAaqr/pOdg3QS+gMQ95o81Ek+yiV+PgVkvh nVoKqui03M5WE7ffC0Hxg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrfeejiedgtdehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurheptgfghfggufffkfhfvegjvffosehtqhhmtdhhtdejnecuhfhrohhmpeflohhs thgvihhnpgfmjhppnhhighhsvghnuceojhhoshhtvghinhesshgvtghurhgvrdhkjhhonh highhsvghnrdhnvghtqeenucggtffrrghtthgvrhhnpeevieevhfehveegtdeiudehfedt heegffetfeevtedugfekueehvdehjeefjeevjeenucevlhhushhtvghrufhiiigvpedtne curfgrrhgrmhepmhgrihhlfhhrohhmpehjohhsthgvihhnsehsvggtuhhrvgdrkhhjohhn ihhgshgvnhdrnhgvth X-ME-Proxy: Feedback-ID: ib2f84088:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 11 Oct 2022 04:44:09 -0400 (EDT) In-Reply-To: <874jwake9u.fsf@yahoo.com> X-Mailer: iPhone Mail (20B5064c) Received-SPF: pass client-ip=64.147.123.21; envelope-from=jostein@secure.kjonigsen.net; helo=wout5-smtp.messagingengine.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:297456 Archived-At: > On 11 Oct 2022, at 09:46, Po Lu wrote: >=20 > =EF=BB=BFEli Zaretskii writes: >=20 >> Yes, I will. >>=20 >> It isn't our business to develop language parsers, and we have no hope >> of keeping up with the technology developments in that area and with >> new languages being invented, well enough to offer reasonable and >> modern support in Emacs for editing programs in those languages. >=20 > It isn't quite our business to develop mini-gmp either, yet it's also > kept in-tree. >=20 >> It's the same reason why we depend on text shaping engines like >> HarfBuzz for rendering complex scripts and other typography-related >> features, on image libraries to display images, on GnuTLS to support >> TLS connections, etc. etc. >=20 > But editing text mostly does not involve editing complex shaped text, > and likewise, TLS connections and image display is mostly orthogonal to > the business of editing text. >=20 >> We cannot possibly be experts in all of those fields, we don't have >> the manpower for that. In the case of support for editing programs we >> actually tried for many years, and the result is before our eyes -- >> and it's unsatisfactory at best. We should advance Emacs towards >> leveraging existing technologies, not try developing those >> technologies in-house, or replacing them by cheap hacks and kludges. >=20 > Then why is mini-gmp kept in-tree? Portable multiple precision > arithmetic is at least as complicated (if not more) than tree-sitter, > yet mini-gmp is a completely satisfactory solution for people who do not > have libgmp installed. >=20 > mini-tree-sitter would fill the same role: it would expose the same API > as tree-sitter, but be small enough for Emacs to include and use when > tree-sitter is not available. >=20 >> It is completely impractical to expect us to be able to develop this >> in-house. >=20 > Why? The tree-sitter runtime library is only about 16,000 lines of C > source code and headers, which is less than xdisp.c alone, and only > slightly more than keyboard.c. In addition, the path to the current > implementation of tree-sitter is well-trodden, so it can't be that hard > to walk down that path again. >=20 > Such an implementation would be able to make use of existing and new > tree-sitter grammars without changes. >=20 >> If tree-sitter becomes defunct (something that doesn't seem like it >> will happen any time soon) >=20 > Famous last words. Absolutely true. But I don=E2=80=99t consider that a big enough problem to b= e worried.=20 tree-sitter will just be one way to implement a major-mode. There=E2=80=99s s= till others. There can be even more.=20 tree-sitter has momentum outside Emacs and high quality parsers for quite a l= ot of complex languages which are troublesome to implement parsers for in El= isp, as evident by our current best-effort parsers.=20 While there=E2=80=99s no guarantee it will be maintained forever, like Emacs= , it doesn=E2=80=99t have to either.=20 It just needs to be maintained for as long as the languages whose parsers ar= e implemented using it are maintained.=20 Once a language stops developing, it shouldn=E2=80=99t be a problem for us t= hat tree-sitter based parsers for it stops developing as well.=20 If a new, better thing comes along ten years down the line, there=E2=80=99s n= othing preventing us from adapting that for the languages and major-modes wh= ich at that time will benefit from it.=20 Just like we=E2=80=99re doing with tree-sitter now, for the major-modes and l= anguages with complex grammar which will benefits from it.=20 Disclaimer: I may be biased here, because I=E2=80=99ve spent the last ten+ y= ears trying to maintain parsers for C# and typescript in Elisp (and failing a= t that). And TSX has been a thorough no-go.=20 For some of these languages, tree-sitter really is the only realistic option= we have now if we want to support these languages at the level our users(an= d programmers in general) expect.=20 I see little to be gained by being philosophical at this point when the real= ity is as clear and harsh as it is.=20 >=20 >> , we will have to find a suitable replacement -- like we did with >> text-shaping engine, when the FLT library stopped being actively >> developed. Dealing with these changes in the outside world is an >> important part of the job of the Emacs maintainers >=20 > Text shaping is, at least IME, immensely more complicated than > interactive language parsing. >=20 >> The present font-lock and indentation features based on our own code >> can remain in Emacs forever, if someone wants a safe fallback for when >> worse comes to worst. >=20 > That logic could also have been applied to bignums, yet there was no > objection to including mini-gmp. >=20 >> But these implementations are from my POV a dead end: they cannot be >> developed any further, and for some languages, like C++, are already >> all but failing to reasonably and efficiently support their complex >> grammars. Using expertise of others is simply a logical step based on >> past experience; the decision to develop this in-house would be a step >> back and a disaster in the long run, IMNSHO. >=20 > I'm not disputing that fact, which is why I proposed to write an > *implementation* of tree-sitter for inclusion with Emacs, and not to > develop an equivalent in-house.