From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.devel Subject: Re: Suggesting that feature/tree-sitter be merged (was Re: Tree-sitter and major mode inheritance) Date: Fri, 18 Nov 2022 14:52:02 -0800 Message-ID: <6DDC3B43-8B34-41A8-9BCA-77EEAD0EB124@gmail.com> References: <0249C656-21C8-49F2-B979-A1894BF80637@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="14824"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel , Theodor Thornhill , Eli Zaretskii To: jostein@kjonigsen.net Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Nov 18 23:53:05 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1owAEJ-0003T5-Fr for ged-emacs-devel@m.gmane-mx.org; Fri, 18 Nov 2022 23:53:03 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1owADS-0004vm-Mh; Fri, 18 Nov 2022 17:52:10 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1owADR-0004vP-9Z for emacs-devel@gnu.org; Fri, 18 Nov 2022 17:52:09 -0500 Original-Received: from mail-pg1-x529.google.com ([2607:f8b0:4864:20::529]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1owADP-0006HJ-Ct; Fri, 18 Nov 2022 17:52:09 -0500 Original-Received: by mail-pg1-x529.google.com with SMTP id f9so1996593pgf.7; Fri, 18 Nov 2022 14:52:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=bT7gCWlVa09oOPwNueu1yGMy1x3+SirId9Z3Wnleda4=; b=qq+pk8mG5C1mXDy+ORpq/b6rR9t2s9R21qMMKospaoEMz+9tCsi4SB3qSYkOL6njX5 1G0fQJVp/zmAecZyowF/XuhnepR3I2MTZlHF2uqAxwrjFa/adpdOm7GQL57RyNu53dTH UQbp0D3sj7hd+2dH7KV92qHGY4mlGEAOOdqRbqb9JxjuhbtUYzw7CWwVA5VHRgDg5YEZ M5b/SPlg+6tkUwZLx6D5epf3boApZ5f8fBOh23HfKBYG2CqajjpTpkZWME9busU+UgwU RjTH8Ejshp98VDkztwHe4ko+Z0HqDdHvKWOBJ/NjeKOxxG3uwL8cMnURG6UvJA5SwL4t nwvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bT7gCWlVa09oOPwNueu1yGMy1x3+SirId9Z3Wnleda4=; b=maS5vC0Zlm5sfEm7a8WIJMoPwEtLcm1TiIg7warmO/1kpgIfXPMILVbPUtuwmu9gWX rQNjy9mAoL39slhne9wAinRoEtkqQuK9ON8+vjMAF7y5XCtV7021S6VMLXh9Co0lBukD GoFtcufHIisEACIoaEQ7YAF9byLjlZpc226+kO62EJybBb6RySxSPDufI7dhGC2s87Qr WMrfZUNHrszUV/Lrl4CQrvGA/qq/eEfWWMwTHCKXgbDoChyiiSX5jjEbtANadUGccgKL iE/uK0zG5xFiSQJ4BI1ZNfY6FDLgE2Viw8hUQB4Xb1+dLfGKq+sxH0GYV9ddvM0kAUF7 0kNw== X-Gm-Message-State: ANoB5pmM26TGqP9dBv31YH0e+4eMD9zB5EhifTbsvDwl6Pe+mOMV1w8B qXkFIm3JQQEcKUW9njaGYWc= X-Google-Smtp-Source: AA0mqf7ytedbQZFeqtgncwAzPQ0IbLvkxxHsb5S+XP+HMymNPr8SwxoetNYv109fSz7MyHGlt4IngA== X-Received: by 2002:a63:5c1e:0:b0:46e:96ba:494d with SMTP id q30-20020a635c1e000000b0046e96ba494dmr8439145pgb.404.1668811925305; Fri, 18 Nov 2022 14:52:05 -0800 (PST) Original-Received: from smtpclient.apple (cpe-172-117-161-177.socal.res.rr.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id l8-20020a170903244800b00186b0ac12c5sm4324659pls.172.2022.11.18.14.52.03 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 Nov 2022 14:52:04 -0800 (PST) In-Reply-To: X-Mailer: Apple Mail (2.3696.120.41.1.1) Received-SPF: pass client-ip=2607:f8b0:4864:20::529; envelope-from=casouri@gmail.com; helo=mail-pg1-x529.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:300126 Archived-At: > On Nov 18, 2022, at 1:54 PM, Jostein Kj=C3=B8nigsen = wrote: >=20 > Hey everyone. >=20 > I know this has been said before, by people which by far has been = contributing much more than me... But I still don't think it's a good = idea to replace the implementation in existing major-modes with = tree-sitter implementations, nor selectively activate tree-sitter in = major-modes prone to inhetitence and derivation. >=20 > Me and Theodor faced these same issues with "our" C# and TypeScript = major-modes, and the only "clean" way we agreed we could make this work = was to create wholly new implementations. I can come up with many good, = objective reasons for this, but I think Theodor has already represented = this view fairly well. >=20 > While for the sake of brevity, I'll not diving deeply into this = particular thing, I will say this: A new tree-sitter based major-mode = free of compatibility concerns allowed us to create entirely new = major-modes fixing most of our existing bugs, faster than we before = would be able to fix a single bug. My personal view is that mixing = existing major-modes with tree-sitter represents absolutely the worst of = all worlds. It maintains all existing complexities, provides us with = very few benefits, and at the same time adds complexities we didn't use = to have. To me, that's a net negative. >=20 > Somewhat surprising to me, I see this is a somewhat controversial = point of view and not as clear cut a matter as I would have expected it = to be. I realize and respect that final decisions in these matter might = take some time to mature. Time which given our current approach detracts = from the momentum tree-sitter has been having. Hey Jostein, Thank you very much for your thoughtful input! I originally thought of = tree-sitter as merely a mean/tool that major modes could use to enhance = their features. But in reality it seems that tree-sitter replaces a = large enough chunk of a major-mode=E2=80=99s responsibility, which has = caused all these compatibility problems we=E2=80=99ve encountered. = Examples include undoing cc-mode=E2=80=99s setup when turning on = tree-sitter, invalidating many existing major-mode variables, and the = major mode inheritance problem I presented in the previous message.=20 These difficulties has nudged us to decouple the tree-sitter and = non-tree-sitter version further and further, from trying to make = tree-sitter a feature that enables/disabled in a major mode by a = command, to a major mode configuration that causes the major mode to = setup differently, to my suggestion of using separate major mode but = share a single virtual parent mode. I think using separate major mode but share a single virtual parent mode = gets the benefit of both worlds: 1. Tree-sitter and non-tree-sitter are separated in different modes, = meaning it gets all the benefit Jostein and Theo described. 2. Backward-compatible. Existing configuration to the non-tree-sitter = version works as they do before. (Here I=E2=80=99m mainly thinking about = hooks: Hooks added to c-mode-hook still runs in c-native-mode.) 3. Solves the inheritance problem I described. 4. Minimal changes to the existing modes. > At this point poor Yuan Fu here has spent quite a bit of time and = effort getting a core tree-sitter interface into Emacs. I would really = hate to see all that effort be for nothing, because we end up conflating = a creating a core tree-sitter feature with how this feature should best = be employed in subsequent major-modes. >=20 > So in the name of pragmatism, I propose a compromise of sorts. >=20 > Instead of waiting for "every" major-mode to be re-implemented into a = tree-sitter derivative in the feature/tree-sitter branch before we = merge... How about we just accept the current "core" tree-sitter = implementation as good enough, and consider merging that to git master = as is. >=20 > This will allow us to land one important mile-stone, while giving us = the head-room to further discuss how we should best = implement/reimplement/"upgrade" existing major-modes to take advantage = of tree-sitter. >=20 > It will also allow third-party packages to make use of tree-sitter in = Emacs core instead coming up with its own implementation, beginning a = defacto standardization of this new API (which I may note already has a = competing implementation in MELPA). >=20 > That is to say: We should land the core library, with holding it = hostage to all its possible consumers being implemented. >=20 > How about it? Are there any good arguments for NOT merging = feature/tree-sitter at this point? :) The good news is, feature/tree-sitter is merging in a few days! And = we=E2=80=99ll make further improvements on the master. So rest assured! = :-) I think we can at least get C, Python, Javascript, Typescript, Bash, = JONS and CSS to go with the up coming release, largely thanks to = Theo=E2=80=99s (and tree-sitter=E2=80=99s) productivity. I personally = don=E2=80=99t know enough of C++ and Java to polish them, but they have = a good chance too. Taking a step back, I think developing major mode support with the = tree-sitter API is a good idea. We found countless shortcomings of the = API when developing major modes with it. Yuan=