From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Danny Freeman Newsgroups: gmane.emacs.devel Subject: Re: Update on tree-sitter structure navigation Date: Tue, 05 Sep 2023 22:51:02 -0400 Message-ID: <87o7igc80a.fsf@dfreeman.email> References: <5E7F2A94-4377-45C0-8541-7F59F3B54BA1@gmail.com> <8a5b3b3e-f091-3f38-09d4-c4e26bec97f9@yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="14150"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Yuan Fu , emacs-devel , Theodor Thornhill , Jostein =?utf-8?Q?Kj=C3=B8nigsen?= , Randy Taylor , Wilhelm Kirschbaum , Perry Smith To: Dmitry Gutov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Sep 06 05:10:42 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qdiwD-0003XE-IR for ged-emacs-devel@m.gmane-mx.org; Wed, 06 Sep 2023 05:10:41 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qdivR-0005Kf-S4; Tue, 05 Sep 2023 23:09:53 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdivQ-0005KI-Em for emacs-devel@gnu.org; Tue, 05 Sep 2023 23:09:52 -0400 Original-Received: from out-224.mta0.migadu.com ([2001:41d0:1004:224b::e0]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qdivM-0005P0-Tt for emacs-devel@gnu.org; Tue, 05 Sep 2023 23:09:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dfreeman.email; s=key1; t=1693969786; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zW5c28fboV/R9lfmzVTa9sZKWLaNnFuUOsU92d+G5SA=; b=EP8NgdByB5/qLLTxlPKbIZLbiOuJY1upxn43boDUe38nRJgVSpbMT7aEKh4/6Y3f3Doenb QpS2zjhc1xPux7Eqz3KwBn+VCkXrN0uaynI/EgNw267qVVcVw7iuHUNZt8uYUk5zu5FUwZ CGBiXK0DpG5Q1unBe+L3w857oQWsbYo= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. In-reply-to: <8a5b3b3e-f091-3f38-09d4-c4e26bec97f9@yandex.ru> X-Migadu-Flow: FLOW_OUT Received-SPF: pass client-ip=2001:41d0:1004:224b::e0; envelope-from=danny@dfreeman.email; helo=out-224.mta0.migadu.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:310169 Archived-At: Dmitry Gutov writes: > Hi Yuan, > > On 02/09/2023 08:01, Yuan Fu wrote: >> - Solve the grammar versioning/breaking-change problem: tree-sitter gram= mar don=E2=80=99t have a version >> number, so every time the author changes the grammar, our queries break,= and loading the mode only >> produces a giant error. > > I don't have a better idea than basically copying NeoVim and others: to m= aintain the urls to parser > repositories and the ref of the latest known good revision, for the curre= nt version of the major > mode. That info could be filled in by major modes themselves, e.g. in an = autoload block (similarly > to how auto-mode-alist is appended to). clojure-ts-mode keeps a URL for the parser, but doesn't do anything about the git revision. It easily could but I don't feel the need (yet) since I am also a maintainer of the clojure grammar and know when we're about to break grammar consumers. It's not quite that simple though. Some distributions (nixos for example) are already providing pre-compiled grammars. That is how I discovered a couple recent bugs in js-ts-mode, because the grammars distributed with nixos 23.05 no longer worked on Emacs 30 after a patch was applied that was supposed to be backwards compatible (a real pain to verify in my experience). With the way Emacs can load a grammar provided by the user's distribution, keeping information about the version of the grammar in the major mode doesn't help all that much. Even if we did it we have no idea what version might be have been built used the user's .emacs.d/tree-sitter folder. That would require something like putting a version number in the file name, or maybe applying a patch to the grammar's C source that allowed us to get a version, SHA, something at runtime.=20 I'm not so sure we can have a great way to do this without a change to the tree-sitter libraries. I would love to see some kind of increasing version number generated in the grammar's C source that we could then access. It could be used to make decisions about what queries to use, or to warn the user they need to use a different grammar (maybe offering to install a compatible version). Tree-sitter grammar changes are almost always breaking changes. Adding nodes can break things, re-naming them and removing them definitely can. I'm not sure any grammar consumer has a great way to deal with this without always compiling the exact grammar they need and only ever using it. --=20 Danny Freeman