From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.devel Subject: Re: Update on tree-sitter structure navigation Date: Thu, 7 Sep 2023 15:52:25 +0300 Message-ID: <5dc1cb76-c453-670a-db1a-1b29842abbe0@gutov.dev> References: <5E7F2A94-4377-45C0-8541-7F59F3B54BA1@gmail.com> <8a5b3b3e-f091-3f38-09d4-c4e26bec97f9@yandex.ru> <87o7igc80a.fsf@dfreeman.email> <87y1hizl9b.fsf@dfreeman.email> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="23403"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: Yuan Fu , emacs-devel , Theodor Thornhill , =?UTF-8?Q?Jostein_Kj=c3=b8nigsen?= , Randy Taylor , Wilhelm Kirschbaum , Perry Smith To: Danny Freeman Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Sep 07 14:53:37 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qeEVs-0005rm-2C for ged-emacs-devel@m.gmane-mx.org; Thu, 07 Sep 2023 14:53:36 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qeEUs-0007tj-8R; Thu, 07 Sep 2023 08:52:34 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qeEUq-0007tA-Jl for emacs-devel@gnu.org; Thu, 07 Sep 2023 08:52:32 -0400 Original-Received: from out2-smtp.messagingengine.com ([66.111.4.26]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qeEUo-0000yE-1m for emacs-devel@gnu.org; Thu, 07 Sep 2023 08:52:32 -0400 Original-Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 89D215C0161; Thu, 7 Sep 2023 08:52:29 -0400 (EDT) Original-Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Thu, 07 Sep 2023 08:52:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1694091149; x=1694177549; bh=qnM+rBw5VU+ZkYe53p2LaoEYDLC0vtQylHh qTnPk3YU=; b=YgpKfMaGOCsups50030a4vLjTiRRi2bRavFoRDZtck3x6q/9TxP qX6jAkhKpMEC3batE7s6vGM5TlVeDT627+5TXwM/guK8jjEI2zKn4gr1G90KNcfJ DQNrc24AYSh/sk15drrccl4vB1QFEGAsLbVaOuslroq9STswcy/tSJSYXlvUeNE6 pmKQyDbWLIAcNsP85r9ejApScjCfz3L/rxu4ZCliZweUrqcFXjGL273OWFxb936Q wOOE0KOtmYuivNzt6kR2/mhrQjHUfgVV1CbYOFIjyCCK9lCtPLE5VDwLs9r6Qf6/ 0Ovu0DS/wDJ5mwJZ2gh44U6UUmjrQ3kusJw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1694091149; x=1694177549; bh=qnM+rBw5VU+ZkYe53p2LaoEYDLC0vtQylHh qTnPk3YU=; b=WeX0sGAVAJr6AX+6j9hADstTO3cNpJ9RyIRsSVa/YAdlrTX/GXB e7WKmO1RD09z4nX+vQ1qzUfdz+lIu4iPhs9At3zi9CPVK/GFDu+icTyqzmLmEQfs Xivh/vklWoTMdbndGS/r3pzG8h7xfNSin7B+mxiMDMvfQSD5IsMc9awyqccvvvwr ya51S7Zu74F+lB+Chc2ES5ciGbWls4Os3iOLD98T8eyM4/7yHMB/Hphdd2njSl5l vmjkTMXkzKkLy+GA16uYugpDl2EW3+auW2ORNTm90jpED3CqHJfJWuJu7og3SHAI GD/rcoUr37QWUvCRgQ2CK+XGsbhSlcDe5pw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudehhedgheeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhephffhleeifffgveevudeugfeifeeuffevgfeutdeitefhiefgtedvheeuvedv vdefnecuffhomhgrihhnpehgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumhhithhrhiesghhuthhovhdruggvvh X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 7 Sep 2023 08:52:26 -0400 (EDT) Content-Language: en-US In-Reply-To: <87y1hizl9b.fsf@dfreeman.email> Received-SPF: pass client-ip=66.111.4.26; envelope-from=dmitry@gutov.dev; helo=out2-smtp.messagingengine.com X-Spam_score_int: -42 X-Spam_score: -4.3 X-Spam_bar: ---- X-Spam_report: (-4.3 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, NICE_REPLY_A=-1.473, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:310259 Archived-At: On 07/09/2023 06:18, Danny Freeman wrote: > > Dmitry Gutov writes: > >>> clojure-ts-mode keeps a URL for the parser, but doesn't do anything >>> about the git revision. It easily could but I don't feel the need (yet) >>> since I am also a maintainer of the clojure grammar and know when we're >>> about to break grammar consumers. >> >> Sure, that's easy enough to do when the package is only in ELPA: upgrade the grammar, upgrade the >> package, all in lockstep. > > Yeah, soon after I sent that email I realized there is no reason for me > not to specify a version for the grammar so I pushed a change doing just > that. Nice. >> Grammars distributed from distros are more of a problem, because it's not always a good idea to >> abort with "wrong version". But perhaps we could do that and recommend installing from Git in such >> cases anyway? > > In some cases, distros might place the grammars in a strange location > made accessible on `treesit-extra-load-path`, which takes precedence > over the grammars that are installed from git in the user's Emacs > directory. This is what nix does, but is probably an outlier. I would > guess more conventional distributions might just make them accessible > where dynamic libraries are normally located and the grammars installed > from git would take precedence. Perhaps the user's Emacs directory should take precendence over treesit-extra-load-path. Or treesit-install-language-grammar should pick a higher-priority place instead. It just makes sense that the user-installed grammar would be loaded first. >> Another problem is that grammars don't have good versioning, and even if they did, we'd have to >> sometimes update the "upper bound" (we'd need coarse ranges, right? rather that one fixed version >> requirement) more frequently than Emacs is released. Less of a problem for modes in ELPA, though. > > Yeah I think ranges would be right. It would be good to say, we tested > this with versions N through M, anything else might not work. There > would still need to be some checks and patches like what exists in > js-ts-mode now. But that seems unavoidable, but could be cleaner if we > had a good way to ID grammars. Not sure about how we'd keep up with > grammars. Maybe we just can't and would need to have users install older > versions. That seems okay? Basically, yes: if the current available grammar is outside of the compatibility range (and/or we get query errors, I'm not sure where to put the balance: I suppose sometimes the query will succeed but it wouldn't match some elements which it matched before), we issue a warning to the user that they're recommended to use treesit-install-language-grammar - installing the last-known good hash, which might as well be older than the current installed grammar. >>> I'm not so sure we can have a great way to do this without a change to >>> the tree-sitter libraries. I would love to see some kind of increasing >>> version number generated in the grammar's C source that we could then >>> access. It could be used to make decisions about what queries to use, or >>> to warn the user they need to use a different grammar (maybe offering to >>> install a compatible version). >> >> Yes, that would be an improvement, worth being up on the issue tracker maybe. > > Yeah, I think this is a good move. I opened up one here > https://github.com/tree-sitter/tree-sitter/issues/2611 > Of course, anyone feel free to chime in. Thanks! I left a note too.