From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Theodor Thornhill Newsgroups: gmane.emacs.devel Subject: Re: Creating a paradigm for leveraging Tree Sitter's power Date: Sat, 24 Dec 2022 11:07:22 +0100 Message-ID: <78FAB063-180C-4700-81E8-81BC9D657D4A@thornhill.no> References: <948662B3-F284-4E89-BFE2-4BCB5007D4EA@easesoftware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="40036"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel To: emacs-devel@gnu.org, Yuan Fu , Perry Smith Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Dec 24 11:08:21 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p91S0-000AEA-Cb for ged-emacs-devel@m.gmane-mx.org; Sat, 24 Dec 2022 11:08:20 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p91RC-0000bh-Lg; Sat, 24 Dec 2022 05:07:30 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p91RB-0000bI-En for emacs-devel@gnu.org; Sat, 24 Dec 2022 05:07:29 -0500 Original-Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p91R9-0005wf-35 for emacs-devel@gnu.org; Sat, 24 Dec 2022 05:07:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=thornhill.no; s=key1; t=1671876443; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AClsnZk368WQ8d5JM56L3zE818lTcRNaC0rr6rrkm50=; b=NKPxFLu0jHfnw1AL8tqFLJqUTx/VNcKu8QD2j3Zv7/itm0hz4/FabQcsgrWT+82lbMJCo/ 7jHbjchPj27Hekp3PuRuUar86/HIY9s38XmdLr+n2iUjj9rqL4de6hlemVGn1F+9q3r6La QKKVTJBozX8pnECX7Eo9Eh2SxqHHbxLY9fHbR/HJ/dV50BCd194JpBrvWPOJhEts0aP9ww fz2Avjs0zFE81dGNrqQAN4ck36DBRXwN/A6VqVx2ELCquUewlnKaLsjZxZWK5wT4C+Hs98 VnxbqE7f/OyT/PxPNWIP0t8hNtH5R+Rf0iUTa20ONdEqUJ6rTpggPkNBE5p4qA== X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. In-Reply-To: X-Migadu-Flow: FLOW_OUT Received-SPF: pass client-ip=2001:41d0:2:aacc::; envelope-from=theo@thornhill.no; helo=out2.migadu.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:301833 Archived-At: On 24 December 2022 10:09:17 CET, Yuan Fu wrote: > > >> On Dec 23, 2022, at 5:32 PM, Perry Smith wrot= e: >>=20 >> I've seen on this list talk about "adding" Tree Sitter concepts to >> such things as mark-defun and other existing Emacs commands=2E >>=20 >> I've just spent a few days writing rtsn-mark-method that I intend on >> adding to ruby-ts-mode which implements all the features of mark-defun >> but I did it from scratch=2E This is mostly out of ignorance on how to >> leverage existing Emacs features and facilities=2E >>=20 >> One more concrete reason is mark-defun will include the comments >> before the defun=2E I wanted the same for mark-method but (as far as I >> can tell) the hooks back into mark-defun and its associated routines >> are simple regular expressions and that, to me, walks away form the >> power that Tree Sitter is providing=2E >>=20 >> I've got rtsn-mark-method working but I plan to rework it over the >> next few days=2E The "arg" as well as the "interactive" features of >> mark-defun which I also put into rtsn-mark-method I believe can be >> pulled out to a wrapper routine=2E There is actually quite a few >> features in the code that I did not know about=2E For example, if >> mark-defun is called with -1 (or -N) it marks N back=2E Ok=2E But now= if >> it is called again with no argument, mark-defun knows to go backwards >> to add in the previous defun=2E The same is true in the forward >> direction as well=2E (Although it does have a subtle bug in my opinion >> but I digress=2E) >>=20 >> To repeat myself, I believe these higher level features could be >> separated into a wrapper function so that all that would be needed for >> the language specific piece is a routine that would be passed a point >> and a direction=2E > >I think it makes a lot of sense=2E > >>=20 >> I'll call this the "primitive routine"=2E The routine would be >> responsible for returning a beginning and end (in a cons cell) and it >> would be the routine's responsibility to make sure that the beginning >> and end lie after (in the forward case) or before (in the backward >> case) the point that is passed in=2E > >You mean beginning and end of (symbol | string | statement | =E2=80=A6)? > >>From my experience implementing defun navigation for tree-sitter, it migh= t be more helpful to return three ranges: the thing before point, the thing= at point, and the thing after point, and either one could be nil if there = doesn=E2=80=99t exist one=2E For nested things it can be prev-sibling, par= ent, next-sibling instead=2E The point is that the user can move back and f= orward and make decisions easily with this =E2=80=9Cfield of view=E2=80=9D= =2E > >>=20 >> The wrapper routine would then know how to properly adjust the mark >> and point, execute counts, add to regions, catch errors, etc=2E >>=20 >> Broadening the picture: navigation, transpose, and other Emacs >> commands could likewise call the same primitive routine to provide >> transpose-method, forward-method, etc=2E And, broadening the picture >> once more: primitive routines could be written not just for methods >> (defun) but also for statements, arguments, expressions, classes, etc= =2E >> In all cases, the primitive routine would be relatively simple=2E It >> would be given a point and return a ( begin =2E end ) cons cell leaving >> all the harder work of expanding the region, remembering the >> direction, etc to the wrapper routines=2E >>=20 >> To be clear, there would be a wrapper routine for mark, a wrapper for >> forward, transpose, etc=2E We would end up with the classic: >>=20 >> For A + B number of routines we would have A x B number of commands=2E >>=20 >> Does this strike others as a good idea or insanity? > >Yeah it sounds pretty good=2E Probably no one needs/wants all the A x B c= ombinations, but the framework that allows this miss and match is good, sin= ce it allows us easily create the few combinations we care about=2E We need= more brains thinking about it and more experiments to fully flesh it out= =2E > >We also had a related discussion in another thread =E2=80=9CPlug treesit= =2Eel into other emacs constructs=E2=80=9D=2E > >Yuan Yeah=2E One shortcoming of tree-sitter imo is that the parser author decid= es what the nodes are named=2E So I think we need to create a framework so = that every mode can map over ast-names to Emacs concepts=2E The goal must b= e for the normal Emacs things to require little to no changes, but get the = benefits from treesit=2E I think we should just start doing that immediately on the master branch a= nd allow for "big" changes going forward=2E We should settle on something g= ood for Emacs 30, hopefully=2E I'm a little worried we feel we need "complete" proposals too soon=2E =20 Let's get all good ideas on the table, implemented and installed, then we = can consolidate after we discover pain points etc=2E I'm working on changing the forward/backward thing and transpose=2E Not on= ly for tree-sitter, but for others as well :) What do you think? Happy holidays! Theo