From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?Q?Jostein_Kj=C3=B8nigsen?= Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter integration in python.el Date: Mon, 26 Sep 2022 21:10:07 +0200 Message-ID: <761f064e-3cbf-5ca7-2edb-4de6670345bf@secure.kjonigsen.net> References: <3A68545C-2E40-4BB4-8563-8041A5452634@gmail.com> Reply-To: jostein@kjonigsen.net Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=3da8ab7e4fd840dbb4f9483bc37a8826 Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="27476"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Cyrus-JMAP/3.7.0-alpha0-968-g04df58079d-fm-20220921.001-g04df5807 To: "Yuan Fu" , emacs-devel Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Sep 26 21:17:01 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1octb8-0006q4-Ah for ged-emacs-devel@m.gmane-mx.org; Mon, 26 Sep 2022 21:16:59 +0200 Original-Received: from localhost ([::1]:48554 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1octb7-0004y3-9l for ged-emacs-devel@m.gmane-mx.org; Mon, 26 Sep 2022 15:16:57 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:43700) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1octV5-0000Sy-M4 for emacs-devel@gnu.org; Mon, 26 Sep 2022 15:10:50 -0400 Original-Received: from wout3-smtp.messagingengine.com ([64.147.123.19]:50279) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1octUx-00081d-HG for emacs-devel@gnu.org; Mon, 26 Sep 2022 15:10:39 -0400 Original-Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id E159E3200971; Mon, 26 Sep 2022 15:10:29 -0400 (EDT) Original-Received: from imap50 ([10.202.2.100]) by compute2.internal (MEProxy); Mon, 26 Sep 2022 15:10:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= secure.kjonigsen.net; h=cc:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:reply-to:sender:subject:subject:to:to; s=fm1; t= 1664219429; x=1664305829; bh=dEW7VARSf14ci6xunov/opBY1PEnN8ZC23P FZss7T54=; b=eg/anS9ULIrstdYN4brm3UtTNPH1N1SejUsYY51fbPP3Gx2eh5b vRIVNqgv9wF68L2EHAIyB7+vOlnruK76fb/+44VLc7KEx5st+m0HdfPOj/lMONTJ apvZu7dj/C3xT/hhl/YBb6nmaiITOUYC9iVGX4ynns6QpRzCYtqDlWE+QcfUxAHI vMlVE9WVVzMqudUECocPu/SfKOeXrrlRcgf7WRiY5KdISXuyT6J6B/UeOKro8p90 0KA13QrQ14iKQjw85jxHlEhfK/0sH9t2lWkj7QGjC4QkIGMlNsBxhxsn2PRgLPp9 M6NQw4DAzHXObNMUoRsV+NtuDYhdO3oH5jg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; t=1664219429; x=1664305829; bh=dEW7VARSf14ci 6xunov/opBY1PEnN8ZC23PFZss7T54=; b=WaSh1abad2ClUJPesDaR7AlazA19T C9ssibtm7mMRfgfjjfiaGuv8Uav6NYD9Ppfx1oIzLL1k0XBPd08lnMFwnFM3VQWf mBhBi6A7D+mpCTenR9dbrBxfKiS5PXekuqJVVcmeFdWPrDOKFawTZuJYFPeGTGbv 2a+Zf8vLRrDkOqBIrEU/MXPDySp06I00bpvawlwdmh2AFc8C22rct8CBsfE0Y7Ab eRsmhRIIencDgXU/x3fbqAofjvEWUCNOkB70eyCdTDTlb2gG0A4kvEyj6sGm4nVC RX1oYqttjUJjeQzvcUBsiCJArUOjm7L+O49c77qPp0sjnkZ292YIwEe+Q== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrfeegvddgudefhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepofgfggfkjghffffhrhfvufgtse grtderreerreejnecuhfhrohhmpeflohhsthgvihhnpgfmjhppnhhighhsvghnuceojhho shhtvghinhesshgvtghurhgvrdhkjhhonhhighhsvghnrdhnvghtqeenucggtffrrghtth gvrhhnpeetjefhgeekieetfffhhfevhfefffdvkedtgedvvefgueekveejgeeludehudef feenucffohhmrghinhepghhnuhdrohhrghdpkhhjnhhighhsvghnrdhnohenucevlhhush htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehjohhsthgvihhnsehs vggtuhhrvgdrkhhjohhnihhgshgvnhdrnhgvth X-ME-Proxy: Feedback-ID: ib2f84088:Fastmail Original-Received: by mailuser.nyi.internal (Postfix, from userid 501) id 245A21700089; Mon, 26 Sep 2022 15:10:29 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface In-Reply-To: <3A68545C-2E40-4BB4-8563-8041A5452634@gmail.com> Received-SPF: pass client-ip=64.147.123.19; envelope-from=jostein@secure.kjonigsen.net; helo=wout3-smtp.messagingengine.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:296302 Archived-At: --3da8ab7e4fd840dbb4f9483bc37a8826 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable On 22.09.2022 20:42, Yuan Fu wrote: > Hi,=20 >=20 > I=E2=80=99ve added tree-sitter version for font-lock and which-func in= python.el. And I=E2=80=99d love to hear some feedback from python.el ma= intainers. Specifically, does it look right and which other part of pyth= on.el could actually benefit from a parse tree? I wrote a tree-sitter im= enu indexer for python and it performed worse than the current one, pres= umably because it traverses the whole parse tree whereas the current one= only scans the buffer once or so and do some regex matching. >=20 > Here is the commit: https://git.savannah.gnu.org/cgit/emacs.git/commit= /?h=3Dfeature/tree-sitter&id=3D1cdb24fe35a9ff2e4f92c5acc93a5a5b0e70d93f >=20 > Yuan >=20 >=20 Hey Yuan. Thanks for putting in all this work into tree-sitter in Emacs! Trying the latest Emacs-version in feature/tree-sitter I got an error I = tend to get "a lot" with tree-sitter based modes, which I hoped bundling= things with Emacs would solve, namely obtaining the original shared-obj= ect containing the compiled grammer. Like for your python-mode, I get this: File mode specification error: (treesit-load-language-error python (/hom= e/jostein/.emacs.d/tree-sitter/libtree-sitter-python: cannot open shared= object file: No such file or directory /home/jostein/.emacs.d/tree-sitt= er/libtree-sitter-python.so: cannot open shared object file: No such fil= e or directory libtree-sitter-python: cannot open shared object file: No= such file or directory libtree-sitter-python.so: cannot open shared obj= ect file: No such file or directory)) I realize third-party modes are on their own, but when tree-sitter is co= mpiled with Emacs, I would at least expect the Emacs-build to also produ= ce these .so-files. What are your thoughts on how we can best, across the Emacs-verse, provi= de these libraries? Or at least for the modes which are bundled with Ema= cs itself? Being a tree-sitter based-developer myself, I know where I can go to get= this compiled to make the mode runnable, but surely that's not how we c= an deploy this en-masse. Most people will be stuck at this point, and we= will need to come up with a better answer. --=20 Kind regards *Jostein Kj=C3=B8nigsen* jostein@kjonigsen.net =F0=9F=8D=B5 jostein@gmail.com https://jostein.kj=C3=B8nigsen.no --3da8ab7e4fd840dbb4f9483bc37a8826 Content-Type: text/html;charset=utf-8 Content-Transfer-Encoding: quoted-printable
On 22.09.2022 20:42, Yuan Fu wrote:
Hi,=20 I=E2=80=99ve added tree-sitter version for font-lock and which-func in p= ython.el. And I=E2=80=99d love to hear some feedback from python.el main= tainers. Specifically, does it look right and which other part of python= .el could actually benefit from a parse tree? I wrote a tree-sitter imen= u indexer for python and it performed worse than the current one, presum= ably because it traverses the whole parse tree whereas the current one o= nly scans the buffer once or so and do some regex matching. Here is the commit: https://git.savannah.gnu.o= rg/cgit/emacs.git/commit/?h=3Dfeature/tree-sitter&id=3D1cdb24fe35a9f= f2e4f92c5acc93a5a5b0e70d93f Yuan
Hey Yuan.
<= div class=3D"moz-signature">
Thank= s for putting in all this work into tree-sitter in Emacs!

Trying the latest Emacs-version in feature/tree-sitter I got an error I tend to get "a lot" with tree-sitter based modes, which I hoped bundling things with Emacs would solve, namely obtaining the original shared-object containing the compiled grammer.

Like for your python-mode, I= get this:

File mode specification error: (treesit-load-language-er= ror python (/home/jostein/.emacs.d/tree-sitter/libtree-sitter-python: ca= nnot open shared object file: No such file or directory /home/jostein/.e= macs.d/tree-sitter/libtree-sitter-python.so: cannot open shared object f= ile: No such file or directory libtree-sitter-python: cannot open shared= object file: No such file or directory libtree-sitter-python.so: cannot= open shared object file: No such file or directory))

I realize thi= rd-party modes are on their own, but when tree-sitter is compiled with E= macs, I would at least expect the Emacs-build to also produce these .so-= files.

What are your thoughts on how we can best, across the Emacs-= verse, provide these libraries? Or at least for the modes which are bund= led with Emacs itself?

<= div class=3D"moz-signature">Being a tree-sitter based-developer myself, = I know where I can go to get this compiled to make the mode runnable, bu= t surely that's not how we can deploy this en-masse. Most people will be= stuck at this point, and we will need to come up with a better answer.<= br>

--
 Kind regards
Jostein= Kj=C3=B8nigsen


--3da8ab7e4fd840dbb4f9483bc37a8826--