From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Theodor Thornhill Newsgroups: gmane.emacs.devel Subject: Re: Tree-sitter api Date: Sun, 05 Sep 2021 23:15:55 +0200 Message-ID: References: <83r1f7hydn.fsf@gnu.org> <95F37923-5BF9-4D81-B361-267CF119FBCA@gmail.com> <735AF34C-FD18-4A6A-A99D-E5D8EB4DE4F3@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="12041"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Stefan Monnier , Eli Zaretskii , =?utf-8?Q?Cl=C3=A9ment?= Pit-Claudel , Stephen Leake , emacs-devel To: Yuan Fu Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Sep 05 23:17:17 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mMzVs-0002pt-Lo for ged-emacs-devel@m.gmane-mx.org; Sun, 05 Sep 2021 23:17:17 +0200 Original-Received: from localhost ([::1]:56316 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mMzVr-0002VT-FW for ged-emacs-devel@m.gmane-mx.org; Sun, 05 Sep 2021 17:17:15 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:47140) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mMzUl-0001mS-JV for emacs-devel@gnu.org; Sun, 05 Sep 2021 17:16:07 -0400 Original-Received: from out1.migadu.com ([91.121.223.63]:41204) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mMzUh-00072z-6I; Sun, 05 Sep 2021 17:16:07 -0400 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=thornhill.no; s=key1; t=1630876558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CuYwGhuhl5kChEBdhfopw2aLqQGMuY2lIEIRGl+Jreo=; b=gjlmvqJksZHHxVkbZ5OBrwIwpFqG1qT5IK7y2o7p8RriZH/C8/RJpElV2f/WDw5PnZP6Fv 5Y+tw4f1IsngUmXoTP0ygoHQmokpgD6V+OjJseYqF7BPbGpLp8OGPvcet+sX/8lGm/FYN5 UI/Bmd4Y8Cd506FIiW1pkEEhzG8ON5/N+p4ix9jzqPudMINYGHccIXSc406h5nVlAuV1qZ MDf9yfD/NiOBBeQdQVBhftxXQT9YA4+gtnqb077ks7kC7J02t6s0VQYWMGPCobhUbp/OGa Q5E3jIsIr/EaNADjCn4N7i9FsJwY2L4fsdxTCqdsacyRwMrKfC4J8ioMwDARBQ== In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: theo@thornhill.no Received-SPF: pass client-ip=91.121.223.63; envelope-from=theo@thornhill.no; helo=out1.migadu.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:274038 Archived-At: Yuan Fu writes: > You can find the code at https://github.com/casouri/emacs.git, in =E2=80= =9Cts=E2=80=9D branch. As long as tree-sitter library is in the standard pa= th, Emacs will compile with tree-sitter support. Language definitions are l= oaded by dynamic modules. You can find a script for building dynamic module= s at https://github.com/casouri/tree-sitter-module, you can even just grab = the release file. I need to add C# to the list of languages in the build sc= ript. > > Now you have Emacs and dynamic modules. You can also build the manual, I = just wrote the manual entries for tree-sitter API, its a draft but should e= xplain everything the Emacs tree-sitter API provides. To build the manual y= ou want to go to /doc/listpref, and do =E2=80=9Cmake -e HTML_OPTS=3D"--html= =E2=80=9D elisp.html=E2=80=9D, that should compile a manual in elisp.html d= irectory. The tree-sitter part is in =E2=80=9C37 Parsing Program Source=E2= =80=9D. I attached a zip file containing the compiled manual on my machine,= you can just use that. > > I haven=E2=80=99t written manual for font-lock and indent support, becaus= e they are not settled yet. To see how they work, you can read: > > 1. the source of ts-c-mode in /lisp/tree-sitter.el, > 2. doctoring of font-lock-tree-sitter-defaults and font-lock-tree-sitter-= settings, and=20 > 3. docstring of tree-sitter-simple-indent-rules > > BTW, tree-sitter-inspect-mode could be helpful. > >> Right now we support the rust version on melpa, but I'd rather >> move to this core-supported package. How far are we from including this >> in core, > > Some blockers that I can think of are 1) tree-sitter lacks a way to chang= e its malloc behavior in run-time, I commented on their road-map issue, but= no one has replied yet, 2) font-lock and indent support hasn=E2=80=99t set= tled, 3) writing, reviewing and editing the manual will take some time, 4) = I need to work on a better way to build and distribute language dynamic mod= ules. > >> and what can I do to help? > > For a starter, could you perhaps have a look at the indentation system (t= ree-sitter-simple-indent-rules and friends), and tell me if anything is lac= king? Too complex, not powerful enough, etc. Do you have any suggestions? T= he same goes for font-lock. > > Also, I=E2=80=99m happy to hear your suggestions on the general tree-sitt= er API and the manual. > Thank you for your thorough instructions. I've been able to compile it on my system, but I'm having trouble with the c-sharp module. I get this error: -------------------------------------------- Cloning into 'tree-sitter-c-sharp'... remote: Enumerating objects: 62, done. remote: Counting objects: 100% (62/62), done. remote: Compressing objects: 100% (57/57), done. remote: Total 62 (delta 17), reused 19 (delta 0), pack-reused 0 Receiving objects: 100% (62/62), 831.92 KiB | 2.12 MiB/s, done. Resolving deltas: 100% (17/17), done. tree-sitter-c-sharp.c:7:33: error: expected ';' after top level declarator extern TSLanguage *tree_sitter_c-sharp(void); ^ ; tree-sitter-c-sharp.c:16:40: error: implicit declaration of function 'sharp= ' is invalid in C99 [-Werror,-Wimplicit-function-declaration] TSLanguage *language =3D tree_sitter_c-sharp(); ^ 2 errors generated. binding.cc:2:10: fatal error: 'node.h' file not found #include ^~~~~~~~ 1 error generated. ---------------------------------------------- I'm guessing this is due to the hyphen in the function name. I remember we had to do some shenanigans in the rust variant some time ago to translate this properly. In the C files we need to use underscore rather than hyphen, yes? If this isn't too hard to do I guess I can try to make a PR to your project, otherwise you at least have a bugreport here :) I'm also looking into the code now, and it looks nice so far. I'll come back to you when I have something more! Theodor