From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id iNEbEk5oEGHzIAEAgWs5BA (envelope-from ) for ; Mon, 09 Aug 2021 01:27:10 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id iMrMDU5oEGENMgAA1q6Kng (envelope-from ) for ; Sun, 08 Aug 2021 23:27:10 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id B24C1D033 for ; Mon, 9 Aug 2021 01:27:09 +0200 (CEST) Received: from localhost ([::1]:38670 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mCsCC-0005jj-MP for larch@yhetil.org; Sun, 08 Aug 2021 19:27:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56110) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mCsC6-0005ja-EZ for guix-patches@gnu.org; Sun, 08 Aug 2021 19:27:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:43389) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mCsC6-0000XE-8A for guix-patches@gnu.org; Sun, 08 Aug 2021 19:27:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mCsC6-0000dI-0h for guix-patches@gnu.org; Sun, 08 Aug 2021 19:27:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#49946] [PATCH 00/31] Tree-sitter, node-gyp addon support and emacs-tree-sitter Resent-From: Pierre Langlois Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 08 Aug 2021 23:27:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 49946 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 49946@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.16284651852386 (code B ref -1); Sun, 08 Aug 2021 23:27:01 +0000 Received: (at submit) by debbugs.gnu.org; 8 Aug 2021 23:26:25 +0000 Received: from localhost ([127.0.0.1]:54935 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mCsBU-0000cQ-GQ for submit@debbugs.gnu.org; Sun, 08 Aug 2021 19:26:24 -0400 Received: from lists.gnu.org ([209.51.188.17]:54406) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mCsBS-0000cH-M2 for submit@debbugs.gnu.org; Sun, 08 Aug 2021 19:26:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56108) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mCsBS-0005hc-BU for guix-patches@gnu.org; Sun, 08 Aug 2021 19:26:22 -0400 Received: from mout.gmx.net ([212.227.15.19]:42791) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mCsBQ-0008Qj-6Q for guix-patches@gnu.org; Sun, 08 Aug 2021 19:26:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1628465176; bh=+yntOmXptqFSwgFtpzEf1tsbxlIZN27mdaABHfTpEO0=; h=X-UI-Sender-Class:From:To:Subject:Date; b=RMp3hTRSaOs26fxCiI/ZVRklg5luXr6ddNe5jjrzt1uajVRV4bGAcPqZwi+oQpHqc 6EEId016qBLLZSmXYtU50/4K0m2SZvQz6j+/DvSrR+xMwUrtLey8rn1FX+3PXCQbG0 i8lfp1E1e6nzfO7ZJkrSNdbVfR5g77Ivf3up5di0= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from labiere ([82.69.64.142]) by mail.gmx.net (mrgmx004 [212.227.17.184]) with ESMTPSA (Nemesis) id 1Mn2aD-1mwuFj0or9-00k4Ke for ; Mon, 09 Aug 2021 01:26:16 +0200 User-agent: mu4e 1.6.1; emacs 27.2 From: Pierre Langlois Date: Sun, 08 Aug 2021 23:29:43 +0100 Message-ID: <87h7fztt60.fsf@gmx.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Provags-ID: V03:K1:Y5BYsE5Kdb4srnwpPFVWNe/+I8pcxFIANPUb2OR2FrCZGNXNz0T I40eI/SJbewEh6iL83cOPbZwMI8JOQaT633/HMVcirNe2d4MwWOPmZ47oAaT4KQINBodyJ4 slgDaoLQcUgmM81YbkP888wroOXkAxXtFZTHoP1AZNPKjOTu3uyvYwKuCPmB8yOF71Ri4Ot pTxVBKboZSGuO6RINFfrw== X-UI-Out-Filterresults: notjunk:1;V03:K0:+X6p37ikJJI=:uJFdd2nh1S7prerLS1+nKo 2e2/AaqgrcY8IBEqo5CTFu/YtAtS9GHWIUbfDDuauETiIC9aoCfr2LZ5iuNDwmV0W/X8iB06f 6PgIh2sb95QPh71EzA5QLpp++F+PCs+4fBZf8Y6UayU1PzpLpoV+fLGU+tsJVJKoVhInR2fXZ NBGb/ww7PiW1uRp2jaEy6QrlmIheqAKxAqsPk4GsLMI1xTLk4AlQv3wIl/WQUBDMEwbiGt2nN NTfOpf+ZJ1kdubDQ2IYsNQqkC04ZNmqbSgjjSbQDhNWOLMzR9T461tvyG+46HJmQ3NXCHVP5k DwMc91a/T109LjKDqTc+XsbSm8EH4dbBE2021IA1s4DuBMSdRhxMwxKZqkZezZiLxFGUNmaCi UpSecUf/tXhCIE3bLyqXOvK2F9bngHqumuGSaHa3w3HI3xi1jVZIiwDj9B0aCG+OvzCw8YWsa 9CAJ9yVrTS6hpPWmCzfFaaM0sy7Udru/nsOCzjMATLvPzQ2ZRPxU2SRw/8qUe+VD6Diw8DOms yQ2sKnvC0G7JfXIQJYIxelAlheSJIT6ggDNz1NhGDbtWzVeQOqimF+819W78jfQNjo8RAKDf2 RvjPCmTbvQ38m75hoHzKZ3C+qe09nhC/7dCa5dLu5r3Dga0Uub/q2i4lDafWTjvwIr6qe6G30 NN/zHJ3C82lUdMChgBCi0pXJC5/JUTCutS071FMTou6bZ4Z3BlxsluQgxxQfrnkAN/SJbNOw3 c6GQVVO20GZqMzxN/cfnT1EbYj3gT6gOwLZKn4VNYcUMWMrRdpy87ECHCUcqZMfSD10JqJJlJ tpKjJfo+EqrV69tLhEdljUkT5HmvjeLHfYixrtp1hOQl2ykJbJBNAHp1RWz+nx7EnfyCM11he 5+iF/aCHthAWWJNU/qQKFVsNE4MktcJwua3A6a1naL/aZlm/RUwhC/wLD5wN5BRHNPCVE8bji hmXq5Lsv3/yV1pGaajS/ce4ZMryqciD/adbz/ZVsaeJRqELjxh74/SyMDwPABKpPnAWSKShmQ IsIG+jYv9uIwfv+vaNl5hab5G25vStqTllRcI9asLu7LEqhSx7RH+qR8Elk6bN41JC2boLCD8 nMGldSDGPdZKVa6L6H4Gg5Cg/D025YpopDTlvV0hmCZiyfYZhevTnmpEQ== Received-SPF: pass client-ip=212.227.15.19; envelope-from=pierre.langlois@gmx.com; helo=mout.gmx.net X-Spam_score_int: -25 X-Spam_score: -2.6 X-Spam_bar: -- X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1628465230; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:list-id:list-help:list-unsubscribe:list-subscribe: list-post:dkim-signature; bh=f+vXkxiPnXK7z6/2dyKG3zaTedp2CAYYTAqfE5RVGiQ=; b=QO+DpHcGVYXLvMA7r8BEnjTv1LjF92s0Lr/I/niKCWKSFhysXVcele8Mgg4slXXNFx9CkB kToK8VrYiqapnvUrXOpe1VHA0vuaiugciNyOvX3sF+zPMlSB9s1AB09ZURhzLEXliLZ4Y4 eY6VukXAXluIg+FMz0OCM119UdSulmyJSZNNoXvSLf6b+zdDHjBIQ2ylmUgy8ZwhOtVWsV W71F+zk7SjAGFTfnsMCzItCVmuPxEeE5JKWccw8A19tEuN6Q3Y56GXeXfGi4QEvZUTpeqy gvyo6+OHa2fbNZdzwislTdX1bIUHBhox6skMOpRAG2PGs6JoWTWj2967/Pqcgg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1628465230; a=rsa-sha256; cv=none; b=ePJpRTR9T3vjzPJ6I547itGI15YNkfPsZ/0W3aU3BjNcyBrxM0GzJhg7KmCLz1lxIqfMEt ijKBvbfzLtmZsWtqKy8/aNa8Cg976wfvqvbyA0Mciq1g4zhjE4X2PQwehgOGd3JEww+brQ eIneatS98zAdTXb2THlN3097T0tIsWzYdcVMnBfgGk26ycj2BLDOHTg5ufaLjoprHQz3IH NlS6lmdSoHzEwTwpjAoUfdc6X6DcFoeH2m9dhpr8O0HFowGyLnhxxITah96PKMjjRAnJW1 WSAc4SErUJBkkIyHw/SyvBiaFShiOFR46oFfKyZXN0gGBGnTg4IdhVZh55lyew== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmx.net header.s=badeba3b8450 header.b=RMp3hTRS; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmx.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: -0.91 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmx.net header.s=badeba3b8450 header.b=RMp3hTRS; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmx.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: B24C1D033 X-Spam-Score: -0.91 X-Migadu-Scanner: scn1.migadu.com X-TUID: ioeWYazNpJSz --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi Guix! Here's a patch series that adds support for the Tree-sitter library, command-line tool and its support in emacs: https://tree-sitter.github.io/tree-sitter The series in split into multiple parts: 1. Add the tree-sitter package, which is both a command-line tool for generating parsers and a runtime library. The runtime is written in C but the tooling in Rust. * gnu: rust-chunked-transfer: Update to 1.4.0. * gnu: Add rust-fdlimit. * gnu: Add rust-html-escape. * gnu: Add rust-smallbitvec. * gnu: rust-tiny-http: Update to 0.8. * gnu: rust-spin: Update to 0.7. * gnu: Add tree-sitter. 2. Add support for native addons to the node build-system. * gnu: node: Patch /usr/bin/env in node-gyp. * guix: node-build-system: Support compiling addons with node-gyp. I've not touched the node build system before so it'd be great to get some review here! The idea is that when a package has a `binding.gyp' file then there's native code that needs to be compiled against node. This uses node-gyp, which is a build system written in python. Normally, there shouldn't be anything to do, the package.json file should invoke `node-gyp` tp compile C code. However, `node-gyp' needs node headers to compile against, packaged as a tarball, which it tries to download. Instead, we can run a `node-gyp --tarball <> configure' step to manually provide the tarball, which we can package separately for any given node version. I could have added this into the packages themselves, but I thought it made more sense to be a part of the build system, let me know what you think! 3. Packaging grammars for multiple languages. Tree-sitter grammars are Nodejs packages, they consist of a grammar.js file which the tree-sitter command-line program uses to generate C code for the parser/scanner. Using npm as a packaging mechanism for this seems overkill, but it's true some grammars depend on each other (cpp on c, typescript and javascript). * gnu: Add node-nan. * gnu: Add tree-sitter-c. * gnu: Add tree-sitter-cpp. * gnu: Add tree-sitter-css. * gnu: Add tree-sitter-go. * gnu: Add tree-sitter-html. * gnu: Add tree-sitter-java. * gnu: Add tree-sitter-javascript. * gnu: Add tree-sitter-json. * gnu: Add tree-sitter-julia. * gnu: Add tree-sitter-php. * gnu: Add tree-sitter-python. * gnu: Add tree-sitter-rust. * gnu: Add tree-sitter-typescript. 4. Finally, emacs support! emacs-tree-sitter can be used for syntax highlighting, replacing font-lock. The tree-sitter runtime library is built as a Rust crate and exposed to emacs using a module thanks to the rust-emacs crate. Then emacs-tree-sitter-core provides just the core APIs as exposed by the rust module, then emacs-tree-sitter integrates it with emacs' syntax highlighting (and probably other things). Finally, we bundle together the various language grammars into a single emacs-tree-sitter-langs package so that they can get loaded along with their respective major modes. Note that there are grammars left to support which I've not yet managed to package, but what we have already is a good start I think :-). So far I've tested C++, Rust, Python and JavaScript and JSON. * gnu: Add rust-bindgen@0.56. * gnu: Add rust-tree-sitter. * gnu: rust-emacs-module: Update to 0.16. * gnu: rust-emacs-macros: Update to 0.17. * gnu: rust-emacs: Update to 0.17. * gnu: Add emacs-tree-sitter-core. * gnu: Add emacs-tree-sitter. * gnu: Add emacs-tree-sitter-langs. Let me know what you think! This rebuilds all node packages, but there aren't that many so hopefully that'll be ok on master. Thanks, Pierre Pierre Langlois (31): gnu: rust-chunked-transfer: Update to 1.4.0. gnu: Add rust-fdlimit. gnu: Add rust-html-escape. gnu: Add rust-smallbitvec. gnu: rust-tiny-http: Update to 0.8. gnu: rust-spin: Update to 0.7. gnu: Add tree-sitter. gnu: node: Patch /usr/bin/env in node-gyp. guix: node-build-system: Support compiling addons with node-gyp. gnu: Add node-nan. gnu: Add tree-sitter-c. gnu: Add tree-sitter-cpp. gnu: Add tree-sitter-css. gnu: Add tree-sitter-go. gnu: Add tree-sitter-html. gnu: Add tree-sitter-java. gnu: Add tree-sitter-javascript. gnu: Add tree-sitter-json. gnu: Add tree-sitter-julia. gnu: Add tree-sitter-php. gnu: Add tree-sitter-python. gnu: Add tree-sitter-rust. gnu: Add tree-sitter-typescript. gnu: Add rust-bindgen@0.56. gnu: Add rust-tree-sitter. gnu: rust-emacs-module: Update to 0.16. gnu: rust-emacs-macros: Update to 0.17. gnu: rust-emacs: Update to 0.17. gnu: Add emacs-tree-sitter-core. gnu: Add emacs-tree-sitter. gnu: Add emacs-tree-sitter-langs. gnu/local.mk | 1 + gnu/packages/crates-io.scm | 290 +++++++++++-- gnu/packages/node-xyz.scm | 30 ++ gnu/packages/node.scm | 29 +- gnu/packages/tree-sitter.scm | 716 +++++++++++++++++++++++++++++++ guix/build-system/node.scm | 16 + guix/build/node-build-system.scm | 15 + 7 files changed, 1064 insertions(+), 33 deletions(-) create mode 100644 gnu/packages/tree-sitter.scm =2D-=20 2.32.0 --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmEQaBcYHHBpZXJyZS5s YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31Uk64H/RTjvpOAq1dLgQVt69syGPZO cx2KRhhKea+vAnjzkcau71H+ekqv2faeUCeRYJATBViOsavxbs0cMp4lqTS+7YQY 1K0rH/sR0Kc4D5Reb6OhUBkp1QjbrETydrAhp9vdX8KPwsMf8qApaxZT7KC49eYj UeiCFytDQ0f7fcELUWMnIsy3ca1F9Q/FarUofITWGNiJ5biWxkr0xhdO4qe7roo/ xEElSYGbVp7qNeXcyku7kRWQES7F2ijsc354Hv1G6HAA76tkyDF6r2OOzKtGLFPg rpfduwfzWo4DENtOvSjekIHtkVSWuW3g5s6ThN+jSoAZf0IUE4XoxSn5i5b/Keg= =vVea -----END PGP SIGNATURE----- --=-=-=--