From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id sO3wB+C/KGIQLQEAgWs5BA (envelope-from ) for ; Wed, 09 Mar 2022 15:55:28 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id sHPDBOC/KGKlFgAAauVa8A (envelope-from ) for ; Wed, 09 Mar 2022 15:55:28 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 94C932DEE3 for ; Wed, 9 Mar 2022 15:55:27 +0100 (CET) Received: from localhost ([::1]:37192 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nRxio-00005N-9v for larch@yhetil.org; Wed, 09 Mar 2022 09:55:26 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40940) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nRxiQ-000058-T6 for guix-patches@gnu.org; Wed, 09 Mar 2022 09:55:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:36730) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nRxiQ-0002Of-IB for guix-patches@gnu.org; Wed, 09 Mar 2022 09:55:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nRxiQ-0004ew-F5 for guix-patches@gnu.org; Wed, 09 Mar 2022 09:55:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#49946] [WIP PATCH v3 00/26] gnu: Add tree-sitter for emacs (and neovim as well?). Resent-From: Pierre Langlois Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 09 Mar 2022 14:55:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49946 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Luis Henrique Gomes Higino Cc: Pierre Langlois , 49946@debbugs.gnu.org Received: via spool by 49946-submit@debbugs.gnu.org id=B49946.164683766017842 (code B ref 49946); Wed, 09 Mar 2022 14:55:02 +0000 Received: (at 49946) by debbugs.gnu.org; 9 Mar 2022 14:54:20 +0000 Received: from localhost ([127.0.0.1]:58858 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nRxhd-0004dZ-LY for submit@debbugs.gnu.org; Wed, 09 Mar 2022 09:54:20 -0500 Received: from mout.gmx.net ([212.227.17.20]:52461) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nRxha-0004dG-4B for 49946@debbugs.gnu.org; Wed, 09 Mar 2022 09:54:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1646837643; bh=P5tHrL4CQgzRpczxitO8Fg3unaQqWZ1Er7jC0pzMj84=; h=X-UI-Sender-Class:References:From:To:Cc:Subject:Date:In-reply-to; b=cubUYNj4j6dahNGS13P09jeIs5cYb9fHtYxETZCsdsZHwCYu8EBq/e98SyCGB1i93 31+Xw51QaDnaibXE9RqZf8KypdCWt0MRAptBPuYPxay+mUC5QgIb1ceSc5XlOc+0/l Ie/cEhuiGXpCZy6v6Oi3h7jznv9csxu8ZdFV2CmU= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from labiere ([82.69.64.142]) by mail.gmx.net (mrgmx104 [212.227.17.174]) with ESMTPSA (Nemesis) id 1MVNB1-1nciIX47MN-00SRKM; Wed, 09 Mar 2022 15:54:03 +0100 References: <87h7fztt60.fsf@gmx.com> <20210808233354.6745-1-pierre.langlois@gmx.com> <20210808233354.6745-7-pierre.langlois@gmx.com> <87czpwzg9e.fsf@gmx.com> <87leyinxly.fsf@gmx.com> <87lexl1goc.fsf@gmail.com> User-agent: mu4e 1.6.10; emacs 27.2 From: Pierre Langlois Date: Wed, 09 Mar 2022 13:46:29 +0000 In-reply-to: <87lexl1goc.fsf@gmail.com> Message-ID: <87sfrrnpme.fsf@gmx.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Provags-ID: V03:K1:yGdqPkv0q1dJyqigF6Ro3vwKo9MrVyLHBCtSo8t4F6C6TdgwSe6 S1cO2tcCqyqPQp1kkaFxJSXDZgsN5EhIkQ50yNoLXnOUXRVkPE1XCTJDN/zvntqkWDKG8P5 3gikmNk9/UVUBZMtU24GlU7iJDOt2Gf91urcSyj9C3mkL+cvHWVCHiAZtg0QolyNL/OXKz8 o7RH0p0ojMUxMpxwwT7cw== X-UI-Out-Filterresults: notjunk:1;V03:K0:doJAjd3ZZ5Y=:M3CE7Q66q9K3/OcxgGX35M M+ayUBSGZdq3/+5GPcYHNVTFN+dUAvTMgE6IhXypGyA4nbFOKECEcc5dstuS2I+27e27yP7nW g3N6fpzUdoVHDPdYW9pBToZ1+mTlbseB3UrfvgHmUaDDfo7zWSdiNFm+cHukf1+FNpUT7dy2v 5XwO5dmhSeTmMLLaHyRQhxViZxVosxNAhQl69iajq6ckADw5rum72qSfp/TSlrRGoDt9HH16Z oZqJ8LMqBlj0ZOhPAnYWpOgjR3+yLIMoTOpMscIkFiI9FqGSDa2dMyikUgph5Gcxdh8QOALas 4oA12PVFEkTlhRF0B2zxYksZ22B3hz4acuYXeiViYpBZmL1z3+gSKg1GkXe/Z24GkYJRX0toI JZbPXVOyhJZxJDKnhnQNH5jetUxbMnseKXLis1p/6TnwPo9ouabIyLWdzazoSmW/wPuXOxB5u MT0ul8uokYyoVmz6kS6AeF3lB+AkKtrubV6a6C8Aodm/CdT4S31R5ar4Qh3u1scOSNrlJd+fb TdDlsB77EtEKSeBmeNFzONlh7G08mgV91Dsow659oqJpAInPMAIYqGSjxzlaU/lOfpzeKbHsJ 3AZa6iKTMHGv2omeheHc36LZJXtezwDrUa4lW4W6p/hHmhKGMki4ehN5TAypiP/uK/lPKSOzK +/rhUf5rewJeH4+3+58QE7iAJldvfoHykw/JYbg/9xP7r0GDIpvIqkWds3EaHgmcuhJZBMPEu pbR1rFNdXs1ZNgcHgLW9kmpn/49RruhiApTXtjIAvlgwPaw6e/92apozdPvwx1tbPsQQ22PoN n6bQeVrHkg4OI1ZelOjRAlnM0iAjPzShR3wamSmQeT8m6jiXbj4uNgaXImErZ3AfDdzn8KSmY 8dwqEpXMGGTSPw8+8ccdHV52dd4FNL6Z0SopA8MpApfyodTR5qvgUAoY9Pl6jJX2UAGvELq4q 9yJW50dVl0wyR3W3YEkiM7IF3W24X6vvOKEtW0qwBEr6SONOO2wPJdI+9LCY38scoeKF+7CZi lUGqb+WIa5HgpbtUQmDzxs1rPUvxPymp0tNitkdPulTtcVeIFasqyuHbHnj7MvpEPFmxBn+7q C3AJcW2zM5EEUk= X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1646837727; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=Vh4tcDFM0ZuShvTReuWqTEgvzN+cf5QLgDHPrvcUfFI=; b=NA0tBG73FVlDW388rw4iffP/zaSOBGQ4Q4MV9K1ABWOplALdlmhf7tM3VhzcGjTOsdtyM8 vIdQc6Ft2kiXN+Tzwq2k63lXiBG978aXSNUbiCbeaI6dcMQtdgdIbS6DptCS3S2VzDCSMF xIRl4hqVoWOW6HsWtVRrcV8QwOh2vsVS8Cpy/g2LEmP9xGzlBHLsoB6N0/JnjXoPIlkJLv H96hXrSSv26sm0iuoCKOF8tEgu+I5C9uwkCbuRPt9DPF3ToiMxRt6+YSKZA4iLQEozATZz fUiwy+NCWnCUmjLeZj4+i/tMp7YrhFQaPCdmCQPAH3yPpghnhgQRhba3yDllIA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1646837727; a=rsa-sha256; cv=none; b=dylN/65fVtiNLaSCuWaj6nmFE0kAVlAJDV2hHFPouoge0mkXene2Yust7zc/Z+LLfmOQNn LnYeJvm15ZHf4sH3xRwRU5nAkNYdlAR7V4+t/59MG40tma9/3vGgzYkdfmEHZidQeSAV/p Iq0QXpY+U/CuVZxV4DbKmhB0Z4GILEQocFyn4ppxxBDN1B07J/Q2BPhKZEbpBjF3AeOL3Y FVUKMfKjpZEibj9Z4Wo8Uq2/48+lldS5v0QXgawVyYgNy67XOOG1MIRvhZpEdHC7iXF9wb g8jCA2DW+AOl4TK7vm7XmFlfBtymFfngOWa85wGIQmx372mS80CgCmYQeIKXSw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmx.net header.s=badeba3b8450 header.b=cubUYNj4; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmx.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 2.99 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmx.net header.s=badeba3b8450 header.b=cubUYNj4; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmx.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 94C932DEE3 X-Spam-Score: 2.99 X-Migadu-Scanner: scn0.migadu.com X-TUID: u3kHJL82hy/1 --=-=-= Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" --==-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hi Luis, Luis Henrique Gomes Higino writes: > Hi Guix, Hi Pierre, > > I apologize for my delay, the last few weeks were quite busy :-/. No worries! :-). > > Pierre Langlois writes: > >> I'm sharing it early as a WIP first, as I just noticed the other day >> that Luis also contributed a patches for tree-sitter for neovim[0]. The >> series I have here is quite a lot bigger though so I'm happy to rebase >> it on top of yours, Luis :-). The package I had for tree-sitter looks >> basically the same, I would then move it to a new file with all >> tree-sitter related packages, if you're happy with that. > > That's OK by me! > >> So I'm wondering, are these tree-sitter grammar packages also useful for >> the neovim package for tree-sitter? > > I'm not exactly sure about this. Currently, most Neovim users install the= ir > grammars through the installer included in > https://github.com/nvim-treesitter/nvim-treesitter. That ensures=20 > the user gets the version that is ABI compatible with nvim-treesitter. Th= at > makes me uncertain if we could use the same grammar packages for Neovim a= nd > Emacs, at least for now. I see, I suppose for the moment we're not packaging nvim-treesitter so we don't have to concern ourselves with how language grammars are delivered to users. In the future it might be cool to package it, out of curiosity I took a look at it and it: - [0]: Clones the repo with `git'. - [1]: Run `npm install' and `tree-sitter generate'. - [2]: Compiles the generated grammar with gcc. [0]: https://github.com/nvim-treesitter/nvim-treesitter/blob/90485c890503f9= 73271db1ae02ddba6d6fd46397/lua/nvim-treesitter/install.lua#L378 [1]: https://github.com/nvim-treesitter/nvim-treesitter/blob/90485c890503f9= 73271db1ae02ddba6d6fd46397/lua/nvim-treesitter/install.lua#L228 [2]: https://github.com/nvim-treesitter/nvim-treesitter/blob/3aac7f9db9ee19= 73152426c097216e0071dd2293/lua/nvim-treesitter/shell_command_selectors.lua#= L96 The only concern I can think of is that when it runs `npm install', it's likely that it downloads a `tree-sitter' binary, rather than use Guix's tree-sitter-cli package. In my series I had to patch package.json files for each grammar to remove `tree-sitter-cli' as a dependency: https://github.com/tree-sitter/tree-sitter-c/blob/e348e8ec5efd3aac020020e4a= f53d2ff18f393a9/package.json#L20 Otherwise it would pull in the official tree-sitter-cli node package, which then downloads a `tree-sitter' binary :-/ https://github.com/tree-sitter/tree-sitter/blob/master/cli/npm/package.json Anyways, I don't think that's something we should concern ourselves with for now. If we ever want to package nvim-treesitter, I think we might have to patch it so it uses Guix-provided grammars instead of building them locally. For emacs that wasn't so difficult to do, but the real issue is making sure the grammar is compabible, and luckily the emacs package had tests we could run for that. We would also need to run tests if we're to package nvim-treesitter. > >> I do wonder if I'm going about it the right way for Guix though, for >> instance I'm wondering if I should rewrite the grammar packages using a >> new custom build system, do people think it would be worth it? That >> could be done as a follow-up of course. > > I'm not a very experienced packager myself, but from what I saw in your p= atch, > it seems rather unnecessary, as the grammar packages are very simple. That was also my impression :-). Since then I did play with building one to see what it would look like, also as an escuse to go and learn how they work. I'm still on the fence about it though, I've attached what it could look like, if anybody else is also unsure. I think I'd prefer to work on it as a potential follow-up after an initial set of patches get merged. Thanks, Pierre --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmIov4kYHHBpZXJyZS5s YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31USi8H/0ImEqfzU20ae2q2Cb4BgTR3 v16AjRWRFrM9rU6Q2bwoj6tx3Coe/xpMmtVTHOwlwzunwKLttdmyzbFAVGIt7pOb MLLuxBf8LsF42/BJdnFmwDS9sP+PAq2mVpSt0zG2VBs8QO/NhuuvTZwpsea8pBtL kOF22UfKfrKwu7f/PyntMS5pubBWMxl4x0qvDhzeiMyVzvlJvoM/4EkG34fgt566 9eLsvjvjPpLVvNlvIy9sRNfEJKSUp8Ql0LADjnYYJa7JchVP74uBWUicIW0jRaWl cnHnek6zjmtSKoxsP4L3W/+lY4/+UhIUg+fWAa0lps2ligbe20cqWGbq0c+XKvE= =r8Qv -----END PGP SIGNATURE----- --==-=-=-- --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-wip-build-system-for-tree-sitter-grammars.patch Content-Transfer-Encoding: quoted-printable >From a38e3e66788129bbd441f9d28e450dba8a7438d1 Mon Sep 17 00:00:00 2001 From: Pierre Langlois Date: Wed, 23 Feb 2022 20:38:51 +0000 Subject: [PATCH] wip build-system for tree-sitter grammars. --- Makefile.am | 2 + gnu/packages/tree-sitter.scm | 312 ++++++++---------------- guix/build-system/tree-sitter.scm | 109 +++++++++ guix/build/tree-sitter-build-system.scm | 124 ++++++++++ 4 files changed, 342 insertions(+), 205 deletions(-) create mode 100644 guix/build-system/tree-sitter.scm create mode 100644 guix/build/tree-sitter-build-system.scm diff --git a/Makefile.am b/Makefile.am index 8850c4562c..c5da931041 100644 --- a/Makefile.am +++ b/Makefile.am @@ -167,6 +167,7 @@ MODULES =3D \ guix/build-system/ruby.scm \ guix/build-system/scons.scm \ guix/build-system/texlive.scm \ + guix/build-system/tree-sitter.scm \ guix/build-system/trivial.scm \ guix/ftp-client.scm \ guix/http-client.scm \ @@ -220,6 +221,7 @@ MODULES =3D \ guix/build/ruby-build-system.scm \ guix/build/scons-build-system.scm \ guix/build/texlive-build-system.scm \ + guix/build/tree-sitter-build-system.scm \ guix/build/waf-build-system.scm \ guix/build/haskell-build-system.scm \ guix/build/julia-build-system.scm \ diff --git a/gnu/packages/tree-sitter.scm b/gnu/packages/tree-sitter.scm index 0186518b23..725ab16910 100644 --- a/gnu/packages/tree-sitter.scm +++ b/gnu/packages/tree-sitter.scm @@ -23,6 +23,7 @@ (define-module (gnu packages tree-sitter) #:use-module (guix build-system emacs) #:use-module (guix build-system gnu) #:use-module (guix build-system node) + #:use-module (guix build-system tree-sitter) #:use-module (guix build-system trivial) #:use-module (guix download) #:use-module (guix git-download) @@ -224,70 +225,7 @@ (define-public tree-sitter-c "0454jziys33i4kbwnvi9xcck0fzya792ghy32ahgk1hhv96xga9w")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) - (build-system node-build-system) - (native-inputs - (list tree-sitter tree-sitter-cli)) - (outputs '("out" "js")) - (arguments - (list - #:phases - (with-imported-modules '((guix build json)) - #~(modify-phases %standard-phases - (add-after 'patch-dependencies 'delete-dependencies - (lambda _ - (delete-dependencies '("tree-sitter-cli" - "nan" - "node-gyp")))) - (replace 'build - (lambda _ - (invoke "tree-sitter" "generate" "--no-bindings"))) - (replace 'check - (lambda _ - (invoke "tree-sitter" "test"))) - (replace 'install - (lambda _ - (use-modules (guix build json) - (ice-9 regex)) - (let* ((name (assoc-ref - (call-with-input-file "package.json" read-js= on) - "name")) - (lang (cond - ((string-match "^(@.*/)?tree-sitter-(.*)$" n= ame) - =3D> (lambda (m) - (match:substring m 2))) - (else #f))) - (lib (string-append #$output "/lib/tree-sitter"))) - (mkdir-p lib) - (define (source-file path) - (if (file-exists? path) - path - #f)) - (apply invoke - `(,#$(cxx-for-target) - "-shared" - "-fPIC" - "-fno-exceptions" - "-O2" - "-g" - "-o" ,(string-append lib "/" lang ".so") - ,@(cond - ((source-file "src/scanner.c") - =3D> (lambda (file) (list "-xc" "-std=3Dc99= " file))) - ((source-file "src/scanner.cc") - =3D> (lambda (file) (list file))) - (else '())) - "-xc" "src/parser.c"))))) - (add-after 'install 'install-js-module - (lambda* (#:key inputs #:allow-other-keys) - (invoke (search-input-file inputs "/bin/npm") - "--prefix" #$output:js - "--global" - "--offline" - "--loglevel" "info" - "--production" - ;; Skip scripts to prevent building bindings via G= YP. - "--ignore-scripts" - "install" "../package.tgz"))))))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-c") (synopsis "Tree-sitter C grammar") (description @@ -296,7 +234,6 @@ (define (source-file path) =20 (define-public tree-sitter-cpp (package - (inherit tree-sitter-c) (name "tree-sitter-cpp") (version "0.20.0") (source (origin @@ -311,7 +248,8 @@ (define-public tree-sitter-cpp (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) (inputs - (list `(,tree-sitter-c "js"))) + (list tree-sitter-c)) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-cpp") (synopsis "Tree-sitter C++ grammar") (description @@ -319,7 +257,7 @@ (define-public tree-sitter-cpp (license license:expat))) =20 (define-public tree-sitter-c-sharp - (package (inherit tree-sitter-c) + (package (name "tree-sitter-c-sharp") (version "0.19.1") (source (origin @@ -333,6 +271,7 @@ (define-public tree-sitter-c-sharp "054fmpf47cwh59gbg00sc0nl237ba4rnxi73miz39yqzcs87055r")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-c-sharp") (synopsis "Tree-sitter C# grammar") (description @@ -340,7 +279,7 @@ (define-public tree-sitter-c-sharp (license license:expat))) =20 (define-public tree-sitter-bash - (package (inherit tree-sitter-c) + (package (name "tree-sitter-bash") (version "0.19.0") (source (origin @@ -354,17 +293,7 @@ (define-public tree-sitter-bash "18c030bb65r50i6z37iy7jb9z9i8i36y7b08dbc9bchdifqsijs5")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) - (arguments - (substitute-keyword-arguments (package-arguments tree-sitter-c) - ((#:phases phases) - `(modify-phases ,phases - (replace 'delete-dependencies - (lambda _ - (delete-dependencies '("tree-sitter-cli" - "nan" - "node-gyp" - "prebuild" - "prebuild-install")))))))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-bash") (synopsis "Tree-sitter Bash grammar") (description @@ -372,7 +301,7 @@ (define-public tree-sitter-bash (license license:expat))) =20 (define-public tree-sitter-css - (package (inherit tree-sitter-c) + (package (name "tree-sitter-css") (version "0.19.0") (source (origin @@ -386,6 +315,7 @@ (define-public tree-sitter-css "014jrlgi7zfza9g38hsr4vlbi8964i5p7iglaih6qmzaiml7bja2")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-css") (synopsis "Tree-sitter CSS grammar") (description @@ -395,7 +325,7 @@ (define-public tree-sitter-css (define-public tree-sitter-elixir (let ((commit "de20391afe5cb03ef1e8a8e43167e7b58cc52869") (revision "1")) - (package (inherit tree-sitter-c) + (package (name "tree-sitter-elixir") (version (git-version "0.19.0" revision commit)) (source (origin @@ -409,16 +339,7 @@ (define-public tree-sitter-elixir "0zrkrwhw3g1vazsxcwrfd1fk4wvs9hdwmwp6073mfh370bz4140h")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) - (arguments - (substitute-keyword-arguments (package-arguments tree-sitter-c) - ((#:phases phases) - `(modify-phases ,phases - (replace 'delete-dependencies - (lambda _ - (delete-dependencies '("tree-sitter-cli" - "node-gyp" - "nan" - "prettier")))))))) + (build-system tree-sitter-build-system) (home-page "https://elixir-lang.org/tree-sitter-elixir/") (synopsis "Tree-sitter Elixir grammar") (description @@ -428,7 +349,7 @@ (define-public tree-sitter-elixir license:expat))))) =20 (define-public tree-sitter-elm - (package (inherit tree-sitter-c) + (package (name "tree-sitter-elm") (version "5.5.1") (source (origin @@ -442,18 +363,7 @@ (define-public tree-sitter-elm "10hbi4vyj4hjixqswdcbvzl60prldczz29mlp02if61wvwiwvqrw")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) - (arguments - (substitute-keyword-arguments (package-arguments tree-sitter-c) - ((#:phases phases) - `(modify-phases ,phases - (replace 'delete-dependencies - (lambda _ - (delete-dependencies '("tree-sitter-cli" - "nan" - "node-gyp" - "@asgerf/dts-tree-sitter" - "prebuild" - "prebuild-install")))))))) + (build-system tree-sitter-build-system) (home-page "https://elm-tooling.github.io/tree-sitter-elm/") (synopsis "Tree-sitter Elm grammar") (description @@ -461,7 +371,7 @@ (define-public tree-sitter-elm (license license:expat))) =20 (define-public tree-sitter-go - (package (inherit tree-sitter-c) + (package (name "tree-sitter-go") (version "0.19.1") (source (origin @@ -475,6 +385,7 @@ (define-public tree-sitter-go "0nxs47vd2fc2fr0qlxq496y852rwg39flhg334s7dlyq7d3lcx4x")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-go") (synopsis "Tree-sitter Go grammar") (description @@ -482,7 +393,7 @@ (define-public tree-sitter-go (license license:expat))) =20 (define-public tree-sitter-html - (package (inherit tree-sitter-c) + (package (name "tree-sitter-html") (version "0.19.0") (source (origin @@ -496,6 +407,7 @@ (define-public tree-sitter-html "1hg7vbcy7bir6b8x11v0a4x0glvqnsqc3i2ixiarbxmycbgl3axy")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-html") (synopsis "Tree-sitter HTML grammar") (description @@ -503,7 +415,7 @@ (define-public tree-sitter-html (license license:expat))) =20 (define-public tree-sitter-java - (package (inherit tree-sitter-c) + (package (name "tree-sitter-java") (version "0.19.1") (source (origin @@ -517,6 +429,7 @@ (define-public tree-sitter-java "07zw9ygb45hnvlx9qlz7rlz8hc3byjy03d24v72i5iyhpiiwlhvl")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-java") (synopsis "Tree-sitter Java grammar") (description @@ -524,7 +437,7 @@ (define-public tree-sitter-java (license license:expat))) =20 (define-public tree-sitter-javascript - (package (inherit tree-sitter-c) + (package (name "tree-sitter-javascript") (version "0.20.0") (source (origin @@ -538,6 +451,7 @@ (define-public tree-sitter-javascript "175yrk382n2di0c2xn4gpv8y4n83x1lg4hqn04vabf0yqynlkq67")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-javascript") (synopsis "Tree-sitter Javascript grammar") (description @@ -546,7 +460,7 @@ (define-public tree-sitter-javascript (license license:expat))) =20 (define-public tree-sitter-json - (package (inherit tree-sitter-c) + (package (name "tree-sitter-json") (version "0.19.0") (source (origin @@ -560,6 +474,7 @@ (define-public tree-sitter-json "06pjh31bv9ja9hlnykk257a6zh8bsxg2fqa54al7qk1r4n9ksnff")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-json") (synopsis "Tree-sitter JSON grammar") (description @@ -567,7 +482,7 @@ (define-public tree-sitter-json (license license:expat))) =20 (define-public tree-sitter-julia - (package (inherit tree-sitter-c) + (package (name "tree-sitter-julia") (version "0.19.0") (source (origin @@ -581,6 +496,7 @@ (define-public tree-sitter-julia "1pbnmvhy2gq4vg1b0sjzmjm4s2gsgdjh7h01yj8qrrqbcl29c463")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-julia") (synopsis "Tree-sitter Julia grammar") (description @@ -590,7 +506,7 @@ (define-public tree-sitter-julia (define-public tree-sitter-ocaml (let ((commit "0348562f385bc2bd67ecf181425e1afd6d454192") (revision "1")) - (package (inherit tree-sitter-c) + (package (name "tree-sitter-ocaml") (version (git-version "0.19.0" revision commit)) (source (origin @@ -617,43 +533,44 @@ (define-public tree-sitter-ocaml '("ocaml" "interface")) #t)))) (arguments - (substitute-keyword-arguments (package-arguments tree-sitter-c) - ((#:phases phases) - #~(modify-phases #$phases - (replace 'build - (lambda _ - (for-each (lambda (dir) - (with-directory-excursion dir - (invoke "tree-sitter" "generate" "--no-bind= ings"))) - '("ocaml" "interface")))) - (replace 'check - (lambda _ - (for-each (lambda (dir) - (with-directory-excursion dir - (invoke "tree-sitter" "test"))) - '("ocaml" "interface")))) - (replace 'install - (lambda _ - (let ((lib (string-append #$output "/lib/tree-sitter/"))) - (mkdir-p lib) - (invoke #$(cxx-for-target) - "-shared" - "-fPIC" - "-fno-exceptions" - "-O2" - "-g" - "-o" (string-append lib "/ocaml.so") - "ocaml/src/scanner.cc" - "-xc" "ocaml/src/parser.c") - (invoke #$(cxx-for-target) - "-shared" - "-fPIC" - "-fno-exceptions" - "-O2" - "-g" - "-o" (string-append lib "/ocaml-interface.so") - "interface/src/scanner.cc" - "-xc" "interface/src/parser.c")))))))) + (list + #:phases + #~(modify-phases %standard-phases + (replace 'build + (lambda _ + (for-each (lambda (dir) + (with-directory-excursion dir + (invoke "tree-sitter" "generate" "--no-bindi= ngs"))) + '("ocaml" "interface")))) + (replace 'check + (lambda _ + (for-each (lambda (dir) + (with-directory-excursion dir + (invoke "tree-sitter" "test"))) + '("ocaml" "interface")))) + (replace 'install + (lambda _ + (let ((lib (string-append #$output "/lib/tree-sitter/"))) + (mkdir-p lib) + (invoke #$(cxx-for-target) + "-shared" + "-fPIC" + "-fno-exceptions" + "-O2" + "-g" + "-o" (string-append lib "/ocaml.so") + "ocaml/src/scanner.cc" + "-xc" "ocaml/src/parser.c") + (invoke #$(cxx-for-target) + "-shared" + "-fPIC" + "-fno-exceptions" + "-O2" + "-g" + "-o" (string-append lib "/ocaml-interface.so") + "interface/src/scanner.cc" + "-xc" "interface/src/parser.c"))))))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-ocaml") (synopsis "Tree-sitter OCaml grammar") (description @@ -663,7 +580,7 @@ (define-public tree-sitter-ocaml (define-public tree-sitter-php (let ((commit "435fa00006c0d1515c37fbb4dd6a9de284af75ab") (revision "1")) - (package (inherit tree-sitter-c) + (package (name "tree-sitter-php") (version (git-version "0.19.0" revision commit)) (source (origin @@ -683,15 +600,7 @@ (define-public tree-sitter-php (delete-file "src/parser.c") (delete-file-recursively "src/tree_sitter") #t)))) - (arguments - (substitute-keyword-arguments (package-arguments tree-sitter-c) - ((#:phases phases) - `(modify-phases ,phases - (replace 'delete-dependencies - (lambda _ - (delete-dependencies '("tree-sitter-cli" - "nan" - "shelljs")))))))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-php") (synopsis "Tree-sitter PHP grammar") (description @@ -701,7 +610,7 @@ (define-public tree-sitter-php (define-public tree-sitter-python (let ((commit "ed0fe62e55dc617ed9dec8817ebf771aa7cf3c42") (revision "1")) - (package (inherit tree-sitter-c) + (package (name "tree-sitter-python") (version (git-version "0.19.1" revision commit)) (source (origin @@ -715,6 +624,7 @@ (define-public tree-sitter-python "0wrfpg84mc3pzcrdi6n5fqwijkqr1nj5sqfnayb502krvqpjilal")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-python") (synopsis "Tree-sitter Python grammar") (description @@ -722,7 +632,7 @@ (define-public tree-sitter-python (license license:expat)))) =20 (define-public tree-sitter-ruby - (package (inherit tree-sitter-c) + (package (name "tree-sitter-ruby") (version "0.19.0") (source (origin @@ -736,17 +646,7 @@ (define-public tree-sitter-ruby "0m3h4928rbs300wcb6776h9r88hi32rybbhcaf6rdympl5nzi83v")) (modules '((guix build utils))) (snippet tree-sitter-delete-generated-files))) - (arguments - (substitute-keyword-arguments (package-arguments tree-sitter-c) - ((#:phases phases) - `(modify-phases ,phases - (replace 'delete-dependencies - (lambda _ - (delete-dependencies '("tree-sitter-cli" - "nan" - "node-gyp" - "prebuild" - "prebuild-install")))))))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-ruby") (synopsis "Tree-sitter Ruby grammar") (description @@ -754,7 +654,7 @@ (define-public tree-sitter-ruby (license license:expat))) =20 (define-public tree-sitter-rust - (package (inherit tree-sitter-c) + (package (name "tree-sitter-rust") (version "0.20.1") (source (origin @@ -771,6 +671,7 @@ (define-public tree-sitter-rust (native-inputs (modify-inputs (package-native-inputs tree-sitter-c) (prepend bc))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-rust") (synopsis "Tree-sitter Rust grammar") (description @@ -780,7 +681,7 @@ (define-public tree-sitter-rust (define-public tree-sitter-typescript (let ((commit "111b07762e86efab9a918b7c721f720c37e76b0a") (revision "1")) - (package (inherit tree-sitter-c) + (package (name "tree-sitter-typescript") (version (git-version "0.20.0" revision commit)) (source (origin @@ -806,39 +707,40 @@ (define-public tree-sitter-typescript '("typescript" "tsx")) #t)))) (inputs - (list `(,tree-sitter-javascript "js"))) + (list tree-sitter-javascript)) (arguments - (substitute-keyword-arguments (package-arguments tree-sitter-c) - ((#:phases phases) - #~(modify-phases #$phases - (replace 'build - (lambda _ - (for-each (lambda (dir) - (with-directory-excursion dir - (invoke "tree-sitter" "generate" "--no-bind= ings"))) - '("typescript" "tsx")))) - (replace 'check - (lambda _ - (for-each (lambda (dir) - (with-directory-excursion dir - (invoke "tree-sitter" "test"))) - '("typescript" "tsx")))) - (replace 'install - (lambda _ - (let ((lib (string-append #$output "/lib/tree-sitter/"))) - (mkdir-p lib) - (for-each - (lambda (lang) - (invoke #$(cxx-for-target) - "-shared" - "-fPIC" - "-fno-exceptions" - "-O2" - "-g" - "-o" (string-append lib "/" lang ".so") - "-xc" (string-append lang "/src/scanner.c") - "-xc" (string-append lang "/src/parser.c"))) - '("typescript" "tsx"))))))))) + (list + #:phases + #~(modify-phases %standard-phases + (replace 'build + (lambda _ + (for-each (lambda (dir) + (with-directory-excursion dir + (invoke "tree-sitter" "generate" "--no-bindi= ngs"))) + '("typescript" "tsx")))) + (replace 'check + (lambda _ + (for-each (lambda (dir) + (with-directory-excursion dir + (invoke "tree-sitter" "test"))) + '("typescript" "tsx")))) + (replace 'install + (lambda _ + (let ((lib (string-append #$output "/lib/tree-sitter/"))) + (mkdir-p lib) + (for-each + (lambda (lang) + (invoke #$(cxx-for-target) + "-shared" + "-fPIC" + "-fno-exceptions" + "-O2" + "-g" + "-o" (string-append lib "/" lang ".so") + "-xc" (string-append lang "/src/scanner.c") + "-xc" (string-append lang "/src/parser.c"))) + '("typescript" "tsx")))))))) + (build-system tree-sitter-build-system) (home-page "https://github.com/tree-sitter/tree-sitter-typescript") (synopsis "Tree-sitter Typescript grammar") (description diff --git a/guix/build-system/tree-sitter.scm b/guix/build-system/tree-sit= ter.scm new file mode 100644 index 0000000000..bfccea4007 --- /dev/null +++ b/guix/build-system/tree-sitter.scm @@ -0,0 +1,109 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2022 Pierre Langlois +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix build-system tree-sitter) + #:use-module (guix store) + #:use-module (guix utils) + #:use-module (guix packages) + #:use-module (guix gexp) + #:use-module (guix monads) + #:use-module (guix search-paths) + #:use-module (guix build-system) + #:use-module (guix build-system gnu) + #:use-module (guix build-system node) + #:use-module (ice-9 match) + #:export (%tree-sitter-build-system-modules + tree-sitter-build + tree-sitter-build-system)) + +(define %tree-sitter-build-system-modules + ;; Build-side modules imported by default. + `((guix build tree-sitter-build-system) + ;; (guix build json) + ,@%node-build-system-modules)) + +(define* (lower name + #:key source inputs native-inputs outputs system target + #:allow-other-keys + #:rest arguments) + "" + (define private-keywords + '(#:target #:inputs #:native-inputs #:outputs)) + + (and (not target) ;XXX: no cross-compilation + (bag + (name name) + (system system) + (host-inputs `(,@(if source + `(("source" ,source)) + '()) + ,@(map (match-lambda + ((name package) + `(,name ,package "js"))) + inputs) + ;; Keep the standard inputs of 'gnu-build-system'. + ,@(standard-packages))) + (build-inputs `(("node" ,(module-ref (resolve-interface '(gnu pac= kages node)) + 'node-lts)) + ("tree-sitter" ,(module-ref (resolve-interface '(= gnu packages tree-sitter)) + 'tree-sitter)) + ("tree-sitter-cli" ,(module-ref (resolve-interfac= e '(gnu packages tree-sitter)) + 'tree-sitter-cli)) + ,@native-inputs)) + (outputs (match outputs + (("out") (cons "js" outputs)) + (_ outputs))) + (build tree-sitter-build) + (arguments (strip-keyword-arguments private-keywords arguments)))= )) + +(define* (tree-sitter-build name inputs + #:key + source + (phases '%standard-phases) + (outputs '("out" "js")) + (search-paths '()) + (system (%current-system)) + (guile #f) + (imported-modules %tree-sitter-build-system-mo= dules) + (modules '((guix build utils) + (guix build tree-sitter-build-syste= m)))) + (define builder + (with-imported-modules imported-modules + #~(begin + (use-modules #$@(sexp->gexp modules)) + (tree-sitter-build #:name #$name + #:source #+source + #:system #$system + #:phases #$phases + #:outputs #$(outputs->gexp outputs) + #:search-paths '#$(sexp->gexp + (map search-path-specifica= tion->sexp + search-paths)) + #:inputs #$(input-tuples->gexp inputs))))) + + (mlet %store-monad ((guile (package->derivation (or guile (default-guile= )) + system #:graft? #f))) + (gexp->derivation name builder + #:system system + #:guile-for-build guile))) + +(define tree-sitter-build-system + (build-system + (name 'tree-sitter) + (description "") + (lower lower))) diff --git a/guix/build/tree-sitter-build-system.scm b/guix/build/tree-sitt= er-build-system.scm new file mode 100644 index 0000000000..bf1d2e363d --- /dev/null +++ b/guix/build/tree-sitter-build-system.scm @@ -0,0 +1,124 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2022 Pierre Langlois +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (guix build tree-sitter-build-system) + #:use-module ((guix build node-build-system) #:prefix node:) + #:use-module (guix build json) + #:use-module (guix build utils) + #:use-module (ice-9 match) + #:use-module (ice-9 regex) + #:use-module (srfi srfi-1) + #:export (%standard-phases + tree-sitter-build)) + +(define (tree-sitter-node-inputs inputs) + "" + (alist-delete "node" + (alist-delete "source" + (filter (match-lambda + ((label . directory) + (directory-exists? (string-append directory + "/lib/node_modules"))) + (_ #f)) + inputs)))) + +;;; +;;; Phases. +;;; + +(define* (adjust-dependencies #:key inputs #:allow-other-keys) + "" + (node:with-atomic-json-file-replacement "package.json" + (match-lambda + (('@ . pkg-meta-alist) + (cons '@ (map (match-lambda + (("dependencies" @ . deps) + '("dependencies" @)) + (("devDependencies" @ . deps) + `("devDependencies" @ + ,@(map + (lambda (input) + `(,(car input) . "latest")) + (tree-sitter-node-inputs inputs)))) + (other other)) + pkg-meta-alist)))))) + +(define (build . _) + (invoke "tree-sitter" "generate" "--no-bindings") + #t) + +(define (check . _) + (invoke "tree-sitter" "test") + #t) + +(define* (install #:key outputs #:allow-other-keys) + (use-modules (guix build json) + (ice-9 regex)) + (let* ((name (assoc-ref + (call-with-input-file "package.json" read-json) + "name")) + (lang (cond + ((string-match "^(@.*/)?tree-sitter-(.*)$" name) + =3D> (lambda (m) + (match:substring m 2))) + (else #f))) + (lib (string-append (assoc-ref outputs "out") + "/lib/tree-sitter"))) + (mkdir-p lib) + (define (source-file path) + (if (file-exists? path) + path + #f)) + (apply invoke + `("g++" + "-shared" + "-fPIC" + "-fno-exceptions" + "-O2" + "-g" + "-o" ,(string-append lib "/" lang ".so") + ,@(cond + ((source-file "src/scanner.c") + =3D> (lambda (file) (list "-xc" "-std=3Dc99" file))) + ((source-file "src/scanner.cc") + =3D> (lambda (file) (list file))) + (else '())) + "-xc" "src/parser.c")))) + +(define* (install-js #:key inputs outputs #:allow-other-keys) + (invoke (search-input-file inputs "/bin/npm") + "--prefix" (assoc-ref outputs "js") + "--global" + "--offline" + "--loglevel" "info" + "--production" + ;; Skip scripts to prevent building bindings via GYP. + "--ignore-scripts" + "install" "../package.tgz")) + +(define %standard-phases + (modify-phases node:%standard-phases + (add-before 'patch-dependencies 'adjust-dependencies adjust-dependenci= es) + (replace 'build build) + (replace 'check check) + (replace 'install install) + (add-after 'install 'install-js install-js))) + +(define* (tree-sitter-build #:key inputs (phases %standard-phases) + #:allow-other-keys #:rest args) + (apply node:node-build #:inputs inputs #:phases phases args)) --=20 2.34.0 --=-=-=--