From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id qEoYDsm9YmDmywAAgWs5BA (envelope-from ) for ; Tue, 30 Mar 2021 07:57:29 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id +HDuB8m9YmDEUgAA1q6Kng (envelope-from ) for ; Tue, 30 Mar 2021 05:57:29 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 939AA1FB5C for ; Tue, 30 Mar 2021 07:57:28 +0200 (CEST) Received: from localhost ([::1]:52070 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lR6wD-0000iO-E6 for larch@yhetil.org; Tue, 30 Mar 2021 01:29:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39690) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lR6w2-0000hw-LX for guix-patches@gnu.org; Tue, 30 Mar 2021 01:29:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:38530) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lR6w2-0000YC-ET for guix-patches@gnu.org; Tue, 30 Mar 2021 01:29:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lR6w2-0005jC-BN for guix-patches@gnu.org; Tue, 30 Mar 2021 01:29:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#47282] [PATCH v2 01/13] build-system: Rewrite node build system. Resent-From: Timothy Sample Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 30 Mar 2021 05:29:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47282 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47282@debbugs.gnu.org Cc: Timothy Sample , Jelle Licht Received: via spool by 47282-submit@debbugs.gnu.org id=B47282.161708209421823 (code B ref 47282); Tue, 30 Mar 2021 05:29:02 +0000 Received: (at 47282) by debbugs.gnu.org; 30 Mar 2021 05:28:14 +0000 Received: from localhost ([127.0.0.1]:50034 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lR6vB-0005fq-UJ for submit@debbugs.gnu.org; Tue, 30 Mar 2021 01:28:14 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:59861) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lR6v9-0005fP-4Q for 47282@debbugs.gnu.org; Tue, 30 Mar 2021 01:28:08 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 005605C01EF; Tue, 30 Mar 2021 01:28:02 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Tue, 30 Mar 2021 01:28:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=mex3pCsBrrb2bjbmrcXoAQSr0v4Cfd61jE7DjHPiP dE=; b=YYhC/zD3RHzLT7O8CTPEWJJ/nSPyYm+X4P+sCn98VqJdZCHsVdKjqqMch NPwGu+cOePBx5YXNoLWuEXA8AnMSe6Qnxgs608j/1xcu+XN9e5PZtMNFghrPGrBg 8TuMlWwDIEAE6d5V70e3jG6Im12LC7YCXquxluzCG+qtGwdJFPKJMTuPKjFXuA9C txMyPB6FqAjwo5grupAhv/RY3v8JV6eNU8LtYAPsMC27Bu7DM1neuRPKbx2/bUx1 7f9vj/ES2N3N3LGMJNY+5O8ZOa4ufYxfc3NsQBbriTIGF8KN02P4wBwCL4WwUee9 TxZxrw6mNQsFvxpR/Jh2vsowve3FA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrudehledgleekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkffojghfgggtgfesthekredtredtjeenucfhrhhomhepvfhimhho thhhhicuufgrmhhplhgvuceoshgrmhhplhgvthesnhhghihrohdrtghomheqnecuggftrf grthhtvghrnhepieeiheegtdfhhfetgfffjeegvdekvdevteefueeffeeggefhvdevteeu gfdviefgnecuffhomhgrihhnpehgihhthhhusgdrtghomhdpghhnuhdrohhrghdpnhhpmh hjshdrohhrghenucfkphepjeegrdduudeirddukeeirdeggeenucevlhhushhtvghrufhi iigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsrghmphhlvghtsehnghihrhhord gtohhm X-ME-Proxy: Received: from mrblack.lan (74-116-186-44.qc.dsl.ebox.net [74.116.186.44]) by mail.messagingengine.com (Postfix) with ESMTPA id 4CAA624005B; Tue, 30 Mar 2021 01:28:01 -0400 (EDT) From: Timothy Sample Date: Tue, 30 Mar 2021 01:27:31 -0400 Message-Id: <20210330052743.575-1-samplet@ngyro.com> X-Mailer: git-send-email 2.31.0 In-Reply-To: <87k0pprz6n.fsf@ngyro.com> References: <87k0pprz6n.fsf@ngyro.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1617083848; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=mex3pCsBrrb2bjbmrcXoAQSr0v4Cfd61jE7DjHPiPdE=; b=s3AUEFpKLpfJAMXbNBPXZhe1RxDGwhVxqN5AfcLcyqUMd9c1plP47D4Iog8D6atky0F3KP m9497M6Ks+A3gFe+PAMP/Gg4kSttkYIE6MBoShMJ/HR2dasHTkI3heCoY2X2j6CeDVSY+m 9nrv467Pez5ppkqMbKyTk3vjqhOz2wM8feH7DUlvfYhdOsCySayk3KGm63XVC3zYfTVT/+ GC88JlCIRE3ETh3beVuWv3dzMihgFDXuJBHPo0WhtHzUwJNtac6+MXNRFapgBLaXS+fPhc +JGtwfWoIQeI/9kb1B6ketKQUuaKOv7e8Gxefw4bjRtM29d1p4ot78S51WaHRA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1617083848; a=rsa-sha256; cv=none; b=awBnGsesk5Kkr7UhvkHM88nxW51/oky4HN7Dt53mRg+H+PKw/larQdc97YeTGunzR9rnsj CVSkupW9ew+ojykTHpCHrWGxeB2SjW4EJjXKVOgY6UmcwwkSKOONHDAiCEC90REHGJ/5z1 Rut9cGXvgGbPaLXFy6zDdRR+b/+TQNWkh/D6f1Ut5Gfij2lub/A7kz44cXqy6/ScLxedtx T2uKoK2KDifiLM9rmCJNvv3E8ZaKznWILDT2y/7aMinRd3rnM5i8m1sT7uJDj2wldApd4N yIGoSPeEi6zdKX60OlV7rC6H312bnyPZ/cDTgkXiga+Hk0B/fl1BD1IqwG6vaA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b="YYhC/zD3"; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: 1.08 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b="YYhC/zD3"; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 939AA1FB5C X-Spam-Score: 1.08 X-Migadu-Scanner: scn0.migadu.com X-TUID: VYrCDe1lk+le From: Jelle Licht * guix/build/node-build-system.scm: Rewrite it. * guix/build-system/node.scm: Adjust accordingly. * gnu/packages/node-xyz.scm (node-semver): Likewise. Co-authored-by: Timothy Sample --- gnu/packages/node-xyz.scm | 6 +- guix/build-system/node.scm | 27 ++-- guix/build/node-build-system.scm | 207 +++++++++++++++---------------- 3 files changed, 110 insertions(+), 130 deletions(-) diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm index b1d6d4ce59..60cc005ea4 100644 --- a/gnu/packages/node-xyz.scm +++ b/gnu/packages/node-xyz.scm @@ -261,7 +261,11 @@ function with browser support.") "06biknqb05r9xsmcflm3ygh50pjvdk84x6r79w43kmck4fn3qn5p")))) (build-system node-build-system) (arguments - `(#:tests? #f)) ;; FIXME: Tests depend on node-tap + '(#:tests? #f ; FIXME: Tests depend on node-tap + #:phases + (modify-phases %standard-phases + ;; The only dependency to check for is tap, which we don't have. + (delete 'configure)))) (home-page "https://github.com/npm/node-semver") (synopsis "Parses semantic versions strings") (description diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm index a8c5eed09b..4991ed53a5 100644 --- a/guix/build-system/node.scm +++ b/guix/build-system/node.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2016 Jelle Licht +;;; Copyright © 2019 Timothy Sample ;;; ;;; This file is part of GNU Guix. ;;; @@ -17,7 +18,6 @@ ;;; along with GNU Guix. If not, see . (define-module (guix build-system node) - #:use-module (guix store) #:use-module (guix utils) #:use-module (guix packages) #:use-module (guix derivations) @@ -25,22 +25,15 @@ #:use-module (guix build-system) #:use-module (guix build-system gnu) #:use-module (ice-9 match) - #:export (npm-meta-uri - %node-build-system-modules + #:export (%node-build-system-modules node-build node-build-system)) -(define (npm-meta-uri name) - "Return a URI string for the metadata of node module NAME found in the npm -registry." - (string-append "https://registry.npmjs.org/" name)) - (define %node-build-system-modules ;; Build-side modules imported by default. `((guix build node-build-system) (guix build json) - (guix build union) - ,@%gnu-build-system-modules)) ;; TODO: Might be not needed + ,@%gnu-build-system-modules)) (define (default-node) "Return the default Node package." @@ -76,7 +69,7 @@ registry." (define* (node-build store name inputs #:key - (npm-flags ''()) + (test-target "test") (tests? #t) (phases '(@ (guix build node-build-system) %standard-phases)) @@ -86,8 +79,6 @@ registry." (guile #f) (imported-modules %node-build-system-modules) (modules '((guix build node-build-system) - (guix build json) - (guix build union) (guix build utils)))) "Build SOURCE using NODE and INPUTS." (define builder @@ -97,12 +88,10 @@ registry." #:source ,(match (assoc-ref inputs "source") (((? derivation? source)) (derivation->output-path source)) - ((source) - source) - (source - source)) + ((source) source) + (source source)) #:system ,system - #:npm-flags ,npm-flags + #:test-target ,test-target #:tests? ,tests? #:phases ,phases #:outputs %outputs @@ -129,5 +118,5 @@ registry." (define node-build-system (build-system (name 'node) - (description "The standard Node build system") + (description "The Node build system") (lower lower))) diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm index 7799f03595..a55cab237c 100644 --- a/guix/build/node-build-system.scm +++ b/guix/build/node-build-system.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2015 David Thompson -;;; Copyright © 2016 Jelle Licht +;;; Copyright © 2016, 2020 Jelle Licht +;;; Copyright © 2019, 2021 Timothy Sample ;;; ;;; This file is part of GNU Guix. ;;; @@ -19,144 +20,130 @@ (define-module (guix build node-build-system) #:use-module ((guix build gnu-build-system) #:prefix gnu:) - #:use-module (guix build json) - #:use-module (guix build union) #:use-module (guix build utils) + #:use-module (guix build json) + #:use-module (ice-9 ftw) #:use-module (ice-9 match) - #:use-module (ice-9 popen) - #:use-module (ice-9 regex) #:use-module (srfi srfi-1) - #:use-module (srfi srfi-26) #:export (%standard-phases node-build)) ;; Commentary: ;; -;; Builder-side code of the standard Node/npm package build procedure. +;; Builder-side code of the standard Node/NPM package install procedure. ;; ;; Code: -(define* (read-package-data #:key (filename "package.json")) - (call-with-input-file filename - (lambda (port) - (read-json port)))) +(define (set-home . _) + (with-directory-excursion ".." + (let loop ((i 0)) + (let ((dir (string-append "npm-home-" (number->string i)))) + (if (directory-exists? dir) + (loop (1+ i)) + (begin + (mkdir dir) + (setenv "HOME" (string-append (getcwd) "/" dir)) + (format #t "set HOME to ~s~%" (getenv "HOME"))))))) + #t) -(define* (build #:key inputs #:allow-other-keys) - (define (build-from-package-json? package-file) - (let* ((package-data (read-package-data #:filename package-file)) - (scripts (assoc-ref package-data "scripts"))) - (assoc-ref scripts "build"))) - "Build a new node module using the appropriate build system." - ;; XXX: Develop a more robust heuristic, allow override - (cond ((file-exists? "gulpfile.js") - (invoke "gulp")) - ((file-exists? "gruntfile.js") - (invoke "grunt")) - ((file-exists? "Makefile") - (invoke "make")) - ((and (file-exists? "package.json") - (build-from-package-json? "package.json")) - (invoke "npm" "run" "build"))) +(define (module-name module) + (let* ((package.json (string-append module "/package.json")) + (package-meta (call-with-input-file package.json read-json))) + (assoc-ref package-meta "name"))) + +(define (index-modules input-paths) + (define (list-modules directory) + (append-map (lambda (x) + (if (string-prefix? "@" x) + (list-modules (string-append directory "/" x)) + (list (string-append directory "/" x)))) + (filter (lambda (x) + (not (member x '("." "..")))) + (or (scandir directory) '())))) + (let ((index (make-hash-table (* 2 (length input-paths))))) + (for-each (lambda (dir) + (let ((nm (string-append dir "/lib/node_modules"))) + (for-each (lambda (module) + (hash-set! index (module-name module) module)) + (list-modules nm)))) + input-paths) + index)) + +(define* (patch-dependencies #:key inputs #:allow-other-keys) + + (define index (index-modules (map cdr inputs))) + + (define (resolve-dependencies package-meta meta-key) + (fold (lambda (key+value acc) + (match key+value + ('@ acc) + ((key . value) (acons key (hash-ref index key value) acc)))) + '() + (or (assoc-ref package-meta meta-key) '()))) + + (with-atomic-file-replacement "package.json" + (lambda (in out) + (let ((package-meta (read-json in))) + (assoc-set! package-meta "dependencies" + (append + '(@) + (resolve-dependencies package-meta "dependencies") + (resolve-dependencies package-meta "peerDependencies"))) + (assoc-set! package-meta "devDependencies" + (append + '(@) + (resolve-dependencies package-meta "devDependencies"))) + (write-json package-meta out)))) #t) -(define* (link-npm-dependencies #:key inputs #:allow-other-keys) - (define (inputs->node-inputs inputs) - "Filter the directory part from INPUTS." - (filter (lambda (input) - (match input - ((name . _) (node-package? name)))) - inputs)) - (define (inputs->directories inputs) - "Extract the directory part from INPUTS." - (match inputs - (((names . directories) ...) - directories))) - (define (make-node-path root) - (string-append root "/lib/node_modules/")) - - (let ((input-node-directories (inputs->directories - (inputs->node-inputs inputs)))) - (union-build "node_modules" - (map make-node-path input-node-directories)) +(define* (configure #:key outputs inputs #:allow-other-keys) + (let ((npm (string-append (assoc-ref inputs "node") "/bin/npm"))) + (invoke npm "--offline" "--ignore-scripts" "install") #t)) -(define configure link-npm-dependencies) +(define* (build #:key inputs #:allow-other-keys) + (let ((package-meta (call-with-input-file "package.json" read-json))) + (if (and=> (assoc-ref package-meta "scripts") + (lambda (scripts) + (assoc-ref scripts "build"))) + (let ((npm (string-append (assoc-ref inputs "node") "/bin/npm"))) + (invoke npm "run" "build")) + (format #t "there is no build script to run~%")) + #t)) -(define* (check #:key tests? #:allow-other-keys) +(define* (check #:key tests? inputs #:allow-other-keys) "Run 'npm test' if TESTS?" (if tests? - ;; Should only be enabled once we know that there are tests - (invoke "npm" "test")) + (let ((npm (string-append (assoc-ref inputs "node") "/bin/npm"))) + (invoke npm "test")) + (format #t "test suite not run~%")) #t) -(define (node-package? name) - "Check if NAME correspond to the name of an Node package." - (string-prefix? "node-" name)) +(define* (repack #:key inputs #:allow-other-keys) + (invoke "tar" "-czf" "../package.tgz" ".") + #t) (define* (install #:key outputs inputs #:allow-other-keys) - "Install the node module to the output store item. The module itself is -installed in a subdirectory of @file{node_modules} and its runtime dependencies -as defined by @file{package.json} are symlinked into a @file{node_modules} -subdirectory of the module's directory. Additionally, binaries are installed in -the @file{bin} directory." - (let* ((out (assoc-ref outputs "out")) - (target (string-append out "/lib")) - (binaries (string-append out "/bin")) - (data (read-package-data)) - (modulename (assoc-ref data "name")) - (binary-configuration (match (assoc-ref data "bin") - (('@ configuration ...) configuration) - ((? string? configuration) configuration) - (#f #f))) - (dependencies (match (assoc-ref data "dependencies") - (('@ deps ...) deps) - (#f #f)))) - (mkdir-p target) - (copy-recursively "." (string-append target "/node_modules/" modulename)) - ;; Remove references to dependencies - (delete-file-recursively - (string-append target "/node_modules/" modulename "/node_modules")) - (cond - ((string? binary-configuration) - (begin - (mkdir-p binaries) - (symlink (string-append target "/node_modules/" modulename "/" - binary-configuration) - (string-append binaries "/" modulename)))) - ((list? binary-configuration) - (for-each - (lambda (conf) - (match conf - ((key . value) - (begin - (mkdir-p (dirname (string-append binaries "/" key))) - (symlink (string-append target "/node_modules/" modulename "/" - value) - (string-append binaries "/" key)))))) - binary-configuration))) - (when dependencies - (mkdir-p - (string-append target "/node_modules/" modulename "/node_modules")) - (for-each - (lambda (dependency) - (let ((dependency (car dependency))) - (symlink - (string-append (assoc-ref inputs (string-append "node-" dependency)) - "/lib/node_modules/" dependency) - (string-append target "/node_modules/" modulename - "/node_modules/" dependency)))) - dependencies)) + "Install the node module to the output store item." + (let ((out (assoc-ref outputs "out")) + (npm (string-append (assoc-ref inputs "node") "/bin/npm"))) + (invoke npm "--prefix" out + "--global" + "--offline" + "--loglevel" "info" + "--production" + "install" "../package.tgz") #t)) - (define %standard-phases (modify-phases gnu:%standard-phases + (add-after 'unpack 'set-home set-home) + (add-before 'configure 'patch-dependencies patch-dependencies) (replace 'configure configure) (replace 'build build) - (replace 'install install) - (delete 'check) - (add-after 'install 'check check) - (delete 'strip))) + (replace 'check check) + (add-before 'install 'repack repack) + (replace 'install install))) (define* (node-build #:key inputs (phases %standard-phases) #:allow-other-keys #:rest args) -- 2.31.0