unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / Atom feed
* [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp.
@ 2021-11-14 12:41 Philip McGrath
  2021-11-14 12:58 ` [bug#51838] [PATCH 01/11] gnu: node: Avoid duplicating build phases Philip McGrath
                   ` (4 more replies)
  0 siblings, 5 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 12:41 UTC (permalink / raw)
  To: 51838; +Cc: Philip McGrath, Pierre Langlois

This patch series adds support to `node-build-system` for building native
add-ons using `node-gyp`. To confirm that the changes to the build system
work, the patch series also adds two packages (plus a few dependencies)
representing two major ways of writing Node.js native add-ons:

 1. `node-openzwave-shared` uses `node-nan`: “Native Abstractions for Node.js”.

 2. `node-sqlite3` uses the Node-API via `node-addon-api`.

The patches are adapted from part of a series by Pierre Langlois
at <https://issues.guix.gnu.org/49946>: you can find discussion of an earlier
version of this spun-off series from <https://issues.guix.gnu.org/49946#71>
to <https://issues.guix.gnu.org/49946#83>.
They are also available in a Git repository
at <https://gitlab.com/philip1/guix-patches/-/tree/wip-node-npm-gyp-hist-9>.

In the earlier discussion, Pierre and I both thought it was quite confusing
that the Scheme variable `node` refers to the bootstrap Node.js, while
`node-lts` refers to the newer Node.js that a package specification of
`"node"` would resolve to. My analysis is
at <https://issues.guix.gnu.org/49946#75> (tl;dr it seems like an unintentional
mistake), and Pierre suggested a solution
in <https://issues.guix.gnu.org/49946#81>. I haven't taken that on in this patch
series, but I do think its worth considering.

Philip McGrath (11):
  gnu: node: Avoid duplicating build phases.
  gnu: node: Update to 10.24.1 for bootstrapping.
  guix: node-build-system: Support compiling add-ons with node-gyp.
  gnu: Add node-inherits.
  gnu: Add node-safe-buffer.
  gnu: Add node-string-decoder.
  gnu: Add node-readable-stream.
  gnu: Add node-nan.
  gnu: Add node-openzwave-shared.
  gnu: Add node-addon-api.
  gnu: Add node-sqlite3.

 gnu/packages/node-xyz.scm        | 335 +++++++++++++++++++++++++++++++
 gnu/packages/node.scm            | 143 +++++--------
 gnu/packages/zwave.scm           |  55 +++++
 guix/build-system/node.scm       |  11 +-
 guix/build/node-build-system.scm |  23 +++
 5 files changed, 473 insertions(+), 94 deletions(-)

-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 01/11] gnu: node: Avoid duplicating build phases.
  2021-11-14 12:41 [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
@ 2021-11-14 12:58 ` Philip McGrath
  2021-11-14 13:04   ` [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
  2021-11-20 17:38 ` [bug#51838] [PATCH v2 04/26] gnu: node: Add an npmrc file to set nodedir Timothy Sample
                   ` (3 subsequent siblings)
  4 siblings, 1 reply; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 12:58 UTC (permalink / raw)
  To: 51838; +Cc: Philip McGrath

* gnu/packages/node.scm (node)[arguments]: Split 'patch-files phase
into 'patch-hardcoded-program-references and
'delete-problematic-tests. Adapt 'patch-hardcoded-program-references
and 'configure to work unmodified on node-lts, but don't try to
share 'delete-problematic-tests, because those details seem to change
too much between node versions.
(node, node-lts)[inputs]: Use bash-minimal rather than bash.
(node-lts)[arguments]: Inherit 'patch-hardcoded-program-references,
and 'configure phases from the bootstrap node.
Remove the 'patch-files phase, keeping its remaining
non-inherited work in a new 'replace-llhttp-sources phase.
---
 gnu/packages/node.scm | 105 +++++++++++++-----------------------------
 1 file changed, 32 insertions(+), 73 deletions(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index d0ffe8a398..db1658f03d 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -9,6 +9,7 @@
 ;;; Copyright © 2020, 2021 Pierre Langlois <pierre.langlois@gmx.com>
 ;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -105,14 +106,22 @@ (define-public node
        #:test-target "test-ci-js"
        #:phases
        (modify-phases %standard-phases
-         (add-before 'configure 'patch-files
+         (add-before 'configure 'patch-hardcoded-program-references
            (lambda* (#:key inputs #:allow-other-keys)
+
              ;; Fix hardcoded /bin/sh references.
-             (substitute* '("lib/child_process.js"
-                            "lib/internal/v8_prof_polyfill.js"
-                            "test/parallel/test-child-process-spawnsync-shell.js"
-                            "test/parallel/test-stdio-closed.js"
-                            "test/sequential/test-child-process-emfile.js")
+             (substitute*
+                 (let ((common
+                        '("lib/child_process.js"
+                          "lib/internal/v8_prof_polyfill.js"
+                          "test/parallel/test-child-process-spawnsync-shell.js"
+                          "test/parallel/test-stdio-closed.js"
+                          "test/sequential/test-child-process-emfile.js"))
+                       ;; not in bootstap node:
+                       (sigxfsz "test/parallel/test-fs-write-sigxfsz.js"))
+                   (if (file-exists? sigxfsz)
+                       (cons sigxfsz common)
+                       common))
                (("'/bin/sh'")
                 (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
 
@@ -122,7 +131,10 @@ (define-public node
                             "test/parallel/test-child-process-exec-env.js")
                (("'/usr/bin/env'")
                 (string-append "'" (assoc-ref inputs "coreutils")
-                               "/bin/env'")))
+                               "/bin/env'")))))
+         (add-after 'patch-hardcoded-program-references
+             'delete-problematic-tests
+           (lambda* (#:key inputs #:allow-other-keys)
 
              ;; FIXME: These tests fail in the build container, but they don't
              ;; seem to be indicative of real problems in practice.
@@ -217,9 +229,13 @@ (define-public node
                (setenv "CXX" ,(cxx-for-target))
                (setenv "PKG_CONFIG" ,(pkg-config-for-target))
                (apply invoke
-                      (string-append (assoc-ref (or native-inputs inputs)
-                                                "python")
-                                     "/bin/python")
+                      (let ((python
+                             (string-append (assoc-ref (or native-inputs inputs)
+                                                       "python")
+                                            "/bin/python")))
+                        (if (file-exists? python)
+                            python
+                            (string-append python "3")))
                       "configure" flags))))
          (add-after 'patch-shebangs 'patch-npm-shebang
            (lambda* (#:key outputs #:allow-other-keys)
@@ -256,7 +272,7 @@ (define-public node
             (variable "NODE_PATH")
             (files '("lib/node_modules")))))
     (inputs
-     `(("bash" ,bash)
+     `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
        ("c-ares" ,c-ares)
        ("http-parser" ,http-parser)
@@ -711,66 +727,8 @@ (define-public node-lts
                                    libuv "/lib:"
                                    zlib "/lib"
                                    "'],"))))))
-           (replace 'configure
-             ;; Node's configure script is actually a python script, so we can't
-             ;; run it with bash.
-             (lambda* (#:key outputs (configure-flags '()) native-inputs inputs
-                       #:allow-other-keys)
-               (let* ((prefix (assoc-ref outputs "out"))
-                      (xflags ,(if (%current-target-system)
-                                   `'("--cross-compiling"
-                                     ,(string-append
-                                       "--dest-cpu="
-                                       (match (%current-target-system)
-                                         ((? (cut string-prefix? "arm" <>))
-                                          "arm")
-                                         ((? (cut string-prefix? "aarch64" <>))
-                                          "arm64")
-                                         ((? (cut string-prefix? "i686" <>))
-                                          "ia32")
-                                         ((? (cut string-prefix? "x86_64" <>))
-                                          "x64")
-                                         ((? (cut string-prefix? "powerpc64" <>))
-                                          "ppc64")
-                                         (_ "unsupported"))))
-                                   ''()))
-                      (flags (cons
-                               (string-append "--prefix=" prefix)
-                               (append xflags configure-flags))))
-                 (format #t "build directory: ~s~%" (getcwd))
-                 (format #t "configure flags: ~s~%" flags)
-                 ;; Node's configure script expects the CC environment variable to
-                 ;; be set.
-                 (setenv "CC_host" "gcc")
-                 (setenv "CXX_host" "g++")
-                 (setenv "CC" ,(cc-for-target))
-                 (setenv "CXX" ,(cxx-for-target))
-                 (setenv "PKG_CONFIG" ,(pkg-config-for-target))
-                 (apply invoke
-                        (string-append (assoc-ref (or native-inputs inputs)
-                                                  "python")
-                                       "/bin/python3")
-                        "configure" flags))))
-           (replace 'patch-files
+           (replace 'delete-problematic-tests
              (lambda* (#:key inputs #:allow-other-keys)
-               ;; Fix hardcoded /bin/sh references.
-               (substitute* '("lib/child_process.js"
-                              "lib/internal/v8_prof_polyfill.js"
-                              "test/parallel/test-child-process-spawnsync-shell.js"
-                              "test/parallel/test-fs-write-sigxfsz.js"
-                              "test/parallel/test-stdio-closed.js"
-                              "test/sequential/test-child-process-emfile.js")
-                 (("'/bin/sh'")
-                  (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
-
-               ;; Fix hardcoded /usr/bin/env references.
-               (substitute* '("test/parallel/test-child-process-default-options.js"
-                              "test/parallel/test-child-process-env.js"
-                              "test/parallel/test-child-process-exec-env.js")
-                 (("'/usr/bin/env'")
-                  (string-append "'" (assoc-ref inputs "coreutils")
-                                 "/bin/env'")))
-
                ;; FIXME: These tests fail in the build container, but they don't
                ;; seem to be indicative of real problems in practice.
                (for-each delete-file
@@ -806,8 +764,9 @@ (define-public node-lts
                ;; TODO: Regenerate certs instead.
                (for-each delete-file
                          '("test/parallel/test-tls-passphrase.js"
-                           "test/parallel/test-tls-server-verify.js"))
-
+                           "test/parallel/test-tls-server-verify.js"))))
+           (add-after 'delete-problematic-tests 'replace-llhttp-sources
+             (lambda* (#:key inputs #:allow-other-keys)
                ;; Replace pre-generated llhttp sources
                (let ((llhttp (assoc-ref inputs "llhttp")))
                  (copy-file (string-append llhttp "/src/llhttp.c")
@@ -834,7 +793,7 @@ (define-public node-lts
        ("python" ,python)
        ("util-linux" ,util-linux)))
     (inputs
-     `(("bash" ,bash)
+     `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
        ("c-ares" ,c-ares)
        ("icu4c" ,icu4c-67)
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping.
  2021-11-14 12:58 ` [bug#51838] [PATCH 01/11] gnu: node: Avoid duplicating build phases Philip McGrath
@ 2021-11-14 13:04   ` Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
                       ` (8 more replies)
  0 siblings, 9 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 13:04 UTC (permalink / raw)
  To: 51838; +Cc: Philip McGrath

* gnu/packages/node.scm (node): Update to 10.24.1.
---
 gnu/packages/node.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index db1658f03d..89a9bc7533 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -57,14 +57,14 @@ (define-module (gnu packages node)
 (define-public node
   (package
     (name "node")
-    (version "10.24.0")
+    (version "10.24.1")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://nodejs.org/dist/v" version
                                   "/node-v" version ".tar.xz"))
               (sha256
                (base32
-                "1k1srdis23782hnd1ymgczs78x9gqhv77v0am7yb54gqcspp70hm"))
+                "032801kg24j04xmf09m0vxzlcz86sv21s24lv9l4cfv08k1c4byp"))
               (modules '((guix build utils)))
               (snippet
                `(begin
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-14 13:04   ` [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
@ 2021-11-14 13:04     ` Philip McGrath
  2021-11-14 20:44       ` Liliana Marie Prikler
  2021-11-14 13:04     ` [bug#51838] [PATCH 04/11] gnu: Add node-inherits Philip McGrath
                       ` (7 subsequent siblings)
  8 siblings, 1 reply; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 13:04 UTC (permalink / raw)
  To: 51838; +Cc: Pierre Langlois, Philip McGrath

* gnu/packages/node.scm (node)[arguments]: Replace 'patch-npm-shebang
and 'patch-node-shebang with a new 'patch-nested-shebangs that also
handles node-gyp and other shebangs under "/lib/node_modules".
[inputs]: Add Python for node-gyp as "python-for-target".
(node-lts)[inputs]: Likewise.
(libnode)[arguments]: Adjust to delete 'patch-nested-shebangs rather
than 'patch-npm-shebang and 'patch-node-shebang.
* guix/build-system/node.scm (lower): Add optional #:python argument
and corresponding implicit input. Add the version of libuv used
as an input to the #:node package as a new implicit input.
* guix/build/node-build-system.scm (set-node-gyp-paths): New
function. Sets the "npm_config_nodedir" and "npm_config_python"
environment variables. Adds the "node-gyp-bin" directory to "PATH".
(configure-gyp): New function. Run `node-gyp configure` if we see
a `binding.gyp` file.
(%standard-phases): Add 'set-node-gyp-paths after 'set-paths.
Add 'configure-gyp after 'configure.

Co-authored-by: Pierre Langlois <pierre.langlois@gmx.com>
---
 gnu/packages/node.scm            | 34 ++++++++++++++++----------------
 guix/build-system/node.scm       | 11 +++++++++--
 guix/build/node-build-system.scm | 23 +++++++++++++++++++++
 3 files changed, 49 insertions(+), 19 deletions(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 89a9bc7533..ad5179426a 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -237,21 +237,20 @@ (define-public node
                             python
                             (string-append python "3")))
                       "configure" flags))))
-         (add-after 'patch-shebangs 'patch-npm-shebang
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((bindir (string-append (assoc-ref outputs "out")
-                                           "/bin"))
-                    (npm    (string-append bindir "/npm"))
-                    (target (readlink npm)))
-               (with-directory-excursion bindir
-                 (patch-shebang target (list bindir))))))
-         (add-after 'install 'patch-node-shebang
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((bindir (string-append (assoc-ref outputs "out")
-                                           "/bin"))
-                    (npx    (readlink (string-append bindir "/npx"))))
-               (with-directory-excursion bindir
-                 (patch-shebang npx (list bindir)))))))))
+         (add-after 'patch-shebangs 'patch-nested-shebangs
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let* ((prefix (assoc-ref outputs "out"))
+                    (path (map (lambda (dir)
+                                 (string-append dir "/bin"))
+                               (list prefix
+                                     (assoc-ref inputs "python-for-target")))))
+               (for-each
+                (lambda (file)
+                  (patch-shebang file path))
+                (find-files (string-append prefix "/lib/node_modules")
+                            (lambda (file stat)
+                              (executable-file? file))
+                            #:stat lstat))))))))
     (native-inputs
      `(;; Runtime dependencies for binaries used as a bootstrap.
        ("c-ares" ,c-ares)
@@ -274,6 +273,7 @@ (define-public node
     (inputs
      `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
+       ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
        ("c-ares" ,c-ares)
        ("http-parser" ,http-parser)
        ("icu4c" ,icu4c)
@@ -795,6 +795,7 @@ (define-public node-lts
     (inputs
      `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
+       ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
        ("c-ares" ,c-ares)
        ("icu4c" ,icu4c-67)
        ("libuv" ,libuv-for-node)
@@ -813,5 +814,4 @@ (define-public libnode
         `(cons* "--shared" "--without-npm" ,flags))
        ((#:phases phases '%standard-phases)
         `(modify-phases ,phases
-           (delete 'patch-npm-shebang)
-           (delete 'patch-node-shebang)))))))
+           (delete 'patch-nested-shebangs)))))))
diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 98f63f87ef..fee4142a99 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -1,6 +1,8 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
 ;;; Copyright © 2019 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -24,6 +26,7 @@ (define-module (guix build-system node)
   #:use-module (guix search-paths)
   #:use-module (guix build-system)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system python)
   #:use-module (ice-9 match)
   #:export (%node-build-system-modules
             node-build
@@ -44,11 +47,12 @@ (define (default-node)
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
                 (node (default-node))
+                (python (default-python)) ;; for node-gyp
                 #:allow-other-keys
                 #:rest arguments)
   "Return a bag for NAME."
   (define private-keywords
-    '(#:source #:target #:node #:inputs #:native-inputs))
+    '(#:source #:target #:node #:python #:inputs #:native-inputs))
 
   (and (not target)                    ;XXX: no cross-compilation
        (bag
@@ -58,10 +62,13 @@ (define private-keywords
                               `(("source" ,source))
                               '())
                         ,@inputs
-
                         ;; Keep the standard inputs of 'gnu-build-system'.
                         ,@(standard-packages)))
          (build-inputs `(("node" ,node)
+                         ("python" ,python)
+                        ;; We don't always need libuv, but the libuv and
+                        ;; node versions need to match:
+                        ("libuv" ,@(assoc-ref (package-inputs node) "libuv"))
                          ,@native-inputs))
          (outputs outputs)
          (build node-build)
diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 70a367618e..6aeb0149dd 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -2,6 +2,8 @@
 ;;; Copyright © 2015 David Thompson <davet@gnu.org>
 ;;; Copyright © 2016, 2020 Jelle Licht <jlicht@fsfe.org>
 ;;; Copyright © 2019, 2021 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -46,6 +48,19 @@ (define (set-home . _)
               (format #t "set HOME to ~s~%" (getenv "HOME")))))))
   #t)
 
+(define* (set-node-gyp-paths #:key inputs #:allow-other-keys)
+  "Initialize environment variables needed for building native addons."
+  (setenv "npm_config_nodedir" (assoc-ref inputs "node"))
+  (setenv "npm_config_python" (assoc-ref inputs "python"))
+  (setenv "PATH"
+          (string-append (getenv "PATH")
+                         ":"
+                         ;; Put this at the end to make it easier to override,
+                         ;; just in case that should ever be necessary:
+                         (assoc-ref inputs "node")
+                         "/lib/node_modules/npm/bin/node-gyp-bin"))
+  #t)
+
 (define (module-name module)
   (let* ((package.json (string-append module "/package.json"))
          (package-meta (call-with-input-file package.json read-json)))
@@ -101,6 +116,12 @@ (define* (configure #:key outputs inputs #:allow-other-keys)
     (invoke npm "--offline" "--ignore-scripts" "install")
     #t))
 
+(define* (configure-gyp #:key inputs #:allow-other-keys)
+  "Run 'node-gyp configure' if we see a 'binding.gyp' file."
+  (if (file-exists? "binding.gyp")
+      (invoke (which "node-gyp") "configure")
+      #t))
+
 (define* (build #:key inputs #:allow-other-keys)
   (let ((package-meta (call-with-input-file "package.json" read-json)))
     (if (and=> (assoc-ref package-meta "scripts")
@@ -144,9 +165,11 @@ (define* (install #:key outputs inputs #:allow-other-keys)
 
 (define %standard-phases
   (modify-phases gnu:%standard-phases
+    (add-after 'set-paths 'set-node-gyp-paths set-node-gyp-paths)
     (add-after 'unpack 'set-home set-home)
     (add-before 'configure 'patch-dependencies patch-dependencies)
     (replace 'configure configure)
+    (add-after 'configure 'configure-gyp configure-gyp)
     (replace 'build build)
     (replace 'check check)
     (add-before 'install 'repack repack)
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 04/11] gnu: Add node-inherits.
  2021-11-14 13:04   ` [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
@ 2021-11-14 13:04     ` Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 05/11] gnu: Add node-safe-buffer Philip McGrath
                       ` (6 subsequent siblings)
  8 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 13:04 UTC (permalink / raw)
  To: 51838; +Cc: Philip McGrath

* gnu/packages/node-xyz.scm (node-inherits): New variable.
---
 gnu/packages/node-xyz.scm | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 98c611f227..da6affeb1f 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2020 Giacomo Leidi <goodoldpaul@autistici.org>
 ;;; Copyright © 2021 Noisytoot <noisytoot@disroot.org>
 ;;; Copyright © 2021 Charles <charles.b.jackson@protonmail.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -358,6 +359,37 @@ (define-public node-once
 if desired.")
     (license license:isc)))
 
+(define-public node-inherits
+  (package
+    (name "node-inherits")
+    (version "2.0.4")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/isaacs/inherits")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0cpsr5yqwkxpbbbbl0rwk4mcby6zbx841k2zb4c3gb1579i5wq9p"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f
+       #:phases
+       (modify-phases %standard-phases
+         ;; Ignore our lack of tap (for tests) as with node-once.
+         (delete 'configure))))
+    (home-page
+     "https://github.com/isaacs/inherits")
+    (synopsis
+     "Browser-friendly inheritance Node.js")
+    (description
+     "Browser-friendly inheritance fully compatible with standard Node.js
+@code{inherits()}.")
+    (license license:isc)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 05/11] gnu: Add node-safe-buffer.
  2021-11-14 13:04   ` [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 04/11] gnu: Add node-inherits Philip McGrath
@ 2021-11-14 13:04     ` Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 06/11] gnu: Add node-string-decoder Philip McGrath
                       ` (5 subsequent siblings)
  8 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 13:04 UTC (permalink / raw)
  To: 51838; +Cc: Philip McGrath

* gnu/packages/node-xyz.scm (node-safe-buffer): New variable.
---
 gnu/packages/node-xyz.scm | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index da6affeb1f..44000e0568 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -390,6 +390,36 @@ (define-public node-inherits
 @code{inherits()}.")
     (license license:isc)))
 
+(define-public node-safe-buffer
+  (package
+    (name "node-safe-buffer")
+    (version "5.2.1")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/feross/safe-buffer")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0r26m0nl41h90ihnl2xf0cqs6z9z7jb87dl5j8yqb7887r9jlbpi"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f
+       #:phases
+       (modify-phases %standard-phases
+         ;; Skip checking devDependencies, since we don't have tape.
+         (delete 'configure))))
+    (home-page
+     "https://github.com/feross/safe-buffer")
+    (synopsis "Safer Node.js Buffer API")
+    (description "A safe drop-in replacement the Node.js @code{Buffer} API
+that works in all versions of Node.js, using the built-in implementation when
+available.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 06/11] gnu: Add node-string-decoder.
  2021-11-14 13:04   ` [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
                       ` (2 preceding siblings ...)
  2021-11-14 13:04     ` [bug#51838] [PATCH 05/11] gnu: Add node-safe-buffer Philip McGrath
@ 2021-11-14 13:04     ` Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 07/11] gnu: Add node-readable-stream Philip McGrath
                       ` (4 subsequent siblings)
  8 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 13:04 UTC (permalink / raw)
  To: 51838; +Cc: Philip McGrath

* gnu/packages/node-xyz.scm (node-string-decoder): New variable.
---
 gnu/packages/node-xyz.scm | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 44000e0568..a5ca5631c7 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -420,6 +420,40 @@ (define-public node-safe-buffer
 available.")
     (license license:expat)))
 
+(define-public node-string-decoder
+  (package
+    (name "node-string-decoder")
+    (version "1.3.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/string_decoder")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0xxvyya9fl9rlkqwmxzqzbz4rdr3jgw4vf37hff7cgscxkhg266k"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f
+       #:phases
+       (modify-phases %standard-phases
+         ;; Ignore missing tap dependency (for tests).
+         (delete 'configure))))
+    (inputs
+     `(("node-safe-buffer" ,node-safe-buffer)
+       ("node-inherits" ,node-inherits)))
+    (home-page
+     "https://github.com/nodejs/string_decoder")
+    (synopsis
+     "Node.js core @code{string_decoder} for userland")
+    (description
+     "This package is a mirror of the @code{string_decoder} implementation in
+Node-core.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 07/11] gnu: Add node-readable-stream.
  2021-11-14 13:04   ` [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
                       ` (3 preceding siblings ...)
  2021-11-14 13:04     ` [bug#51838] [PATCH 06/11] gnu: Add node-string-decoder Philip McGrath
@ 2021-11-14 13:04     ` Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 08/11] gnu: Add node-nan Philip McGrath
                       ` (3 subsequent siblings)
  8 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 13:04 UTC (permalink / raw)
  To: 51838; +Cc: Philip McGrath

* gnu/packages/node-xyz.scm (node-readable-stream): New variable.
---
 gnu/packages/node-xyz.scm | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index a5ca5631c7..d0b4d1fa98 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -454,6 +454,44 @@ (define-public node-string-decoder
 Node-core.")
     (license license:expat)))
 
+(define-public node-readable-stream
+  (package
+    (name "node-readable-stream")
+    (version "3.6.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/readable-stream")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0ybl4cdgsm9c5jq3xq8s01201jk8w0yakh63hlclsfbcdfqhd9ri"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f
+       #:phases
+       (modify-phases %standard-phases
+         ;; Ignore numerous missing devDependencies
+         (delete 'configure))))
+    (inputs
+     `(("node-util-deprecate" ,node-util-deprecate)
+       ("node-string-decoder" ,node-string-decoder)
+       ("node-inherits" ,node-inherits)))
+    (home-page
+     "https://github.com/nodejs/readable-stream")
+    (synopsis
+     "Node.js core streams for userland")
+    (description
+     "This package is a mirror of the streams implementations in Node.js.
+
+If you want to guarantee a stable streams base, regardless of what version of
+Node you (or the users of your libraries) are using, use
+@code{readable-stream} only and avoid the @code{stream} module in Node-core.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 08/11] gnu: Add node-nan.
  2021-11-14 13:04   ` [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
                       ` (4 preceding siblings ...)
  2021-11-14 13:04     ` [bug#51838] [PATCH 07/11] gnu: Add node-readable-stream Philip McGrath
@ 2021-11-14 13:04     ` Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 09/11] gnu: Add node-openzwave-shared Philip McGrath
                       ` (2 subsequent siblings)
  8 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 13:04 UTC (permalink / raw)
  To: 51838; +Cc: Philip McGrath

* gnu/packages/node-xyz.scm (node-nan): New variable.
---
 gnu/packages/node-xyz.scm | 41 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index d0b4d1fa98..29ad14f810 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -549,3 +549,44 @@ (define-public node-irc
     (description "@code{node-irc} is an IRC client library for Node.js.
 It has functions for joining, parting, talking, and many other IRC commands.")
     (license license:gpl3+)))
+
+(define-public node-nan
+  (package
+    (name "node-nan")
+    (version "2.15.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/nan")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "18xslh9va5ld872scrp5y4251ax9s3c6qh0lnl1200lpzbsxy7yd"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       ;; We can't run the npm-driven tests,
+       ;; but a number of C++ tests do run anyway:
+       #f
+       #:phases
+       (modify-phases %standard-phases
+         ;; Ignore missing devDependencies (e.g. "tap", for tests):
+         (delete 'configure))))
+    (inputs
+     `(("readable-stream" ,node-readable-stream)))
+    (home-page "https://github.com/nodejs/nan")
+    (synopsis "Native Abstractions for Node.js")
+    (description "A header file filled with macro and utility goodness for
+making add-on development for Node.js easier across versions 0.8, 0.10, 0.12,
+1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 and 16.
+
+Thanks to the crazy changes in V8 (and some in Node core), keeping native
+addons compiling happily across versions, particularly 0.10 to 0.12 to 4.0, is
+a minor nightmare.  The goal of this project is to store all logic necessary
+to develop native Node.js addons without having to inspect
+@code{NODE_MODULE_VERSION} and get yourself into a macro-tangle.
+
+This project also contains some helper utilities that make addon development a
+bit more pleasant.")
+    (license license:expat)))
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 09/11] gnu: Add node-openzwave-shared.
  2021-11-14 13:04   ` [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
                       ` (5 preceding siblings ...)
  2021-11-14 13:04     ` [bug#51838] [PATCH 08/11] gnu: Add node-nan Philip McGrath
@ 2021-11-14 13:04     ` Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 10/11] gnu: Add node-addon-api Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 11/11] gnu: Add node-sqlite3 Philip McGrath
  8 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 13:04 UTC (permalink / raw)
  To: 51838; +Cc: Philip McGrath

* gnu/packages/zwave.scm (node-openzwave-shared): New variable.
---
 gnu/packages/zwave.scm | 55 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/gnu/packages/zwave.scm b/gnu/packages/zwave.scm
index 2019ec32df..39028b4f34 100644
--- a/gnu/packages/zwave.scm
+++ b/gnu/packages/zwave.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2019 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -21,8 +22,10 @@ (define-module (gnu packages zwave)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system node)
   #:use-module (gnu packages)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages node-xyz)
   #:use-module (gnu packages libusb)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages perl)
@@ -88,3 +91,55 @@ (define-public open-zwave
 and respond to devices on a Z-Wave network, without requiring in-depth
 knowledge of the Z-Wave protocol.")
     (license license:lgpl3+)))
+
+(define-public node-openzwave-shared
+  (package
+    (name "node-openzwave-shared")
+    (version "1.7.2")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/OpenZWave/node-openzwave-shared")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "1bqqy12dzqj05j9wsh50dmn84dddmhz0gjzvd3y20z4hpy1v8rsc"))))
+    (inputs
+     `(("open-zwave" ,open-zwave)
+       ("node-nan" ,node-nan)))
+    (native-inputs
+     `(("which" ,which)
+       ("pkg-config" ,pkg-config)))
+    (build-system node-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (replace 'build
+           (lambda* (#:key inputs #:allow-other-keys)
+             (invoke (which "node-gyp") "build"))))))
+    (home-page "https://github.com/OpenZWave/node-openzwave-shared")
+    (synopsis "Node.js bindings for OpenZWave")
+    (description
+     "With the @code{node-openzwave-shared} package, you can easily control
+and manage your Z-Wave devices (lights, dimmers, blinds, you name it) from
+within Node.js applications.  This library also supports secure
+devices (e.g. door locks) that require encryption.  All widely used Node.js
+versions are supported with the help of @code{node-nan}.
+
+This library is currently able to:
+@itemize @bullet
+@item
+scan a Z-Wave network and report on connected devices;
+@item
+write values to Z-Wave nodes;
+@item
+monitor the network for changes;
+@item
+heal nodes and/or the network; and
+@item
+perform management tasks: add or remove nodes, replace failed nodes,
+manage their group associations, etc.
+@end itemize")
+    (license license:isc)))
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 10/11] gnu: Add node-addon-api.
  2021-11-14 13:04   ` [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
                       ` (6 preceding siblings ...)
  2021-11-14 13:04     ` [bug#51838] [PATCH 09/11] gnu: Add node-openzwave-shared Philip McGrath
@ 2021-11-14 13:04     ` Philip McGrath
  2021-11-14 13:04     ` [bug#51838] [PATCH 11/11] gnu: Add node-sqlite3 Philip McGrath
  8 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 13:04 UTC (permalink / raw)
  To: 51838; +Cc: Philip McGrath

* gnu/packages/node-xyz.scm (node-addon-api): New variable.
---
 gnu/packages/node-xyz.scm | 46 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 29ad14f810..8aa93122df 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -590,3 +590,49 @@ (define-public node-nan
 This project also contains some helper utilities that make addon development a
 bit more pleasant.")
     (license license:expat)))
+
+(define-public node-addon-api
+  (package
+    (name "node-addon-api")
+    (version "4.2.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/node-addon-api")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "1bhvfi2m9nxfz418s619914vmidcnrzbjv6l9nid476c3zlpazch"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         ;; Ignore numerous missing devDependencies:
+         (delete 'configure)
+         (add-after 'unpack 'skip-js-tests
+           ;; We can't run the js-based tests,
+           ;; but we can still do the C++ parts
+           (lambda args
+             (substitute* "package.json"
+               (("\"test\": \"node test\"")
+                "\"test\": \"echo stopping after pretest on Guix\"")))))))
+    (home-page "https://github.com/nodejs/node-addon-api")
+    (synopsis "Node.js API (Node-API) header-only C++ wrappers")
+    (description "This module contains header-only C++ wrapper classes which
+simplify the use of the C based Node-API provided by Node.js when using C++.
+It provides a C++ object model and exception handling semantics with low
+overhead.
+
+Node-API is an ABI stable C interface provided by Node.js for building native
+addons.  It is intended to insulate native addons from changes in the
+underlying JavaScript engine and allow modules compiled for one version to run
+on later versions of Node.js without recompilation.  The @code{node-addon-api}
+module, which is not part of Node.js, preserves the benefits of the Node-API
+as it consists only of inline code that depends only on the stable API
+provided by Node-API.
+
+It is important to remember that @emph{other} Node.js interfaces such as
+@code{libuv} (included in a project via @code{#include <uv.h>}) are not
+ABI-stable across Node.js major versions.")
+    (license license:expat)))
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 11/11] gnu: Add node-sqlite3.
  2021-11-14 13:04   ` [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
                       ` (7 preceding siblings ...)
  2021-11-14 13:04     ` [bug#51838] [PATCH 10/11] gnu: Add node-addon-api Philip McGrath
@ 2021-11-14 13:04     ` Philip McGrath
  8 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-14 13:04 UTC (permalink / raw)
  To: 51838; +Cc: Philip McGrath

* gnu/packages/node-xyz.scm (node-sqlite3): New variable.
---
 gnu/packages/node-xyz.scm | 114 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 114 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 8aa93122df..d30b6f6e04 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -22,6 +22,8 @@
 
 (define-module (gnu packages node-xyz)
   #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (gnu packages sqlite)
+  #:use-module (guix gexp)
   #:use-module (guix packages)
   #:use-module (guix git-download)
   #:use-module (guix build-system node))
@@ -636,3 +638,115 @@ (define-public node-addon-api
 @code{libuv} (included in a project via @code{#include <uv.h>}) are not
 ABI-stable across Node.js major versions.")
     (license license:expat)))
+
+(define-public node-sqlite3
+  (package
+    (name "node-sqlite3")
+    (version "5.0.2")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/mapbox/node-sqlite3")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "0sbbzzli282nxyfha10zx0k5m8hdp0sf3ipl59khjb7wm449j86h"))
+       (snippet
+        (with-imported-modules '((guix build utils))
+          #~(begin
+              (use-modules (guix build utils))
+              ;; unbundle sqlite
+              '(for-each delete-file-recursively
+                         (find-files "deps" gzip-file?)))))))
+    (inputs
+     `(("node-addon-api" ,node-addon-api)
+       ("sqlite" ,sqlite)))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f ; FIXME: tests depend on node-mocha
+       #:modules
+       ((guix build node-build-system)
+        (guix build json)
+        (srfi srfi-1)
+        (ice-9 match)
+        (guix build utils))
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'npm-config-sqlite
+           (lambda* (#:key inputs #:allow-other-keys)
+             (setenv "npm_config_sqlite" (assoc-ref inputs "sqlite"))))
+         (add-after 'unpack 'un-cloud-ify
+           ;; Normally, this is "built" using @mapbox/node-pre-gyp,
+           ;; which publishes or downloads pre-built binaries
+           ;; or falls back to building from source.
+           ;; Here, we patch out all of that and just build directly.
+           ;; It would be better to patch a version of @mapbox/node-pre-gyp
+           ;; that always builds from source, but there are a number
+           ;; of dependencies that need to be packaged or removed.
+           (lambda args
+             (with-atomic-file-replacement "package.json"
+               (lambda (in out)
+                 (let ((js (read-json in)))
+                   (match (assoc-ref js "binary")
+                     (('@ . alist)
+                      (setenv "GYP_DEFINES"
+                              (string-append
+                               "module_name="
+                               (assoc-ref alist "module_name")
+                               " "
+                               "module_path="
+                               (assoc-ref alist "module_path")))))
+                   (write-json
+                    (fold (match-lambda*
+                            (((key update) js)
+                             (assoc-set! js
+                                         key
+                                         (match (assoc-ref js key)
+                                           (('@ . alist)
+                                            (cons '@ (update alist)))
+                                           (other
+                                            (update other))))))
+                          js
+                          `(("dependencies"
+                             ,(lambda (deps)
+                                (assoc-remove!
+                                 (assoc-remove! deps "@mapbox/node-pre-gyp")
+                                 "node-pre-gyp")))
+                            ("devDependencies"
+                             ,(lambda (deps)
+                                (assoc-remove!
+                                 (assoc-remove! deps "aws-sdk")
+                                 "@mapbox/cloudfriend")))
+                            ("scripts"
+                             ,(lambda (scripts)
+                                ;; install script would use node-pre-gyp
+                                (assoc-remove! scripts "install")))))
+                    out))))))
+         (add-after 'un-cloud-ify 'remove-missing-dev-deps
+           ;; Remove some missing testing/linting dependencies
+           ;; so we don't have to skip the configure phase entirely.
+           (lambda args
+             (with-atomic-file-replacement "package.json"
+               (lambda (in out)
+                 (write-json
+                  (let ((js (read-json in)))
+                    (assoc-set!
+                     (assoc-remove!
+                      (assoc-remove! (assoc-remove! js "peerDependencies")
+                                     "peerDependenciesMeta")
+                      "optionalDependencies")
+                     "devDependencies"
+                     (match (assoc-ref js "devDependencies")
+                       (('@ . deps)
+                        (cons '@ (assoc-remove! (assoc-remove! deps "eslint")
+                                                "mocha"))))))
+                  out))))))))
+    (home-page "https://github.com/mapbox/node-sqlite3")
+    (synopsis "Asynchronous, non-blocking SQLite3 bindings for Node.js")
+    (description
+     "The Node.js add-on @code{node-sqlite3} provides a set of a asynchronous,
+non-blocking bindings for SQLite3, written in modern C++ and tested for memory
+leaks.")
+     (license license:bsd-3)))
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-14 13:04     ` [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
@ 2021-11-14 20:44       ` Liliana Marie Prikler
  2021-11-20  4:26         ` Philip McGrath
  0 siblings, 1 reply; 60+ messages in thread
From: Liliana Marie Prikler @ 2021-11-14 20:44 UTC (permalink / raw)
  To: Philip McGrath, 51838; +Cc: Pierre Langlois

Hi Philip

Am Sonntag, den 14.11.2021, 08:04 -0500 schrieb Philip McGrath:
> * gnu/packages/node.scm (node)[arguments]: Replace 'patch-npm-shebang
> and 'patch-node-shebang with a new 'patch-nested-shebangs that also
> handles node-gyp and other shebangs under "/lib/node_modules".
> [inputs]: Add Python for node-gyp as "python-for-target".
> (node-lts)[inputs]: Likewise.
> (libnode)[arguments]: Adjust to delete 'patch-nested-shebangs rather
> than 'patch-npm-shebang and 'patch-node-shebang.
> * guix/build-system/node.scm (lower): Add optional #:python argument
> and corresponding implicit input. Add the version of libuv used
> as an input to the #:node package as a new implicit input.
> * guix/build/node-build-system.scm (set-node-gyp-paths): New
> function. Sets the "npm_config_nodedir" and "npm_config_python"
> environment variables. Adds the "node-gyp-bin" directory to "PATH".
> (configure-gyp): New function. Run `node-gyp configure` if we see
> a `binding.gyp` file.
> (%standard-phases): Add 'set-node-gyp-paths after 'set-paths.
> Add 'configure-gyp after 'configure.
> 
> Co-authored-by: Pierre Langlois <pierre.langlois@gmx.com>
Looking at this patch, it does *a lot* at once and could probably be
separated into more than one.  Particularly, I'd suggest providing
capabilities in node-build-system first, then switching over to the new
thing in node.

> [...]
> --- a/guix/build-system/node.scm
> +++ b/guix/build-system/node.scm
> @@ -1,6 +1,8 @@
>  ;;; GNU Guix --- Functional package management for GNU
>  ;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
>  ;;; Copyright © 2019 Timothy Sample <samplet@ngyro.com>
> +;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
> +;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -24,6 +26,7 @@ (define-module (guix build-system node)
>    #:use-module (guix search-paths)
>    #:use-module (guix build-system)
>    #:use-module (guix build-system gnu)
> +  #:use-module (guix build-system python)
>    #:use-module (ice-9 match)
>    #:export (%node-build-system-modules
>              node-build
> @@ -44,11 +47,12 @@ (define (default-node)
>  (define* (lower name
>                  #:key source inputs native-inputs outputs system
> target
>                  (node (default-node))
> +                (python (default-python)) ;; for node-gyp
>                  #:allow-other-keys
>                  #:rest arguments)
>    "Return a bag for NAME."
>    (define private-keywords
> -    '(#:source #:target #:node #:inputs #:native-inputs))
> +    '(#:source #:target #:node #:python #:inputs #:native-inputs))
>  
>    (and (not target)                    ;XXX: no cross-compilation
>         (bag
> @@ -58,10 +62,13 @@ (define private-keywords
>                                `(("source" ,source))
>                                '())
>                          ,@inputs
> -
>                          ;; Keep the standard inputs of 'gnu-build-
> system'.
>                          ,@(standard-packages)))
>           (build-inputs `(("node" ,node)
> +                         ("python" ,python)
> +                        ;; We don't always need libuv, but the libuv
> and
> +                        ;; node versions need to match:
> +                        ("libuv" ,@(assoc-ref (package-inputs node)
> "libuv"))
>                           ,@native-inputs))
>           (outputs outputs)
>           (build node-build)
Will this python input always or often enough be needed?  What's the
build system ratio on node like, gyp vs. anything else, particularly
with packages close to the node core?

> diff --git a/guix/build/node-build-system.scm b/guix/build/node-
> build-system.scm
> index 70a367618e..6aeb0149dd 100644
> --- a/guix/build/node-build-system.scm
> +++ b/guix/build/node-build-system.scm
> @@ -2,6 +2,8 @@
>  ;;; Copyright © 2015 David Thompson <davet@gnu.org>
>  ;;; Copyright © 2016, 2020 Jelle Licht <jlicht@fsfe.org>
>  ;;; Copyright © 2019, 2021 Timothy Sample <samplet@ngyro.com>
> +;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
> +;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -46,6 +48,19 @@ (define (set-home . _)
>                (format #t "set HOME to ~s~%" (getenv "HOME")))))))
>    #t)
>  
> +(define* (set-node-gyp-paths #:key inputs #:allow-other-keys)
> +  "Initialize environment variables needed for building native
> addons."
> +  (setenv "npm_config_nodedir" (assoc-ref inputs "node"))
> +  (setenv "npm_config_python" (assoc-ref inputs "python"))
> +  (setenv "PATH"
> +          (string-append (getenv "PATH")
> +                         ":"
> +                         ;; Put this at the end to make it easier to
> override,
> +                         ;; just in case that should ever be
> necessary:
> +                         (assoc-ref inputs "node")
> +                         "/lib/node_modules/npm/bin/node-gyp-bin"))
> +  #t)
> +
Is this a necessary step to build node modules?  If not can we skip
this step when packages don't need gyp?

>  (define (module-name module)
>    (let* ((package.json (string-append module "/package.json"))
>           (package-meta (call-with-input-file package.json read-
> json)))
> @@ -101,6 +116,12 @@ (define* (configure #:key outputs inputs
> #:allow-other-keys)
>      (invoke npm "--offline" "--ignore-scripts" "install")
>      #t))
>  
> +(define* (configure-gyp #:key inputs #:allow-other-keys)
> +  "Run 'node-gyp configure' if we see a 'binding.gyp' file."
> +  (if (file-exists? "binding.gyp")
> +      (invoke (which "node-gyp") "configure")
> +      #t))
> +
You might want to make this part of configure itself, though I'm not
sure what the consensus is there when mixing different build system
styles.  (invoke (which ...) ) is also a pretty rare pattern, used in
only four locations so far.

Also, while better than the previous thing in that it actually checks
whether we have something gyp-esque at hand, I'd still prefer users
being able to not run this portion through some flag.  See e.g. #:use-
setuptools? in python or #:glib-or-gtk? in meson.

Cheers





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-14 20:44       ` Liliana Marie Prikler
@ 2021-11-20  4:26         ` Philip McGrath
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
  2021-11-20  5:10           ` [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
  0 siblings, 2 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:26 UTC (permalink / raw)
  To: Liliana Marie Prikler, 51838; +Cc: Pierre Langlois

Hi!

On 11/14/21 15:44, Liliana Marie Prikler wrote:
> Looking at this patch, it does *a lot* at once and could probably be
> separated into more than one.  Particularly, I'd suggest providing
> capabilities in node-build-system first, then switching over to the new
> thing in node.

Thanks for these comments. Some of the things to which you drew my 
attention seem to have been workarounds for problems that had since been 
solved at a deeper level. Then, in particular, this comment:


 >> +(define* (configure-gyp #:key inputs #:allow-other-keys)
 >> +  "Run 'node-gyp configure' if we see a 'binding.gyp' file."
 >> +  (if (file-exists? "binding.gyp")
 >> +      (invoke (which "node-gyp") "configure")
 >> +      #t))
 >> +
 > You might want to make this part of configure itself, though I'm not
 > sure what the consensus is there when mixing different build system
 > styles.  (invoke (which ...) ) is also a pretty rare pattern, used in
 > only four locations so far.

prompted me to look more closely at why so much manual work was needed 
in the first place.

It turns out that the `npm install` in the `'configure` phase should 
have handled most of it automatically, but the Guix packages were 
deleting the configure phase to avoid checking for devDependencies that 
aren't in Guix (or that we just don't want, e.g. some dependencies of 
node-sqlite3).

In v2 of this series (which will follow this email), I've removed all of 
the `node-gyp`-specific build-side code and tried a more general 
solution, adding an `#:absent-packages` argument to instruct the 
`'patch-dependencies` phase to remove the specified packages from the 
"package.json" file. This means that the Guix package can still run 
`'configure`/`npm install`, checking properly for the packages that we 
*do* have/want and doing all of the other automatic work `npm install` 
does. I also like that listing the missing packages in the Guix package 
definition provides a sort of documentation of what is missing: for 
example, it is clear which packages could have their tests enabled with 
the addition of a `node-tap` package, without having to inspect all of 
the individual "package.json" files.

I've updated all of the existing Node.js packages that deleted their 
`'configure` phase to use `#:absent-dependencies` instead.

>> @@ -58,10 +62,13 @@ (define private-keywords
>>                                 `(("source" ,source))
>>                                 '())
>>                           ,@inputs
>> -
>>                           ;; Keep the standard inputs of 'gnu-build-
>> system'.
>>                           ,@(standard-packages)))
>>            (build-inputs `(("node" ,node)
>> +                         ("python" ,python)
>> +                        ;; We don't always need libuv, but the libuv
>> and
>> +                        ;; node versions need to match:
>> +                        ("libuv" ,@(assoc-ref (package-inputs node)
>> "libuv"))
>>                            ,@native-inputs))
>>            (outputs outputs)
>>            (build node-build)
> Will this python input always or often enough be needed?  What's the
> build system ratio on node like, gyp vs. anything else, particularly
> with packages close to the node core?

GYP is a Python program, and it (or at least node's fork of it) expects 
to have a python executable available to invoke with sub-programs. Since 
npm depends on node-gyp and GYP, it is pretty close to the core.

In v2, I've just had packages that use node-gyp add Python to their 
inputs. IIRC, that used to not be enough, but it seems underlying 
problems were fixed in the mean time.

An alternative approach would be to configure it using the npmrc file, 
as I do for `nodedir` in v2. I'm not sure that makes much difference 
either way, but in v2 I've tried to minimize the amount of 
`node-gyp`-specific handling.

-Philip




^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases.
  2021-11-20  4:26         ` Philip McGrath
@ 2021-11-20  4:33           ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 02/26] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
                               ` (24 more replies)
  2021-11-20  5:10           ` [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
  1 sibling, 25 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/node.scm (node)[arguments]: Split 'patch-files phase
into 'patch-hardcoded-program-references and
'delete-problematic-tests. Adapt 'patch-hardcoded-program-references
and 'configure to work unmodified on node-lts, but don't try to
share 'delete-problematic-tests, because those details seem to change
too much between node versions.
(node, node-lts)[inputs]: Use bash-minimal rather than bash.
(node-lts)[arguments]: Inherit 'patch-hardcoded-program-references,
and 'configure phases from the bootstrap node.
Remove the 'patch-files phase, keeping its remaining
non-inherited work in a new 'replace-llhttp-sources phase.
---
 gnu/packages/node.scm | 105 +++++++++++++-----------------------------
 1 file changed, 32 insertions(+), 73 deletions(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index dccf871d2c..7ebbbc3060 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -9,6 +9,7 @@
 ;;; Copyright © 2020, 2021 Pierre Langlois <pierre.langlois@gmx.com>
 ;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -105,14 +106,22 @@ (define-public node
        #:test-target "test-ci-js"
        #:phases
        (modify-phases %standard-phases
-         (add-before 'configure 'patch-files
+         (add-before 'configure 'patch-hardcoded-program-references
            (lambda* (#:key inputs #:allow-other-keys)
+
              ;; Fix hardcoded /bin/sh references.
-             (substitute* '("lib/child_process.js"
-                            "lib/internal/v8_prof_polyfill.js"
-                            "test/parallel/test-child-process-spawnsync-shell.js"
-                            "test/parallel/test-stdio-closed.js"
-                            "test/sequential/test-child-process-emfile.js")
+             (substitute*
+                 (let ((common
+                        '("lib/child_process.js"
+                          "lib/internal/v8_prof_polyfill.js"
+                          "test/parallel/test-child-process-spawnsync-shell.js"
+                          "test/parallel/test-stdio-closed.js"
+                          "test/sequential/test-child-process-emfile.js"))
+                       ;; not in bootstap node:
+                       (sigxfsz "test/parallel/test-fs-write-sigxfsz.js"))
+                   (if (file-exists? sigxfsz)
+                       (cons sigxfsz common)
+                       common))
                (("'/bin/sh'")
                 (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
 
@@ -122,7 +131,10 @@ (define-public node
                             "test/parallel/test-child-process-exec-env.js")
                (("'/usr/bin/env'")
                 (string-append "'" (assoc-ref inputs "coreutils")
-                               "/bin/env'")))
+                               "/bin/env'")))))
+         (add-after 'patch-hardcoded-program-references
+             'delete-problematic-tests
+           (lambda* (#:key inputs #:allow-other-keys)
 
              ;; FIXME: These tests fail in the build container, but they don't
              ;; seem to be indicative of real problems in practice.
@@ -217,9 +229,13 @@ (define-public node
                (setenv "CXX" ,(cxx-for-target))
                (setenv "PKG_CONFIG" ,(pkg-config-for-target))
                (apply invoke
-                      (string-append (assoc-ref (or native-inputs inputs)
-                                                "python")
-                                     "/bin/python")
+                      (let ((python
+                             (string-append (assoc-ref (or native-inputs inputs)
+                                                       "python")
+                                            "/bin/python")))
+                        (if (file-exists? python)
+                            python
+                            (string-append python "3")))
                       "configure" flags))))
          (add-after 'patch-shebangs 'patch-npm-shebang
            (lambda* (#:key outputs #:allow-other-keys)
@@ -256,7 +272,7 @@ (define-public node
             (variable "NODE_PATH")
             (files '("lib/node_modules")))))
     (inputs
-     `(("bash" ,bash)
+     `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
        ("c-ares" ,c-ares)
        ("http-parser" ,http-parser)
@@ -712,66 +728,8 @@ (define-public node-lts
                                    libuv "/lib:"
                                    zlib "/lib"
                                    "'],"))))))
-           (replace 'configure
-             ;; Node's configure script is actually a python script, so we can't
-             ;; run it with bash.
-             (lambda* (#:key outputs (configure-flags '()) native-inputs inputs
-                       #:allow-other-keys)
-               (let* ((prefix (assoc-ref outputs "out"))
-                      (xflags ,(if (%current-target-system)
-                                   `'("--cross-compiling"
-                                     ,(string-append
-                                       "--dest-cpu="
-                                       (match (%current-target-system)
-                                         ((? (cut string-prefix? "arm" <>))
-                                          "arm")
-                                         ((? (cut string-prefix? "aarch64" <>))
-                                          "arm64")
-                                         ((? (cut string-prefix? "i686" <>))
-                                          "ia32")
-                                         ((? (cut string-prefix? "x86_64" <>))
-                                          "x64")
-                                         ((? (cut string-prefix? "powerpc64" <>))
-                                          "ppc64")
-                                         (_ "unsupported"))))
-                                   ''()))
-                      (flags (cons
-                               (string-append "--prefix=" prefix)
-                               (append xflags configure-flags))))
-                 (format #t "build directory: ~s~%" (getcwd))
-                 (format #t "configure flags: ~s~%" flags)
-                 ;; Node's configure script expects the CC environment variable to
-                 ;; be set.
-                 (setenv "CC_host" "gcc")
-                 (setenv "CXX_host" "g++")
-                 (setenv "CC" ,(cc-for-target))
-                 (setenv "CXX" ,(cxx-for-target))
-                 (setenv "PKG_CONFIG" ,(pkg-config-for-target))
-                 (apply invoke
-                        (string-append (assoc-ref (or native-inputs inputs)
-                                                  "python")
-                                       "/bin/python3")
-                        "configure" flags))))
-           (replace 'patch-files
+           (replace 'delete-problematic-tests
              (lambda* (#:key inputs #:allow-other-keys)
-               ;; Fix hardcoded /bin/sh references.
-               (substitute* '("lib/child_process.js"
-                              "lib/internal/v8_prof_polyfill.js"
-                              "test/parallel/test-child-process-spawnsync-shell.js"
-                              "test/parallel/test-fs-write-sigxfsz.js"
-                              "test/parallel/test-stdio-closed.js"
-                              "test/sequential/test-child-process-emfile.js")
-                 (("'/bin/sh'")
-                  (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
-
-               ;; Fix hardcoded /usr/bin/env references.
-               (substitute* '("test/parallel/test-child-process-default-options.js"
-                              "test/parallel/test-child-process-env.js"
-                              "test/parallel/test-child-process-exec-env.js")
-                 (("'/usr/bin/env'")
-                  (string-append "'" (assoc-ref inputs "coreutils")
-                                 "/bin/env'")))
-
                ;; FIXME: These tests fail in the build container, but they don't
                ;; seem to be indicative of real problems in practice.
                (for-each delete-file
@@ -810,8 +768,9 @@ (define-public node-lts
                ;; TODO: Regenerate certs instead.
                (for-each delete-file
                          '("test/parallel/test-tls-passphrase.js"
-                           "test/parallel/test-tls-server-verify.js"))
-
+                           "test/parallel/test-tls-server-verify.js"))))
+           (add-after 'delete-problematic-tests 'replace-llhttp-sources
+             (lambda* (#:key inputs #:allow-other-keys)
                ;; Replace pre-generated llhttp sources
                (let ((llhttp (assoc-ref inputs "llhttp")))
                  (copy-file (string-append llhttp "/src/llhttp.c")
@@ -838,7 +797,7 @@ (define-public node-lts
        ("python" ,python)
        ("util-linux" ,util-linux)))
     (inputs
-     `(("bash" ,bash)
+     `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
        ("c-ares" ,c-ares-for-node)
        ("icu4c" ,icu4c-67)
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 02/26] gnu: node: Update to 10.24.1 for bootstrapping.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 03/26] gnu: node: Patch shebangs in node_modules Philip McGrath
                               ` (23 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/node.scm (node): Update to 10.24.1.
---
 gnu/packages/node.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 7ebbbc3060..71b66774a6 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -57,14 +57,14 @@ (define-module (gnu packages node)
 (define-public node
   (package
     (name "node")
-    (version "10.24.0")
+    (version "10.24.1")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://nodejs.org/dist/v" version
                                   "/node-v" version ".tar.xz"))
               (sha256
                (base32
-                "1k1srdis23782hnd1ymgczs78x9gqhv77v0am7yb54gqcspp70hm"))
+                "032801kg24j04xmf09m0vxzlcz86sv21s24lv9l4cfv08k1c4byp"))
               (modules '((guix build utils)))
               (snippet
                `(begin
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 03/26] gnu: node: Patch shebangs in node_modules.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 02/26] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 04/26] gnu: node: Add an npmrc file to set nodedir Philip McGrath
                               ` (22 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/node.scm (node)[arguments]: Replace 'patch-npm-shebang
and 'patch-node-shebang with a new 'patch-nested-shebangs that also
handles node-gyp and other shebangs under "/lib/node_modules".
[inputs]: Add Python for node-gyp as "python-for-target".
(node-lts)[inputs]: Likewise.
(libnode)[arguments]: Adjust to delete 'patch-nested-shebangs rather
than 'patch-npm-shebang and 'patch-node-shebang.
---
 gnu/packages/node.scm | 34 +++++++++++++++++-----------------
 1 file changed, 17 insertions(+), 17 deletions(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 71b66774a6..a57a74fb81 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -237,21 +237,20 @@ (define-public node
                             python
                             (string-append python "3")))
                       "configure" flags))))
-         (add-after 'patch-shebangs 'patch-npm-shebang
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((bindir (string-append (assoc-ref outputs "out")
-                                           "/bin"))
-                    (npm    (string-append bindir "/npm"))
-                    (target (readlink npm)))
-               (with-directory-excursion bindir
-                 (patch-shebang target (list bindir))))))
-         (add-after 'install 'patch-node-shebang
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((bindir (string-append (assoc-ref outputs "out")
-                                           "/bin"))
-                    (npx    (readlink (string-append bindir "/npx"))))
-               (with-directory-excursion bindir
-                 (patch-shebang npx (list bindir)))))))))
+         (add-after 'patch-shebangs 'patch-nested-shebangs
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let* ((prefix (assoc-ref outputs "out"))
+                    (path (map (lambda (dir)
+                                 (string-append dir "/bin"))
+                               (list prefix
+                                     (assoc-ref inputs "python-for-target")))))
+               (for-each
+                (lambda (file)
+                  (patch-shebang file path))
+                (find-files (string-append prefix "/lib/node_modules")
+                            (lambda (file stat)
+                              (executable-file? file))
+                            #:stat lstat))))))))
     (native-inputs
      `(;; Runtime dependencies for binaries used as a bootstrap.
        ("c-ares" ,c-ares)
@@ -274,6 +273,7 @@ (define-public node
     (inputs
      `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
+       ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
        ("c-ares" ,c-ares)
        ("http-parser" ,http-parser)
        ("icu4c" ,icu4c)
@@ -799,6 +799,7 @@ (define-public node-lts
     (inputs
      `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
+       ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
        ("c-ares" ,c-ares-for-node)
        ("icu4c" ,icu4c-67)
        ("libuv" ,libuv-for-node)
@@ -817,5 +818,4 @@ (define-public libnode
         `(cons* "--shared" "--without-npm" ,flags))
        ((#:phases phases '%standard-phases)
         `(modify-phases ,phases
-           (delete 'patch-npm-shebang)
-           (delete 'patch-node-shebang)))))))
+           (delete 'patch-nested-shebangs)))))))
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 04/26] gnu: node: Add an npmrc file to set nodedir.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 02/26] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 03/26] gnu: node: Patch shebangs in node_modules Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument Philip McGrath
                               ` (21 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/node.scm (node, node-lis)[arguments]: Add a phase
'install-npmrc to create a "built-in" npmrc file that configures
"nodedir" to point to the output store path.
(libnode)[arguments]: Delete the 'install-npmrc phase.
---
 gnu/packages/node.scm | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index a57a74fb81..0f67fe79c2 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -250,7 +250,21 @@ (define-public node
                 (find-files (string-append prefix "/lib/node_modules")
                             (lambda (file stat)
                               (executable-file? file))
-                            #:stat lstat))))))))
+                            #:stat lstat)))))
+         (add-after 'install 'install-npmrc
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out")))
+               (with-output-to-file
+                   ;; Use the config file "primarily for distribution
+                   ;; maintainers" rather than "{prefix}/etc/npmrc",
+                   ;; especially because node-build-system uses --prefix
+                   ;; to install things to their store paths:
+                   (string-append out "/lib/node_modules/npm/npmrc")
+                 (lambda ()
+                   ;; Tell npm (mostly node-gyp) where to find our
+                   ;; installed headers so it doesn't try to
+                   ;; download them from the internet:
+                   (format #t "nodedir=~a\n" out)))))))))
     (native-inputs
      `(;; Runtime dependencies for binaries used as a bootstrap.
        ("c-ares" ,c-ares)
@@ -818,4 +832,5 @@ (define-public libnode
         `(cons* "--shared" "--without-npm" ,flags))
        ((#:phases phases '%standard-phases)
         `(modify-phases ,phases
+           (delete 'install-npmrc)
            (delete 'patch-nested-shebangs)))))))
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (2 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 04/26] gnu: node: Add an npmrc file to set nodedir Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  7:41               ` Liliana Marie Prikler
  2021-11-28 19:27               ` [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Timothy Sample
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 06/26] gnu: node-semver-bootstrap: Use #:absent-dependencies Philip McGrath
                               ` (20 subsequent siblings)
  24 siblings, 2 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

Many of Guix's Node.js packages are built without some of the
dependencies they specify in their "package-lock.json" files,
either because we don't have them packaged yet (e.g. test
utilities) or because we don't want them (e.g. to reduce the
closure size). Previously, Guix package definitions would work
around this situation by deleting the `'configure`
phase (i.e. the initial `npm install`).

This commit adds an optional #:absent-dependencies argument to
`node-build-system` to list Node.js packages that should be
removed from the "package.json" file.Retaining the `'configure`
phase avoids skipping checks for the dependencies that are
intended to be present and other actions performed by `npm
install`, such as automatically building native add-ons with
`node-gyp` when the "gypfile" key is present.

* guix/build-system/node.scm (lower, node-build): Add optional
argument #:absent-dependencies with default of ''(). Pass it on
to the build-side code.
* guix/build/node-build-system.scm (patch-dependencies): Respect
the #:absent-dependencies argument, removing specified npm
packages from the "dependencies" or "devDependencies" tables
in "package.json". Also, strictly follow the linearity rules
for `assoc-set!` and friends.
---
 guix/build-system/node.scm       |  3 ++
 guix/build/node-build-system.scm | 55 ++++++++++++++++++++++----------
 2 files changed, 41 insertions(+), 17 deletions(-)

diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 98f63f87ef..75ae34508f 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -44,6 +44,7 @@ (define (default-node)
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
                 (node (default-node))
+                (absent-dependencies ''())
                 #:allow-other-keys
                 #:rest arguments)
   "Return a bag for NAME."
@@ -73,6 +74,7 @@ (define* (node-build store name inputs
                      (tests? #t)
                      (phases '(@ (guix build node-build-system)
                                  %standard-phases))
+                     (absent-dependencies ''())
                      (outputs '("out"))
                      (search-paths '())
                      (system (%current-system))
@@ -94,6 +96,7 @@ (define builder
                    #:test-target ,test-target
                    #:tests? ,tests?
                    #:phases ,phases
+                   #:absent-dependencies ,absent-dependencies
                    #:outputs %outputs
                    #:search-paths ',(map search-path-specification->sexp
                                          search-paths)
diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 70a367618e..32d6807e3e 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -69,30 +69,51 @@ (define (list-modules directory)
               input-paths)
     index))
 
-(define* (patch-dependencies #:key inputs #:allow-other-keys)
+(define* (patch-dependencies #:key inputs absent-dependencies
+                             #:allow-other-keys)
 
   (define index (index-modules (map cdr inputs)))
 
-  (define (resolve-dependencies package-meta meta-key)
-    (fold (lambda (key+value acc)
-            (match key+value
-              ('@ acc)
-              ((key . value) (acons key (hash-ref index key value) acc))))
+  (define (resolve-dependencies meta-alist meta-key)
+    (match (assoc-ref meta-alist meta-key)
+      (#f
+       '())
+      (('@ . orig-deps)
+       (fold (match-lambda*
+               (('@ acc)
+                acc)
+                (((key . value) acc)
+                 (if (member key absent-dependencies)
+                     acc
+                     (acons key (hash-ref index key value) acc))))
           '()
-          (or (assoc-ref package-meta meta-key) '())))
+          orig-deps))))
 
   (with-atomic-file-replacement "package.json"
     (lambda (in out)
-      (let ((package-meta (read-json in)))
-        (assoc-set! package-meta "dependencies"
-                    (append
-                     '(@)
-                     (resolve-dependencies package-meta "dependencies")
-                     (resolve-dependencies package-meta "peerDependencies")))
-        (assoc-set! package-meta "devDependencies"
-                    (append
-                     '(@)
-                     (resolve-dependencies package-meta "devDependencies")))
+      ;; It is unsafe to rely on 'assoc-set!' to update an
+      ;; existing assosciation list variable:
+      ;; see 'info "(guile)Adding or Setting Alist Entries"'.
+      (let* ((package-meta (read-json in))
+             (alist (match package-meta
+                      ((@ . alist) alist)))
+             ;; Other relevant keys may include peerDependenciesMeta
+             ;; and optionalDependencies, but it seems to work out fine
+             ;; just to leave those alone.
+             (alist
+              (assoc-set!
+               alist "dependencies"
+               (append
+                '(@)
+                (resolve-dependencies alist "dependencies")
+                (resolve-dependencies alist "peerDependencies"))))
+             (alist
+              (assoc-set!
+               alist "devDependencies"
+               (append
+                '(@)
+                (resolve-dependencies alist "devDependencies"))))
+             (package-meta (cons '@ alist)))
         (write-json package-meta out))))
   #t)
 
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 06/26] gnu: node-semver-bootstrap: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (3 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  7:43               ` Liliana Marie Prikler
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 07/26] gnu: node-ms-bootstrap: " Philip McGrath
                               ` (19 subsequent siblings)
  24 siblings, 1 reply; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node.scm (node-semver-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
 gnu/packages/node.scm | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 0f67fe79c2..6c958eebec 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -330,9 +330,8 @@ (define-public node-semver-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       '("tap")))
     (home-page "https://github.com/npm/node-semver")
     (properties '((hidden? . #t)))
     (synopsis "Parses semantic versions strings")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 07/26] gnu: node-ms-bootstrap: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (4 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 06/26] gnu: node-semver-bootstrap: Use #:absent-dependencies Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 08/26] gnu: node-binary-search-bootstrap: " Philip McGrath
                               ` (18 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node.scm (node-ms-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
 gnu/packages/node.scm | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 6c958eebec..ca4bf1ad23 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -358,9 +358,12 @@ (define-public node-ms-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       `("eslint"
+         "expect.js"
+         "husky"
+         "lint-staged"
+         "mocha")))
     (home-page "https://github.com/zeit/ms#readme")
     (properties '((hidden? . #t)))
     (synopsis "Tiny millisecond conversion utility")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 08/26] gnu: node-binary-search-bootstrap: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (5 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 07/26] gnu: node-ms-bootstrap: " Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 09/26] gnu: node-debug-bootstrap: " Philip McGrath
                               ` (17 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node.scm (node-binary-search-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
 gnu/packages/node.scm | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index ca4bf1ad23..2cdae34ae1 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -389,9 +389,9 @@ (define-public node-binary-search-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       `("chai"
+         "mocha")))
     (home-page "https://github.com/darkskyapp/binary-search#readme")
     (properties '((hidden? . #t)))
     (synopsis "Tiny binary search function with comparators")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 09/26] gnu: node-debug-bootstrap: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (6 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 08/26] gnu: node-binary-search-bootstrap: " Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: " Philip McGrath
                               ` (16 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node.scm (node-debug-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
 gnu/packages/node.scm | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 2cdae34ae1..98a51276e7 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -416,9 +416,18 @@ (define-public node-debug-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       `("brfs"
+         "browserify"
+         "coveralls"
+         "istanbul"
+         "karma"
+         "karma-browserify"
+         "karma-chrome-launcher"
+         "karma-mocha"
+         "mocha"
+         "mocha-lcov-reporter"
+         "xo")))
     (inputs `(("node-ms" ,node-ms-bootstrap)))
     (home-page "https://github.com/visionmedia/debug#readme")
     (properties '((hidden? . #t)))
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (7 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 09/26] gnu: node-debug-bootstrap: " Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  7:44               ` Liliana Marie Prikler
  2021-11-23 11:04               ` Jelle Licht
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 11/26] gnu: node-llparse-frontend-bootstrap: Use #:absent-dependencies Philip McGrath
                               ` (15 subsequent siblings)
  24 siblings, 2 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node.scm (node-llparse-builder-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
Add a new phase `#:delete-package-lock` to remove the
problematic "package-lock.json".
---
 gnu/packages/node.scm | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 98a51276e7..9d4903a8ca 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -479,9 +479,21 @@ (define-public node-llparse-builder-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
+       #:absent-dependencies
+       `("@types/mocha"
+         "@types/node"
+         "mocha"
+         "ts-node"
+         "tslint"
+         "typescript")
        #:phases
        (modify-phases %standard-phases
-         (delete 'configure)
+         (add-before 'configure 'remove-package-lock
+           ;; Having package-lock.json seems to cause npm
+           ;; to look for things on the internet in the configure phase,
+           ;; even if we have them properly installed.
+           (lambda args
+             (delete-file-recursively "package-lock.json")))
          (replace 'build
            (lambda* (#:key inputs #:allow-other-keys)
              (let ((esbuild (string-append (assoc-ref inputs "esbuild")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 11/26] gnu: node-llparse-frontend-bootstrap: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (8 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: " Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 12/26] gnu: node-llparse-bootstrap: " Philip McGrath
                               ` (14 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node.scm (node-llparse-frontend-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
 gnu/packages/node.scm | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 9d4903a8ca..298b9376a8 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -547,9 +547,16 @@ (define-public node-llparse-frontend-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
+       #:absent-dependencies
+       `("@types/debug"
+         "@types/mocha"
+         "@types/node"
+         "mocha"
+         "ts-node"
+         "tslint"
+         "typescript")
        #:phases
        (modify-phases %standard-phases
-         (delete 'configure)
          (replace 'build
            (lambda* (#:key inputs #:allow-other-keys)
              (let ((esbuild (string-append (assoc-ref inputs "esbuild")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 12/26] gnu: node-llparse-bootstrap: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (9 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 11/26] gnu: node-llparse-frontend-bootstrap: Use #:absent-dependencies Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 13/26] gnu: node-semver: " Philip McGrath
                               ` (13 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node.scm (node-llparse-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
Add a new phase `#:delete-package-lock` to remove the
problematic "package-lock.json".
---
 gnu/packages/node.scm | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 298b9376a8..6d48816c77 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -609,9 +609,24 @@ (define-public node-llparse-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
+       #:absent-dependencies
+       `("@types/debug"
+         "@types/mocha"
+         "@types/node"
+         "esm"
+         "llparse-test-fixture"
+         "mocha"
+         "ts-node"
+         "tslint"
+         "typescript")
        #:phases
        (modify-phases %standard-phases
-         (delete 'configure)
+         (add-before 'configure 'remove-package-lock
+           ;; Having package-lock.json seems to cause npm
+           ;; to look for things on the internet in the configure phase,
+           ;; even if we have them properly installed.
+           (lambda args
+             (delete-file-recursively "package-lock.json")))
          (replace 'build
            (lambda* (#:key inputs #:allow-other-keys)
              (let ((esbuild (string-append (assoc-ref inputs "esbuild")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 13/26] gnu: node-semver: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (10 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 12/26] gnu: node-llparse-bootstrap: " Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 14/26] gnu: node-wrappy: " Philip McGrath
                               ` (12 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node-xyz.scm (node-semver)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
 gnu/packages/node-xyz.scm | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 98c611f227..93a033d33e 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -289,11 +289,10 @@ (define-public node-semver
                 "06biknqb05r9xsmcflm3ygh50pjvdk84x6r79w43kmck4fn3qn5p"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f ; FIXME: Tests depend on node-tap
-       #:phases
-       (modify-phases %standard-phases
-         ;; The only dependency to check for is tap, which we don't have.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       '("tap")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
     (home-page "https://github.com/npm/node-semver")
     (synopsis "Parses semantic versions strings")
     (description
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 14/26] gnu: node-wrappy: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (11 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 13/26] gnu: node-semver: " Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 15/26] gnu: node-once: " Philip McGrath
                               ` (11 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node-xyz.scm (node-wrappy)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
 gnu/packages/node-xyz.scm | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 93a033d33e..77577cb315 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -316,10 +316,8 @@ (define-public node-wrappy
     (build-system node-build-system)
     (arguments
      '(#:tests? #f ; FIXME: Tests depend on node-tap
-       #:phases
-       (modify-phases %standard-phases
-         ;; The only dependency to check for is tap, which we don't have.
-         (delete 'configure))))
+       #:absent-dependencies
+       '("tap")))
     (home-page "https://github.com/npm/wrappy")
     (synopsis "Callback wrapping utility")
     (description "@code{wrappy} is a utility for Node.js to wrap callbacks.")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 15/26] gnu: node-once: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (12 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 14/26] gnu: node-wrappy: " Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 16/26] gnu: node-irc-colors: " Philip McGrath
                               ` (10 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node-xyz.scm (node-once)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
 gnu/packages/node-xyz.scm | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 77577cb315..eb071320cd 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -338,13 +338,10 @@ (define-public node-once
                 "1z8dcbf28dqdcp4wb0c53wrs90a07nkrax2c9kk26dsk1dhrnxav"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f                      ; FIXME: tests depend on node-tap
-       #:phases
-       (modify-phases %standard-phases
-         ;; The default configure phase fails due to tap being missing, as we do
-         ;; not have tap packaged yet.  It is used only for tests.  This package
-         ;; still works as a dependency of node-glob and node-inflight.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       '("tap")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
     (inputs
      `(("node-wrappy" ,node-wrappy)))
     (home-page "https://github.com/isaacs/once")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 16/26] gnu: node-irc-colors: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (13 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 15/26] gnu: node-once: " Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 17/26] gnu: node-irc: " Philip McGrath
                               ` (9 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node-xyz.scm (node-irc-colors)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
 gnu/packages/node-xyz.scm | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index eb071320cd..9646c14243 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -367,12 +367,10 @@ (define-public node-irc-colors
                 "0q3y34rbnlc55jcakmdxkicwazyvyph9r6gaf6hi8k7wj2nfwfli"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f                      ; FIXME: tests depend on node-istanbul
-       #:phases
-       (modify-phases %standard-phases
-         ;; The default configure phase fails due to various packages
-         ;; being missing, as we don't have them packaged yet.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       `("istanbul"
+         "vows")
+       #:tests? #f))
     (home-page "https://github.com/fent/irc-colors.js")
     (synopsis "Node.js module providing color and formatting for IRC")
     (description "@code{node-irc-colors} is a Node.js module that
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 17/26] gnu: node-irc: Use #:absent-dependencies.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (14 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 16/26] gnu: node-irc-colors: " Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 18/26] guix: node-build-system: Add optional #:libuv? argument Philip McGrath
                               ` (8 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

gnu/packages/node-xyz.scm (node-irc)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
 gnu/packages/node-xyz.scm | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 9646c14243..998d0a9a90 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -394,12 +394,12 @@ (define-public node-irc
                 "1ln4qfx20jbwg4cp8lp0vf27m5281z2sz16d15xd6150n26cbi4x"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f                      ; FIXME: tests depend on node-faucet
-       #:phases
-       (modify-phases %standard-phases
-         ;; The default configure phase fails due to various packages
-         ;; being missing, as we don't have them packaged yet.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       `("ansi-color"
+         "faucet"
+         "jscs"
+         "tape")
+       #:tests? #f))
     (inputs
      `(("node-irc-colors" ,node-irc-colors)))
     (home-page "https://github.com/martynsmith/node-irc")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 18/26] guix: node-build-system: Add optional #:libuv? argument.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (15 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 17/26] gnu: node-irc: " Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  7:46               ` Liliana Marie Prikler
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 19/26] gnu: Add node-inherits Philip McGrath
                               ` (7 subsequent siblings)
  24 siblings, 1 reply; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* guix/build-system/node.scm (lower): Add an optional #:libuv?
argument to tell the build system to add the version of libuv
used as an input to the #:node package as an additional implicit
input, so that packages needing libuv always get the correct version.
---
 guix/build-system/node.scm | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 75ae34508f..f83a7f64ce 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -1,6 +1,8 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
 ;;; Copyright © 2019 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -44,12 +46,13 @@ (define (default-node)
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
                 (node (default-node))
+                (libuv? #f)
                 (absent-dependencies ''())
                 #:allow-other-keys
                 #:rest arguments)
   "Return a bag for NAME."
   (define private-keywords
-    '(#:source #:target #:node #:inputs #:native-inputs))
+    '(#:source #:target #:node #:libuv? #:inputs #:native-inputs))
 
   (and (not target)                    ;XXX: no cross-compilation
        (bag
@@ -59,10 +62,18 @@ (define private-keywords
                               `(("source" ,source))
                               '())
                         ,@inputs
-
                         ;; Keep the standard inputs of 'gnu-build-system'.
                         ,@(standard-packages)))
          (build-inputs `(("node" ,node)
+                         ;; Many packages with native addons need
+                         ;; libuv headers. The libuv version must
+                         ;; be exactly the same as for the node
+                         ;; package we are adding implicitly,
+                         ;; so we take care of adding libuv, too.
+                         ,@(if libuv?
+                               `(("libuv" ,@(assoc-ref (package-inputs node)
+                                                       "libuv")))
+                               '())
                          ,@native-inputs))
          (outputs outputs)
          (build node-build)
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 19/26] gnu: Add node-inherits.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (16 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 18/26] guix: node-build-system: Add optional #:libuv? argument Philip McGrath
@ 2021-11-20  4:33             ` Philip McGrath
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 20/26] gnu: Add node-safe-buffer Philip McGrath
                               ` (6 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:33 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/node-xyz.scm (node-inherits): New variable.
---
 gnu/packages/node-xyz.scm | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 998d0a9a90..9d8bf27852 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2020 Giacomo Leidi <goodoldpaul@autistici.org>
 ;;; Copyright © 2021 Noisytoot <noisytoot@disroot.org>
 ;;; Copyright © 2021 Charles <charles.b.jackson@protonmail.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -352,6 +353,35 @@ (define-public node-once
 if desired.")
     (license license:isc)))
 
+(define-public node-inherits
+  (package
+    (name "node-inherits")
+    (version "2.0.4")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/isaacs/inherits")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0cpsr5yqwkxpbbbbl0rwk4mcby6zbx841k2zb4c3gb1579i5wq9p"))))
+    (build-system node-build-system)
+    (arguments
+     '(#:absent-dependencies
+       '("tap")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
+    (home-page
+     "https://github.com/isaacs/inherits")
+    (synopsis
+     "Browser-friendly inheritance Node.js")
+    (description
+     "Browser-friendly inheritance fully compatible with standard Node.js
+@code{inherits()}.")
+    (license license:isc)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 20/26] gnu: Add node-safe-buffer.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (17 preceding siblings ...)
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 19/26] gnu: Add node-inherits Philip McGrath
@ 2021-11-20  4:34             ` Philip McGrath
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 21/26] gnu: Add node-string-decoder Philip McGrath
                               ` (5 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:34 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/node-xyz.scm (node-safe-buffer): New variable.
---
 gnu/packages/node-xyz.scm | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 9d8bf27852..6dff0ddba5 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -382,6 +382,34 @@ (define-public node-inherits
 @code{inherits()}.")
     (license license:isc)))
 
+(define-public node-safe-buffer
+  (package
+    (name "node-safe-buffer")
+    (version "5.2.1")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/feross/safe-buffer")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0r26m0nl41h90ihnl2xf0cqs6z9z7jb87dl5j8yqb7887r9jlbpi"))))
+    (build-system node-build-system)
+    (arguments
+     '(#:absent-dependencies
+       '("tape"
+         "standard")
+       #:tests? #f))
+    (home-page
+     "https://github.com/feross/safe-buffer")
+    (synopsis "Safer Node.js Buffer API")
+    (description "A safe drop-in replacement the Node.js @code{Buffer} API
+that works in all versions of Node.js, using the built-in implementation when
+available.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 21/26] gnu: Add node-string-decoder.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (18 preceding siblings ...)
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 20/26] gnu: Add node-safe-buffer Philip McGrath
@ 2021-11-20  4:34             ` Philip McGrath
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 22/26] gnu: Add node-readable-stream Philip McGrath
                               ` (4 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:34 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/node-xyz.scm (node-string-decoder): New variable.
---
 gnu/packages/node-xyz.scm | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 6dff0ddba5..c6cce64f7d 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -410,6 +410,40 @@ (define-public node-safe-buffer
 available.")
     (license license:expat)))
 
+(define-public node-string-decoder
+  (package
+    (name "node-string-decoder")
+    (version "1.3.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/string_decoder")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0xxvyya9fl9rlkqwmxzqzbz4rdr3jgw4vf37hff7cgscxkhg266k"))))
+    (build-system node-build-system)
+    (arguments
+     '(#:absent-dependencies
+       '("tap"
+         "core-util-is"
+         "babel-polyfill")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
+    (inputs
+     `(("node-safe-buffer" ,node-safe-buffer)
+       ("node-inherits" ,node-inherits)))
+    (home-page
+     "https://github.com/nodejs/string_decoder")
+    (synopsis
+     "Node.js core @code{string_decoder} for userland")
+    (description
+     "This package is a mirror of the @code{string_decoder} implementation in
+Node-core.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 22/26] gnu: Add node-readable-stream.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (19 preceding siblings ...)
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 21/26] gnu: Add node-string-decoder Philip McGrath
@ 2021-11-20  4:34             ` Philip McGrath
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 23/26] gnu: Add node-nan Philip McGrath
                               ` (3 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:34 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/node-xyz.scm (node-readable-stream): New variable.
---
 gnu/packages/node-xyz.scm | 54 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 54 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index c6cce64f7d..3e06413908 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -444,6 +444,60 @@ (define-public node-string-decoder
 Node-core.")
     (license license:expat)))
 
+(define-public node-readable-stream
+  (package
+    (name "node-readable-stream")
+    (version "3.6.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/readable-stream")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0ybl4cdgsm9c5jq3xq8s01201jk8w0yakh63hlclsfbcdfqhd9ri"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:absent-dependencies
+       `("@babel/cli"
+         "@babel/core"
+         "@babel/polyfill"
+         "@babel/preset-env"
+         "airtap"
+         "assert"
+         "bl"
+         "deep-strict-equal"
+         "events.once"
+         "glob"
+         "gunzip-maybe"
+         "hyperquest"
+         "lolex"
+         "nyc"
+         "pump"
+         "rimraf"
+         "tap"
+         "tape"
+         "tar-fs"
+         "util-promisify")
+       #:tests? #f))
+    (inputs
+     `(("node-util-deprecate" ,node-util-deprecate)
+       ("node-string-decoder" ,node-string-decoder)
+       ("node-inherits" ,node-inherits)))
+    (home-page
+     "https://github.com/nodejs/readable-stream")
+    (synopsis
+     "Node.js core streams for userland")
+    (description
+     "This package is a mirror of the streams implementations in Node.js.
+
+If you want to guarantee a stable streams base, regardless of what version of
+Node you (or the users of your libraries) are using, use
+@code{readable-stream} only and avoid the @code{stream} module in Node-core.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 23/26] gnu: Add node-nan.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (20 preceding siblings ...)
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 22/26] gnu: Add node-readable-stream Philip McGrath
@ 2021-11-20  4:34             ` Philip McGrath
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 24/26] gnu: Add node-openzwave-shared Philip McGrath
                               ` (2 subsequent siblings)
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:34 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/node-xyz.scm (node-nan): New variable.
---
 gnu/packages/node-xyz.scm | 44 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 3e06413908..ed169c0778 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -553,3 +553,47 @@ (define-public node-irc
     (description "@code{node-irc} is an IRC client library for Node.js.
 It has functions for joining, parting, talking, and many other IRC commands.")
     (license license:gpl3+)))
+
+(define-public node-nan
+  (package
+    (name "node-nan")
+    (version "2.15.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/nan")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "18xslh9va5ld872scrp5y4251ax9s3c6qh0lnl1200lpzbsxy7yd"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       ;; tests need tap and other dependencies
+       #f
+       #:absent-dependencies
+       '("bindings"
+         "commander"
+         "glob"
+         "request"
+         "node-gyp" ;; would be needed for tests
+         "tap"
+         "xtend")))
+    (inputs
+     `(("readable-stream" ,node-readable-stream)))
+    (home-page "https://github.com/nodejs/nan")
+    (synopsis "Native Abstractions for Node.js")
+    (description "A header file filled with macro and utility goodness for
+making add-on development for Node.js easier across versions 0.8, 0.10, 0.12,
+1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 and 16.
+
+Thanks to the crazy changes in V8 (and some in Node core), keeping native
+addons compiling happily across versions, particularly 0.10 to 0.12 to 4.0, is
+a minor nightmare.  The goal of this project is to store all logic necessary
+to develop native Node.js addons without having to inspect
+@code{NODE_MODULE_VERSION} and get yourself into a macro-tangle.
+
+This project also contains some helper utilities that make addon development a
+bit more pleasant.")
+    (license license:expat)))
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 24/26] gnu: Add node-openzwave-shared.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (21 preceding siblings ...)
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 23/26] gnu: Add node-nan Philip McGrath
@ 2021-11-20  4:34             ` Philip McGrath
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 25/26] gnu: Add node-addon-api Philip McGrath
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 26/26] gnu: Add node-sqlite3 Philip McGrath
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:34 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/zwave.scm (node-openzwave-shared): New variable.
---
 gnu/packages/zwave.scm | 67 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/gnu/packages/zwave.scm b/gnu/packages/zwave.scm
index 2019ec32df..8757ddecf2 100644
--- a/gnu/packages/zwave.scm
+++ b/gnu/packages/zwave.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2019 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -21,11 +22,14 @@ (define-module (gnu packages zwave)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system node)
   #:use-module (gnu packages)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages node-xyz)
   #:use-module (gnu packages libusb)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages perl)
+  #:use-module (gnu packages python)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages xml))
 
@@ -88,3 +92,66 @@ (define-public open-zwave
 and respond to devices on a Z-Wave network, without requiring in-depth
 knowledge of the Z-Wave protocol.")
     (license license:lgpl3+)))
+
+(define-public node-openzwave-shared
+  (package
+    (name "node-openzwave-shared")
+    (version "1.7.2")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/OpenZWave/node-openzwave-shared")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "1bqqy12dzqj05j9wsh50dmn84dddmhz0gjzvd3y20z4hpy1v8rsc"))))
+    (inputs
+     `(("open-zwave" ,open-zwave)
+       ("node-nan" ,node-nan)))
+    (native-inputs
+     `(("which" ,which)
+       ("python" ,python)
+       ("pkg-config" ,pkg-config)))
+    (build-system node-build-system)
+    (arguments
+     `(#:libuv?
+       #t
+       #:phases
+       (modify-phases %standard-phases
+         (replace 'build
+           (lambda* (#:key inputs #:allow-other-keys)
+             (define node-dir
+               (assoc-ref inputs "node"))
+             (invoke (string-append node-dir "/bin/npx")
+                     "--call"
+                     (string-append
+                      node-dir
+                      "/lib/node_modules/npm/bin/node-gyp-bin/node-gyp"
+                      " "
+                      "rebuild")))))))
+    (home-page "https://github.com/OpenZWave/node-openzwave-shared")
+    (synopsis "Node.js bindings for OpenZWave")
+    (description
+     "With the @code{node-openzwave-shared} package, you can easily control
+and manage your Z-Wave devices (lights, dimmers, blinds, you name it) from
+within Node.js applications.  This library also supports secure
+devices (e.g. door locks) that require encryption.  All widely used Node.js
+versions are supported with the help of @code{node-nan}.
+
+This library is currently able to:
+@itemize @bullet
+@item
+scan a Z-Wave network and report on connected devices;
+@item
+write values to Z-Wave nodes;
+@item
+monitor the network for changes;
+@item
+heal nodes and/or the network; and
+@item
+perform management tasks: add or remove nodes, replace failed nodes,
+manage their group associations, etc.
+@end itemize")
+    (license license:isc)))
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 25/26] gnu: Add node-addon-api.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (22 preceding siblings ...)
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 24/26] gnu: Add node-openzwave-shared Philip McGrath
@ 2021-11-20  4:34             ` Philip McGrath
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 26/26] gnu: Add node-sqlite3 Philip McGrath
  24 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:34 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/node-xyz.scm (node-addon-api): New variable.
---
 gnu/packages/node-xyz.scm | 63 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index ed169c0778..60dbfc163c 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -22,6 +22,9 @@
 
 (define-module (gnu packages node-xyz)
   #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (gnu packages sqlite)
+  #:use-module (gnu packages python)
+  #:use-module (guix gexp)
   #:use-module (guix packages)
   #:use-module (guix git-download)
   #:use-module (guix build-system node))
@@ -597,3 +600,63 @@ (define-public node-nan
 This project also contains some helper utilities that make addon development a
 bit more pleasant.")
     (license license:expat)))
+
+(define-public node-addon-api
+  (package
+    (name "node-addon-api")
+    (version "4.2.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/node-addon-api")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "1bhvfi2m9nxfz418s619914vmidcnrzbjv6l9nid476c3zlpazch"))))
+    (inputs
+     `(("python" ,python)))
+    (build-system node-build-system)
+    (arguments
+     `(#:absent-dependencies
+       `("benchmark"
+         "bindings"
+         "clang-format"
+         "eslint"
+         "eslint-config-semistandard"
+         "eslint-config-standard"
+         "eslint-plugin-import"
+         "eslint-plugin-node"
+         "eslint-plugin-promise"
+         "fs-extra"
+         "path"
+         "pre-commit"
+         "safe-buffer")
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'skip-js-tests
+           ;; We can't run the js-based tests,
+           ;; but we can still do the C++ parts
+           (lambda args
+             (substitute* "package.json"
+               (("\"test\": \"node test\"")
+                "\"test\": \"echo stopping after pretest on Guix\"")))))))
+    (home-page "https://github.com/nodejs/node-addon-api")
+    (synopsis "Node.js API (Node-API) header-only C++ wrappers")
+    (description "This module contains header-only C++ wrapper classes which
+simplify the use of the C based Node-API provided by Node.js when using C++.
+It provides a C++ object model and exception handling semantics with low
+overhead.
+
+Node-API is an ABI stable C interface provided by Node.js for building native
+addons.  It is intended to insulate native addons from changes in the
+underlying JavaScript engine and allow modules compiled for one version to run
+on later versions of Node.js without recompilation.  The @code{node-addon-api}
+module, which is not part of Node.js, preserves the benefits of the Node-API
+as it consists only of inline code that depends only on the stable API
+provided by Node-API.
+
+It is important to remember that @emph{other} Node.js interfaces such as
+@code{libuv} (included in a project via @code{#include <uv.h>}) are not
+ABI-stable across Node.js major versions.")
+    (license license:expat)))
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 26/26] gnu: Add node-sqlite3.
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
                               ` (23 preceding siblings ...)
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 25/26] gnu: Add node-addon-api Philip McGrath
@ 2021-11-20  4:34             ` Philip McGrath
  2021-11-20  7:48               ` Liliana Marie Prikler
  24 siblings, 1 reply; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  4:34 UTC (permalink / raw)
  To: 51838; +Cc: Liliana Marie Prikler

* gnu/packages/node-xyz.scm (node-sqlite3): New variable.
---
 gnu/packages/node-xyz.scm | 107 ++++++++++++++++++++++++++++++++++++--
 1 file changed, 104 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 60dbfc163c..a56281fe18 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -615,7 +615,8 @@ (define-public node-addon-api
        (sha256
         (base32 "1bhvfi2m9nxfz418s619914vmidcnrzbjv6l9nid476c3zlpazch"))))
     (inputs
-     `(("python" ,python)))
+     `(("python" ,python)
+       ("node-safe-buffer" ,node-safe-buffer)))
     (build-system node-build-system)
     (arguments
      `(#:absent-dependencies
@@ -630,8 +631,7 @@ (define-public node-addon-api
          "eslint-plugin-promise"
          "fs-extra"
          "path"
-         "pre-commit"
-         "safe-buffer")
+         "pre-commit")
        #:phases
        (modify-phases %standard-phases
          (add-after 'unpack 'skip-js-tests
@@ -660,3 +660,104 @@ (define-public node-addon-api
 @code{libuv} (included in a project via @code{#include <uv.h>}) are not
 ABI-stable across Node.js major versions.")
     (license license:expat)))
+
+(define-public node-sqlite3
+  (package
+    (name "node-sqlite3")
+    (version "5.0.2")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/mapbox/node-sqlite3")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "0sbbzzli282nxyfha10zx0k5m8hdp0sf3ipl59khjb7wm449j86h"))
+       (snippet
+        (with-imported-modules '((guix build utils))
+          #~(begin
+              (use-modules (guix build utils))
+              ;; unbundle sqlite
+              '(for-each delete-file-recursively
+                         (find-files "deps" gzip-file?)))))))
+    (inputs
+     `(("node-addon-api" ,node-addon-api)
+       ("python" ,python)
+       ("sqlite" ,sqlite)))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f ; FIXME: tests depend on node-mocha
+       #:modules
+       ((guix build node-build-system)
+        (guix build json)
+        (srfi srfi-1)
+        (ice-9 match)
+        (guix build utils))
+       #:libuv? #t
+       #:absent-dependencies
+       `(;; Normally, this is "built" using @mapbox/node-pre-gyp,
+         ;; which publishes or downloads pre-built binaries
+         ;; or falls back to building from source.
+         ;; Here, we patch out all of that and just build directly.
+         ;; It would be better to patch a version of @mapbox/node-pre-gyp
+         ;; that always builds from source, as Debian does,
+         ;; but there are a number of dependencies that need
+         ;; to be packaged or removed.
+         "@mapbox/node-pre-gyp"
+         "node-pre-gyp" ;; deprecated name still used in some places
+         "aws-sdk"
+         "@mapbox/cloudfriend"
+         ;; Confusingly, this is only a dependency beceuse of
+         ;; @mapbox/node-pre-gyp: with that removed,
+         ;; npm will use its own copy:
+         "node-gyp"
+         ;; These we'd like, we just don't have them yet:
+         "eslint"
+         "mocha")
+       #:phases
+       (modify-phases %standard-phases
+         ;; We need this step even if we do replace @mapbox/node-pre-gyp
+         ;; because the package expects to build its bundled sqlite
+         (add-before 'configure 'npm-config-sqlite
+           (lambda* (#:key inputs #:allow-other-keys)
+             (setenv "npm_config_sqlite" (assoc-ref inputs "sqlite"))))
+         (add-after 'patch-dependencies 'avoid-node-pre-gyp
+           (lambda args
+             (with-atomic-file-replacement "package.json"
+               (lambda (in out)
+                 (let* ((js (read-json in))
+                        (alist (match js
+                                 (('@ . alist) alist)))
+                        (scripts-alist (match (assoc-ref alist "scripts")
+                                         (('@ . alist) alist)))
+                        (scripts-alist
+                         ;; install script would use node-pre-gyp
+                         (assoc-remove! scripts-alist "install"))
+                        (alist
+                         (assoc-set! alist "scripts" (cons '@ scripts-alist)))
+                        (alist
+                         ;; causes `npm install` (our 'configure phase)
+                         ;; run the appropriate parts of node-gyp
+                         (assoc-set! alist "gypfile" #t))
+                        (binary-alist (match (assoc-ref alist "binary")
+                                        (('@ . alist) alist)))
+                        (js (cons '@ alist)))
+                   ;; compensate for lack of @mapbox/node-pre-gyp
+                   (setenv "GYP_DEFINES"
+                           (string-append
+                            "module_name="
+                            (assoc-ref binary-alist "module_name")
+                            " "
+                            "module_path="
+                            (assoc-ref binary-alist "module_path")))
+                   (write-json js
+                               out)))))))))
+    (home-page "https://github.com/mapbox/node-sqlite3")
+    (synopsis "Asynchronous, non-blocking SQLite3 bindings for Node.js")
+    (description
+     "The Node.js add-on @code{node-sqlite3} provides a set of a asynchronous,
+non-blocking bindings for SQLite3, written in modern C++ and tested for memory
+leaks.")
+     (license license:bsd-3)))
-- 
2.32.0





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-20  4:26         ` Philip McGrath
  2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
@ 2021-11-20  5:10           ` Philip McGrath
  2021-11-20  7:28             ` Liliana Marie Prikler
  1 sibling, 1 reply; 60+ messages in thread
From: Philip McGrath @ 2021-11-20  5:10 UTC (permalink / raw)
  To: Liliana Marie Prikler, 51838; +Cc: Pierre Langlois

On 11/19/21 23:26, Philip McGrath wrote> In v2 of this series (which 
will follow this email)

I'm not sure what went wrong, but I, at least, received these patches 
out of order. Let me know if I should resend them as v3. They are 
definitely in the right order here: 
https://gitlab.com/philip1/guix-patches/-/tree/guix-issue-51838-v2

-Philip




^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-20  5:10           ` [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
@ 2021-11-20  7:28             ` Liliana Marie Prikler
  0 siblings, 0 replies; 60+ messages in thread
From: Liliana Marie Prikler @ 2021-11-20  7:28 UTC (permalink / raw)
  To: Philip McGrath, 51838; +Cc: Pierre Langlois

Am Samstag, den 20.11.2021, 00:10 -0500 schrieb Philip McGrath:
> On 11/19/21 23:26, Philip McGrath wrote> In v2 of this series (which 
> will follow this email)
> 
> I'm not sure what went wrong, but I, at least, received these
> patches 
> out of order. Let me know if I should resend them as v3. They are 
> definitely in the right order here: 
> https://gitlab.com/philip1/guix-patches/-/tree/guix-issue-51838-v2
> 
> -Philip
Our mumi frontend to debbugs should be able to deliver them to you as
one convenient mbox: 
https://issues.guix.gnu.org/issue/51838/patch-set/2

Cheers





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument.
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument Philip McGrath
@ 2021-11-20  7:41               ` Liliana Marie Prikler
  2021-11-20 17:04                 ` Philip McGrath
  2021-11-28 19:27               ` [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Timothy Sample
  1 sibling, 1 reply; 60+ messages in thread
From: Liliana Marie Prikler @ 2021-11-20  7:41 UTC (permalink / raw)
  To: Philip McGrath, 51838

Hi,

Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
> Many of Guix's Node.js packages are built without some of the
> dependencies they specify in their "package-lock.json" files,
> either because we don't have them packaged yet (e.g. test
> utilities) or because we don't want them (e.g. to reduce the
> closure size). Previously, Guix package definitions would work
> around this situation by deleting the `'configure`
> phase (i.e. the initial `npm install`).
> 
> This commit adds an optional #:absent-dependencies argument to
> `node-build-system` to list Node.js packages that should be
> removed from the "package.json" file.Retaining the `'configure`
> phase avoids skipping checks for the dependencies that are
> intended to be present and other actions performed by `npm
> install`, such as automatically building native add-ons with
> `node-gyp` when the "gypfile" key is present.
> 
> [...]
This is a somewhat decent approach, but I wonder whether we could
improve this.  Does nodejs complain if a dependency exists in the code,
but is not present in the packages.json?

In the resolve-dependencies subprocedure, we could check whether we
have a matching input somewhere and only include the dependency if we
do.  WDYT?





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 06/26] gnu: node-semver-bootstrap: Use #:absent-dependencies.
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 06/26] gnu: node-semver-bootstrap: Use #:absent-dependencies Philip McGrath
@ 2021-11-20  7:43               ` Liliana Marie Prikler
  0 siblings, 0 replies; 60+ messages in thread
From: Liliana Marie Prikler @ 2021-11-20  7:43 UTC (permalink / raw)
  To: Philip McGrath, 51838

Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
> gnu/packages/node.scm (node-semver-bootstrap)[arguments]: Add
> `#:absent-dependencies`. Stop deleting the `'configure` phase.
> ---
>  gnu/packages/node.scm | 5 ++---
>  1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
> index 0f67fe79c2..6c958eebec 100644
> --- a/gnu/packages/node.scm
> +++ b/gnu/packages/node.scm
> @@ -330,9 +330,8 @@ (define-public node-semver-bootstrap
>      (arguments
>       `(#:node ,node-bootstrap
>         #:tests? #f
> -       #:phases
> -       (modify-phases %standard-phases
> -         (delete 'configure))))
> +       #:absent-dependencies
> +       '("tap")))
>      (home-page "https://github.com/npm/node-semver")
>      (properties '((hidden? . #t)))
>      (synopsis "Parses semantic versions strings")
Not deleting configure SGTM, but recall my note about #:absent-
dependencies.





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: Use #:absent-dependencies.
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: " Philip McGrath
@ 2021-11-20  7:44               ` Liliana Marie Prikler
  2021-11-20 17:09                 ` Philip McGrath
  2021-11-23 11:04               ` Jelle Licht
  1 sibling, 1 reply; 60+ messages in thread
From: Liliana Marie Prikler @ 2021-11-20  7:44 UTC (permalink / raw)
  To: Philip McGrath, 51838

Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
> gnu/packages/node.scm (node-llparse-builder-bootstrap)[arguments]:
> Add `#:absent-dependencies`. Stop deleting the `'configure` phase.
> Add a new phase `#:delete-package-lock` to remove the
> problematic "package-lock.json".
> ---
>  gnu/packages/node.scm | 14 +++++++++++++-
>  1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
> index 98a51276e7..9d4903a8ca 100644
> --- a/gnu/packages/node.scm
> +++ b/gnu/packages/node.scm
> @@ -479,9 +479,21 @@ (define-public node-llparse-builder-bootstrap
>      (arguments
>       `(#:node ,node-bootstrap
>         #:tests? #f
> +       #:absent-dependencies
> +       `("@types/mocha"
> +         "@types/node"
> +         "mocha"
> +         "ts-node"
> +         "tslint"
> +         "typescript")
>         #:phases
>         (modify-phases %standard-phases
> -         (delete 'configure)
> +         (add-before 'configure 'remove-package-lock
> +           ;; Having package-lock.json seems to cause npm
> +           ;; to look for things on the internet in the configure
> phase,
> +           ;; even if we have them properly installed.
> +           (lambda args
> +             (delete-file-recursively "package-lock.json")))
I think a simple delete-file ought to work.  This should also be done
by configure or similar, compare cargo-build-system.





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 18/26] guix: node-build-system: Add optional #:libuv? argument.
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 18/26] guix: node-build-system: Add optional #:libuv? argument Philip McGrath
@ 2021-11-20  7:46               ` Liliana Marie Prikler
  2021-11-20 17:16                 ` Philip McGrath
  0 siblings, 1 reply; 60+ messages in thread
From: Liliana Marie Prikler @ 2021-11-20  7:46 UTC (permalink / raw)
  To: Philip McGrath, 51838

Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
> * guix/build-system/node.scm (lower): Add an optional #:libuv?
> argument to tell the build system to add the version of libuv
> used as an input to the #:node package as an additional implicit
> input, so that packages needing libuv always get the correct version.
Would it change something if we always had libuv as implicit input?  I
don't think it'd do much for closure size, but it might impact implicit
package search as proposed by my solution to the #:absent-dependencies
thing.





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 26/26] gnu: Add node-sqlite3.
  2021-11-20  4:34             ` [bug#51838] [PATCH v2 26/26] gnu: Add node-sqlite3 Philip McGrath
@ 2021-11-20  7:48               ` Liliana Marie Prikler
  0 siblings, 0 replies; 60+ messages in thread
From: Liliana Marie Prikler @ 2021-11-20  7:48 UTC (permalink / raw)
  To: Philip McGrath, 51838

Warning, this patch does not just add node-sqlite3!

Am Freitag, den 19.11.2021, 23:34 -0500 schrieb Philip McGrath:
> * gnu/packages/node-xyz.scm (node-sqlite3): New variable.
> ---
>  gnu/packages/node-xyz.scm | 107
> ++++++++++++++++++++++++++++++++++++--
>  1 file changed, 104 insertions(+), 3 deletions(-)
> 
> diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
> index 60dbfc163c..a56281fe18 100644
> --- a/gnu/packages/node-xyz.scm
> +++ b/gnu/packages/node-xyz.scm
> @@ -615,7 +615,8 @@ (define-public node-addon-api
>         (sha256
>          (base32
> "1bhvfi2m9nxfz418s619914vmidcnrzbjv6l9nid476c3zlpazch"))))
>      (inputs
> -     `(("python" ,python)))
> +     `(("python" ,python)
> +       ("node-safe-buffer" ,node-safe-buffer)))
>      (build-system node-build-system)
>      (arguments
>       `(#:absent-dependencies
> @@ -630,8 +631,7 @@ (define-public node-addon-api
>           "eslint-plugin-promise"
>           "fs-extra"
>           "path"
> -         "pre-commit"
> -         "safe-buffer")
> +         "pre-commit")
>         #:phases
>         (modify-phases %standard-phases
>           (add-after 'unpack 'skip-js-tests
> @@ -660,3 +660,104 @@ (define-public node-addon-api
>  @code{libuv} (included in a project via @code{#include <uv.h>}) are
> not
>  ABI-stable across Node.js major versions.")
>      (license license:expat)))
> +
> +(define-public node-sqlite3
> +  (package
> +    (name "node-sqlite3")
> +    (version "5.0.2")
> +    (source
> +     (origin
> +       (method git-fetch)
> +       (uri (git-reference
> +             (url "https://github.com/mapbox/node-sqlite3")
> +             (commit (string-append "v" version))))
> +       (file-name (git-file-name name version))
> +       (sha256
> +        (base32
> "0sbbzzli282nxyfha10zx0k5m8hdp0sf3ipl59khjb7wm449j86h"))
> +       (snippet
> +        (with-imported-modules '((guix build utils))
> +          #~(begin
> +              (use-modules (guix build utils))
> +              ;; unbundle sqlite
> +              '(for-each delete-file-recursively
> +                         (find-files "deps" gzip-file?)))))))
> +    (inputs
> +     `(("node-addon-api" ,node-addon-api)
> +       ("python" ,python)
> +       ("sqlite" ,sqlite)))
> +    (build-system node-build-system)
> +    (arguments
> +     `(#:tests?
> +       #f ; FIXME: tests depend on node-mocha
> +       #:modules
> +       ((guix build node-build-system)
> +        (guix build json)
> +        (srfi srfi-1)
> +        (ice-9 match)
> +        (guix build utils))
> +       #:libuv? #t
> +       #:absent-dependencies
> +       `(;; Normally, this is "built" using @mapbox/node-pre-gyp,
> +         ;; which publishes or downloads pre-built binaries
> +         ;; or falls back to building from source.
> +         ;; Here, we patch out all of that and just build directly.
> +         ;; It would be better to patch a version of @mapbox/node-
> pre-gyp
> +         ;; that always builds from source, as Debian does,
> +         ;; but there are a number of dependencies that need
> +         ;; to be packaged or removed.
> +         "@mapbox/node-pre-gyp"
> +         "node-pre-gyp" ;; deprecated name still used in some places
> +         "aws-sdk"
> +         "@mapbox/cloudfriend"
> +         ;; Confusingly, this is only a dependency beceuse of
> +         ;; @mapbox/node-pre-gyp: with that removed,
> +         ;; npm will use its own copy:
> +         "node-gyp"
> +         ;; These we'd like, we just don't have them yet:
> +         "eslint"
> +         "mocha")
> +       #:phases
> +       (modify-phases %standard-phases
> +         ;; We need this step even if we do replace @mapbox/node-
> pre-gyp
> +         ;; because the package expects to build its bundled sqlite
> +         (add-before 'configure 'npm-config-sqlite
> +           (lambda* (#:key inputs #:allow-other-keys)
> +             (setenv "npm_config_sqlite" (assoc-ref inputs
> "sqlite"))))
> +         (add-after 'patch-dependencies 'avoid-node-pre-gyp
> +           (lambda args
> +             (with-atomic-file-replacement "package.json"
> +               (lambda (in out)
> +                 (let* ((js (read-json in))
> +                        (alist (match js
> +                                 (('@ . alist) alist)))
> +                        (scripts-alist (match (assoc-ref alist
> "scripts")
> +                                         (('@ . alist) alist)))
> +                        (scripts-alist
> +                         ;; install script would use node-pre-gyp
> +                         (assoc-remove! scripts-alist "install"))
> +                        (alist
> +                         (assoc-set! alist "scripts" (cons '@
> scripts-alist)))
> +                        (alist
> +                         ;; causes `npm install` (our 'configure
> phase)
> +                         ;; run the appropriate parts of node-gyp
> +                         (assoc-set! alist "gypfile" #t))
> +                        (binary-alist (match (assoc-ref alist
> "binary")
> +                                        (('@ . alist) alist)))
> +                        (js (cons '@ alist)))
> +                   ;; compensate for lack of @mapbox/node-pre-gyp
> +                   (setenv "GYP_DEFINES"
> +                           (string-append
> +                            "module_name="
> +                            (assoc-ref binary-alist "module_name")
> +                            " "
> +                            "module_path="
> +                            (assoc-ref binary-alist "module_path")))
> +                   (write-json js
> +                               out)))))))))
> +    (home-page "https://github.com/mapbox/node-sqlite3")
> +    (synopsis "Asynchronous, non-blocking SQLite3 bindings for
> Node.js")
> +    (description
> +     "The Node.js add-on @code{node-sqlite3} provides a set of a
> asynchronous,
> +non-blocking bindings for SQLite3, written in modern C++ and tested
> for memory
> +leaks.")
> +     (license license:bsd-3)))





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument.
  2021-11-20  7:41               ` Liliana Marie Prikler
@ 2021-11-20 17:04                 ` Philip McGrath
  2021-11-20 20:24                   ` Liliana Marie Prikler
  0 siblings, 1 reply; 60+ messages in thread
From: Philip McGrath @ 2021-11-20 17:04 UTC (permalink / raw)
  To: Liliana Marie Prikler, 51838

Hi,

On 11/20/21 02:41, Liliana Marie Prikler wrote:
> Hi,
> 
> Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
>> Many of Guix's Node.js packages are built without some of the
>> dependencies they specify in their "package-lock.json" files,
>> either because we don't have them packaged yet (e.g. test
>> utilities) or because we don't want them (e.g. to reduce the
>> closure size). Previously, Guix package definitions would work
>> around this situation by deleting the `'configure`
>> phase (i.e. the initial `npm install`).
>>
>> This commit adds an optional #:absent-dependencies argument to
>> `node-build-system` to list Node.js packages that should be
>> removed from the "package.json" file.Retaining the `'configure`
>> phase avoids skipping checks for the dependencies that are
>> intended to be present and other actions performed by `npm
>> install`, such as automatically building native add-ons with
>> `node-gyp` when the "gypfile" key is present.
>>
>> [...]
> This is a somewhat decent approach, but I wonder whether we could
> improve this.  Does nodejs complain if a dependency exists in the code,
> but is not present in the packages.json?

To the best of my understanding, npm doesn't inspect the js files at all 
by default, so no. (A package.json file might define a build script that 
does more checks.) I think a missing and undeclared dependency manifests 
itself as a runtime error when the require() function is evaluated.

> In the resolve-dependencies subprocedure, we could check whether we
> have a matching input somewhere and only include the dependency if we
> do.  WDYT?

I thought about this, but it seems to me there are two problems. First, 
I'm very unsure about this, but, in the last few days, I've gotten the 
impression there may be some "package.json" packages that don't 
correspond directly to Guix packages. I don't really understand how that 
works, but I think it may have something to do with npm packages that 
can run either on Node.js or on the web and need to use some 
functionality that's part of the Node.js core but not the web platform 
(or vice versa?). I could be wrong about that, but I've tried, 
especially in v2, to only have the build-side code do things I'm 
confident are correct in all cases.

The other issue is that deleting packages with no matching input by 
default would replicate some of the drawbacks of the current `(delete 
'configure)` approach. I think it's better to have an explicit list of 
dependencies that Guix is deleting. If eventually we package all of the 
missing dependencies for Guix, it would be much easier to find the 
packages that ought to use it. And, in the highly dynamic JavaScript 
world, I'm reluctant to give up one of the few static checks we have. If 
a missing package that really was required were automatically deleted 
from "package.json", it seems the failure mode would by a mysterious 
runtime error, potentially many steps down a dependency chain.




^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: Use #:absent-dependencies.
  2021-11-20  7:44               ` Liliana Marie Prikler
@ 2021-11-20 17:09                 ` Philip McGrath
  0 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20 17:09 UTC (permalink / raw)
  To: Liliana Marie Prikler, 51838

On 11/20/21 02:44, Liliana Marie Prikler wrote:
> Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
>>          (modify-phases %standard-phases
>> -         (delete 'configure)
>> +         (add-before 'configure 'remove-package-lock
>> +           ;; Having package-lock.json seems to cause npm
>> +           ;; to look for things on the internet in the configure
>> phase,
>> +           ;; even if we have them properly installed.
>> +           (lambda args
>> +             (delete-file-recursively "package-lock.json")))
> I think a simple delete-file ought to work.  This should also be done
> by configure or similar, compare cargo-build-system.

Yes, I thought about having the build system automatically delete 
"package-lock.json", but I'm not 100% sure it's always a problem to have 
it, or if there might even be some circumstance where we want to keep 
it. I'd prefer to wait until we see a significant number of node 
packages deleting their "package-lock.json" files before trying to 
abstract over the pattern.




^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 18/26] guix: node-build-system: Add optional #:libuv? argument.
  2021-11-20  7:46               ` Liliana Marie Prikler
@ 2021-11-20 17:16                 ` Philip McGrath
  0 siblings, 0 replies; 60+ messages in thread
From: Philip McGrath @ 2021-11-20 17:16 UTC (permalink / raw)
  To: Liliana Marie Prikler, 51838

On 11/20/21 02:46, Liliana Marie Prikler wrote:
> Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
>> * guix/build-system/node.scm (lower): Add an optional #:libuv?
>> argument to tell the build system to add the version of libuv
>> used as an input to the #:node package as an additional implicit
>> input, so that packages needing libuv always get the correct version.
> Would it change something if we always had libuv as implicit input?  I
> don't think it'd do much for closure size, but it might impact implicit
> package search as proposed by my solution to the #:absent-dependencies
> thing.

 From a Guix perspective, I don't think it would make much difference, 
since libuv is always needed by node itself. Maybe it would require more 
grafting than would be needed otherwise?

But I think the recommendation for authors of node add-ons is to avoid 
libuv unless it's really needed, because it has fewer stability 
guarantees that other Node.js APIs.





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 04/26] gnu: node: Add an npmrc file to set nodedir.
  2021-11-14 12:41 [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
  2021-11-14 12:58 ` [bug#51838] [PATCH 01/11] gnu: node: Avoid duplicating build phases Philip McGrath
@ 2021-11-20 17:38 ` Timothy Sample
  2021-11-20 19:55 ` [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Timothy Sample
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 60+ messages in thread
From: Timothy Sample @ 2021-11-20 17:38 UTC (permalink / raw)
  To: Philip McGrath; +Cc: 51838, Liliana Marie Prikler

Hi!

Just a little typo:

Philip McGrath <philip@philipmcgrath.com> writes:

> * gnu/packages/node.scm (node, node-lis)[arguments]: Add a phase

node-lis should be node-lts.


-- Tim




^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-14 12:41 [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
  2021-11-14 12:58 ` [bug#51838] [PATCH 01/11] gnu: node: Avoid duplicating build phases Philip McGrath
  2021-11-20 17:38 ` [bug#51838] [PATCH v2 04/26] gnu: node: Add an npmrc file to set nodedir Timothy Sample
@ 2021-11-20 19:55 ` Timothy Sample
  2021-11-20 20:04 ` Timothy Sample
  2021-11-20 20:08 ` Timothy Sample
  4 siblings, 0 replies; 60+ messages in thread
From: Timothy Sample @ 2021-11-20 19:55 UTC (permalink / raw)
  To: Philip McGrath; +Cc: 51838, Liliana Marie Prikler

Hello,

Philip McGrath <philip@philipmcgrath.com> writes:

> On 11/20/21 02:46, Liliana Marie Prikler wrote:
>> Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
>>> * guix/build-system/node.scm (lower): Add an optional #:libuv?
>>> argument to tell the build system to add the version of libuv
>>> used as an input to the #:node package as an additional implicit
>>> input, so that packages needing libuv always get the correct version.
>> Would it change something if we always had libuv as implicit input?  I
>> don't think it'd do much for closure size, but it might impact implicit
>> package search as proposed by my solution to the #:absent-dependencies
>> thing.
>
> From a Guix perspective, I don't think it would make much difference,
> since libuv is always needed by node itself. Maybe it would require
> more grafting than would be needed otherwise?
>
> But I think the recommendation for authors of node add-ons is to avoid
> libuv unless it's really needed, because it has fewer stability 
> guarantees that other Node.js APIs.

My assumption is that most packages would ignore the libuv headers, and
they wouldn’t retain a reference to it (except via Node.js itself).
Hence, I don’t think it would make any difference either to grafting or
closure size to just always add it.


-- Tim




^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-14 12:41 [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
                   ` (2 preceding siblings ...)
  2021-11-20 19:55 ` [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Timothy Sample
@ 2021-11-20 20:04 ` Timothy Sample
  2021-11-20 20:08 ` Timothy Sample
  4 siblings, 0 replies; 60+ messages in thread
From: Timothy Sample @ 2021-11-20 20:04 UTC (permalink / raw)
  To: Philip McGrath; +Cc: 51838, Liliana Marie Prikler

Hello,

Philip McGrath <philip@philipmcgrath.com> writes:

> On 11/20/21 02:41, Liliana Marie Prikler wrote:
>
>> In the resolve-dependencies subprocedure, we could check whether we
>> have a matching input somewhere and only include the dependency if we
>> do.  WDYT?
>
> [...]
>
> The other issue is that deleting packages with no matching input by
> default would replicate some of the drawbacks of the current `(delete 
> 'configure)` approach. I think it's better to have an explicit list of
> dependencies that Guix is deleting. If eventually we package all of
> the missing dependencies for Guix, it would be much easier to find the 
> packages that ought to use it. And, in the highly dynamic JavaScript
> world, I'm reluctant to give up one of the few static checks we
> have. If a missing package that really was required were automatically
> deleted from "package.json", it seems the failure mode would by a
> mysterious runtime error, potentially many steps down a dependency
> chain.

This is well put.  I actually experimented with a similar approach when
we updated the Node.js build system.  This is a big improvement over
deleting the configure phase, which would never scale to more than a
handful of packages.  Having a build-time check that ensures all the
developer-declared dependencies are available (save the “absent” ones)
will be very helpful when we are maintaining hundreds of JavaScript
packages.  :)

For the patch itself, it would be better to move a lot of your commit
message into a comment somewhere in the build system code.  If we had a
section in the manual for Node packages, it would go there, but alas....
I think most people would be happy to see a comment in the build system
code and be saved from having to poke around with ‘git blame’.


-- Tim




^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-14 12:41 [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
                   ` (3 preceding siblings ...)
  2021-11-20 20:04 ` Timothy Sample
@ 2021-11-20 20:08 ` Timothy Sample
  2021-11-23 20:54   ` Pierre Langlois
  4 siblings, 1 reply; 60+ messages in thread
From: Timothy Sample @ 2021-11-20 20:08 UTC (permalink / raw)
  To: Philip McGrath; +Cc: 51838, Pierre Langlois, Liliana Marie Prikler

Hi Philip,

There are some really valuable changes in here, thanks!  I would like to
start understanding and reviewing the changes so that we can get some of
this good stuff merged in.  However, I have one question that I couldn’t
answer from reading here or at <https://issues.guix.gnu.org/49946>: is
there an agreement between you and Pierre that these patches are the
“right” way to do native addons for Node.js.  More importantly, is the
general plan that we merge these changes, and then Pierre rebases their
Tree-sitter changes on top these?

Pierre, maybe you could weigh in here?

Sorry if I missed something.  I assume everything is OK, but I want to
be sure before I start digging into the details of the patches –
especially those first few more complicated ones.  :)

Thanks!


-- Tim




^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument.
  2021-11-20 17:04                 ` Philip McGrath
@ 2021-11-20 20:24                   ` Liliana Marie Prikler
  0 siblings, 0 replies; 60+ messages in thread
From: Liliana Marie Prikler @ 2021-11-20 20:24 UTC (permalink / raw)
  To: Philip McGrath, 51838

Hi,

Am Samstag, den 20.11.2021, 12:04 -0500 schrieb Philip McGrath:
> To the best of my understanding, npm doesn't inspect the js files at
> all by default, so no. (A package.json file might define a build
> script that does more checks.) I think a missing and undeclared
> dependency manifests itself as a runtime error when the require()
> function is evaluated.
Sheesh.

> > In the resolve-dependencies subprocedure, we could check whether we
> > have a matching input somewhere and only include the dependency if
> > we do.  WDYT?
> 
> I thought about this, but it seems to me there are two problems.
> First, I'm very unsure about this, but, in the last few days, I've
> gotten the impression there may be some "package.json" packages that
> don't correspond directly to Guix packages. I don't really understand
> how that works, but I think it may have something to do with npm
> packages that can run either on Node.js or on the web and need to use
> some functionality that's part of the Node.js core but not the web
> platform (or vice versa?). I could be wrong about that, but I've
> tried, especially in v2, to only have the build-side code do things
> I'm confident are correct in all cases.
What does "not directly correspond to Guix packages" mean here?  At the
very least Node must have a way of finding them, probably somewhere in
lib/, no?

While I did say we may want to rewrite these dependencies in
‘configure’, perhaps we can find a middle ground in which we only do so
when a given argument is there.  ant-build-system does that for
instance, so as to cope with packages that actually have no ant build
spec.  WDYT?

> The other issue is that deleting packages with no matching input by 
> default would replicate some of the drawbacks of the current
> `(delete 'configure)` approach. I think it's better to have an
> explicit list of dependencies that Guix is deleting. If eventually we
> package all of the missing dependencies for Guix, it would be much
> easier to find the packages that ought to use it. And, in the highly
> dynamic JavaScript world, I'm reluctant to give up one of the few
> static checks we have. If a missing package that really was required
> were automatically deleted from "package.json", it seems the failure
> mode would by a mysterious runtime error, potentially many steps down
> a dependency chain.
I get where you're coming from, but OTOH if you were to change parts of
the package to suit your own needs – like adding or omitting some
inputs, you now have two places to change, and they're in a tricky
relationship with another.  I think this is setting up some very error-
prone boilerplate.

I hope that in most cases the test suite will cover the important use
cases, but that's not always possible (e.g. with flaky tests,
networking, missing test inputs...)  Furthermore, for better or for
worse, stack traces did become the gold standard of debugging long ago,
so even packages missing "deep down" would probably soon be dug out if
they lead to hard errors.  Soft errors in Javascript though...





^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: Use #:absent-dependencies.
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: " Philip McGrath
  2021-11-20  7:44               ` Liliana Marie Prikler
@ 2021-11-23 11:04               ` Jelle Licht
  2021-11-28 19:35                 ` [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Timothy Sample
  1 sibling, 1 reply; 60+ messages in thread
From: Jelle Licht @ 2021-11-23 11:04 UTC (permalink / raw)
  To: Philip McGrath, 51838; +Cc: Liliana Marie Prikler

Hey Philip,

Philip McGrath <philip@philipmcgrath.com> writes:

> gnu/packages/node.scm (node-llparse-builder-bootstrap)[arguments]: Add
> `#:absent-dependencies`. Stop deleting the `'configure` phase.
> Add a new phase `#:delete-package-lock` to remove the
> problematic "package-lock.json".
> ---
>  gnu/packages/node.scm | 14 +++++++++++++-
>  1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
> index 98a51276e7..9d4903a8ca 100644
> --- a/gnu/packages/node.scm
> +++ b/gnu/packages/node.scm
> @@ -479,9 +479,21 @@ (define-public node-llparse-builder-bootstrap
>      (arguments
>       `(#:node ,node-bootstrap
>         #:tests? #f
> +       #:absent-dependencies
> +       `("@types/mocha"
> +         "@types/node"
> +         "mocha"
> +         "ts-node"
> +         "tslint"
> +         "typescript")
>         #:phases
>         (modify-phases %standard-phases
> -         (delete 'configure)
> +         (add-before 'configure 'remove-package-lock
> +           ;; Having package-lock.json seems to cause npm
> +           ;; to look for things on the internet in the configure phase,
> +           ;; even if we have them properly installed.

package-lock.json lists exact versions _and integrity hashes_; since it
seems unlikely that after node-build-system's finaggling we end up with
an identical hash, we will always have a mismatch and fetch 'proper'
sources online accordingly. As far as npm + package-lock.json are
concerned, we don't have them properly installed.

From what I have seen package-lock.json offers us no benefits (because
we track exact dependency information via the guix store) and can (as
you have seen) prevent builds from working. My 2c: always remove it in a
phase in the build system. It's simply a preference though, so your
please go with what you think is the right choice. You might want to
update the comment nonetheless so it's clear we know what's going on.

> +           (lambda args
> +             (delete-file-recursively "package-lock.json")))
>           (replace 'build
>             (lambda* (#:key inputs #:allow-other-keys)
>               (let ((esbuild (string-append (assoc-ref inputs "esbuild")
> -- 
> 2.32.0




^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-20 20:08 ` Timothy Sample
@ 2021-11-23 20:54   ` Pierre Langlois
  2021-11-28 19:59     ` Timothy Sample
  0 siblings, 1 reply; 60+ messages in thread
From: Pierre Langlois @ 2021-11-23 20:54 UTC (permalink / raw)
  To: Timothy Sample
  Cc: 51838, Pierre Langlois, Philip McGrath, Liliana Marie Prikler

[-- Attachment #1: Type: text/plain, Size: 1384 bytes --]

Hi Timothy,

Timothy Sample <samplet@ngyro.com> writes:

> Hi Philip,
>
> There are some really valuable changes in here, thanks!  I would like to
> start understanding and reviewing the changes so that we can get some of
> this good stuff merged in.  However, I have one question that I couldn’t
> answer from reading here or at <https://issues.guix.gnu.org/49946>: is
> there an agreement between you and Pierre that these patches are the
> “right” way to do native addons for Node.js.  More importantly, is the
> general plan that we merge these changes, and then Pierre rebases their
> Tree-sitter changes on top these?
>
> Pierre, maybe you could weigh in here?

The overall approach looks good to me, it's better than what I
originally proposed for sure :-).  That being said, I'm not very
familiar with the Node.js ecosystem so I don't know if it's necessarily
the right way, but I suspect the correct way for node isn't very Guix-y
so I'm not too worried about that.

It's on my TODO list to take another look at the patches as well :-),
then yes, I'm planning on rebasing my tree-sitter series on top.

> Sorry if I missed something.  I assume everything is OK, but I want to
> be sure before I start digging into the details of the patches –
> especially those first few more complicated ones.  :)

Thanks for taking a look!

Pierre

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 519 bytes --]

^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-20  4:33             ` [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument Philip McGrath
  2021-11-20  7:41               ` Liliana Marie Prikler
@ 2021-11-28 19:27               ` Timothy Sample
  1 sibling, 0 replies; 60+ messages in thread
From: Timothy Sample @ 2021-11-28 19:27 UTC (permalink / raw)
  To: Philip McGrath; +Cc: 51838, Liliana Marie Prikler

Hi Philip,

Philip McGrath <philip@philipmcgrath.com> writes:

> -  (define (resolve-dependencies package-meta meta-key)
> -    (fold (lambda (key+value acc)
> -            (match key+value
> -              ('@ acc)
> -              ((key . value) (acons key (hash-ref index key value) acc))))
> +  (define (resolve-dependencies meta-alist meta-key)
> +    (match (assoc-ref meta-alist meta-key)
> +      (#f
> +       '())
> +      (('@ . orig-deps)
> +       (fold (match-lambda*
> +               (('@ acc)
> +                acc)
> +                (((key . value) acc)
> +                 (if (member key absent-dependencies)
> +                     acc
> +                     (acons key (hash-ref index key value) acc))))
>            '()
> -          (or (assoc-ref package-meta meta-key) '())))
> +          orig-deps))))

There’s a lot of refactoring going here in addition to change at hand.
To me, it needlessly obscures the actual change (which is just the check
for membership in 'absent-dependencies', AFAICS).

>    (with-atomic-file-replacement "package.json"
>      (lambda (in out)
> -      (let ((package-meta (read-json in)))
> -        (assoc-set! package-meta "dependencies"
> -                    (append
> -                     '(@)
> -                     (resolve-dependencies package-meta "dependencies")
> -                     (resolve-dependencies package-meta "peerDependencies")))
> -        (assoc-set! package-meta "devDependencies"
> -                    (append
> -                     '(@)
> -                     (resolve-dependencies package-meta "devDependencies")))
> +      ;; It is unsafe to rely on 'assoc-set!' to update an
> +      ;; existing assosciation list variable:
> +      ;; see 'info "(guile)Adding or Setting Alist Entries"'.

Good catch!

> +      (let* ((package-meta (read-json in))
> +             (alist (match package-meta
> +                      ((@ . alist) alist)))

Why do we have to unwrap (and then re-wrap later on) the alist?  It
would be nice to explain that in the changelog.  Maybe it’s just me, but
I can’t figure it out!  :)

Thanks!


-- Tim




^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-23 11:04               ` Jelle Licht
@ 2021-11-28 19:35                 ` Timothy Sample
  0 siblings, 0 replies; 60+ messages in thread
From: Timothy Sample @ 2021-11-28 19:35 UTC (permalink / raw)
  To: Jelle Licht; +Cc: 51838, Philip McGrath, Liliana Marie Prikler

Hello!

Jelle Licht <jlicht@fsfe.org> writes:

> Philip McGrath <philip@philipmcgrath.com> writes:
>
>> Add a new phase `#:delete-package-lock` to remove the
>> problematic "package-lock.json".
>
> package-lock.json lists exact versions _and integrity hashes_; since it
> seems unlikely that after node-build-system's finaggling we end up with
> an identical hash, we will always have a mismatch and fetch 'proper'
> sources online accordingly. As far as npm + package-lock.json are
> concerned, we don't have them properly installed.
>
> From what I have seen package-lock.json offers us no benefits (because
> we track exact dependency information via the guix store) and can (as
> you have seen) prevent builds from working. My 2c: always remove it in a
> phase in the build system.

I’m inclined to agree with Jelle and Liliana.  I can’t imagine a
situation in which we would want the lock files.  We could be wrong, but
we can always adjust the build system later if something surprising
happens (e.g., ‘#:keep-lock-file?’ or whatever).


-- Tim




^ permalink raw reply	[flat|nested] 60+ messages in thread

* [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp.
  2021-11-23 20:54   ` Pierre Langlois
@ 2021-11-28 19:59     ` Timothy Sample
  0 siblings, 0 replies; 60+ messages in thread
From: Timothy Sample @ 2021-11-28 19:59 UTC (permalink / raw)
  To: Pierre Langlois; +Cc: 51838, Philip McGrath, Liliana Marie Prikler

Hello,

Pierre Langlois <pierre.langlois@gmx.com> writes:

> Hi Timothy,
>
> Timothy Sample <samplet@ngyro.com> writes:
>
>> More importantly, is the general plan that we merge these changes,
>> and then Pierre rebases their Tree-sitter changes on top these?
>>
>> Pierre, maybe you could weigh in here?
>
> The overall approach looks good to me, it's better than what I
> originally proposed for sure :-).  That being said, I'm not very
> familiar with the Node.js ecosystem so I don't know if it's necessarily
> the right way, but I suspect the correct way for node isn't very Guix-y
> so I'm not too worried about that.

The whole Node.js bundles NPM, which bundles node-gyp, which bundles a
fork of GYP [1] is not very Guix-y at all, no.  :/  This is one of those
problems (like bootstrapping GCC) that will take years of incremental
improvements and side projects and all that.

[1] Not to get too off topic, but isn’t “gyp” a slur?  How did Google
ever call something that?

> It's on my TODO list to take another look at the patches as well :-),
> then yes, I'm planning on rebasing my tree-sitter series on top.

Excellent!

>> Sorry if I missed something.  I assume everything is OK, but I want to
>> be sure before I start digging into the details of the patches –
>> especially those first few more complicated ones.  :)
>
> Thanks for taking a look!

I have an idea to simplify the patch series a bit: if we can answer my
question here <https://issues.guix.gnu.org/51838#57> and come to a
conclusion about deleting lock files
<https://issues.guix.gnu.org/51838#58>, I could merge the
‘#:absent-dependencies’ part of the patch series.  I think this might
make future re-rolls easier and help rein in the scope a bit.

Thoughts?  Philip?

Thanks!


-- Tim




^ permalink raw reply	[flat|nested] 60+ messages in thread

end of thread, other threads:[~2021-11-28 20:00 UTC | newest]

Thread overview: 60+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-14 12:41 [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
2021-11-14 12:58 ` [bug#51838] [PATCH 01/11] gnu: node: Avoid duplicating build phases Philip McGrath
2021-11-14 13:04   ` [bug#51838] [PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
2021-11-14 13:04     ` [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
2021-11-14 20:44       ` Liliana Marie Prikler
2021-11-20  4:26         ` Philip McGrath
2021-11-20  4:33           ` [bug#51838] [PATCH v2 01/26] gnu: node: Avoid duplicating build phases Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 02/26] gnu: node: Update to 10.24.1 for bootstrapping Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 03/26] gnu: node: Patch shebangs in node_modules Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 04/26] gnu: node: Add an npmrc file to set nodedir Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument Philip McGrath
2021-11-20  7:41               ` Liliana Marie Prikler
2021-11-20 17:04                 ` Philip McGrath
2021-11-20 20:24                   ` Liliana Marie Prikler
2021-11-28 19:27               ` [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Timothy Sample
2021-11-20  4:33             ` [bug#51838] [PATCH v2 06/26] gnu: node-semver-bootstrap: Use #:absent-dependencies Philip McGrath
2021-11-20  7:43               ` Liliana Marie Prikler
2021-11-20  4:33             ` [bug#51838] [PATCH v2 07/26] gnu: node-ms-bootstrap: " Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 08/26] gnu: node-binary-search-bootstrap: " Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 09/26] gnu: node-debug-bootstrap: " Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: " Philip McGrath
2021-11-20  7:44               ` Liliana Marie Prikler
2021-11-20 17:09                 ` Philip McGrath
2021-11-23 11:04               ` Jelle Licht
2021-11-28 19:35                 ` [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Timothy Sample
2021-11-20  4:33             ` [bug#51838] [PATCH v2 11/26] gnu: node-llparse-frontend-bootstrap: Use #:absent-dependencies Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 12/26] gnu: node-llparse-bootstrap: " Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 13/26] gnu: node-semver: " Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 14/26] gnu: node-wrappy: " Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 15/26] gnu: node-once: " Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 16/26] gnu: node-irc-colors: " Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 17/26] gnu: node-irc: " Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 18/26] guix: node-build-system: Add optional #:libuv? argument Philip McGrath
2021-11-20  7:46               ` Liliana Marie Prikler
2021-11-20 17:16                 ` Philip McGrath
2021-11-20  4:33             ` [bug#51838] [PATCH v2 19/26] gnu: Add node-inherits Philip McGrath
2021-11-20  4:34             ` [bug#51838] [PATCH v2 20/26] gnu: Add node-safe-buffer Philip McGrath
2021-11-20  4:34             ` [bug#51838] [PATCH v2 21/26] gnu: Add node-string-decoder Philip McGrath
2021-11-20  4:34             ` [bug#51838] [PATCH v2 22/26] gnu: Add node-readable-stream Philip McGrath
2021-11-20  4:34             ` [bug#51838] [PATCH v2 23/26] gnu: Add node-nan Philip McGrath
2021-11-20  4:34             ` [bug#51838] [PATCH v2 24/26] gnu: Add node-openzwave-shared Philip McGrath
2021-11-20  4:34             ` [bug#51838] [PATCH v2 25/26] gnu: Add node-addon-api Philip McGrath
2021-11-20  4:34             ` [bug#51838] [PATCH v2 26/26] gnu: Add node-sqlite3 Philip McGrath
2021-11-20  7:48               ` Liliana Marie Prikler
2021-11-20  5:10           ` [bug#51838] [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp Philip McGrath
2021-11-20  7:28             ` Liliana Marie Prikler
2021-11-14 13:04     ` [bug#51838] [PATCH 04/11] gnu: Add node-inherits Philip McGrath
2021-11-14 13:04     ` [bug#51838] [PATCH 05/11] gnu: Add node-safe-buffer Philip McGrath
2021-11-14 13:04     ` [bug#51838] [PATCH 06/11] gnu: Add node-string-decoder Philip McGrath
2021-11-14 13:04     ` [bug#51838] [PATCH 07/11] gnu: Add node-readable-stream Philip McGrath
2021-11-14 13:04     ` [bug#51838] [PATCH 08/11] gnu: Add node-nan Philip McGrath
2021-11-14 13:04     ` [bug#51838] [PATCH 09/11] gnu: Add node-openzwave-shared Philip McGrath
2021-11-14 13:04     ` [bug#51838] [PATCH 10/11] gnu: Add node-addon-api Philip McGrath
2021-11-14 13:04     ` [bug#51838] [PATCH 11/11] gnu: Add node-sqlite3 Philip McGrath
2021-11-20 17:38 ` [bug#51838] [PATCH v2 04/26] gnu: node: Add an npmrc file to set nodedir Timothy Sample
2021-11-20 19:55 ` [bug#51838] [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp Timothy Sample
2021-11-20 20:04 ` Timothy Sample
2021-11-20 20:08 ` Timothy Sample
2021-11-23 20:54   ` Pierre Langlois
2021-11-28 19:59     ` Timothy Sample

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).