bug#34526: Updating node.js

bug#34526: Updating node.js

[Daniel Gerber]

[-- Attachment #1: Type: text/plain, Size: 58 bytes --]

Trying to build the current upstream version, 11.10.0...


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: WIP 11.10.0 --]
[-- Type: text/x-patch, Size: 2905 bytes --]

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index a0221601d..9d35765eb 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -45,26 +45,17 @@
 (define-public node
   (package
     (name "node")
-    (version "9.11.1")
+    (version "11.10.0")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://nodejs.org/dist/v" version
-                                  "/node-v" version ".tar.gz"))
+                                  "/node-v" version ".tar.xz"))
               (sha256
                (base32
-                "1vjh9zvw7wkdz6b0l99ya7mqjk0l8lbg9isr1q8rxwp400dhkk32"))
+                "1kyi35qkg7h5nk0cjdgy8pfm5v2qmzqc7k3app2c8226mrkarhlz"))
               (modules '((guix build utils)))
               (snippet
                `(begin
-                  ;; Remove bundled software.
-                  (for-each delete-file-recursively
-                            '("deps/cares"
-                              "deps/http_parser"
-                              "deps/icu-small"
-                              "deps/nghttp2"
-                              "deps/openssl"
-                              "deps/uv"
-                              "deps/zlib"))
                   (substitute* "Makefile"
                     ;; Remove references to bundled software.
                     (("deps/http_parser/http_parser.gyp") "")
@@ -74,7 +65,6 @@
                   #t))))
     (build-system gnu-build-system)
     (arguments
-     ;; TODO: Purge the bundled copies from the source.
      '(#:configure-flags '("--shared-cares"
                            "--shared-http-parser"
                            "--shared-libuv"
@@ -110,14 +100,6 @@
                (("'/usr/bin/env'")
                 (string-append "'" (which "env") "'")))

-             ;; FIXME: These tests depend on being able to install eslint.
-             ;; See https://github.com/nodejs/node/issues/17098.
-             (for-each delete-if-exists
-                       '("test/parallel/test-eslint-alphabetize-errors.js"
-                         "test/parallel/test-eslint-buffer-constructor.js"
-                         "test/parallel/test-eslint-documented-errors.js"
-                         "test/parallel/test-eslint-inspector-check.js"))
-
              ;; FIXME: These tests fail in the build container, but they don't
              ;; seem to be indicative of real problems in practice.
              (for-each delete-if-exists
@@ -184,9 +166,9 @@
      `(("c-ares" ,c-ares)
        ("http-parser" ,http-parser)
        ("icu4c" ,icu4c)
-       ("libuv" ,libuv-1.19)
+       ("libuv" ,libuv)
        ("nghttp2" ,nghttp2 "lib")
-       ("openssl" ,openssl)
+       ("openssl" ,openssl-next)
        ("zlib" ,zlib)))
     (synopsis "Evented I/O for V8 JavaScript")
     (description "Node.js is a platform built on Chrome's JavaScript runtime

[-- Attachment #3: Type: text/plain, Size: 2124 bytes --]


Notes on v11.10.0:
- it does support openssl@1.1.1
- it ships with libuv 1.26.0 (1.24.0 in guix)
- some previously bundled deps are absent from tarball
- NODE_EXPERIMENTAL_HTTP is a no-op / always defined

There is an issue with the alternative http parser, `llhttp`. The 
choice of parser is at runtime, and one compile flag, 
--shared-http-parser, configures both. Building fails with:

```
g++ -o 
/tmp/guix-build-node-11.10.0.drv-0/node-v11.10.0/out/Release/obj.target/node_lib/src/node_http_parser_llhttp.o 
../src/node_http_parser_llhttp.cc '-DNODE_ARCH="x64"' 
'-DNODE_PLATFORM="linux"' '-DNODE_WANT_INTERNALS=1' 
'-DV8_DEPRECATION_WARNINGS=1' '-DNODE_OPENSSL_SYSTEM_CERT_PATH=""' 
'-DHAVE_INSPECTOR=1' '-DNODE_REPORT' '-D__POSIX__' 
'-DNODE_USE_V8_PLATFORM=1' '-DNODE_HAVE_I18N_SUPPORT=1' 
'-DHAVE_OPENSSL=1' -I../src 
-I/tmp/guix-build-node-11.10.0.drv-0/node-v11.10.0/out/Release/obj/gen 
-I/tmp/guix-build-node-11.10.0.drv-0/node-v11.10.0/out/Release/obj/gen/include 
-I/tmp/guix-build-node-11.10.0.drv-0/node-v11.10.0/out/Release/obj/gen/src 
-I../deps/histogram/src -I../deps/v8/include 
-I../deps/brotli/c/include  -pthread -Wall -Wextra 
-Wno-unused-parameter -m64 -Wall -Wextra -Wno-unused-parameter -O3 
-fno-omit-frame-pointer -fno-rtti -fno-exceptions -std=gnu++1y 
-MMD -MF 
/tmp/guix-build-node-11.10.0.drv-0/node-v11.10.0/out/Release/.deps//tmp/guix-build-node-11.10.0.drv-0/node-v11.10.0/out/Release/obj.target/node_lib/src/node_http_parser_llhttp.o.d.raw 
-c
In file included from ../src/node_http_parser_impl.h:41:0,
                 from ../src/node_http_parser_llhttp.cc:3:
../src/http_parser_adaptor.h:5:21: fatal error: llhttp.h: No such 
file or directory
```

AFAIU, either llhttp has to be made a separate package and listed 
in inputs, or http-parser linked statically. Or should the missing 
-I../deps/llhttp/include argument be passed here somehow -- maybe 
patching node.gypi?

I have not tried to build 10.15.1(LTS), which presumably has the 
same issues as in #32095.

Also, should previous version branches (8.x, 9.x) be kept in guix?

In the meantime, these minor updates work fine:

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: v9.11.2 --]
[-- Type: text/x-patch, Size: 815 bytes --]

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index a0221601d..8ffd60133 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -45,14 +45,14 @@
 (define-public node
   (package
     (name "node")
-    (version "9.11.1")
+    (version "9.11.2")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://nodejs.org/dist/v" version
-                                  "/node-v" version ".tar.gz"))
+                                  "/node-v" version ".tar.xz"))
               (sha256
                (base32
-                "1vjh9zvw7wkdz6b0l99ya7mqjk0l8lbg9isr1q8rxwp400dhkk32"))
+                "04y2dnbf6jl8j0ykfkdwhir09h274d13k843d7lqfz3bgyn4wj06"))
               (modules '((guix build utils)))
               (snippet
                `(begin

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #5: v8.15.0(LTS) --]
[-- Type: text/x-patch, Size: 692 bytes --]

diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index a0221601d..c52e21cf6 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -202,11 +202,11 @@ devices.")
   (package
     (inherit node)
     (name "node-lts")
-    (version "8.12.0")
+    (version "8.15.0")
     (source (origin
               (inherit (package-source node))
               (uri (string-append "https://nodejs.org/dist/v" version
                                   "/node-v" version ".tar.xz"))
               (sha256
                (base32
-                "16j1rrxkhmvpcw689ndw1raql1gz4jqn7n82z55zn63c05cgz7as"))))))
+                "0cy6lzk9sn545kkc0jviv0k0hn30kindrpkkkmv3zk2774rj71cn"))))))

bug#34526: Updating node.js

[Jelle Licht]

Daniel Gerber <dg@atufi.org> writes:

> Notes on v11.10.0:
> - it does support openssl@1.1.1
> - it ships with libuv 1.26.0 (1.24.0 in guix)
> - some previously bundled deps are absent from tarball
> - NODE_EXPERIMENTAL_HTTP is a no-op / always defined
>
> There is an issue with the alternative http parser, `llhttp`. The
> choice of parser is at runtime, and one compile flag,
> --shared-http-parser, configures both. Building fails with:

> [snip]

> ../src/http_parser_adaptor.h:5:21: fatal error: llhttp.h: No such
> file or directory
> ```
>
> AFAIU, either llhttp has to be made a separate package and listed
> in inputs, or http-parser linked statically. Or should the missing
> -I../deps/llhttp/include argument be passed here somehow -- maybe
> patching node.gypi?

It seems that llhttp includes a build step for generating C-files using
TypeScript, making it a non-starter for proper packaging in Guix.

See https://github.com/nodejs/llhttp/issues/14 for more details, but
sadly no solution.

>
> I have not tried to build 10.15.1(LTS), which presumably has the
> same issues as in #32095.
>
> Also, should previous version branches (8.x, 9.x) be kept in guix?

As long as they are still supported by upstream, I see no issue with
this. The 8.x LTS is still maintained through the end of 2019.

I am not sure the 9.X series is still supported. If not, it might make
more sense to remove it instead of updating it.

bug#34526: Updating node.js

[Björn Höfling]

[-- Attachment #1: Type: text/plain, Size: 341 bytes --]

On Mon, 18 Feb 2019 21:50:41 +0100
Jelle Licht <jlicht@fsfe.org> wrote:

> See https://github.com/nodejs/llhttp/issues/14 for more details, but
> sadly no solution.

Thanks for looking into these things, really sounds sad. It would be
nice if the JavaScript/node.js people would care more about
bootstrapping from source.

Björn

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

bug#34526: Updating node.js

[Daniel Gerber]

[-- Attachment #1: Type: text/plain, Size: 386 bytes --]

 
2019-02-18, Jelle Licht: 
 
> It seems that llhttp includes a build step for generating 
> C-files using TypeScript, making it a non-starter for proper 
> packaging in Guix. 
> 
> See https://github.com/nodejs/llhttp/issues/14 for more details, 
> but sadly no solution. 

What about statically linking llhttp's C "sources" included in 
node?   Building v11.10.0 succeeds with this:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: node-llhttp.patch --]
[-- Type: text/x-patch, Size: 383 bytes --]

diff --git a/node.gypi b/node.gypi
index c07b5ea704..c08493efbe 100644
--- a/node.gypi
+++ b/node.gypi
@@ -174,6 +174,12 @@
       ],
     } ],
 
+    [ 'node_shared_http_parser=="true"', {
+      'dependencies': [
+        'deps/llhttp/llhttp.gyp:llhttp'
+      ],
+    } ],
+
     [ 'node_shared_cares=="false"', {
       'dependencies': [ 'deps/cares/cares.gyp:cares' ],
     }],

bug#34526: Updating node.js

[Daniel Gerber]

[-- Attachment #1: Type: text/plain, Size: 47 bytes --]

I mean, it builds after also updating libuv: 


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: gnu-libuv-update-1.26.0.patch --]
[-- Type: text/x-patch, Size: 785 bytes --]

--- gnu/packages/libevent.scm.orig	2019-02-13 10:04:31.913458810 +0100
+++ gnu/packages/libevent.scm	2019-02-19 13:30:49.496780516 +0100
@@ -124,14 +124,14 @@
 (define-public libuv
   (package
     (name "libuv")
-    (version "1.24.0")
+    (version "1.26.0")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://dist.libuv.org/dist/v" version
                                   "/libuv-v" version ".tar.gz"))
               (sha256
                (base32
-                "01pg0zsfr8mxlpipkbpw0dpsl26x5s966f5br7dx9ac29abk419q"))))
+                "1rqlh0ag02fni8ildz3anc26z9vz69vwf5qmzsid2gvzzfkigy6a"))))
     (build-system gnu-build-system)
     (arguments
      '(;; XXX: Some tests want /dev/tty, attempt to make connections, etc.

[-- Attachment #3: Type: text/plain, Size: 61 bytes --]


Then, tests from test/cctest/test_inspector_socket.cc fail.

bug#34526: Updating node.js

[Jelle Licht]

Daniel Gerber <dg@atufi.org> writes:

>   [snip]
> What about statically linking llhttp's C "sources" included in
> node?   Building v11.10.0 succeeds with this:
>

You could do this, of course, but afaics this is not acceptable for
inclusion in Guix proper.

I don't really see any way forward between convincing the fine node
folks to see the 'error of their ways', or to implement a ABI-compatible
replacement for llhttp that we can actually bootstrap.

bug#34526: Updating node.js

[Daniel Gerber]

Hi,

2019-02-20, Jelle Licht:
> Daniel Gerber <dg@atufi.org> writes:
>
>>   [snip]
>> What about statically linking llhttp's C "sources" included in
>> node?   Building v11.10.0 succeeds with this:
>
> You could do this, of course, but afaics this is not acceptable 
> for
> inclusion in Guix proper.
>
> I don't really see any way forward between convincing the fine 
> node
> folks to see the 'error of their ways', or to implement a 
> ABI-compatible
> replacement for llhttp that we can actually bootstrap.

Although I would prefer the convincing-the-fine-node-folks 
solution, here are two more ways to avoid dropping node with the 
EOL of 8.x(LTS) at the end of 2019.

- Remove llhttp and keep only the "legacy" http-parser, or

- Accept to bootstrap it -- I mean use intermediary self-compiling 
  steps, like ccl, golang, java, or haskell do.
The build-time dependencies are: node@11.x -> llhttp -> ts-node -> 
typescript -> self (typescript), plus quite a few npm packages.
It seems that node@8.x or 9.x should be a native-input to later 
versions, but I do not know enough of Guile / Guix packaging to do 
it myself anytime soon.

bug#34526: Updating node.js

[Christopher Lemmer Webber]

Daniel Gerber writes:

> Hi,
>
> 2019-02-20, Jelle Licht:
>> Daniel Gerber <dg@atufi.org> writes:
>>
>>>   [snip]
>>> What about statically linking llhttp's C "sources" included in
>>> node?   Building v11.10.0 succeeds with this:
>>
>> You could do this, of course, but afaics this is not acceptable for
>> inclusion in Guix proper.
>>
>> I don't really see any way forward between convincing the fine node
>> folks to see the 'error of their ways', or to implement a
>> ABI-compatible
>> replacement for llhttp that we can actually bootstrap.
>
> Although I would prefer the convincing-the-fine-node-folks solution,
> here are two more ways to avoid dropping node with the EOL of 8.x(LTS)
> at the end of 2019.
>
> - Remove llhttp and keep only the "legacy" http-parser, or
>
> - Accept to bootstrap it -- I mean use intermediary self-compiling
> steps, like ccl, golang, java, or haskell do.
> The build-time dependencies are: node@11.x -> llhttp -> ts-node ->
> typescript -> self (typescript), plus quite a few npm packages.
> It seems that node@8.x or 9.x should be a native-input to later
> versions, but I do not know enough of Guile / Guix packaging to do it
> myself anytime soon.

Hello,

Went through the process of trying to update node myself, not having
remembered this bug.  Ran into the same issue.

The bug was closed; I doubt we are going to convince the Node folks.

Quite a few high-importance projects rely on Node at this point, and we
are running an out of date Node which I suspect probably has quite a few
insecurities.

Our version of Node:   v10.16.0
LTS Node:              v12.13.0
Latest Node:           v13.1.0

One way or another, we will probably need to update.  Both Chromium and
Icecat depend on Node at this point.  I'm not sure if either of them use
Node in any active way that an insecruity could manifest or if it's
"just for packaging" but I think there's good reason to be nervous about
being so out of date.

bug#34526: Updating node.js

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 2252 bytes --]

Christopher Lemmer Webber <cwebber@dustycloud.org> writes:

> Daniel Gerber writes:
>
>> Hi,
>>
>> 2019-02-20, Jelle Licht:
>>> Daniel Gerber <dg@atufi.org> writes:
>>>
>>>>   [snip]
>>>> What about statically linking llhttp's C "sources" included in
>>>> node?   Building v11.10.0 succeeds with this:
>>>
>>> You could do this, of course, but afaics this is not acceptable for
>>> inclusion in Guix proper.
>>>
>>> I don't really see any way forward between convincing the fine node
>>> folks to see the 'error of their ways', or to implement a
>>> ABI-compatible
>>> replacement for llhttp that we can actually bootstrap.
>>
>> Although I would prefer the convincing-the-fine-node-folks solution,
>> here are two more ways to avoid dropping node with the EOL of 8.x(LTS)
>> at the end of 2019.
>>
>> - Remove llhttp and keep only the "legacy" http-parser, or
>>
>> - Accept to bootstrap it -- I mean use intermediary self-compiling
>> steps, like ccl, golang, java, or haskell do.
>> The build-time dependencies are: node@11.x -> llhttp -> ts-node ->
>> typescript -> self (typescript), plus quite a few npm packages.
>> It seems that node@8.x or 9.x should be a native-input to later
>> versions, but I do not know enough of Guile / Guix packaging to do it
>> myself anytime soon.
>
> Hello,
>
> Went through the process of trying to update node myself, not having
> remembered this bug.  Ran into the same issue.
>
> The bug was closed; I doubt we are going to convince the Node folks.
>
> Quite a few high-importance projects rely on Node at this point, and we
> are running an out of date Node which I suspect probably has quite a few
> insecurities.
>
> Our version of Node:   v10.16.0
> LTS Node:              v12.13.0
> Latest Node:           v13.1.0
>
> One way or another, we will probably need to update.  Both Chromium and
> Icecat depend on Node at this point.  I'm not sure if either of them use
> Node in any active way that an insecruity could manifest or if it's
> "just for packaging" but I think there's good reason to be nervous about
> being so out of date.

Node 10.x is maintained until April 2021 though:

https://nodejs.org/en/about/releases/

...so we still have some time to figure out how to bootstrap Node 12.x
and later.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#34526: Updating node.js

[Christopher Lemmer Webber]

Marius Bakke writes:

> Christopher Lemmer Webber <cwebber@dustycloud.org> writes:
>
>> Daniel Gerber writes:
>>
>>> Hi,
>>>
>>> 2019-02-20, Jelle Licht:
>>>> Daniel Gerber <dg@atufi.org> writes:
>>>>
>>>>>   [snip]
>>>>> What about statically linking llhttp's C "sources" included in
>>>>> node?   Building v11.10.0 succeeds with this:
>>>>
>>>> You could do this, of course, but afaics this is not acceptable for
>>>> inclusion in Guix proper.
>>>>
>>>> I don't really see any way forward between convincing the fine node
>>>> folks to see the 'error of their ways', or to implement a
>>>> ABI-compatible
>>>> replacement for llhttp that we can actually bootstrap.
>>>
>>> Although I would prefer the convincing-the-fine-node-folks solution,
>>> here are two more ways to avoid dropping node with the EOL of 8.x(LTS)
>>> at the end of 2019.
>>>
>>> - Remove llhttp and keep only the "legacy" http-parser, or
>>>
>>> - Accept to bootstrap it -- I mean use intermediary self-compiling
>>> steps, like ccl, golang, java, or haskell do.
>>> The build-time dependencies are: node@11.x -> llhttp -> ts-node ->
>>> typescript -> self (typescript), plus quite a few npm packages.
>>> It seems that node@8.x or 9.x should be a native-input to later
>>> versions, but I do not know enough of Guile / Guix packaging to do it
>>> myself anytime soon.
>>
>> Hello,
>>
>> Went through the process of trying to update node myself, not having
>> remembered this bug.  Ran into the same issue.
>>
>> The bug was closed; I doubt we are going to convince the Node folks.
>>
>> Quite a few high-importance projects rely on Node at this point, and we
>> are running an out of date Node which I suspect probably has quite a few
>> insecurities.
>>
>> Our version of Node:   v10.16.0
>> LTS Node:              v12.13.0
>> Latest Node:           v13.1.0
>>
>> One way or another, we will probably need to update.  Both Chromium and
>> Icecat depend on Node at this point.  I'm not sure if either of them use
>> Node in any active way that an insecruity could manifest or if it's
>> "just for packaging" but I think there's good reason to be nervous about
>> being so out of date.
>
> Node 10.x is maintained until April 2021 though:
>
> https://nodejs.org/en/about/releases/
>
> ...so we still have some time to figure out how to bootstrap Node 12.x
> and later.

That's fair.

I have a personal project that requires that I use a newer version of
Node (at least version 11).  So if anyone has a recipe on how to get
Node running, even the wrong way per Guix standards, maybe useful to
post to this bug in the meanwhile?  It might also still help advance
this bug.

bug#34526: Updating node.js

[Jelle Licht]

Christopher Lemmer Webber <cwebber@dustycloud.org> writes:

> That's fair.
>
> I have a personal project that requires that I use a newer version of
> Node (at least version 11).  So if anyone has a recipe on how to get
> Node running, even the wrong way per Guix standards, maybe useful to
> post to this bug in the meanwhile?  It might also still help advance
> this bug.

I tried my hand at building llhttp (again...) using our existing nodejs
+ Sucrase (an alternative TypeScript transpiler that does not have too
many dependencies), but it seems that the devs of llhttp use
semi-advanced TypeScript constructs that are at the moment not supported
by Sucrase's transformers. To be specific, I am talking about "Moving
types"[1].

Does anyone know of any other TypeScript transpilers? They do not need
to do typechecking, as long as they allow generation of (valid)
JavaScript files.

- Jelle

[1]: https://basarat.gitbooks.io/typescript/docs/types/moving-types.html