State of JS runtime environment in Guix

State of JS runtime environment in Guix

[Ashvith Shetty]

[-- Attachment #1: Type: text/plain, Size: 2002 bytes --]

Hello,
I've not come across any new issues in a long time with regards to the 
state of JS runtime environment in Guix, hence this mail.

 From my observation:

- NodeJS is severely lagging behind at 18.9.0. The current version is 
23.1.0, I've tried bumpin to 22.0.0, incrementing per patch version, 
but I had to give up, because of the failing test cases which continued 
even after disabling several test cases.

- NodeJS-bootstrap is also severely lagging behind at 10.24.1. It 
should be bumped to somewhere in between 14.21.3 or 16.20.2 - I don't 
know much about bootstraping, but my recent attempt to bump failed 
again, because of the test cases - in this case, failures from 
sequential tests.

- Some of the related packages (llhttp-bootstrap, llhttp) seem to be 
contaminated

- Internal source dependencies (that is, stuff inside `deps/` folder, 
as opposed to the inputs themselves) are painful to deal with, because 
they re-build every time on test-failure and take a lot of resource - 
some of them can be replaced with inputs, thankfully, but some of these 
dependencies (v8, simdutf, simdjson, uvwasi, etc) can contribute to 
increased build time when testing locally. Now, I was able to build 
some of them as a separate package variable, but the main culprit here 
(v8) still isn't a external source dependency - for this, I've also 
opened a issue at GitHub: <https://github.com/nodejs/node/issues/53509>

- Deno's patch needs to be bumped, but the work is still pending, and I 
don't really know much about Rust or Cargo

- I've tried adding Bun, but gave up later because of build failures. 
Nixpkgs simply ELF-patches the binaries, which is probably unacceptable 
in Guix?

- Yarn, PNPM and Lerna are still not available for NodeJS

 - There is a lack of JavaScript or NodeJS-specific maintainer team

Given how the JS development tool is to be in a poor state, I wanted to 
have a discussion about the actions that can be taken to improve this 
situation.

Regards,
Ashvith


[-- Attachment #2: Type: text/html, Size: 2358 bytes --]

Re: State of JS runtime environment in Guix

[Jelle Licht]

Hi Ashvith,


Ashvith Shetty <ashvithshetty10@gmail.com> writes:

> Hello,
> I've not come across any new issues in a long time with regards to the 
> state of JS runtime environment in Guix, hence this mail.
>
>  From my observation:
>
> - NodeJS is severely lagging behind at 18.9.0. The current version is 
> 23.1.0, I've tried bumpin to 22.0.0, incrementing per patch version, 
> but I had to give up, because of the failing test cases which continued 
> even after disabling several test cases.

I've just sent a patch series (https://issues.guix.gnu.org/74187)
bumping our node-lts package to 20.18.0. This would be a first step to
bumping it to 22.X.Y, after which I think it should be fairly easy to
create and maintain an up-to-date node-current package.

> - NodeJS-bootstrap is also severely lagging behind at 10.24.1. It 
> should be bumped to somewhere in between 14.21.3 or 16.20.2 - I don't 
> know much about bootstraping, but my recent attempt to bump failed 
> again, because of the test cases - in this case, failures from 
> sequential tests.

AFAICT node@10 is the last version that can still be used to properly
bootstrap the llhttp packages without itself depending on llhttp; We use
esbuild to compile all the TypeScript to JS, and subsequently use
node@10 to run this JS which in turn generates the C code that is used
to build llhttp (and subsequently, Node versions after 10).

If this all sounds like an example of the XY problem, that's because it
is :). What we really need is _some_ way to (re)generate the C code that
is a part of llhttp. If there is some other (minimalist) JS runtime that
could do this, that would be a major improvement over what we currently
do.

>
> - Some of the related packages (llhttp-bootstrap, llhttp) seem to be 
> contaminated

WDYM exactly?

>
> - Internal source dependencies (that is, stuff inside `deps/` folder, 
> as opposed to the inputs themselves) are painful to deal with, because 
> they re-build every time on test-failure and take a lot of resource - 
> some of them can be replaced with inputs, thankfully, but some of these 
> dependencies (v8, simdutf, simdjson, uvwasi, etc) can contribute to 
> increased build time when testing locally. Now, I was able to build 
> some of them as a separate package variable, but the main culprit here 
> (v8) still isn't a external source dependency - for this, I've also 
> opened a issue at GitHub: <https://github.com/nodejs/node/issues/53509>

Are there any things you've already started working on? I'd love to see
what we can do to make this happen.

> - Deno's patch needs to be bumped, but the work is still pending, and I 
> don't really know much about Rust or Cargo
>
> - I've tried adding Bun, but gave up later because of build failures. 
> Nixpkgs simply ELF-patches the binaries, which is probably unacceptable 
> in Guix?
>
> - Yarn, PNPM and Lerna are still not available for NodeJS

I don't use any of the non-npm options, but could you check if they can
be installed using (e.g.) `npm install -g --prefix="$HOME/.local` after
applying the patch series that bumps node to v20? 

Not really relevant for guix proper, but earlier this year I pushed the
npm-binary importer that could perhaps be used to package some of
{yarn,pnpm,lerna} in a channel or local guix.scm.
>
>  - There is a lack of JavaScript or NodeJS-specific maintainer team
>
> Given how the JS development tool is to be in a poor state, I wanted to 
> have a discussion about the actions that can be taken to improve this 
> situation.

Agreed. I think the first order of business should be getting up-to-date
package(s) for Node.js.
From there, we could branch out to several distinct topics, such as:

- Simplifying the bootstrap story of llhttp
- Unbundling more dependencies in the Node.js package
- Keeping a secure LTS version of Node.js in the package repo.
- Keeping a secure and recent version of Node.js in the package repo.

Thanks for starting this discussion
- Jelle

Re: State of JS runtime environment in Guix

[Ashvith Shetty]

[-- Attachment #1: Type: text/plain, Size: 2800 bytes --]

Hello Jelle,

I've just sent a patch series (https://issues.guix.gnu.org/74187) bumping
our node-lts package to 20.18.0. This would be a first step to bumping it
to 22.X.Y, after which I think it should be fairly easy to create and
maintain an up-to-date node-current package.


Thank you for your patches. This will also be a good exercise for me to
read package definitions for `node-lts`.

AFAICT node@10 is the last version that can still be used to properly
bootstrap the llhttp packages without itself depending on llhttp; We use
esbuild to compile all the TypeScript to JS, and subsequently use node@10
to run this JS which in turn generates the C code that is used to build
llhttp (and subsequently, Node versions after 10). If this all sounds like
an example of the XY problem, that's because it is :). What we really need
is _some_ way to (re)generate the C code that is a part of llhttp. If there
is some other (minimalist) JS runtime that could do this, that would be a
major improvement over what we currently do.


I had this in my head for quite a while, and maybe I could be wrong here,
but could we use anything like QuickJS, tinyJS, etc? Since they do not
support NPM, I highly doubt if they can be used as a substitute.

WDYM exactly?


Perhaps I have misread your comments at https://issues.guix.gnu.org/71581#13
.
I was under the impression that llhttp may have some
TypeScript/JavaScript/C
code meddling with building `node-lts`.

Are there any things you've already started working on? I'd love to see
what we can do to make this happen.


Unfortunately, no. Working with compilers and interpreters is beyond my
expertise. I have just recently started to delve into system programming
seriously.

I don't use any of the non-npm options, but could you check if they can be
installed using (e.g.) `npm install -g --prefix="$HOME/.local` after
applying the patch series that bumps node to v20? Not really relevant for
guix proper, but earlier this year I pushed the npm-binary importer that
could perhaps be used to package some of {yarn,pnpm,lerna} in a channel or
local guix.scm.


Personally, I liked using `pnpm`, but I guess that this is not that serious
of
an issue. I think I'll try the command you've shared.

I was also of the opinion that we may remove `http_parser`, but it looks
like
there are several dependencies - some have moved over to `llhttp`, but some
packages, like `psi` and `psi-plus` are still dependent on `qhttp`, which
in turn
depends on `http_parser`.

With respect to removing internal source dependencies, do you think it
would be
possible to remove dependency for `simdutf` and `uvwasi`? I think I've
written
package definitions for both of them earlier - maybe I could rewrite them
again, and
try adding them as inputs this time.

Regards,
Ashvith

[-- Attachment #2: Type: text/html, Size: 4567 bytes --]