unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* bringing npm packages to Guix
@ 2022-11-19 21:27 zamfofex
  2022-11-21 11:13 ` Ludovic Courtès
  0 siblings, 1 reply; 3+ messages in thread
From: zamfofex @ 2022-11-19 21:27 UTC (permalink / raw)
  To: guix-devel

Hello, Guix!

A few months ago, I picked up the work towards importing npm packages to Guix by Jelle et al. in the hopes of continuing it, and I felt disheartened when I concluded that it does indeed seem like a very large part of npm is necessary to build even a simple package like jQuery.

But, recently, I decided to spend some more time thinking about it, and I realized that not all of the ‘devDependencies’ of a given package are actually necessary to build it. A lot of packages have ‘devDependencies’ for things such as tests and watching the filesystem for changes for ease of development.

If you think about it, what the vast majority of npm packages need to build successfully comes down to (optionally) compile TypeScript, then (optionally) perform some kind of bundling or transpilation. And that doesn’t really require TypeScript or a build tool from npm, since esbuild (which is already packaged) can do both of those things by itself.

Now, taking away the ‘devDependencies’ when recursing the dependency tree makes the whole endeavor of importing an npm package seem *much less* difficult and scary! The biggest unfortunate issue is that for packages with a build step (TypeScript and/or bundling+transpilation), it is necessary to use esbuild ad‐hoc (i.e. in a case-by-case basis), because each package has its own build peculiarities, and esbuild doesn’t acknowledge them. (One solution could be to create an esbuild wrapper that mimics ‘tsc’ and other tools.)

With that approach in mind, I was able to package sucrase as a proof of concept! Note that sucrase doesn’t have a lot of transitive dependencies, so it was easy to just modify the phases of each package in an ad‐hoc way. However, also note that sucrase depends on itself, so I decided to bootstrap it with esbuild, then use the bootstrapped version to build sucrase again. (The bootstrapped sucrase seems to also work as expected!)

The import script I wrote, alongside the sucrase package itself can be found here:
- <https://gist.github.com/zamfofex/eac93bc0e00477a8b79f5ca4dc1a34ff>

Any kinds of thoughts or feedback on this would be appreciated!

A few notes of potential future work:

- Keep tests on packages, after packaging the necessary test libraries.
- Maybe: Write a wrapper for esbuild for mimicking ‘tsc’ and other tools. (As mentioned above.)
- Figure out a way to detect vendored dependencies and other forms of bundles in repositories.
- Polish the import script, and make it fit Guix’ style and formatting practices.
- Use the ‘version’ variable in the source field of package definitions when importing. (An extension of the item above.)


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: bringing npm packages to Guix
  2022-11-19 21:27 bringing npm packages to Guix zamfofex
@ 2022-11-21 11:13 ` Ludovic Courtès
  2022-11-21 22:52   ` zamfofex
  0 siblings, 1 reply; 3+ messages in thread
From: Ludovic Courtès @ 2022-11-21 11:13 UTC (permalink / raw)
  To: zamfofex; +Cc: guix-devel

Hi,

zamfofex <zamfofex@twdb.moe> skribis:

> A few months ago, I picked up the work towards importing npm packages to Guix by Jelle et al. in the hopes of continuing it, and I felt disheartened when I concluded that it does indeed seem like a very large part of npm is necessary to build even a simple package like jQuery.

Christine had that great summary of the npm mess at the time:

  https://dustycloud.org/blog/javascript-packaging-dystopia/

With all its shortcomings, I think it would still be useful to merge
Jelle’s npm importer (the “binary” importer), as discussed last year:

  https://lists.gnu.org/archive/html/guix-devel/2021-09/msg00245.html

Thumbs up to whoever submits it for inclusion!  :-)

(I won’t answer on ‘devDependencies’ issues because JS is outside my
area of expertise.)

Thanks,
Ludo’.


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: bringing npm packages to Guix
  2022-11-21 11:13 ` Ludovic Courtès
@ 2022-11-21 22:52   ` zamfofex
  0 siblings, 0 replies; 3+ messages in thread
From: zamfofex @ 2022-11-21 22:52 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

Hello! Thanks for responding.

The premise of the idea I’m proposing is that it wouldn’t be necessary to traverse into (most of) the ‘devDependencies’ if you build the packages ad‐hoc using esbuild. So, the idea is to sidestep the build process that the package has, and just invoke esbuild on the appropriate files of the package. My hope is that this is sufficient to bring most (or at least a lot of) useful packages from npm to Guix without the need of a binary importer.

I set up a package for sucrase as a proof of concept, since sucrase is a CLI tool that can be run and tried out easily (as opposed to jQuery, which is a library meant to be used in browsers). However, note that following my proposed approach, packaging jQuery becomes entirely trivial — it is a matter of simply running esbuild on jQuery’s input files. For the sake of example, I have included a Guix package I created for jQuery in the GitHub Gist I sent above. (And I can anecdotally verify that it works when tring it on a browser.) <https://gist.github.com/zamfofex/eac93bc0e00477a8b79f5ca4dc1a34ff#file-jquery-scm>

I will admit that I cheated a bit, since I used the currently unreleased 4.0.0 version, as opposed to the latest released 3.x versions, since the realeased 3.x versions use the old and outdated AMD format (“asynchronous module definition”) for representing dependencies between files in a project. (See <https://github.com/evanw/esbuild/issues/1035>) However, most packages nowadays have already moved away from AMD (and jQuery is getting there), so this shouldn’t be an issue in practice.


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2022-11-21 22:53 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-19 21:27 bringing npm packages to Guix zamfofex
2022-11-21 11:13 ` Ludovic Courtès
2022-11-21 22:52   ` zamfofex

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).