Re: Idea: a meta language for (language) build systems - npm, Racket, Rust cargo

[Leo Prikler <leo.prikler@student.tugraz.at>]

Am Dienstag, den 01.06.2021, 13:28 +0200 schrieb Adriano Peluso:
> Il giorno mar, 01/06/2021 alle 13.03 +0200, Leo Prikler ha scritto:
> > Am Dienstag, den 01.06.2021, 12:52 +0200 schrieb Adriano Peluso:
> > > Il giorno mar, 01/06/2021 alle 11.11 +0200, Leo Prikler ha
> > > scritto:
> > > > > [...]
> > > Probably the Fractal package will depend on some others, so it's
> > > gonna be a collection 🤷️
> > > 
> > > Doesn't that happen already for traditional tarballs ?
> > We don't stuff tarball collections into packages.  We stuff inputs
> > into
> > packages and one input equals one tarball.
> > 
> > > >   We already drop all
> > > > vendored dependencies from tarballs, that aren't created by
> > > > Rust
> > > > et
> > > > al., this does the exact opposite.
> > > 
> > > I'm not sure I understand
> > > 
> > > This does the opposite ?
> > > 
> > > How so ?
> > Let's assume we form this sexp-pack and use it as input to some
> > package.  What happens?
> 
> we wouldn't use a sexp-pack as an input to a guix package in the same
> way as, as you noticed, we don't feed tarballs as inputs to guix
> packages
> 
> A Guix package doesn't depend on some tarballs. It depends on some
> other Guix packages
That's only half-true.  You can stuff origins into inputs and people do
that.
> In the same way, a Guix package wouldn't depend on a sexp-pack.
> 
> You can think of sexp-pack as an alternative format to tarball
> 
> With, maybe, some more metadata
Tarball + metadata is a standard of package management.[1]
> For example, a sexp-pack could contain a hash of itself and hashes of
> other _sexp-packs_ it depends on
> 
> Similarly to how, for example, python packages on pypi express
> dependecies on other packages in pypi
> 
> The difference is that, as far as I understand, python packages
> (those in pypi, not those in Guix) express dependencies in a somewhat
> loose way
> 
> Guix packages are stricter
> 
> sexp-packs could be stricter too, bringing part of the data
> reconciliation outside of Guix
I don't see the benefit here.  At best, sexp-packs are actual packages,
at worst they're closer to origins.  When bringing "dependencies" into
the mix, one might want to implement them as packages with near-empty
build, that have propagated-inputs.

I'm really struggling to see the use case here.  Packages are useful
abstractions to both developers and maintainers – the former need not
really care about the build step if all they want is an environment.

> A Guix importer could recursively import sexp-packs the same way the
> python importer...
You do know we can already import packages from JSON, right?

> I'm assuming that a Rust package can be built in a sane way, with
> dependencies properly sorted out.
> 
> I know that's possible for javascript packages, I'm not sure about
> Rust
The mathematical term you're seeking is "conjecture".

> Such a data/packages collection could be used by mainstream linux
> distributions too, as far as I understand 🤷️
Guess what, distros already use it.  Not the hypothetical data
collection you want to establish, the ones that actually exist as
published by pip, cpan, npm, cargo…

Whatever fancy intermediate format you want to think up, the fact that
this won't make anyone's lives easier follows from the data processing
inequality.  In other words, how you decide to lay out that data *does
not matter* as long as it can somehow be parsed (and remember that your
own tools will also need to parse it) and you don't omit anything – at
best importing data from your layout will be as good as importing it
from the original source.

In order to improve this situation, you would either need to improve
the tools in question (e.g. make it so that cargo actually produces
shared libraries) or replace them with technologies, that have the
properties we want (perhaps cargo by ninja).  If you furthermore want a
ubiquitous solution, guess what, that'd be Guix itself.  So you're
looking for something that's either similar to Guix or more
accommodating to Guix, that will get accepted by some language
community.  Note, that you can't really sell that solution on the same
grounds as Guix – as otherwise Guix would already be used instead – and
you will also have to compete against whatever technology already
exists, e.g. cargo or npm.

W.r.t. npm, I think an npm importer would already be a great
improvement.  IIUC the biggest problem with npm is the sheer number of
inputs.  We might also try to deduplicate dependencies, but alas, let's
not get ahead of ourselves.

Regards,
Leo

[1] https://xkcd.com/927/