Re: Current state of cargo-build-system

[Ivan Petkov <ivanppetkov@gmail.com>]

Thank you all for the responses!

> Cycles.  Also, often Cargo.lock specifies exact versions of dependencies (in
> programs, at least)

Yes, this is pretty much the main issue I immediately hit when doing a naive
crate import, more on this further down.

> First, we'd have to find out what kind of things Cargo can build and how to
> detect it.

Cargo supports a subcommand called `metadata` which essentially dumps its
manifest in JSON format, which will include build scripts, binary outputs, etc.
I think this should give us enough information to figure out which packages have
targets to install (right now we try to install every single package which
doesn't work for libraries) among other things.

> After that, we can decide whether to compile libraries or just ship them as
> source (right now, we do the latter pretty often - almost everything in Guix
> Rust library packages ends up in the "src" output).
>  
> Getting the unit tests to run often requires breaking many cycles, so maybe
> skip that at first.

I tried breaking up cycles by hand for a couple days on another sample import,
but its just too tedious and error prone, especially when cycles span multiple
packages...  I'd really like for us to solve the issue altogether if possible.

Cargo (thankfully) checks and rejects an true dependency cycles, but its
notion of dev-dependencies (deps only used for running tests) is what throws
guix in for a loop. Dev-depencency cycles are permitted by cargo because it
builds the target crate independently, then it builds any extra dependencies
which may or may not depend on it, and then it builds a test crate which depends
on both. There are potentially legitimate cases which trigger this scenario
(such as testing your own public API through an upstream crate, see proc-macro2
for an example), or are just exacerbated by optional dependencies.

I think depending on the "src" output of crates is the right way to get things
started (this is what cargo essentially does anyway, it figures out what crates
are in the dependency, pulls their sources down and builds them), we can always
optimize this in the future to avoid recursively rebuilding crates over and
over.

However, it appears that guix still insists on building the entire package even
if we only depend on the "src" output. Is it possible to lazily build packages
based on the type of dependency? Is this something the build system can finagle,
or is this an inherent limitation to guix?

--Ivan