Overhauling the cargo-build-system

[Efraim Flashner <efraim@flashner.co.il>]

[-- Attachment #1: Type: text/plain, Size: 3555 bytes --]

I've spent a lot of time over the past few months hacking at packaging
rust crates. I've found it time-consuming, soothing and, ultimately,
likely a waste of time. When we started with the cargo-build-system the
premise was 'packages are both libraries and source, we need them in
source form to build the next library in the chain' and the example was
our first three rust- packages, rust-quote, rust-syn and
rust-proc-macro2, all three of which depend on the other two.

I'd like to challenge the assumption that packages are both libraries
and source. A 'library' in rust compiles into one of three types: a
static library (libfoo.a), a shared library (libfoo.so), or a
'rust-library' (libfoo.rlib). A 'rust library', an rlib file, is like a
shared library but with rust-specific metadata. I haven't looked into
what exactly that means, but I can tell you practically how it works.
Through some in depth searching online it seems it is possible to
pre-compile rlibs and link them to a final binary, but it is not
possible to link an rlib to another rlib. What this means is currently
we have $(guix package -A ^rust- | wc -l) ~> 192 rust libraries packaged
that no one will ever use in any form EXCEPT as source inputs for a
future library or binary.

Let me repeat that. We have 192 rust packages that no one needs or
wants, except in source form.

We have a couple of packages/variables which are defined in source-form
only. We normally see them as inputs or native inputs as
("foo" ,(origin .... This is basically what we have with the rust
packages. Upstream to build a binary they call 'cargo build' and it
downloads the sources of the dependent libraries, and some of their
dependants based on the use-case, compiles them, and then compiles the
binary. In Guix, when we eventually have packaged enough libraries to
get to a target binary, we'll be pulling in hundreds of extra libraries
(transitive inputs of inputs) so a few dozen can be compiled.

PROPOSAL:
Change all the rust packages we have now to be source-only. Rename them
from rust-foo to rust-foo-src or rust-src-foo. The package 'alacritty'
is perhaps not a great example choice, but it's Cargo.lock file, which
describes the dependencies and their versions, lists 278 dependencies.
(Generally you can care only about the major and minor part of the
version.) It's package definition would explicitly list the 278
dependent libraries it wanted so they could be put in the 'guix-vendor'
directory like now. It makes for a very large package definition, but we
wouldn't have to ensure thousands of other rust libraries built so we
can throw away the results and build alacritty in the end. Another
package, starship, would have 124 rust dependencies, but I'm betting
there's a fair amount of overlap between them.

Since we never actually use the output of any of the rust packages we
currently have it doesn't really make sense to 'build them' per se as we
currently do.

$ tree $(guix build rust-proc-macro2)
/gnu/store/jjinj0dvvzqnlpn9li3nxcyrpv1fbbfp-rust-proc-macro2-0.4.30
`-- share
    `-- doc
         `-- rust-proc-macro2-0.4.30
             |-- LICENSE-APACHE
             `-- LICENSE-MIT

My plan is to hack on the cargo-build-system on a separate branch and
see if I can pull all of this together.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]