Discussion: How to package rust crates now and in future?

[Hartmut Goebel <h.goebel@crazy-compilers.com>]

[-- Attachment #1: Type: text/plain, Size: 3760 bytes --]

Hi,

I suggest to discuss this with a broader community.

*TIL: Shall rust packages be packaged with #:skip-build #t? Shall tests 
be run for all crates?*
*My vote: skip-build #t, no tests*

Background: I just submitted some patches for some crates, setting 
#:skip-build for those I touched or added. My rational:

1) Building crate "libraries" is of no use. Rust still has no notion of 
"libraries", neither shared not static. it does not even provide any 
means to use "object"-files from another package. All crates will be 
build again and again for each package using it. And you will notice 
that the output of most crates will be almost empty (only exception: if 
the crate build a program).

2) This is what the crates importer does: It sets skip-build for all 
packages it imports as dependencies. It also does not add the 
crate-build-dependencies for these packages. (Please note that while I 
made the crates importer to honor semver versions, this has already been 
prepared in other patches and was not argued about.


Am 17.12.20 um 21:08 schrieb Efraim Flashner:
> I'm in favor of building the packages anyway, it serves as a check that
> the inputs are actually correct.

When I started packaging crates, I did this too. But then I learned, 
that others do not. So we should define how this should be handled in 
the future - and adjust the importer accordingly.

This might not be possible - as there is another issue in the Rust 
ecosystem: The language is still moving fast.

3) If some packages requires rustc 1.46, while our default rustc is 
still 1.40, we need to add rustc-1.46 as an input to this package and to 
many of it's dependents. (AFAIR the package will even depend on *both* 
version then.) Now if we move on to rustc 1.50, extra care has to be 
taken to remove these dependencies.

Even worse: All packages depending on such a package on will also depend 
on rustc 1.46, and all changes to rustc 1.46 will trigger a rebuild - 
without *any* use.

4) Since (2) building rust packages costs *a lot* of resources: time, 
memory and electrical power. As an example, building sequoia takes about 
20 Minutes on my machine. Most of the time is spend compiling 
dependencies of dependencies. And all these dependencies of dependencies 
will be compiled over and over again.

5) *If* we decide to build dependencies, we should restrict this to 
*one* build. This means: either not run the tests or only do a test 
build. The reason is: When running the tests, all the code, including 
the dependencies, is compiled again with some "test" flag set. This will 
add yet another huge amount of time, memory and electrical power.

To give you some figure: A release and test build for sequoia takes 
about 45 minutes on my machine, requiring 9 GB of space in /tmp. So this 
is double the time if the release build only.

I can't imaging how many hours it would take to rebuild sequoia is one 
of the lower level dependencies changes - which is quite often the case 
in rust.

6) This not only effect berlin, but also every user out there requiring 
a rebuild for some reason. This will lead to a very, very bad user 
experience - practically kicking out users with less powerful equipment.

7) Given the rushing climate crisis, we MUST NOT waste this gigantic 
amount of electrical power. We are in a position of huge impact. If we 
decide to save power, hundreds of Guix users will save power (and 
money). If we decide to waste power, this will multiply by the number of 
Guix users.

*It's our responsibility to protect the earth!**Yes to #:skip-build #t.*

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |


[-- Attachment #2: Type: text/html, Size: 5235 bytes --]