Re: Discussion: How to package rust crates now and in future?

[Hartmut Goebel <h.goebel@crazy-compilers.com>]

Hi Pjotr,

Am 19.12.20 um 07:42 schrieb Pjotr Prins:
> They are talking about librification of the language
> https://rust-lang.github.io/compiler-team/minutes/design-meeting/2020-03-12-shared-library-for-types/
> which may lead to a wider idea of libraries.

For your needs static libraries would suffice. Anyway: Unfortunately
this feature did not make i to the 2021 roadmap, AFAICS.

I'd be happy enable building crates when rust has some notion of
library. Until then I strongly propose to not build intermediate creates
per default.

> The language may still be changing - but I think it has gotten to the
> point that they can't change too much without hurting large software
> projects. It probably is a bad idea to mix different compilers to
> build crates in one software stack. Maybe I am misreading your idea,
> but we either have a 1.40 build or a 1.46 build.

Unfortunately this is not just an idea, but a sad reality :-( As of
today, the rust used by cargo-build-system is still 1.45, while some
crates (e.g. sequoia) require 1.46 language features. So in fact we
already/currently have a mixed stack. Maybe this is just since nobody
did update default rust to rust 1.46?!

Anyhow, my argument about rebuilding without any use still holds: When
upgrading rust, all crates will be be rebuild (which is part of guix's
concept) and the results are thrown away right away - which is a wast of
time and electrical power.


>>    6) This not only effect berlin, but also every user out there requiring
>>    a rebuild for some reason. This will lead to a very, very bad user
>>    experience - practically kicking out users with less powerful
>>    equipment.
> The Rust user experience is that Rust builds all crates or installs a
> single (static) binary. Even for Guix developers, the default is to
> use cargo because of the insane number of dependencies it typically
> pulls in. We package dependencies when we want to deploy, not to
> develop. 

I assume you are referring "Rust user" to someone using rust for
developing some rust software? I'm referring "user" to someone using
Guix for managing their system.

> The user here is a guix user, of course. So between compiler versions
> we rebuild the stack and then distribute binaries. No rebuilds get
> triggered normally. I think the use case you are referring to is users
> as developers (right?) who are actually used to waiting for cargo. I
> wait for cargo when I start a new Guix development environment.

FMPOV there is not much difference between a user and a developer here:
If substitutes are not available, a user - and this includes developers
- needs to build the packages. And even as a developer I don't want to
build the whole stack myself, but use substitutes - like I want to use
pre-built libraries for other languages.

My point is another one: While for programming languages supporting
(static) libraries the number of compiles is O(n), for rust – assuming
#:skip-build #f – it is O(n*(num_average_dependencies+1)) - since each
crate needs to compile all of its dependencies, even if 20 other creates
did this already.

To give you some numbers: sequoia 0.20.0, while directly depending on
only 55 crates, pulls in more then 430 crates (in words: four hundred
and thirty) and compiles about 280 (in words: two hundred and eighty),
including some duplicates compiles. Assuming each package depends on one
other crate (which is a low assumption), this will be 2*n=860 compiles.
Assuming each compile takes 30 seconds, and half of the packages need
rebuild, this is more the 3,5 hours - right away into the trashcan.


-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |