Re: Overhauling the cargo-build-system

[Martin Becze <mjbecze@riseup.net>]

Sorry for digging up and old issue, but i just saw commit
86e443c71d4d19e6f80cad9ca15b9c3a301c738c

> It makes for a very large package definition, but we
wouldn't have to ensure thousands of other rust libraries built so we

The whole point of package management is that you can use module
building blocks. By having to specify the sub-dependencies in a top
level definition kinda breaks the whole modular thing. In commit
86e443c71d4d19e6f80cad9ca15b9c3a301c738c all the inputs got removed for
all the libraries. So if I'm trying to use guix as package manager for a
rust project and I want to use one of the rust libraries in
crates-io.scm, how am i suppose to do this? I can't just include it as
an input to my project because now I have to look up all of it
dependencies as well?

> > PROPOSAL:
> > Change all the rust packages we have now to be source-only. Rename them
> > from rust-foo to rust-foo-src or rust-src-foo.

Is there no way we can do this and keep the have a dependancy graph?
 
On 2019-10-11 14:13, Efraim Flashner wrote:
> On Fri, Oct 11, 2019 at 12:33:18AM +0200, Ludovic Courtès wrote:
>> Hello!
>>
>> Efraim Flashner <efraim@flashner.co.il> skribis:
>>
>> > I'd like to challenge the assumption that packages are both libraries
>> > and source. A 'library' in rust compiles into one of three types: a
>> > static library (libfoo.a), a shared library (libfoo.so), or a
>> > 'rust-library' (libfoo.rlib).
>>
>> Why don’t we create .so files, then?  They have NEEDED and RUNPATH, so
>> that could work like for C, no?
>>
> 
> It is possible to force a library to produce .so files, but then we'd
> have to force all the other packages to link against them, and then at
> that point we're basically rewriting 'cargo build'. For some packages,
> like icecat, that does seem to be what they do.
> 
>> > Let me repeat that. We have 192 rust packages that no one needs or
>> > wants, except in source form.
>>
>> Ouch!  So the rlib file is never actually used?!
>>
> 
> As I understand it, it gets treated as a build artifact. It'll be
> compiled as libfoo_XXXXXXXXX.rlib and then, if some other program comes
> around as asks for "foo with feature baz" it'll be reused. At some point
> it gets compiled directly into the final binary, so it's really more of
> an intermediary stage.
> 
>> You said “it is not possible to link an rlib to another rlib”, but
>> that’s not necessarily a problem, it’s like .a libraries, no?
>>
> 
> if libfoo depends on libbar, then pre-building libbar_XXXXX.rlib doesn't
> allow us to call "rustc --crate-name foo src/lib.rs --crate-type lib
> --extern (string-append bar=(assoc-ref %build-inputs bar)
> /lib/libbar.rlib)" (note the --crate-type lib in there), it'll just not
> work and want to recompile libbar for just the few functions it uses
> from it. The only difference I've seen is if our foo is also a "binary
> crate", ie produces a binary, then it'll sometimes accept rlibs for its
> own rlib lib output.
> 
> Going back to forcing it to produce .so files, if we changed the
> crate-type to dylib then we'd get libfoo.so. The two parts of the
> problem is now we need to change all the other libraries and programs
> that depend on foo to take our libfoo.so and not expect an rlib, and
> that if foo has 3 options then we have to decide how to build libfoo so
> it's useful for everything (obviously all 3) but without dragging in
> everything (easiest with none of the 3), or to have some combination,
> which becomes a nightmare very quickly.
> 
> That sentence was way too long.
> 
>> > PROPOSAL:
>> > Change all the rust packages we have now to be source-only. Rename them
>> > from rust-foo to rust-foo-src or rust-src-foo.
>>
>> In the current scheme, can you actually do, say:
>>
>>   guix environment --ad-hoc rust rust-foo rust-bar
>>
>> and then (pseudo syntax):
>>
>>   rustc mystuff.rust -lfoo -lbar
>>
>> ?
> 
> Ignoring that we don't actually install the libraries here's a copy of
> a check phase I wrote trying to use installed libraries. I think in the
> end it did consume the rlibs as requested, but it never did link to
> them, it just absorbed them into the final binary.
> 
> (replace 'check
>   (lambda* (#:key inputs #:allow-other-keys)
>     (let ((ndarray    (assoc-ref inputs "ndarray"))
>           (rand       (assoc-ref inputs "rand"))
>           (rayon      (assoc-ref inputs "rayon"))
>           (serde      (assoc-ref inputs "serde"))
>           (serde-json (assoc-ref inputs "serde-json"))
>           (structopt  (assoc-ref inputs "structopt")))
>       (invoke "rustc" "--edition=2018" "--crate-name" "qtlreaper"
>               "src/lib.rs"
>               "-C" "opt-level=3"
>               "--test" "-C" "metadata=test" "-C" "extra-filename=-test"
>               "--out-dir" "target/release/deps"
>               "--extern" (string-append "ndarray=" ndarray
> "/lib/libndarray.rlib")
>               "--extern" (string-append "rand=" rand "/lib/librand.rlib")
>               "--extern" (string-append "rayon=" rayon "/lib/librayon.rlib")
>               "--extern" (string-append "serde=" serde "/lib/libserde.rlib")
>               "--extern" (string-append "serde_json=" serde-json
> "/lib/libserde_json.rlib")
>               "--extern" (string-append "structopt=" structopt
> "/lib/structopt.rlib")
>               "--cap-lints" "allow" "-C" "rpath" "-C" "prefer-dynamic")
>       (invoke "rustc" "--edition=2018" "--crate-name" "qtlreaper"
>               "src/main.rs"
>               "-C" "opt-level=3"
>               "--test" "-C" "metadata=test" "-C" "extra-filename=-test"
>               "--out-dir" "target/release/deps"
>               "--extern" "qtlreaper=target/release/deps/libqtlreaper-test.rlib"
>               "--extern" (string-append "ndarray=" ndarray
> "/lib/libndarray.rlib")
>               "--extern" (string-append "rand=" rand "/lib/librand.rlib")
>               "--extern" (string-append "rayon=" rayon "/lib/librayon.rlib")
>               "--extern" (string-append "serde=" serde "/lib/libserde.rlib")
>               "--extern" (string-append "serde_json=" serde-json
> "/lib/libserde_json.rlib")
>               "--extern" (string-append "structopt=" structopt
> "/lib/structopt.rlib")
>               "--cap-lints" "allow" "-C" "rpath" "-C" "prefer-dynamic")
>       (invoke "./target/release/deps/qtlreaper-test")
>       (invoke "rustdoc" "--edition=2018" "--test" "src/lib"
> "--crate-name" "qtlreaper"
>               "--extern" "qtlreaper=target/release/deps/libqtlreaper-test.rlib"
>               "--extern" (string-append "ndarray=" ndarray "/libndarray.rlib")
>               "--extern" (string-append "rand=" rand "/librand.rlib")
>               "--extern" (string-append "rayon=" rayon "/lib/librayon.rlib")
>               "--extern" (string-append "serde=" serde "/lib/libserde.rlib")
>               "--extern" (string-append "serde_json=" serde-json
> "/lib/libserde_json.rlib")
>               "--extern" (string-append "structopt=" structopt
> "/lib/structopt.rlib"))))))))
> 
>>
>> Thanks,
>> Ludo’.
> 
> I was thinking of making the build phases for the cargo-build-system use
> the --verbose flag, I found it very informative to see what it was doing
> along the way.