#:cargo-inputs don't honor --with-input

#:cargo-inputs don't honor --with-input

[Hartmut Goebel]

Hi,

FYI: yet another rust issue: #:cargo-inputs don't honor --with-input.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: #:cargo-inputs don't honor --with-input

[Ludovic Courtès]

Hi Hartmut,

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

> FYI: yet another rust issue: #:cargo-inputs don't honor --with-input.

Uh.  More generally, Rust packages kinda create a “shadow dependency
graph” via #:cargo-inputs & co., which breaks all the tools that are
unaware of it.  It was discussed several times on this list, and
apparently it’s unfortunately unavoidable at this time.  :-/

Thanks,
Ludo’.

Re: #:cargo-inputs don't honor --with-input

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 2787 bytes --]

Hi Ludo,

Am 30.04.21 um 12:45 schrieb Ludovic Courtès:

> Uh.  More generally, Rust packages kinda create a “shadow dependency
> graph” via #:cargo-inputs & co., which breaks all the tools that are
> unaware of it.  It was discussed several times on this list, and
> apparently it’s unfortunately unavoidable at this time.  :-/

Maybe we can get rid of #:cargo-inputs at least:

guix/build-system/cargo.scm says: "Although cargo does not permit cyclic 
dependencies between crates,
however, it permits cycles to occur via dev-dependencies"

So we could change #:cargo-inputs into normal inputs and get at least 
part of the dependencies right.

I'm aware of the "special treatment" of cargo-inputs. Anyhow we could 
apply the following changes to the cargo build-system:

  *

    The cargo build-system copies the "pre-built crate" (more on this
    below) into a new output called "rlib" or "crate". There already is
    a phase "packaging" which only needs to be changed to use the other
    output.

  *

    All of today's #:cargo-inputs will be changed into normal inputs
    using the "rlib/crate" output. (To avoid duplicate assoc-rec keys we
    might need to change the name/keys, but this should be a minor issue.)

  *

    If required, the cargo build-system can easily identify former
    #:cargo-inputs  by being inputs from a "rlib/crate" output.

Benefits up to here:

  * The dependency graph would be much more complete - although
    "#:cargo-development-inputs" would still be missing.
  * Package transformation options would work -again except for
    "#:cargo-development-inputs".
  * If(!) we actually manage to make cargo pick "pre-built" crates,
    package definition will already be adjusted to use them.

|Drawbacks up to here:|

  * ||Since the "packaging" phase copies the source, there is not much
    benefit in having a "rlib/crate" output yet. Actually, when a
    "rlib/crate" output needs to be build, the user will end up with two
    copies of the source (one from the git-checkout, one from packaging)

About "pre-built" crate: Given the many possible ways to build crates 
(e.g. switching on and off "features", different crate types), we might 
never be able to provide pre-built packages for all cases. Thus we might 
end up always providing the source, even if we manage to make cargo pick 
of pre-built artifacts.

About the output name: Rust has a notion of "rlib" (a specialized .a 
file), which seems to be the pre-built artifacts we are seeking. Thus 
the proposed name.

WDYT?

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |


[-- Attachment #2: Type: text/html, Size: 3811 bytes --]

Re: #:cargo-inputs don't honor --with-input

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 3878 bytes --]

On Sat, May 01, 2021 at 11:20:51AM +0200, Hartmut Goebel wrote:
> Hi Ludo,
> 
> Am 30.04.21 um 12:45 schrieb Ludovic Courtès:
> 
> > Uh.  More generally, Rust packages kinda create a “shadow dependency
> > graph” via #:cargo-inputs & co., which breaks all the tools that are
> > unaware of it.  It was discussed several times on this list, and
> > apparently it’s unfortunately unavoidable at this time.  :-/
> 
> Maybe we can get rid of #:cargo-inputs at least:
> 
> guix/build-system/cargo.scm says: "Although cargo does not permit cyclic
> dependencies between crates,
> however, it permits cycles to occur via dev-dependencies"

That I don't remember, but it would make it easier.

> So we could change #:cargo-inputs into normal inputs and get at least part
> of the dependencies right.
> 
> I'm aware of the "special treatment" of cargo-inputs. Anyhow we could apply
> the following changes to the cargo build-system:
> 
>  *
> 
>    The cargo build-system copies the "pre-built crate" (more on this
>    below) into a new output called "rlib" or "crate". There already is
>    a phase "packaging" which only needs to be changed to use the other
>    output.
> 
>  *
> 
>    All of today's #:cargo-inputs will be changed into normal inputs
>    using the "rlib/crate" output. (To avoid duplicate assoc-rec keys we
>    might need to change the name/keys, but this should be a minor issue.)
> 
>  *
> 
>    If required, the cargo build-system can easily identify former
>    #:cargo-inputs  by being inputs from a "rlib/crate" output.
> 
> Benefits up to here:
> 
>  * The dependency graph would be much more complete - although
>    "#:cargo-development-inputs" would still be missing.

This is the biggest one IMO.

>  * Package transformation options would work -again except for
>    "#:cargo-development-inputs".

IIRC they're pulled in as (package-source rust-foo-0.x) so some of the
transformations should work (I would assume).

>  * If(!) we actually manage to make cargo pick "pre-built" crates,
>    package definition will already be adjusted to use them.

And cut down on some of the big build times.

> |Drawbacks up to here:|
> 
>  * ||Since the "packaging" phase copies the source, there is not much
>    benefit in having a "rlib/crate" output yet. Actually, when a
>    "rlib/crate" output needs to be build, the user will end up with two
>    copies of the source (one from the git-checkout, one from packaging)

The benefit of copying the source is that in theory you should be able
to set $GUIX_ENVIRONMENT/share/cargo/registry (or whatever) as a cache for
crates.io when developing, so if you want different features from the
crates you won't have to download the source, it would already be cached
locally.

> About "pre-built" crate: Given the many possible ways to build crates (e.g.
> switching on and off "features", different crate types), we might never be
> able to provide pre-built packages for all cases. Thus we might end up
> always providing the source, even if we manage to make cargo pick of
> pre-built artifacts.

Right now we use the 'default' feature set, which seems to be the
default for most crates when they're used.

> About the output name: Rust has a notion of "rlib" (a specialized .a file),
> which seems to be the pre-built artifacts we are seeking. Thus the proposed
> name.
> 
> WDYT?
> 
> -- 
> Regards
> Hartmut Goebel

When I last touched it I started from rust-apps.scm (or rust-minisign)
and tried transitioning as much as possible, but doing even just the
cargo-inputs would be a very good start.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]