bug#53127: [PATCH RFC] Turning Rust/Cargo inputs into “regular” inputs?

bug#53127: [PATCH RFC] Turning Rust/Cargo inputs into “regular” inputs?

[Ludovic Courtès]

Hello!

I’m opening this issue to discuss the possibility of changing
#:cargo-inputs and #:cargo-development-inputs to regular inputs, as a
followup to:

  https://issues.guix.gnu.org/51845#10

I have a preliminary patch for ‘guix style’ and (guix build-system
cargo), but there’s a couple of stumbling blocks.

First, after the hacky patch in the discussion above, I attempted to
turn #:cargo-inputs into ‘propagated-inputs’ (instead of ‘inputs’),
because that seemed to be somewhat more logical.  That cannot work
though, because then those packages would propagate to non-Rust
packages; for example, librsvg would depend on the “build output” of
rust-* instead of just depending on its source.  Anyway, I’m back to
‘inputs’.

Second, until now, these two things would have a different meaning:

  #:cargo-inputs (list rust-cargo)

vs.

  (inputs (list rust-cargo))

In the latter case, the package depends on the build result of
‘rust-cargo’; in the former case, the package depends on the source of
‘rust-cargo’.  (See ‘rav1e’ for an example where this happens.)

If we turn all #:cargo-inputs into ‘inputs’, how can we distinguish
these two cases?  A package like ‘rust-cargo’ is sometimes depended on
for its source, sometimes for its build result; thus, we cannot just
annotate the ‘rust-cargo’ package itself.

Last, the change to ‘inputs’ would introduce a few cycles at the
<package> level.  Those cycles vanish when we lower to bags and
derivations.  However, because of these cycles, things like ‘guix
refresh -l’ may not work; there might be other unexpected and undesired
side effects.

Some of these cycles could in theory be removed.  For instance,
‘rust-cfg-if’ has an optional dependency on ‘rust-compiler-builtins’,
which leads to a cycle, but Cargo won’t let us actually remove that
dependency, even though it’s optional.

In short: it’s complicated!

Thoughts?  Is status quo a lesser evil, after all?…

Ludo’.

PS: I guess you already knew all this Efraim but I’m kinda
    (re)discovering it and now experiencing frustration firsthand.  :-)

bug#53127: [PATCH RFC] Turning Rust/Cargo inputs into “regular” inputs?

[Liliana Marie Prikler]

Am Samstag, dem 08.01.2022 um 18:57 +0100 schrieb Ludovic Courtès:
> Hello!
> 
> I’m opening this issue to discuss the possibility of changing
> #:cargo-inputs and #:cargo-development-inputs to regular inputs, as a
> followup to:
> 
>   https://issues.guix.gnu.org/51845#10
> 
> I have a preliminary patch for ‘guix style’ and (guix build-system
> cargo), but there’s a couple of stumbling blocks.
> 
> First, after the hacky patch in the discussion above, I attempted to
> turn #:cargo-inputs into ‘propagated-inputs’ (instead of ‘inputs’),
> because that seemed to be somewhat more logical.  That cannot work
> though, because then those packages would propagate to non-Rust
> packages; for example, librsvg would depend on the “build output” of
> rust-* instead of just depending on its source.  Anyway, I’m back to
> ‘inputs’.
> 
> Second, until now, these two things would have a different meaning:
> 
>   #:cargo-inputs (list rust-cargo)
> 
> vs.
> 
>   (inputs (list rust-cargo))
> 
> In the latter case, the package depends on the build result of
> ‘rust-cargo’; in the former case, the package depends on the source
> of ‘rust-cargo’.  (See ‘rav1e’ for an example where this happens.)
I suppose adding (package-source rust-cargo) to inputs to preserve the
old meaning would not make much sense?  If so, what about having a
source output and using (list `(,rust-cargo "source") ...)?

> Last, the change to ‘inputs’ would introduce a few cycles at the
> <package> level.  Those cycles vanish when we lower to bags and
> derivations.  However, because of these cycles, things like ‘guix
> refresh -l’ may not work; there might be other unexpected and
> undesired side effects.
What about making the change incrementally, so that outer layers can
start adopting the new style while inner layers are being
rebootstrapped.  I also think it'd make sense to see how we could
detect cycles through static analysis.

> Some of these cycles could in theory be removed.  For instance,
> ‘rust-cfg-if’ has an optional dependency on ‘rust-compiler-builtins’,
> which leads to a cycle, but Cargo won’t let us actually remove that
> dependency, even though it’s optional.
Could we rewrite the toml file to tell Cargo it has no power over us? 
Could we define bootstrap mockups?

> PS: I guess you already knew all this Efraim but I’m kinda
>     (re)discovering it and now experiencing frustration firsthand. 
> :-)
Let's hope at least someone in our team has overcome Rust fatigue by
the time the GCC frontend for it lands.  Rust is an incredibly good
language, all it needs is a reasonable compiler and build system.

Cheers

bug#53127: [PATCH RFC] Turning Rust/Cargo inputs into “regular” inputs?

[Ludovic Courtès]

Hi!

Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:

> I suppose adding (package-source rust-cargo) to inputs to preserve the
> old meaning would not make much sense?

No because you need transitivity (that is, you need sources of the
dependencies, too).

> If so, what about having a source output and using (list `(,rust-cargo
> "source") ...)?

AFAICS it wouldn’t help solve the problems I mentioned.

>> Last, the change to ‘inputs’ would introduce a few cycles at the
>> <package> level.  Those cycles vanish when we lower to bags and
>> derivations.  However, because of these cycles, things like ‘guix
>> refresh -l’ may not work; there might be other unexpected and
>> undesired side effects.
> What about making the change incrementally, so that outer layers can
> start adopting the new style while inner layers are being
> rebootstrapped.  I also think it'd make sense to see how we could
> detect cycles through static analysis.

Yeah, we could change things incrementally if that helps.

However, given that ‘guix style’ can automate all the changes, I think
we could just as well change everything at once.  The good thing is that
it allows us that the strategy we choose actually works at scale, in
every case.

>> Some of these cycles could in theory be removed.  For instance,
>> ‘rust-cfg-if’ has an optional dependency on ‘rust-compiler-builtins’,
>> which leads to a cycle, but Cargo won’t let us actually remove that
>> dependency, even though it’s optional.
> Could we rewrite the toml file to tell Cargo it has no power over us? 
> Could we define bootstrap mockups?

Maybe, I don’t know!

Thanks,
Ludo’.

bug#53127: [PATCH RFC] Turning Rust/Cargo inputs into “regular” inputs?

[Liliana Marie Prikler]

Am Montag, dem 10.01.2022 um 09:58 +0100 schrieb Ludovic Courtès:
> Hi!
> 
> Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:
> 
> > I suppose adding (package-source rust-cargo) to inputs to preserve
> > the
> > old meaning would not make much sense?
> 
> No because you need transitivity (that is, you need sources of the
> dependencies, too).
> 
> > If so, what about having a source output and using (list `(,rust-
> > cargo
> > "source") ...)?
> 
> AFAICS it wouldn’t help solve the problems I mentioned.
Even if it held the transitive sources?

The only other idea I have is somehow adapting maven-build-system's
parent-poms, but I don't know how that'd look like implementation-wise.
Also, with most of our Rust packages not even being built, it'd all be
parent poms.

> > 
> > > Last, the change to ‘inputs’ would introduce a few cycles at the
> > > <package> level.  Those cycles vanish when we lower to bags and
> > > derivations.  However, because of these cycles, things like ‘guix
> > > refresh -l’ may not work; there might be other unexpected and
> > > undesired side effects.
> > What about making the change incrementally, so that outer layers
> > can start adopting the new style while inner layers are being
> > rebootstrapped.  I also think it'd make sense to see how we could
> > detect cycles through static analysis.
> 
> Yeah, we could change things incrementally if that helps.
> 
> However, given that ‘guix style’ can automate all the changes, I
> think we could just as well change everything at once.  The good
> thing is that it allows us that the strategy we choose actually works
> at scale, in every case.
Ahh, but what if we told ‘guix style’ to automate all the changes
incrementally?  In other words, we'd tell it to detect cycles in inputs
and only commit those changes that it can make without creating one. 
That'd be similar to the new-style inputs change, which also aborts if
even a single input can't be handled.

> > 
Cheers

bug#53127: [PATCH RFC] Turning Rust/Cargo inputs into “regular” inputs?

[Ludovic Courtès]

Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:

> Am Montag, dem 10.01.2022 um 09:58 +0100 schrieb Ludovic Courtès:
>> Hi!
>> 
>> Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:
>> 
>> > I suppose adding (package-source rust-cargo) to inputs to preserve
>> > the
>> > old meaning would not make much sense?
>> 
>> No because you need transitivity (that is, you need sources of the
>> dependencies, too).
>> 
>> > If so, what about having a source output and using (list `(,rust-
>> > cargo
>> > "source") ...)?
>> 
>> AFAICS it wouldn’t help solve the problems I mentioned.
> Even if it held the transitive sources?

Ah yes, that could work.

> The only other idea I have is somehow adapting maven-build-system's
> parent-poms, but I don't know how that'd look like implementation-wise.

I’m not familiar with this.

[...]

>> Yeah, we could change things incrementally if that helps.
>> 
>> However, given that ‘guix style’ can automate all the changes, I
>> think we could just as well change everything at once.  The good
>> thing is that it allows us that the strategy we choose actually works
>> at scale, in every case.
> Ahh, but what if we told ‘guix style’ to automate all the changes
> incrementally?  In other words, we'd tell it to detect cycles in inputs
> and only commit those changes that it can make without creating one. 
> That'd be similar to the new-style inputs change, which also aborts if
> even a single input can't be handled.

Hmm yes, maybe.  As a first step I’d like to see if we can get rid of
those cycles altogether (that seems to be the case at first sight) or if
it’s just not an option.

Thanks,
Ludo’.