Re: Expensive builds when computing channel instance derivations

[Christopher Baines <mail@cbaines.net>]

[-- Attachment #1: Type: text/plain, Size: 4282 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Christopher Baines <mail@cbaines.net> skribis:
>
>> I think I figured out that it's related to grafts. I've attached a test
>> script which computes the channel instance derivation for mips64el-linux
>> which I'm not set up to build things for (if you can build for
>> mips64el-linux, replace this with a system you can't build for). You'll
>> need to use the attached patch (also present on [2]) when running this
>> script.
>>
>> 2: https://git.cbaines.net/guix/log/?h=channel-instances-manifest-graft-control
>>
>> When I run this script locally, it first succeeds in computing the
>> channel instance derivation when grafts are disabled, but then fails
>> when grafts are enabled:
>>
>>   while setting up the build environment: a `mips64el-linux' is required
>>   to build
>>   `/gnu/store/g40shyhsd00r5dq3mm76c2b1krnr1njh-guile-bootstrap-2.0.drv',
>>   but I am a `x86_64-linux'
>>
>> Even though I think this shows that grafts are involved, I'm not sure
>> what this means? I'm guessing that the effects of grafts aren't as clear
>> cut as for packages here, since the grafts are happening here as part of
>> computing a derivation I'm guessing that the derivation is actually
>> built with the grafted outputs, which differs from how grafts affect
>> packages. Given this, I guess computing the derivation without grafts
>> means that the replacements that would be grafted in are just not used
>> at all.
>>
>> To recap on my aim here, I really just want to be able to compute
>> channel instance derivations without performing any expensive builds, or
>> if that's not possible, it would be good to understand why?

For some reason I didn't follow up on this thread at the time, I can't
remember now it's been so long. This issue has remained at the back of
my mind though since within the Guix Data Service, I think it's becoming
more impactful.

> In general, computing the derivation of a package may incur
> builds/downloads due to the way grafts work: depending on the *build
> result* of a package, ‘package->derivation’ might return a grafting
> derivation or it might return the original derivation.
>
> The “Computing Guix derivation” bit of ‘guix pull’ is no exception: it
> computes the derivation of objects that depend on packages that may need
> to be grafted.  Thus, it ends up triggering builds as part of this
> process.

While that's a way of thinking about it to highlight the similarities,
it's the differences I'm interested in here.

For package derivations, I think about grafts as a layer on top. First
you build the packages normally, then you apply any grafts including
building any replacements as required. I don't know if that's an
entirely correct way of thinking about it, but it seems to work as a
mental model. For bordeaux.guix.gnu.org to provide substitues, it's
sufficient to build the packages (including replacements) without any
grafting, because that's a separate concern.

I don't think the same can be said for the channel instance derivations
though, probably because of the difference you highlight above regarding
grafts actually affecting how the derivations involved are constructed,
rather than acting as a layer on top.

> It would be tempting to turn off grafts in this case.  But then, ‘guix
> pull’ would always give us an ungrafted Guix, which is undesirable.  For
> the purposes of the Data Service though, you could consider turning off
> grafts globally.

I get that not applying grafts to guix could be undesirable, but I'm
still don't see why there aren't other options?

What would be the implications of treating it like package grafting. As
in, you'd always build guix without grafts, then you'd apply the
grafting process to the ungrafted output? That would be more consistent
with the way it works for packages, and it would probably go all or some
of the way to resolve the issues for the Guix Data Service.

There's maybe other options as well, although as I raised in my initial
email, it's hard for me to think about this since I haven't worked out
why the grafting here doesn't work like packages grafting for packages
in the first place.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]