Expensive builds when computing channel instance derivations

Expensive builds when computing channel instance derivations

[Christopher Baines]

[-- Attachment #1.1: Type: text/plain, Size: 2627 bytes --]

Hey,

Back in early 2021, I was trying to address the issues the Guix Data
Service has when trying to compute channel instance derivations for
various systems [1].

1: https://issues.guix.gnu.org/47989

Some changes did some out of that, and I believe the helped, but even
with those changes, being able to build things for the system you want
to compute the channel instance derivation for seemed to remain
necessary.

This poses an operational issue for things like data.guix.gnu.org, since
it can only compute channel instance derivations for systems it can
build for, which in practice means that it's limited by the systems
which QEMU emulation is enabled for. Even for those systems it can build
for, because builds can happen when attempting to compute these
derivations, it means that data.guix.gnu.org spends a lot of time
waiting for builds to complete, just so it can store these derivations.

I have a suspicion that this issue is a big contributor to the
data.guix.gnu.org slowness when processing new revisions, so I tried to
dig in to it more recently as I didn't know why these builds were
happening.

I think I figured out that it's related to grafts. I've attached a test
script which computes the channel instance derivation for mips64el-linux
which I'm not set up to build things for (if you can build for
mips64el-linux, replace this with a system you can't build for). You'll
need to use the attached patch (also present on [2]) when running this
script.

2: https://git.cbaines.net/guix/log/?h=channel-instances-manifest-graft-control

When I run this script locally, it first succeeds in computing the
channel instance derivation when grafts are disabled, but then fails
when grafts are enabled:

  while setting up the build environment: a `mips64el-linux' is required
  to build
  `/gnu/store/g40shyhsd00r5dq3mm76c2b1krnr1njh-guile-bootstrap-2.0.drv',
  but I am a `x86_64-linux'

Even though I think this shows that grafts are involved, I'm not sure
what this means? I'm guessing that the effects of grafts aren't as clear
cut as for packages here, since the grafts are happening here as part of
computing a derivation I'm guessing that the derivation is actually
built with the grafted outputs, which differs from how grafts affect
packages. Given this, I guess computing the derivation without grafts
means that the replacements that would be grafted in are just not used
at all.

To recap on my aim here, I really just want to be able to compute
channel instance derivations without performing any expensive builds, or
if that's not possible, it would be good to understand why?

Thanks,

Chris



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-channels-Allow-disabling-grafts-when-computing-deriv.patch --]
[-- Type: text/x-patch, Size: 7881 bytes --]

From 38a12f0f22841b76050a0cf5163cdc65b7f92193 Mon Sep 17 00:00:00 2001
From: Christopher Baines <mail@cbaines.net>
Date: Fri, 18 Feb 2022 23:06:57 +0000
Subject: [PATCH] channels: Allow disabling grafts when computing derivations.

---
 build-aux/build-self.scm | 23 +++++++++++++++--------
 guix/channels.scm        | 19 +++++++++++--------
 2 files changed, 26 insertions(+), 16 deletions(-)

diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index 02822a2ee8..0e7fc2907d 100644
--- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -241,7 +241,8 @@ (define guile-gcrypt
 
 (define* (build-program source version
                         #:optional (guile-version (effective-version))
-                        #:key (pull-version 0) (channel-metadata #f))
+                        #:key (pull-version 0) (channel-metadata #f)
+                        (graft? #t))
   "Return a program that computes the derivation to build Guix from SOURCE."
   (define select?
     ;; Select every module but (guix config) and non-Guix modules.
@@ -316,6 +317,8 @@ (define fake-git
                         (read-disable 'positions))
 
                       (use-modules (guix store)
+                                   (guix grafts)
+                                   (guix monads)
                                    (guix self)
                                    (guix derivations)
                                    (srfi srfi-1))
@@ -348,12 +351,14 @@ (define fake-git
                                              (%make-void-port "w"))
                                             (current-build-output-port sock))
                                (run-with-store store
-                                 (guix-derivation source version
-                                                  #$guile-version
-                                                  #:channel-metadata
-                                                  '#$channel-metadata
-                                                  #:pull-version
-                                                  #$pull-version)
+                                 (mbegin %store-monad
+                                   (set-grafting #$graft?)
+                                   (guix-derivation source version
+                                                    #$guile-version
+                                                    #:channel-metadata
+                                                    '#$channel-metadata
+                                                    #:pull-version
+                                                    #$pull-version))
                                  #:system system))
                              derivation-file-name))))))
                   #:module-path (list source))))
@@ -398,6 +403,7 @@ (define* (build source
                 #:key verbose?
                 (version (date-version-string)) channel-metadata
                 system
+                (graft? #t)
                 (pull-version 0)
 
                 ;; For the standalone Guix, default to Guile 3.0.  For old
@@ -415,7 +421,8 @@ (define* (build source
   ;; SOURCE.
   (mlet %store-monad ((build  (build-program source version guile-version
                                              #:channel-metadata channel-metadata
-                                             #:pull-version pull-version))
+                                             #:pull-version pull-version
+                                             #:graft? graft?))
                       (system (if system (return system) (current-system)))
                       (home -> (getenv "HOME"))
 
diff --git a/guix/channels.scm b/guix/channels.scm
index 5f47834c10..3aba677534 100644
--- a/guix/channels.scm
+++ b/guix/channels.scm
@@ -658,7 +658,7 @@ (define (with-trivial-build-handler mvalue)
               store))))
 
 (define* (build-from-source instance
-                            #:key core verbose? (dependencies '()) system)
+                            #:key core verbose? (dependencies '()) system graft?)
   "Return a derivation to build Guix from INSTANCE, using the self-build
 script contained therein.  When CORE is true, build package modules under
 SOURCE using CORE, an instance of Guix.  By default, build for the current
@@ -703,13 +703,14 @@ (define script
            (build source
                   #:verbose? verbose? #:version commit
                   #:system system
+                  #:graft? graft?
                   #:channel-metadata (channel-instance->sexp instance)
                   #:pull-version %pull-version))))
 
       ;; Build a set of modules that extend Guix using the standard method.
       (standard-module-derivation name source core dependencies)))
 
-(define* (build-channel-instance instance system
+(define* (build-channel-instance instance system graft?
                                  #:optional core (dependencies '()))
   "Return, as a monadic value, the derivation for INSTANCE, a channel
 instance, for SYSTEM.  DEPENDENCIES is a list of extensions providing Guile
@@ -717,7 +718,8 @@ (define* (build-channel-instance instance system
   (build-from-source instance
                      #:core core
                      #:dependencies dependencies
-                     #:system system))
+                     #:system system
+                     #:graft? graft?))
 
 (define (resolve-dependencies instances)
   "Return a procedure that, given one of the elements of INSTANCES, returns
@@ -747,7 +749,7 @@ (define edges
   (lambda (instance)
     (vhash-foldq* cons '() instance edges)))
 
-(define* (channel-instance-derivations instances #:key system)
+(define* (channel-instance-derivations instances #:key system graft?)
   "Return the list of derivations to build INSTANCES, in the same order as
 INSTANCES.  Build for the current system by default, or SYSTEM if specified."
   (define core-instance
@@ -763,11 +765,11 @@ (define edges
   (define (instance->derivation instance)
     (mlet %store-monad ((system (if system (return system) (current-system))))
       (mcached (if (eq? instance core-instance)
-                   (build-channel-instance instance system)
+                   (build-channel-instance instance system graft?)
                    (mlet %store-monad ((core (instance->derivation core-instance))
                                        (deps (mapm %store-monad instance->derivation
                                                    (edges instance))))
-                     (build-channel-instance instance system core deps)))
+                     (build-channel-instance instance system graft? core deps)))
                instance
                system)))
 
@@ -869,7 +871,7 @@ (define (channel-instance->sexp instance)
                     intro))))))
             '()))))
 
-(define* (channel-instances->manifest instances #:key system)
+(define* (channel-instances->manifest instances #:key system (graft? #t))
   "Return a profile manifest with entries for all of INSTANCES, a list of
 channel instances.  By default, build for the current system, or SYSTEM if
 specified."
@@ -889,7 +891,8 @@ (define (instance->entry instance drv)
          `((source ,(channel-instance->sexp instance)))))))
 
   (mlet* %store-monad ((derivations (channel-instance-derivations instances
-                                                                  #:system system))
+                                                                  #:system system
+                                                                  #:graft? graft?))
                        (entries ->  (map instance->entry instances derivations)))
     (return (manifest entries))))
 
-- 
2.34.0


[-- Attachment #1.3: test script --]
[-- Type: text/plain, Size: 1696 bytes --]

(use-modules (srfi srfi-1)
             (guix store)
             (guix profiles)
             (guix derivations)
             (guix utils)
             (guix status)
             (guix grafts)
             (guix channels))

(define channels
  (list (channel
         (name 'guix)
         (branch "channel-instances-manifest-graft-control")
         (url "https://git.cbaines.net/git/guix.git"))))

(let ((drv
       (with-store store
         (let* ((channel-instances
                 (latest-channel-instances store channels #:authenticate? #f))
                (manifest
                 ((channel-instances->manifest channel-instances
                                               #:system
                                               "mips64el-linux"
                                               #:graft? #f)
                  store)))
           (derivation-file-name
            (manifest-entry-item
             (first
              (manifest-entries manifest))))))))
  (peek "Successfully computed derivation without grafts" drv))

(newline)
(newline)
(peek "Now trying with grafts")
(newline)

(let ((drv
       (with-store store
         (let* ((channel-instances
                 (latest-channel-instances store channels #:authenticate? #f))
                (manifest
                 ((channel-instances->manifest channel-instances
                                               #:system
                                               "mips64el-linux")
                  store)))
           (derivation-file-name
            (manifest-entry-item
             (first
              (manifest-entries manifest))))))))
  (peek "Successfully computed derivation with grafts" drv))

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Expensive builds when computing channel instance derivations

[Ludovic Courtès]

Hi,

Christopher Baines <mail@cbaines.net> skribis:

> I think I figured out that it's related to grafts. I've attached a test
> script which computes the channel instance derivation for mips64el-linux
> which I'm not set up to build things for (if you can build for
> mips64el-linux, replace this with a system you can't build for). You'll
> need to use the attached patch (also present on [2]) when running this
> script.
>
> 2: https://git.cbaines.net/guix/log/?h=channel-instances-manifest-graft-control
>
> When I run this script locally, it first succeeds in computing the
> channel instance derivation when grafts are disabled, but then fails
> when grafts are enabled:
>
>   while setting up the build environment: a `mips64el-linux' is required
>   to build
>   `/gnu/store/g40shyhsd00r5dq3mm76c2b1krnr1njh-guile-bootstrap-2.0.drv',
>   but I am a `x86_64-linux'
>
> Even though I think this shows that grafts are involved, I'm not sure
> what this means? I'm guessing that the effects of grafts aren't as clear
> cut as for packages here, since the grafts are happening here as part of
> computing a derivation I'm guessing that the derivation is actually
> built with the grafted outputs, which differs from how grafts affect
> packages. Given this, I guess computing the derivation without grafts
> means that the replacements that would be grafted in are just not used
> at all.
>
> To recap on my aim here, I really just want to be able to compute
> channel instance derivations without performing any expensive builds, or
> if that's not possible, it would be good to understand why?

In general, computing the derivation of a package may incur
builds/downloads due to the way grafts work: depending on the *build
result* of a package, ‘package->derivation’ might return a grafting
derivation or it might return the original derivation.

The “Computing Guix derivation” bit of ‘guix pull’ is no exception: it
computes the derivation of objects that depend on packages that may need
to be grafted.  Thus, it ends up triggering builds as part of this
process.

It would be tempting to turn off grafts in this case.  But then, ‘guix
pull’ would always give us an ungrafted Guix, which is undesirable.  For
the purposes of the Data Service though, you could consider turning off
grafts globally.

HTH!

Ludo’.

Re: Expensive builds when computing channel instance derivations

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 4282 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Christopher Baines <mail@cbaines.net> skribis:
>
>> I think I figured out that it's related to grafts. I've attached a test
>> script which computes the channel instance derivation for mips64el-linux
>> which I'm not set up to build things for (if you can build for
>> mips64el-linux, replace this with a system you can't build for). You'll
>> need to use the attached patch (also present on [2]) when running this
>> script.
>>
>> 2: https://git.cbaines.net/guix/log/?h=channel-instances-manifest-graft-control
>>
>> When I run this script locally, it first succeeds in computing the
>> channel instance derivation when grafts are disabled, but then fails
>> when grafts are enabled:
>>
>>   while setting up the build environment: a `mips64el-linux' is required
>>   to build
>>   `/gnu/store/g40shyhsd00r5dq3mm76c2b1krnr1njh-guile-bootstrap-2.0.drv',
>>   but I am a `x86_64-linux'
>>
>> Even though I think this shows that grafts are involved, I'm not sure
>> what this means? I'm guessing that the effects of grafts aren't as clear
>> cut as for packages here, since the grafts are happening here as part of
>> computing a derivation I'm guessing that the derivation is actually
>> built with the grafted outputs, which differs from how grafts affect
>> packages. Given this, I guess computing the derivation without grafts
>> means that the replacements that would be grafted in are just not used
>> at all.
>>
>> To recap on my aim here, I really just want to be able to compute
>> channel instance derivations without performing any expensive builds, or
>> if that's not possible, it would be good to understand why?

For some reason I didn't follow up on this thread at the time, I can't
remember now it's been so long. This issue has remained at the back of
my mind though since within the Guix Data Service, I think it's becoming
more impactful.

> In general, computing the derivation of a package may incur
> builds/downloads due to the way grafts work: depending on the *build
> result* of a package, ‘package->derivation’ might return a grafting
> derivation or it might return the original derivation.
>
> The “Computing Guix derivation” bit of ‘guix pull’ is no exception: it
> computes the derivation of objects that depend on packages that may need
> to be grafted.  Thus, it ends up triggering builds as part of this
> process.

While that's a way of thinking about it to highlight the similarities,
it's the differences I'm interested in here.

For package derivations, I think about grafts as a layer on top. First
you build the packages normally, then you apply any grafts including
building any replacements as required. I don't know if that's an
entirely correct way of thinking about it, but it seems to work as a
mental model. For bordeaux.guix.gnu.org to provide substitues, it's
sufficient to build the packages (including replacements) without any
grafting, because that's a separate concern.

I don't think the same can be said for the channel instance derivations
though, probably because of the difference you highlight above regarding
grafts actually affecting how the derivations involved are constructed,
rather than acting as a layer on top.

> It would be tempting to turn off grafts in this case.  But then, ‘guix
> pull’ would always give us an ungrafted Guix, which is undesirable.  For
> the purposes of the Data Service though, you could consider turning off
> grafts globally.

I get that not applying grafts to guix could be undesirable, but I'm
still don't see why there aren't other options?

What would be the implications of treating it like package grafting. As
in, you'd always build guix without grafts, then you'd apply the
grafting process to the ungrafted output? That would be more consistent
with the way it works for packages, and it would probably go all or some
of the way to resolve the issues for the Guix Data Service.

There's maybe other options as well, although as I raised in my initial
email, it's hard for me to think about this since I haven't worked out
why the grafting here doesn't work like packages grafting for packages
in the first place.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Expensive builds when computing channel instance derivations

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1153 bytes --]


Christopher Baines <mail@cbaines.net> writes:

>> It would be tempting to turn off grafts in this case.  But then, ‘guix
>> pull’ would always give us an ungrafted Guix, which is undesirable.  For
>> the purposes of the Data Service though, you could consider turning off
>> grafts globally.
>
> I get that not applying grafts to guix could be undesirable, but I'm
> still don't see why there aren't other options?
>
> What would be the implications of treating it like package grafting. As
> in, you'd always build guix without grafts, then you'd apply the
> grafting process to the ungrafted output? That would be more consistent
> with the way it works for packages, and it would probably go all or some
> of the way to resolve the issues for the Guix Data Service.

I've actually surprised myself by making some quick progress on this. I
think I see now why the current situation is not really anything more
than the normal package grafting code applying to the computing of the
guix derivation.

I've sent some patches which seem to superficially work to [1].

1: https://issues.guix.gnu.org/61363

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]