Speeding up 'make check-system'

Speeding up 'make check-system'

[Maxim Cournoyer]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1: 0001-gnu-tests-Reduce-the-time-required-to-run-the-system.patch --]
[-- Type: text/x-patch, Size: 5840 bytes --]

From 82d3266d42390b67341ed5b0a5cc880d0ba3ed77 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date: Sun, 17 Nov 2019 06:01:00 +0900
Subject: [PATCH] gnu: tests: Reduce the time required to run the system tests.

When setting the GUIX_DEV_HACKS environment variable, the Guix package used
inside the instrumented VMs recycles the binaries already found in the Guix
checkout of the developer instead of rebuilding Guix from scratch.  This
brings the time required for this component from 20+ minutes down to 2-3
minutes on an X200 machine.

* gnu/packages/package-management.scm (current-guix/pre-built): New procedure.
* build-aux/run-system-tests.scm (tests-for-channel-instance): Use it, when
GUIX_DEV_HACKS is defined.
---
 build-aux/run-system-tests.scm      | 10 ++++-
 gnu/packages/package-management.scm | 66 +++++++++++++++++++++++++++++
 2 files changed, 75 insertions(+), 1 deletion(-)

diff --git a/build-aux/run-system-tests.scm b/build-aux/run-system-tests.scm
index b0cb3bd2bf..e6fda0546b 100644
--- a/build-aux/run-system-tests.scm
+++ b/build-aux/run-system-tests.scm
@@ -58,8 +58,16 @@ instance."
   ;; of tests to run in the usual way:
   ;;
   ;;   make check-system TESTS=installed-os
+
+  ;; When the GUIX_DEV_HACKS environment variable is defined, override the
+  ;; package returned by `current-guix' with a flavor that saves recompiling
+  ;; Guix from scratch and reuse the developer's checkout binaries.  The
+  ;; override "builds" about 20 times faster than the regular `current-guix'
+  ;; package, which can help speed iterative development.
   (parameterize ((current-guix-package
-                  (channel-instance->package instance)))
+                  (if (getenv "GUIX_DEV_HACKS")
+                      (current-guix/pre-built)
+                      (channel-instance->package instance))))
     (match (getenv "TESTS")
       (#f
        (all-system-tests))
diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 422d4f1959..bd2ed85189 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -468,6 +468,72 @@ out) and returning a package that uses that as its 'source'."
                                 #:recursive? #t
                                 #:select? (force select?))))))))
 
+(define-public (current-guix/pre-built)
+  "Similar to `current-guix', but with a modified build procedure that
+reuses the existing byte compiled artifacts to save recompilation time."
+
+  (let* ( ;; The `current-source-directory' macro doesn't work from the REPL.
+         ;; For testing, you can replace it with a static string pointing to
+         ;; your Guix checkout directory.
+         (repository-root (delay (canonicalize-path
+                                  (string-append (current-source-directory)
+                                                 "/../.."))))
+         (select? (lambda (file stat)
+                    (match (basename file)
+                      ((or ".git"
+                           "configure" "autom4te.cache"
+                           "config.log" "config.status"
+                           "stamp-1" "stamp-2" "stamp-3" "stamp-4" "stamp-5"
+                           "stamp-h1" "stamp-vti"
+                           "Makefile" "Makefile.in" ".libs"
+                           ".deps" ".dirstamp"
+                           "test-tmp"
+                           ) #f)
+                      (_ #t)))))
+    (package
+      (inherit guix)
+      (version (string-append (package-version guix) "+"))
+      (source (local-file (force repository-root) "guix-current"
+                          #:recursive? #t
+                          #:select? select?))
+      (arguments
+       (substitute-keyword-arguments (package-arguments guix)
+         ((#:phases phases)
+          `(modify-phases ,phases
+             ;; XXX: References to tools such as 'mkdir' and 'install' are
+             ;; captured in Makefile.in when 'autoconf' is run.  It'd be nicer
+             ;; to find those at configuration time.
+             (delete 'copy-bootstrap-guile)
+             (delete 'check)
+             (delete 'disable-failing-tests)
+             (delete 'strip)            ;can't strip .go files anyway
+             (replace 'build
+               (lambda _
+                 ;; Set the write permission bit on some files that need to be
+                 ;; touched.
+                 (chmod "nix" #o777)
+                 (for-each (lambda (f)
+                             (chmod f #o666))
+                           (cons* "guix-daemon"
+                                  (find-files "." ".*\\.(a|o)$")))
+
+                 ;; The following prevent 'make install' from rebuilding the
+                 ;; daemon and the documentation.
+                 (invoke "make" "--touch" "info"
+                         ;; TODO: Currently we must rebuild the daemon as it
+                         ;; was linked against external dependencies that
+                         ;; depend on the provenance of the profile (or
+                         ;; environment) that was used to build it.
+
+                         ;; If we could query the provenance of any profile,
+                         ;; we could make this package inherit from the guix
+                         ;; inferior that was used to provide such
+                         ;; dependencies.  The most reliable way would
+                         ;; probably be to record that provenance at build
+                         ;; time (as a make target).
+                         ;"guix-daemon"
+                         ))))))))))
+
 \f
 ;;;
 ;;; Other tools.
-- 
2.23.0


[-- Attachment #1.2: Type: text/plain, Size: 1760 bytes --]

Hello Guix!

The attached patch aims to speed up iterative development with 'make
check-system', by reusing most of the compiled objects from the Guix
checkout you are developing from (which needs to be built anyway for
'make check' to run).

Why most and not all?  Because the C++ daemon refers (links) to external
libraries that were captured at the time it was built, and are most
likely different from those inherited from the Guix package defined in
your checkout.

A solution would be have the "current-guix/pre-built" package inherit
from the *inferior* of the guix package that was used to build the
profile in which the deamon was actually built.  I'm punting on that one
because there doesn't seem to be a robust way to query the provenance of
a profile ('guix environment' generates a profile without said
information, for example).

Anyway, recompiling the daemon only vs the whole Guix .go files already
cuts down the time nicely from 20+ minutes to a couple minutes or so, on
a dated X200 machine, which helps when iterating on Guix code changes
only.

To benchmark the build of just the current-guix/pre-built package, one
can use:

--8<---------------cut here---------------start------------->8---
time ./pre-inst-env guix build -e '((@ (gnu packages package-management) current-guix/pre-built))'
--8<---------------cut here---------------end--------------->8---

To try using that package in an actual system test, you could use:

--8<---------------cut here---------------start------------->8---
GUIX_DEV_HACKS=1 make check-system TESTS=basic
--8<---------------cut here---------------end--------------->8---

or just 'make check-system', if you have lots of time on your hands :-)

Thoughts?

Maxim

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Speeding up 'make check-system'

[Ludovic Courtès]

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> The attached patch aims to speed up iterative development with 'make
> check-system', by reusing most of the compiled objects from the Guix
> checkout you are developing from (which needs to be built anyway for
> 'make check' to run).

Note that this is not about ‘make check-system’ in general but
specifically about the tests in (gnu tests install).  These tests
require a “current Guix”, which is costly.

Interested readers are hereby invited to read Maxim’s benchmarks and the
discussion here:

  https://issues.guix.gnu.org/issue/37305#7

:-)

Thanks,
Ludo’.