[bug#41839] [PATCH 0/2] ci: Build Guix System images.

[bug#41839] [PATCH 0/2] ci: Build Guix System images.

[Mathieu Othacehe]

Hello,

Now that we have a new shiny new "image" record that contains all the required
informations to build or cross-build a Guix System image, it would be nice if
Cuirass could provide associated substitutes.

This serie allows to build a list of Guix System images in the CI. For now
this list only contains the Hurd barebones Guix System image.

Thanks,

Mathieu

Mathieu Othacehe (2):
  image: Move hurd image definition to a dedicated file.
  ci: Build Guix System images.

 gnu/ci.scm                 | 69 +++++++++++++++++-----------------
 gnu/image.scm              |  2 +
 gnu/local.mk               |  2 +
 gnu/system/image.scm       | 23 +++---------
 gnu/system/images/hurd.scm | 76 ++++++++++++++++++++++++++++++++++++++
 5 files changed, 121 insertions(+), 51 deletions(-)
 create mode 100644 gnu/system/images/hurd.scm

-- 
2.26.2

[bug#41839] [PATCH 1/2] image: Move hurd image definition to a dedicated file.

[Mathieu Othacehe]

This moves hurd-disk-image to a dedicated file. It also defines a default
operating-system so that the image can be built standalone.

* gnu/system/images/hurd.scm: New file,
* gnu/local.mk (GNU_SYSTEM_MODULES): add it.
* gnu/image.scm (<image>)[name]: New field.
* gnu/system/image.scm (root-offset, root-label): Export it,
(hurd-disk-image): remove it as this is now defined in the new, Hurd dedicated
file above,
(find-image): adapt to avoid loop dependency.
---
 gnu/image.scm              |  2 +
 gnu/local.mk               |  2 +
 gnu/system/image.scm       | 23 +++---------
 gnu/system/images/hurd.scm | 76 ++++++++++++++++++++++++++++++++++++++
 4 files changed, 86 insertions(+), 17 deletions(-)
 create mode 100644 gnu/system/images/hurd.scm

diff --git a/gnu/image.scm b/gnu/image.scm
index 19b466527b..dc66f2c533 100644
--- a/gnu/image.scm
+++ b/gnu/image.scm
@@ -67,6 +67,8 @@
 (define-record-type* <image>
   image make-image
   image?
+  (name               image-name ;symbol
+                      (default #f))
   (format             image-format) ;symbol
   (target             image-target
                       (default #f))
diff --git a/gnu/local.mk b/gnu/local.mk
index 583274235b..eeeb276478 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -629,6 +629,8 @@ GNU_SYSTEM_MODULES =				\
   %D%/system/uuid.scm				\
   %D%/system/vm.scm				\
 						\
+  %D%/system/images/hurd.scm			\
+						\
   %D%/machine.scm				\
   %D%/machine/digital-ocean.scm			\
   %D%/machine/ssh.scm				\
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 1bda25fd7f..cb8fe18bcc 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -53,10 +53,12 @@
   #:use-module (srfi srfi-35)
   #:use-module (rnrs bytevectors)
   #:use-module (ice-9 match)
-  #:export (esp-partition
+  #:export (root-offset
+            root-label
+
+            esp-partition
             root-partition
 
-            hurd-disk-image
             efi-disk-image
             iso9660-image
 
@@ -101,20 +103,6 @@
                      (list #:make-device-nodes
                            make-hurd-device-nodes)))))
 
-(define hurd-disk-image
-  (image
-   (format 'disk-image)
-   (target "i586-pc-gnu")
-   (partitions
-    (list (partition
-           (size 'guess)
-           (offset root-offset)
-           (label root-label)
-           (file-system "ext2")
-           (file-system-options '("-o" "hurd" "-O" "ext_attr"))
-           (flags '(boot))
-           (initializer hurd-initialize-root-partition))))))
-
 (define efi-disk-image
   (image
    (format 'disk-image)
@@ -569,7 +557,8 @@ addition of the <image> record."
     (_ (cond
         ((and target
               (hurd-triplet? target))
-         hurd-disk-image)
+         (module-ref (resolve-interface '(gnu system images hurd))
+                     'hurd-disk-image))
         (else
          efi-disk-image)))))
 
diff --git a/gnu/system/images/hurd.scm b/gnu/system/images/hurd.scm
new file mode 100644
index 0000000000..4417f03cc8
--- /dev/null
+++ b/gnu/system/images/hurd.scm
@@ -0,0 +1,76 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2020 Mathieu Othacehe <m.othacehe@gmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu system images hurd)
+  #:use-module (guix gexp)
+  #:use-module (gnu bootloader)
+  #:use-module (gnu bootloader grub)
+  #:use-module (gnu image)
+  #:use-module (gnu packages ssh)
+  #:use-module (gnu services)
+  #:use-module (gnu services ssh)
+  #:use-module (gnu system)
+  #:use-module (gnu system file-systems)
+  #:use-module (gnu system hurd)
+  #:use-module (gnu system image)
+  #:export (hurd-barebones-os
+            hurd-disk-image
+            hurd-barebones-disk-image))
+
+(define hurd-barebones-os
+  (operating-system
+    (inherit %hurd-default-operating-system)
+    (bootloader (bootloader-configuration
+                 (bootloader grub-minimal-bootloader)
+                 (target "/dev/sdX")))
+    (file-systems (cons (file-system
+                          (device (file-system-label "my-root"))
+                          (mount-point "/")
+                          (type "ext2"))
+                        %base-file-systems))
+    (host-name "guixygnu")
+    (timezone "Europe/Amsterdam")
+    (packages (cons openssh-sans-x %base-packages/hurd))
+    (services (cons (service openssh-service-type
+                             (openssh-configuration
+                              (openssh openssh-sans-x)
+                              (use-pam? #f)
+                              (port-number 2222)
+                              (permit-root-login #t)
+                              (allow-empty-passwords? #t)
+                              (password-authentication? #t)))
+               %base-services/hurd))))
+
+(define hurd-disk-image
+  (image
+   (format 'disk-image)
+   (target "i586-pc-gnu")
+   (partitions
+    (list (partition
+           (size 'guess)
+           (offset root-offset)
+           (label root-label)
+           (file-system "ext2")
+           (flags '(boot))
+           (initializer (gexp initialize-root-partition)))))))
+
+(define hurd-barebones-disk-image
+  (image
+   (inherit hurd-disk-image)
+   (name 'hurd-barebones-disk-image)
+   (operating-system hurd-barebones-os)))
-- 
2.26.2

[bug#41839] [PATCH 2/2] ci: Build Guix System images.

[Mathieu Othacehe]

Build a list of Guix System images. For now, this list only contains the Hurd
barebones Guix System image.

* gnu/ci.scm (%guixsd-supported-systems): Remove "armhf-linux",
(%u-boot-systems): remove unused variable,
(%guix-system-images): new variable
(qemu-jobs): rename to "image-jobs" and build the Guix Systems listed in the
new "%guix-system-images" variable,
(system-test-jobs): adapt accordingly,
(hydra-jobs): ditto.
---
 gnu/ci.scm | 69 +++++++++++++++++++++++++++---------------------------
 1 file changed, 35 insertions(+), 34 deletions(-)

diff --git a/gnu/ci.scm b/gnu/ci.scm
index fa67168e22..2d49009810 100644
--- a/gnu/ci.scm
+++ b/gnu/ci.scm
@@ -53,6 +53,7 @@
   #:use-module (gnu system image)
   #:use-module (gnu system vm)
   #:use-module (gnu system install)
+  #:use-module (gnu system images hurd)
   #:use-module (gnu tests)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
@@ -185,18 +186,21 @@ SYSTEM."
               (remove (either from-32-to-64? same? pointless?)
                       %cross-targets)))
 
-(define %guixsd-supported-systems
-  '("x86_64-linux" "i686-linux" "armhf-linux"))
+;; Architectures that are able to build or cross-build Guix System images.
+;; This does not mean that other architectures are not supported, only that
+;; they are often not fast enough to support Guix System images building.
+(define %guix-system-supported-systems
+  '("x86_64-linux" "i686-linux"))
 
-(define %u-boot-systems
-  '("armhf-linux"))
+(define %guix-system-images
+  (list hurd-barebones-disk-image))
 
-(define (qemu-jobs store system)
-  "Return a list of jobs that build QEMU images for SYSTEM."
+(define (image-jobs store system)
+  "Return a list of jobs that build images for SYSTEM."
   (define (->alist drv)
     `((derivation . ,(derivation-file-name drv))
-      (description . "Stand-alone QEMU image of the GNU system")
-      (long-description . "This is a demo stand-alone QEMU image of the GNU
+      (description . "Stand-alone image of the GNU system")
+      (long-description . "This is a demo stand-alone image of the GNU
 system.")
       (license . ,(license-name gpl3+))
       (max-silent-time . 600)
@@ -211,29 +215,30 @@ system.")
                    (parameterize ((%graft? #f))
                      (->alist drv))))))
 
+  (define (build-image image)
+    (run-with-store store
+      (mbegin %store-monad
+        (set-guile-for-build (default-guile))
+        (lower-object (system-image image)))))
+
   (define MiB
     (expt 2 20))
 
-  (if (member system %guixsd-supported-systems)
-      (list (->job 'usb-image
-                   (run-with-store store
-                     (mbegin %store-monad
-                       (set-guile-for-build (default-guile))
-                       (lower-object
-                        (system-image
-                         (image
-                          (inherit efi-disk-image)
-                          (size (* 1500 MiB))
-                          (operating-system installation-os)))))))
-            (->job 'iso9660-image
-                   (run-with-store store
-                     (mbegin %store-monad
-                       (set-guile-for-build (default-guile))
-                       (lower-object
-                        (system-image
-                         (image
-                          (inherit iso9660-image)
-                          (operating-system installation-os))))))))
+  (if (member system %guix-system-supported-systems)
+      `(,(->job 'usb-image
+                (build-image
+                 (image
+                  (inherit efi-disk-image)
+                  (size (* 1500 MiB))
+                  (operating-system installation-os))))
+        ,(->job 'iso9660-image
+                (build-image
+                 (image
+                  (inherit iso9660-image)
+                  (operating-system installation-os))))
+        ,@(map (lambda (image)
+                 (->job (image-name image) (build-image image)))
+               %guix-system-images))
       '()))
 
 (define channel-build-system
@@ -305,11 +310,7 @@ system.")
                                 "." system))))
       (cons name (test->thunk test))))
 
-  (if (and (member system %guixsd-supported-systems)
-
-           ;; XXX: Our build farm has too few ARMv7 machines and they are very
-           ;; slow, so skip system tests there.
-           (not (string=? system "armhf-linux")))
+  (if (member system %guix-system-supported-systems)
       ;; Override the value of 'current-guix' used by system tests.  Using a
       ;; channel instance makes tests that rely on 'current-guix' less
       ;; expensive.  It also makes sure we get a valid Guix package when this
@@ -486,7 +487,7 @@ Return #f if no such checkout is found."
                                   (package->job store package
                                                 system))))
                        (append (filter-map job all)
-                               (qemu-jobs store system)
+                               (image-jobs store system)
                                (system-test-jobs store system
                                                  #:source source
                                                  #:commit commit)
-- 
2.26.2

[bug#41839] [PATCH 1/2] image: Move hurd image definition to a dedicated file.

[Ludovic Courtès]

Hi!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

> This moves hurd-disk-image to a dedicated file. It also defines a default
> operating-system so that the image can be built standalone.
>
> * gnu/system/images/hurd.scm: New file,
> * gnu/local.mk (GNU_SYSTEM_MODULES): add it.
> * gnu/image.scm (<image>)[name]: New field.
> * gnu/system/image.scm (root-offset, root-label): Export it,
> (hurd-disk-image): remove it as this is now defined in the new, Hurd dedicated
> file above,
> (find-image): adapt to avoid loop dependency.

LGTM!

It would be good to check by running “make as-derivation” if gnu/image/*
gets included.

Ludo’.

[bug#41839] [PATCH 2/2] ci: Build Guix System images.

[Ludovic Courtès]

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

> Build a list of Guix System images. For now, this list only contains the Hurd
> barebones Guix System image.
>
> * gnu/ci.scm (%guixsd-supported-systems): Remove "armhf-linux",
> (%u-boot-systems): remove unused variable,
> (%guix-system-images): new variable
> (qemu-jobs): rename to "image-jobs" and build the Guix Systems listed in the
> new "%guix-system-images" variable,
> (system-test-jobs): adapt accordingly,
> (hydra-jobs): ditto.

Cool.

> +  (if (member system %guix-system-supported-systems)
> +      `(,(->job 'usb-image
> +                (build-image
> +                 (image
> +                  (inherit efi-disk-image)
> +                  (size (* 1500 MiB))
> +                  (operating-system installation-os))))
> +        ,(->job 'iso9660-image
> +                (build-image
> +                 (image
> +                  (inherit iso9660-image)
> +                  (operating-system installation-os))))
> +        ,@(map (lambda (image)
> +                 (->job (image-name image) (build-image image)))
> +               %guix-system-images))
>        '()))

I think the effect is that we’ll also cross-build GNU/Hurd images from
i686-linux, right?  It would be nice to avoid that and only cross-build
from x86_64.

Otherwise LGTM, thank you!

Ludo’.

[bug#41839] [PATCH 1/2] image: Move hurd image definition to a dedicated file.

[Mathieu Othacehe]

Hey,

> It would be good to check by running “make as-derivation” if gnu/image/*
> gets included.

Yep, its included. Thanks for reviewing!

Mathieu

bug#41839: [PATCH 2/2] ci: Build Guix System images.

[Mathieu Othacehe]

Hey,

> I think the effect is that we’ll also cross-build GNU/Hurd images from
> i686-linux, right?  It would be nice to avoid that and only cross-build
> from x86_64.

Yes, I fixed that (in two tries actually :p). Now working on adding this
image to the website.

Thanks,

Mathieu