* [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests.
@ 2022-03-13 5:40 Maxim Cournoyer
2022-03-13 5:43 ` [bug#54368] [PATCH 1/4] tests: install: Adjust the timeouts on two failing tests Maxim Cournoyer
0 siblings, 1 reply; 11+ messages in thread
From: Maxim Cournoyer @ 2022-03-13 5:40 UTC (permalink / raw)
To: 54368; +Cc: Maxim Cournoyer
Hello Guix!
On the quest to get our Berlin CI machine rebooted on a Btrfs RAID10 array,
these new system tests validate that software-wise, it's supposed to work.
Thanks,
Maxim Cournoyer (4):
tests: install: Adjust the timeouts on two failing tests.
tests: install: Streamline 'qemu-command/writable-image'.
tests: install: Enable the use of multiple disk devices for tests.
tests: install: Add two new Btrfs RAID10 install test.
gnu/tests/base.scm | 6 +-
gnu/tests/install.scm | 382 +++++++++++++++++++++++++++---------------
2 files changed, 254 insertions(+), 134 deletions(-)
--
2.34.0
^ permalink raw reply [flat|nested] 11+ messages in thread
* [bug#54368] [PATCH 1/4] tests: install: Adjust the timeouts on two failing tests.
2022-03-13 5:40 [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests Maxim Cournoyer
@ 2022-03-13 5:43 ` Maxim Cournoyer
2022-03-13 5:43 ` [bug#54368] [PATCH 2/4] tests: install: Streamline 'qemu-command/writable-image' Maxim Cournoyer
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Maxim Cournoyer @ 2022-03-13 5:43 UTC (permalink / raw)
To: 54368; +Cc: Maxim Cournoyer
The 'login on tty1' and 'getlogin on tty1' tests were failing on a machine
equipped with slower hard drives.
* gnu/tests/base.scm (run-basic-test)
["login on tty1", "getlogin on tty1"]: Bump timeout from 10 to 30 s.
---
gnu/tests/base.scm | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/gnu/tests/base.scm b/gnu/tests/base.scm
index 38d4317e52..cfaa736aec 100644
--- a/gnu/tests/base.scm
+++ b/gnu/tests/base.scm
@@ -346,7 +346,8 @@ (define (user-owned? file)
;; It can take a while before the shell commands are executed.
(marionette-eval '(use-modules (rnrs io ports)) marionette)
(wait-for-file "/root/logged-in" marionette
- #:read 'get-string-all)))
+ #:read 'get-string-all
+ #:timeout 30)))
(test-equal "getlogin on tty1"
"\"root\""
@@ -360,7 +361,8 @@ (define (user-owned? file)
;; It can take a while before the shell commands are executed.
(marionette-eval '(use-modules (rnrs io ports)) marionette)
(wait-for-file "/root/login-id" marionette
- #:read 'get-string-all)))
+ #:read 'get-string-all
+ #:timeout 30)))
;; There should be one utmpx entry for the user logged in on tty1.
(test-equal "utmpx entry"
--
2.34.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [bug#54368] [PATCH 2/4] tests: install: Streamline 'qemu-command/writable-image'.
2022-03-13 5:43 ` [bug#54368] [PATCH 1/4] tests: install: Adjust the timeouts on two failing tests Maxim Cournoyer
@ 2022-03-13 5:43 ` Maxim Cournoyer
2022-03-13 5:43 ` [bug#54368] [PATCH 3/4] tests: install: Enable the use of multiple disk devices for tests Maxim Cournoyer
2022-03-13 5:43 ` [bug#54368] [PATCH 4/4] tests: install: Add two new Btrfs RAID10 install test Maxim Cournoyer
2 siblings, 0 replies; 11+ messages in thread
From: Maxim Cournoyer @ 2022-03-13 5:43 UTC (permalink / raw)
To: 54368; +Cc: Maxim Cournoyer
* gnu/tests/install.scm (qemu-command/writable-image): Replace the use of a
writable backing file by the use of the '-snapshot' option, and rename to...
(qemu-command*): ... this, adjusting all calls.
---
gnu/tests/install.scm | 61 +++++++++++++++++--------------------------
1 file changed, 24 insertions(+), 37 deletions(-)
diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index ae8c6051f1..d1f8cc1c6d 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -341,29 +341,16 @@ (define marionette
(gexp->derivation "installation" install
#:substitutable? #f))) ;too big
-(define* (qemu-command/writable-image image
- #:key
- (uefi-support? #f)
- (memory-size 256))
- "Return as a monadic value the command to run QEMU on a writable copy of
-IMAGE, a disk image. The QEMU VM has access to MEMORY-SIZE MiB of RAM."
+(define* (qemu-command* image #:key (uefi-support? #f) (memory-size 256))
+ "Return as a monadic value the command to run QEMU with a writable overlay
+above IMAGE, a disk image. The QEMU VM has access to MEMORY-SIZE MiB of RAM."
(mlet* %store-monad ((system (current-system))
(uefi-firmware -> (and uefi-support?
(uefi-firmware system))))
- (return #~(let ((image #$image))
- ;; First we need a writable copy of the image.
- (format #t "creating writable image from '~a'...~%" image)
- (unless (zero? (system* #+(file-append qemu-minimal
- "/bin/qemu-img")
- "create" "-f" "qcow2" "-F" "qcow2"
- "-o"
- (string-append "backing_file=" image)
- "disk.img"))
- (error "failed to create writable QEMU image" image))
-
- (chmod "disk.img" #o644)
+ (return #~(begin
`(,(string-append #$qemu-minimal "/bin/"
#$(qemu-command system))
+ "-snapshot" ;for the volatile, writable overlay
,@(if (file-exists? "/dev/kvm")
'("-enable-kvm")
'())
@@ -371,7 +358,7 @@ (define* (qemu-command/writable-image image
'("-bios" #$uefi-firmware)
'())
"-no-reboot" "-m" #$(number->string memory-size)
- "-drive" "file=disk.img,if=virtio")))))
+ "-drive" (format #f "file=~a,if=virtio" #$image))))))
(define %test-installed-os
(system-test
@@ -382,7 +369,7 @@ (define %test-installed-os
build (current-guix) and then store a couple of full system images.")
(value
(mlet* %store-monad ((image (run-install %minimal-os %minimal-os-source))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %minimal-os command
"installed-os")))))
@@ -399,7 +386,7 @@ (define %test-installed-extlinux-os
(list syslinux)
#:script
%extlinux-gpt-installation-script))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %minimal-extlinux-os command
"installed-extlinux-os")))))
@@ -476,7 +463,7 @@ (define %test-iso-image-installer
%simple-installation-script-for-/dev/vda
#:installation-image-type
'uncompressed-iso9660))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %minimal-os-on-vda command name)))))
\f
@@ -531,7 +518,7 @@ (define %test-separate-home-os
%separate-home-os-source
#:script
%simple-installation-script))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %separate-home-os command "separate-home-os")))))
\f
@@ -608,7 +595,7 @@ (define %test-separate-store-os
%separate-store-os-source
#:script
%separate-store-installation-script))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %separate-store-os command "separate-store-os")))))
\f
@@ -690,7 +677,7 @@ (define %test-raid-root-os
#:script
%raid-root-installation-script
#:target-size (* 3200 MiB)))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %raid-root-os
`(,@command) "raid-root-os")))))
@@ -823,7 +810,7 @@ (define %test-encrypted-root-os
%encrypted-root-os-source
#:script
%encrypted-root-installation-script))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %encrypted-root-os command "encrypted-root-os"
#:initialization enter-luks-passphrase)))))
@@ -909,7 +896,7 @@ (define %test-lvm-separate-home-os
%lvm-separate-home-installation-script
#:packages (list lvm2-static)
#:target-size (* 3200 MiB)))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %lvm-separate-home-os
`(,@command) "lvm-separate-home-os")))))
@@ -1009,7 +996,7 @@ (define %test-encrypted-root-not-boot-os
%encrypted-root-not-boot-os-source
#:script
%encrypted-root-not-boot-installation-script))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %encrypted-root-not-boot-os command
"encrypted-root-not-boot-os"
#:initialization enter-luks-passphrase)))))
@@ -1085,7 +1072,7 @@ (define %test-btrfs-root-os
%btrfs-root-os-source
#:script
%btrfs-root-installation-script))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %btrfs-root-os command "btrfs-root-os")))))
@@ -1153,7 +1140,7 @@ (define %test-btrfs-raid-root-os
%btrfs-raid-root-os-source
#:script %btrfs-raid-root-installation-script
#:target-size (* 2800 MiB)))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %btrfs-raid-root-os `(,@command) "btrfs-raid-root-os")))))
\f
@@ -1245,7 +1232,7 @@ (define %test-btrfs-root-on-subvolume-os
%btrfs-root-on-subvolume-os-source
#:script
%btrfs-root-on-subvolume-installation-script))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %btrfs-root-on-subvolume-os command
"btrfs-root-on-subvolume-os")))))
@@ -1319,7 +1306,7 @@ (define %test-jfs-root-os
%jfs-root-os-source
#:script
%jfs-root-installation-script))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %jfs-root-os command "jfs-root-os")))))
\f
@@ -1392,7 +1379,7 @@ (define %test-f2fs-root-os
%f2fs-root-os-source
#:script
%f2fs-root-installation-script))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %f2fs-root-os command "f2fs-root-os")))))
\f
@@ -1465,7 +1452,7 @@ (define %test-xfs-root-os
%xfs-root-os-source
#:script
%xfs-root-installation-script))
- (command (qemu-command/writable-image image)))
+ (command (qemu-command* image)))
(run-basic-test %xfs-root-os command "xfs-root-os")))))
\f
@@ -1748,9 +1735,9 @@ (define* (guided-installation-test name
#:desktop? desktop?
#:encrypted? encrypted?
#:uefi-support? uefi-support?))))
- (command (qemu-command/writable-image image
- #:uefi-support? uefi-support?
- #:memory-size 512)))
+ (command (qemu-command* image
+ #:uefi-support? uefi-support?
+ #:memory-size 512)))
(run-basic-test target-os command name
#:initialization (and encrypted? enter-luks-passphrase)
#:root-password %root-password
--
2.34.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [bug#54368] [PATCH 3/4] tests: install: Enable the use of multiple disk devices for tests.
2022-03-13 5:43 ` [bug#54368] [PATCH 1/4] tests: install: Adjust the timeouts on two failing tests Maxim Cournoyer
2022-03-13 5:43 ` [bug#54368] [PATCH 2/4] tests: install: Streamline 'qemu-command/writable-image' Maxim Cournoyer
@ 2022-03-13 5:43 ` Maxim Cournoyer
2022-03-18 9:40 ` [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests Mathieu Othacehe
2022-03-18 9:41 ` [bug#54368] " Mathieu Othacehe
2022-03-13 5:43 ` [bug#54368] [PATCH 4/4] tests: install: Add two new Btrfs RAID10 install test Maxim Cournoyer
2 siblings, 2 replies; 11+ messages in thread
From: Maxim Cournoyer @ 2022-03-13 5:43 UTC (permalink / raw)
To: 54368; +Cc: Maxim Cournoyer
* gnu/tests/install.scm (run-install)[NUMBER-OF-DISKS]: Add argument, update
doc and adjust. The returned gexp output is now a list of images rather than
the image itself.
* gnu/tests/install.scm (qemu-command*): Rename IMAGE argument to IMAGES, to
account for the above change. Adjust doc. Generate a QEMU '-drive' argument
for each disk image.
(%test-installed-os): Rename the IMAGE variable to IMAGES.
(%test-installed-extlinux-os): Likewise.
(%test-iso-image-installer): Likewise.
(%test-separate-home-os): Likewise.
(%test-separate-store-os): Likewise.
(%test-raid-root-os): Likewise.
(%test-encrypted-root-os): Likewise.
(%test-lvm-separate-home-os): Likewise.
(%test-encrypted-root-not-boot-os): Likewise.
(%test-btrfs-root-os): Likewise.
(%test-btrfs-raid-root-os): Likewise.
(%test-btrfs-root-on-subvolume-os): Likewise.
(%test-jfs-root-os): Likewise.
(%test-f2fs-root-os): Likewise.
(%test-xfs-root-os): Likewise.
(guided-installation-test): Likewise.
---
gnu/tests/install.scm | 244 +++++++++++++++++++++++-------------------
1 file changed, 132 insertions(+), 112 deletions(-)
diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index d1f8cc1c6d..59e76c86e7 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -240,12 +240,14 @@ (define* (run-install target-os target-os-source
(uefi-support? #f)
(installation-image-type 'efi-raw)
(install-size 'guess)
- (target-size (* 2200 MiB)))
+ (target-size (* 2200 MiB))
+ (number-of-disks 1))
"Run SCRIPT (a shell script following the system installation procedure) in
-OS to install TARGET-OS. Return a VM image of TARGET-SIZE bytes containing
-the installed system. The packages specified in PACKAGES will be appended to
-packages defined in installation-os."
-
+OS to install TARGET-OS. Return the VM disk images of TARGET-SIZE bytes
+containing the installed system. Unless providing OS, the PACKAGES will be
+added to the packages defined in INSTALLATION-OS (from (gnu system install)).
+NUMBER-OF-DISKS can be used to specify a number of disks different than one,
+such as for RAID systems."
(mlet* %store-monad ((_ (set-grafting #f))
(system (current-system))
@@ -276,13 +278,18 @@ (define install
(gnu build marionette))
#~(begin
(use-modules (guix build utils)
- (gnu build marionette))
+ (gnu build marionette)
+ (srfi srfi-1))
(set-path-environment-variable "PATH" '("bin")
(list #$qemu-minimal))
- (system* "qemu-img" "create" "-f" "qcow2"
- #$output #$(number->string target-size))
+ (mkdir-p #$output)
+ (for-each (lambda (n)
+ (system* "qemu-img" "create" "-f" "qcow2"
+ (format #f "~a/disk~a.qcow2" #$output n)
+ #$(number->string target-size)))
+ (iota #$number-of-disks))
(define marionette
(make-marionette
@@ -303,8 +310,12 @@ (define marionette
(error
"unsupported installation-image-type:"
installation-image-type)))
- "-drive"
- ,(string-append "file=" #$output ",if=virtio")
+ ,@(append-map
+ (lambda (n)
+ (list "-drive"
+ (format #f "file=~a/disk~a.qcow2,if=virtio"
+ #$output n)))
+ (iota #$number-of-disks))
,@(if (file-exists? "/dev/kvm")
'("-enable-kvm")
'()))))
@@ -338,16 +349,23 @@ (define marionette
(exit #$(and gui-test
(gui-test #~marionette)))))))
- (gexp->derivation "installation" install
- #:substitutable? #f))) ;too big
+ (mlet %store-monad ((images-dir (gexp->derivation "installation"
+ install
+ #:substitutable? #f))) ;too big
+ (return (with-imported-modules '((guix build utils))
+ #~(begin
+ (use-modules (guix build utils))
+ (find-files #$images-dir)))))))
-(define* (qemu-command* image #:key (uefi-support? #f) (memory-size 256))
+(define* (qemu-command* images #:key (uefi-support? #f) (memory-size 256))
"Return as a monadic value the command to run QEMU with a writable overlay
-above IMAGE, a disk image. The QEMU VM has access to MEMORY-SIZE MiB of RAM."
+on top of IMAGES, a list disk images. The QEMU VM has access to MEMORY-SIZE
+MiB of RAM."
(mlet* %store-monad ((system (current-system))
(uefi-firmware -> (and uefi-support?
(uefi-firmware system))))
(return #~(begin
+ (use-modules (srfi srfi-1))
`(,(string-append #$qemu-minimal "/bin/"
#$(qemu-command system))
"-snapshot" ;for the volatile, writable overlay
@@ -358,7 +376,10 @@ (define* (qemu-command* image #:key (uefi-support? #f) (memory-size 256))
'("-bios" #$uefi-firmware)
'())
"-no-reboot" "-m" #$(number->string memory-size)
- "-drive" (format #f "file=~a,if=virtio" #$image))))))
+ ,@(append-map (lambda (image)
+ (list "-drive" (format #f "file=~a,if=virtio"
+ image)))
+ #$images))))))
(define %test-installed-os
(system-test
@@ -368,8 +389,8 @@ (define %test-installed-os
This test is expensive in terms of CPU and storage usage since we need to
build (current-guix) and then store a couple of full system images.")
(value
- (mlet* %store-monad ((image (run-install %minimal-os %minimal-os-source))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install %minimal-os %minimal-os-source))
+ (command (qemu-command* images)))
(run-basic-test %minimal-os command
"installed-os")))))
@@ -380,13 +401,13 @@ (define %test-installed-extlinux-os
"Test basic functionality of an OS booted with an extlinux bootloader. As
per %test-installed-os, this test is expensive in terms of CPU and storage.")
(value
- (mlet* %store-monad ((image (run-install %minimal-extlinux-os
- %minimal-extlinux-os-source
- #:packages
- (list syslinux)
- #:script
- %extlinux-gpt-installation-script))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install %minimal-extlinux-os
+ %minimal-extlinux-os-source
+ #:packages
+ (list syslinux)
+ #:script
+ %extlinux-gpt-installation-script))
+ (command (qemu-command* images)))
(run-basic-test %minimal-extlinux-os command
"installed-extlinux-os")))))
@@ -456,14 +477,14 @@ (define %test-iso-image-installer
(description
"")
(value
- (mlet* %store-monad ((image (run-install
- %minimal-os-on-vda
- %minimal-os-on-vda-source
- #:script
- %simple-installation-script-for-/dev/vda
- #:installation-image-type
- 'uncompressed-iso9660))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install
+ %minimal-os-on-vda
+ %minimal-os-on-vda-source
+ #:script
+ %simple-installation-script-for-/dev/vda
+ #:installation-image-type
+ 'uncompressed-iso9660))
+ (command (qemu-command* images)))
(run-basic-test %minimal-os-on-vda command name)))))
\f
@@ -514,11 +535,11 @@ (define %test-separate-home-os
partition. In particular, home directories must be correctly created (see
<https://bugs.gnu.org/21108>).")
(value
- (mlet* %store-monad ((image (run-install %separate-home-os
- %separate-home-os-source
- #:script
- %simple-installation-script))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install %separate-home-os
+ %separate-home-os-source
+ #:script
+ %simple-installation-script))
+ (command (qemu-command* images)))
(run-basic-test %separate-home-os command "separate-home-os")))))
\f
@@ -591,11 +612,11 @@ (define %test-separate-store-os
"Test basic functionality of an OS installed like one would do by hand,
where /gnu lives on a separate partition.")
(value
- (mlet* %store-monad ((image (run-install %separate-store-os
- %separate-store-os-source
- #:script
- %separate-store-installation-script))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install %separate-store-os
+ %separate-store-os-source
+ #:script
+ %separate-store-installation-script))
+ (command (qemu-command* images)))
(run-basic-test %separate-store-os command "separate-store-os")))))
\f
@@ -672,12 +693,12 @@ (define %test-raid-root-os
"Test functionality of an OS installed with a RAID root partition managed
by 'mdadm'.")
(value
- (mlet* %store-monad ((image (run-install %raid-root-os
- %raid-root-os-source
- #:script
- %raid-root-installation-script
- #:target-size (* 3200 MiB)))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install %raid-root-os
+ %raid-root-os-source
+ #:script
+ %raid-root-installation-script
+ #:target-size (* 3200 MiB)))
+ (command (qemu-command* images)))
(run-basic-test %raid-root-os
`(,@command) "raid-root-os")))))
@@ -806,11 +827,11 @@ (define %test-encrypted-root-os
This test is expensive in terms of CPU and storage usage since we need to
build (current-guix) and then store a couple of full system images.")
(value
- (mlet* %store-monad ((image (run-install %encrypted-root-os
- %encrypted-root-os-source
- #:script
- %encrypted-root-installation-script))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install %encrypted-root-os
+ %encrypted-root-os-source
+ #:script
+ %encrypted-root-installation-script))
+ (command (qemu-command* images)))
(run-basic-test %encrypted-root-os command "encrypted-root-os"
#:initialization enter-luks-passphrase)))))
@@ -890,13 +911,13 @@ (define %test-lvm-separate-home-os
(description
"Test functionality of an OS installed with a LVM /home partition")
(value
- (mlet* %store-monad ((image (run-install %lvm-separate-home-os
- %lvm-separate-home-os-source
- #:script
- %lvm-separate-home-installation-script
- #:packages (list lvm2-static)
- #:target-size (* 3200 MiB)))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install %lvm-separate-home-os
+ %lvm-separate-home-os-source
+ #:script
+ %lvm-separate-home-installation-script
+ #:packages (list lvm2-static)
+ #:target-size (* 3200 MiB)))
+ (command (qemu-command* images)))
(run-basic-test %lvm-separate-home-os
`(,@command) "lvm-separate-home-os")))))
@@ -992,11 +1013,11 @@ (define %test-encrypted-root-not-boot-os
store a couple of full system images.")
(value
(mlet* %store-monad
- ((image (run-install %encrypted-root-not-boot-os
- %encrypted-root-not-boot-os-source
- #:script
- %encrypted-root-not-boot-installation-script))
- (command (qemu-command* image)))
+ ((images (run-install %encrypted-root-not-boot-os
+ %encrypted-root-not-boot-os-source
+ #:script
+ %encrypted-root-not-boot-installation-script))
+ (command (qemu-command* images)))
(run-basic-test %encrypted-root-not-boot-os command
"encrypted-root-not-boot-os"
#:initialization enter-luks-passphrase)))))
@@ -1068,11 +1089,11 @@ (define %test-btrfs-root-os
This test is expensive in terms of CPU and storage usage since we need to
build (current-guix) and then store a couple of full system images.")
(value
- (mlet* %store-monad ((image (run-install %btrfs-root-os
- %btrfs-root-os-source
- #:script
- %btrfs-root-installation-script))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install %btrfs-root-os
+ %btrfs-root-os-source
+ #:script
+ %btrfs-root-installation-script))
+ (command (qemu-command* images)))
(run-basic-test %btrfs-root-os command "btrfs-root-os")))))
@@ -1136,11 +1157,11 @@ (define %test-btrfs-raid-root-os
RAID-0 (stripe) root partition.")
(value
(mlet* %store-monad
- ((image (run-install %btrfs-raid-root-os
- %btrfs-raid-root-os-source
- #:script %btrfs-raid-root-installation-script
- #:target-size (* 2800 MiB)))
- (command (qemu-command* image)))
+ ((images (run-install %btrfs-raid-root-os
+ %btrfs-raid-root-os-source
+ #:script %btrfs-raid-root-installation-script
+ #:target-size (* 2800 MiB)))
+ (command (qemu-command* images)))
(run-basic-test %btrfs-raid-root-os `(,@command) "btrfs-raid-root-os")))))
\f
@@ -1227,12 +1248,11 @@ (define %test-btrfs-root-on-subvolume-os
build (current-guix) and then store a couple of full system images.")
(value
(mlet* %store-monad
- ((image
- (run-install %btrfs-root-on-subvolume-os
- %btrfs-root-on-subvolume-os-source
- #:script
- %btrfs-root-on-subvolume-installation-script))
- (command (qemu-command* image)))
+ ((images (run-install %btrfs-root-on-subvolume-os
+ %btrfs-root-on-subvolume-os-source
+ #:script
+ %btrfs-root-on-subvolume-installation-script))
+ (command (qemu-command* images)))
(run-basic-test %btrfs-root-on-subvolume-os command
"btrfs-root-on-subvolume-os")))))
@@ -1302,11 +1322,11 @@ (define %test-jfs-root-os
This test is expensive in terms of CPU and storage usage since we need to
build (current-guix) and then store a couple of full system images.")
(value
- (mlet* %store-monad ((image (run-install %jfs-root-os
- %jfs-root-os-source
- #:script
- %jfs-root-installation-script))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install %jfs-root-os
+ %jfs-root-os-source
+ #:script
+ %jfs-root-installation-script))
+ (command (qemu-command* images)))
(run-basic-test %jfs-root-os command "jfs-root-os")))))
\f
@@ -1375,11 +1395,11 @@ (define %test-f2fs-root-os
This test is expensive in terms of CPU and storage usage since we need to
build (current-guix) and then store a couple of full system images.")
(value
- (mlet* %store-monad ((image (run-install %f2fs-root-os
- %f2fs-root-os-source
- #:script
- %f2fs-root-installation-script))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install %f2fs-root-os
+ %f2fs-root-os-source
+ #:script
+ %f2fs-root-installation-script))
+ (command (qemu-command* images)))
(run-basic-test %f2fs-root-os command "f2fs-root-os")))))
\f
@@ -1448,11 +1468,11 @@ (define %test-xfs-root-os
This test is expensive in terms of CPU and storage usage since we need to
build (current-guix) and then store a couple of full system images.")
(value
- (mlet* %store-monad ((image (run-install %xfs-root-os
- %xfs-root-os-source
- #:script
- %xfs-root-installation-script))
- (command (qemu-command* image)))
+ (mlet* %store-monad ((images (run-install %xfs-root-os
+ %xfs-root-os-source
+ #:script
+ %xfs-root-installation-script))
+ (command (qemu-command* images)))
(run-basic-test %xfs-root-os command "xfs-root-os")))))
\f
@@ -1720,22 +1740,22 @@ (define* (guided-installation-test name
"Install an OS using the graphical installer and test it.")
(value
(mlet* %store-monad
- ((image (run-install target-os '(this is unused)
- #:script #f
- #:os installation-os-for-gui-tests
- #:uefi-support? uefi-support?
- #:install-size install-size
- #:target-size target-size
- #:installation-image-type
- 'uncompressed-iso9660
- #:gui-test
- (lambda (marionette)
- (gui-test-program
- marionette
- #:desktop? desktop?
- #:encrypted? encrypted?
- #:uefi-support? uefi-support?))))
- (command (qemu-command* image
+ ((images (run-install target-os '(this is unused)
+ #:script #f
+ #:os installation-os-for-gui-tests
+ #:uefi-support? uefi-support?
+ #:install-size install-size
+ #:target-size target-size
+ #:installation-image-type
+ 'uncompressed-iso9660
+ #:gui-test
+ (lambda (marionette)
+ (gui-test-program
+ marionette
+ #:desktop? desktop?
+ #:encrypted? encrypted?
+ #:uefi-support? uefi-support?))))
+ (command (qemu-command* images
#:uefi-support? uefi-support?
#:memory-size 512)))
(run-basic-test target-os command name
--
2.34.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [bug#54368] [PATCH 4/4] tests: install: Add two new Btrfs RAID10 install test.
2022-03-13 5:43 ` [bug#54368] [PATCH 1/4] tests: install: Adjust the timeouts on two failing tests Maxim Cournoyer
2022-03-13 5:43 ` [bug#54368] [PATCH 2/4] tests: install: Streamline 'qemu-command/writable-image' Maxim Cournoyer
2022-03-13 5:43 ` [bug#54368] [PATCH 3/4] tests: install: Enable the use of multiple disk devices for tests Maxim Cournoyer
@ 2022-03-13 5:43 ` Maxim Cournoyer
2 siblings, 0 replies; 11+ messages in thread
From: Maxim Cournoyer @ 2022-03-13 5:43 UTC (permalink / raw)
To: 54368; +Cc: Maxim Cournoyer
* gnu/tests/install.scm (%btrfs-raid10-root-os)
(%btrfs-raid10-root-os): New variables.
(%btrfs-raid10-root-installation-script): Likewise.
(%test-btrfs-raid10-root-images): Likewise.
(%test-btrfs-raid10-root-os, %test-btrfs-raid10-root-os-degraded): New tests.
---
gnu/tests/install.scm | 113 +++++++++++++++++++++++++++++++++++++++++-
1 file changed, 112 insertions(+), 1 deletion(-)
diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index 59e76c86e7..79a7490e7a 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -4,7 +4,7 @@
;;; Copyright © 2020 Mathieu Othacehe <m.othacehe@gmail.com>
;;; Copyright © 2020 Danny Milosavljevic <dannym@scratchpost.org>
;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
-;;; Copyright © 2020, 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;; Copyright © 2020, 2021, 2022 Maxim Cournoyer <maxim.cournoyer@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -70,6 +70,8 @@ (define-module (gnu tests install)
%test-btrfs-root-os
%test-btrfs-root-on-subvolume-os
%test-btrfs-raid-root-os
+ %test-btrfs-raid10-root-os
+ %test-btrfs-raid10-root-os-degraded
%test-jfs-root-os
%test-f2fs-root-os
%test-xfs-root-os
@@ -1256,6 +1258,115 @@ (define %test-btrfs-root-on-subvolume-os
(run-basic-test %btrfs-root-on-subvolume-os command
"btrfs-root-on-subvolume-os")))))
+
+;;;
+;;; Btrfs RAID10 root file system.
+;;;
+
+(define-os-with-source (%btrfs-raid10-root-os
+ %btrfs-raid10-root-os-source)
+ ;; The OS we want to install.
+ (use-modules (gnu) (gnu tests) (srfi srfi-1))
+
+ (operating-system
+ (host-name "hurd")
+ (timezone "Europe/Paris")
+ (locale "en_US.UTF-8")
+ (bootloader (bootloader-configuration
+ (bootloader grub-bootloader)
+ (targets (list "/dev/vdb" "/dev/vdc" "/dev/vdd" "/dev/vde"))))
+ (kernel-arguments '("console=ttyS0"))
+ (file-systems (cons* (file-system
+ (device (uuid "16ff18e2-eb41-4324-8df5-80d3b53c411b"))
+ (mount-point "/")
+ (options "compress-force=zstd,degraded")
+ (type "btrfs"))
+ %base-file-systems))
+ (users (cons (user-account
+ (name "charlie")
+ (group "users")
+ (supplementary-groups '("wheel" "audio" "video")))
+ %base-user-accounts))
+ (services (cons (service marionette-service-type
+ (marionette-configuration
+ (imported-modules '((gnu services herd)
+ (guix combinators)))))
+ %base-services))))
+
+(define %btrfs-raid10-root-installation-script
+ ;; Shell script of a simple installation.
+ "\
+. /etc/profile
+set -e -x
+guix --version
+
+export GUIX_BUILD_OPTIONS=--no-grafts
+ls -l /run/current-system/gc-roots
+for d in vdb vdc vdd vde; do
+ parted --script /dev/$d mklabel gpt \\
+ mkpart primary ext2 1M 2M \\
+ mkpart primary ext2 2M 100% \\
+ set 1 boot on \\
+ set 1 bios_grub on
+done
+
+# Create the RAID10 Btrfs array.
+mkfs.btrfs -d raid10 -m raid1c4 /dev/{vdb2,vdc2,vdd2,vde2} \\
+ --uuid 16ff18e2-eb41-4324-8df5-80d3b53c411b
+
+# Mount it, ready for installation.
+mount UUID=16ff18e2-eb41-4324-8df5-80d3b53c411b -o compress-force=zstd /mnt
+
+herd start cow-store /mnt
+mkdir /mnt/etc
+cp /etc/target-config.scm /mnt/etc/config.scm
+guix system build /mnt/etc/config.scm
+guix system init /mnt/etc/config.scm /mnt --no-substitutes
+sync
+reboot\n")
+
+(define %test-btrfs-raid10-root-images
+ (mlet %store-monad
+ ((images (run-install %btrfs-raid10-root-os
+ %btrfs-raid10-root-os-source
+ #:script
+ %btrfs-raid10-root-installation-script
+ #:number-of-disks 4
+ #:target-size (* 1100 MiB))))
+ (return images)))
+
+(define %test-btrfs-raid10-root-os
+ (system-test
+ (name "btrfs-raid10-root-os")
+ (description
+ "Test basic functionality of an OS installed on top of a Btrfs RAID10 file
+system spanning 4 disks. This test is expensive in terms of CPU and storage
+usage since we need to build (current-guix) and then store a couple of full
+system images.")
+ (value
+ (mlet* %store-monad
+ ((images %test-btrfs-raid10-root-images)
+ (command (qemu-command* images)))
+ (run-basic-test %btrfs-raid10-root-os command
+ "btrfs-raid10-root-os")))))
+
+(define %test-btrfs-raid10-root-os-degraded
+ (system-test
+ (name "btrfs-raid10-root-os-degraded")
+ (description
+ "Test basic functionality of an OS installed on top of a Btrfs RAID10 file
+system spanning 4 disks, degraded to 3 disks. This test is expensive in terms
+of CPU and storage usage since we need to build (current-guix) and then store
+a couple of full system images.")
+ (value
+ (mlet* %store-monad
+ ;; Drop the first image; this boots because the root file system uses
+ ;; the Btrfs "degraded" mount option.
+ ((images %test-btrfs-raid10-root-images)
+ (command (qemu-command* #~(cdr #$images))))
+ (run-basic-test %btrfs-raid10-root-os command
+ "btrfs-raid10-root-os")))))
+
\f
;;;
;;; JFS root file system.
--
2.34.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests.
2022-03-13 5:43 ` [bug#54368] [PATCH 3/4] tests: install: Enable the use of multiple disk devices for tests Maxim Cournoyer
@ 2022-03-18 9:40 ` Mathieu Othacehe
2022-03-18 13:28 ` Maxim Cournoyer
2022-03-19 15:31 ` bug#54368: " Maxim Cournoyer
2022-03-18 9:41 ` [bug#54368] " Mathieu Othacehe
1 sibling, 2 replies; 11+ messages in thread
From: Mathieu Othacehe @ 2022-03-18 9:40 UTC (permalink / raw)
To: Maxim Cournoyer; +Cc: 54368
Hello Maxim,
This looks great! I created a dedicated jobset here:
https://ci.guix.gnu.org/jobset/wip-btrfs-raid-tests to check everything
is still running fine.
> +OS to install TARGET-OS. Return the VM disk images of TARGET-SIZE bytes
> +containing the installed system. Unless providing OS, the PACKAGES will be
> +added to the packages defined in INSTALLATION-OS (from (gnu system install)).
> +NUMBER-OF-DISKS can be used to specify a number of disks different than one,
> +such as for RAID systems."
I'm having troubles understanding the "Unless providing OS" part of the
sentence, but that's also because this mechanism of adding packages to
the default OS definition is tricky.
Maybe we should augment the OS definition with those packages
unconditionally in the body of the "run-install" procedure.
Thanks,
Mathieu
^ permalink raw reply [flat|nested] 11+ messages in thread
* [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests.
2022-03-13 5:43 ` [bug#54368] [PATCH 3/4] tests: install: Enable the use of multiple disk devices for tests Maxim Cournoyer
2022-03-18 9:40 ` [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests Mathieu Othacehe
@ 2022-03-18 9:41 ` Mathieu Othacehe
2022-03-18 13:31 ` Maxim Cournoyer
1 sibling, 1 reply; 11+ messages in thread
From: Mathieu Othacehe @ 2022-03-18 9:41 UTC (permalink / raw)
To: Maxim Cournoyer; +Cc: 54368
> -above IMAGE, a disk image. The QEMU VM has access to MEMORY-SIZE MiB of RAM."
> +on top of IMAGES, a list disk images. The QEMU VM has access to MEMORY-SIZE
"a list <of> disk images."?
Thanks,
Mathieu
^ permalink raw reply [flat|nested] 11+ messages in thread
* [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests.
2022-03-18 9:40 ` [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests Mathieu Othacehe
@ 2022-03-18 13:28 ` Maxim Cournoyer
2022-03-19 15:31 ` bug#54368: " Maxim Cournoyer
1 sibling, 0 replies; 11+ messages in thread
From: Maxim Cournoyer @ 2022-03-18 13:28 UTC (permalink / raw)
To: Mathieu Othacehe; +Cc: 54368
Hi Mathieu,
Mathieu Othacehe <othacehe@gnu.org> writes:
> Hello Maxim,
>
> This looks great! I created a dedicated jobset here:
> https://ci.guix.gnu.org/jobset/wip-btrfs-raid-tests to check everything
> is still running fine.
>
>> +OS to install TARGET-OS. Return the VM disk images of TARGET-SIZE bytes
>> +containing the installed system. Unless providing OS, the PACKAGES will be
>> +added to the packages defined in INSTALLATION-OS (from (gnu system install)).
>> +NUMBER-OF-DISKS can be used to specify a number of disks different than one,
>> +such as for RAID systems."
>
> I'm having troubles understanding the "Unless providing OS" part of the
> sentence, but that's also because this mechanism of adding packages to
> the default OS definition is tricky.
>
> Maybe we should augment the OS definition with those packages
> unconditionally in the body of the "run-install" procedure.
Yes, it's a bit weird that we accept an argument whose only purpose is
to affect the default value of another argument! I'll try implementing
your idea; I also think it's better.
Thank you!
Maxim
^ permalink raw reply [flat|nested] 11+ messages in thread
* [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests.
2022-03-18 9:41 ` [bug#54368] " Mathieu Othacehe
@ 2022-03-18 13:31 ` Maxim Cournoyer
2022-03-18 15:26 ` Mathieu Othacehe
0 siblings, 1 reply; 11+ messages in thread
From: Maxim Cournoyer @ 2022-03-18 13:31 UTC (permalink / raw)
To: Mathieu Othacehe; +Cc: 54368
Mathieu Othacehe <othacehe@gnu.org> writes:
>> -above IMAGE, a disk image. The QEMU VM has access to MEMORY-SIZE MiB of RAM."
>> +on top of IMAGES, a list disk images. The QEMU VM has access to MEMORY-SIZE
>
> "a list <of> disk images."?
Indeed, fixed :-).
About the dedicated jobset, there seems to be a couple failures that are
more CI-related? Such as:
--8<---------------cut here---------------start------------->8---
@ substituter-succeeded /gnu/store/qb6n55c7anrilw0pb0lspxxm3xyrmj4x-guix-translated-texinfo.drv
fetching path `/gnu/store/yc1nj4dp0dygr3a4nfa3r1g7dll2kzcg-guix-manual.drv'...
@ substituter-started /gnu/store/yc1nj4dp0dygr3a4nfa3r1g7dll2kzcg-guix-manual.drv substitute
Downloading http://141.80.167.131/nar/gzip/yc1nj4dp0dygr3a4nfa3r1g7dll2kzcg-guix-manual.drv...
[K guix-manual.drv 679B 0B/s 00:00 [ ] 0.0%
[K guix-manual.drv 679B 657KiB/s 00:00 [##################] 100.0%
[K guix-manual.drv 679B 449KiB/s 00:00 [##################] 100.0%
@ substituter-succeeded /gnu/store/yc1nj4dp0dygr3a4nfa3r1g7dll2kzcg-guix-manual.drv
cannot build missing derivation ?/gnu/store/x7z4y134cx5il1ifbzqhdfmqc3b1980w-btrfs-raid10-root-os.drv?
--8<---------------cut here---------------end--------------->8---
Are you able to interpret such failure?
Thanks,
Maxim
^ permalink raw reply [flat|nested] 11+ messages in thread
* [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests.
2022-03-18 13:31 ` Maxim Cournoyer
@ 2022-03-18 15:26 ` Mathieu Othacehe
0 siblings, 0 replies; 11+ messages in thread
From: Mathieu Othacehe @ 2022-03-18 15:26 UTC (permalink / raw)
To: Maxim Cournoyer; +Cc: 54368
Hey,
> @ substituter-succeeded /gnu/store/yc1nj4dp0dygr3a4nfa3r1g7dll2kzcg-guix-manual.drv
> cannot build missing derivation ?/gnu/store/x7z4y134cx5il1ifbzqhdfmqc3b1980w-btrfs-raid10-root-os.drv?
>
> Are you able to interpret such failure?
Nothing to related to your patches I think, reported here:
https://issues.guix.gnu.org/54447.
Quite unfortunate though as the installation tests are all failing
because of this problem :(.
Mathieu
^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#54368: [PATCH 0/4] Add Btrfs RAID10 install tests.
2022-03-18 9:40 ` [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests Mathieu Othacehe
2022-03-18 13:28 ` Maxim Cournoyer
@ 2022-03-19 15:31 ` Maxim Cournoyer
1 sibling, 0 replies; 11+ messages in thread
From: Maxim Cournoyer @ 2022-03-19 15:31 UTC (permalink / raw)
To: Mathieu Othacehe; +Cc: 54368-done
Hi Mathieu,
Mathieu Othacehe <othacehe@gnu.org> writes:
> Hello Maxim,
>
> This looks great! I created a dedicated jobset here:
> https://ci.guix.gnu.org/jobset/wip-btrfs-raid-tests to check everything
> is still running fine.
>
>> +OS to install TARGET-OS. Return the VM disk images of TARGET-SIZE bytes
>> +containing the installed system. Unless providing OS, the PACKAGES will be
>> +added to the packages defined in INSTALLATION-OS (from (gnu system install)).
>> +NUMBER-OF-DISKS can be used to specify a number of disks different than one,
>> +such as for RAID systems."
>
> I'm having troubles understanding the "Unless providing OS" part of the
> sentence, but that's also because this mechanism of adding packages to
> the default OS definition is tricky.
>
> Maybe we should augment the OS definition with those packages
> unconditionally in the body of the "run-install" procedure.
This is now done unconditionally. I've also fixed the typo reported in
another email on this thread, thank you!
Retested with 'make check-system TESTS=btrfs-raid10-root-os' and pushed
with commit b45a301618.
Thanks for the review!
Maxim
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2022-03-19 15:32 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-03-13 5:40 [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests Maxim Cournoyer
2022-03-13 5:43 ` [bug#54368] [PATCH 1/4] tests: install: Adjust the timeouts on two failing tests Maxim Cournoyer
2022-03-13 5:43 ` [bug#54368] [PATCH 2/4] tests: install: Streamline 'qemu-command/writable-image' Maxim Cournoyer
2022-03-13 5:43 ` [bug#54368] [PATCH 3/4] tests: install: Enable the use of multiple disk devices for tests Maxim Cournoyer
2022-03-18 9:40 ` [bug#54368] [PATCH 0/4] Add Btrfs RAID10 install tests Mathieu Othacehe
2022-03-18 13:28 ` Maxim Cournoyer
2022-03-19 15:31 ` bug#54368: " Maxim Cournoyer
2022-03-18 9:41 ` [bug#54368] " Mathieu Othacehe
2022-03-18 13:31 ` Maxim Cournoyer
2022-03-18 15:26 ` Mathieu Othacehe
2022-03-13 5:43 ` [bug#54368] [PATCH 4/4] tests: install: Add two new Btrfs RAID10 install test Maxim Cournoyer
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).