[bug#60373] Kernel job for Cuirass

[bug#60373] Kernel job for Cuirass

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 443 bytes --]

Here are some patches to create a 'kernel-updates' Cuirass
specification, so that CI will build kernel-related things based on
changes to a kernel-updates Git branch.

The first patch creates a manifest of packages named "linux-libre-*" for
the Guix source tree, and the second patch creates the Cuirass spec for
maintenance.git.

I can test the manifest with `guix weather`, but I'm not sure how to
test the Cuirass spec.

Advice is welcome!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#60373] [PATCH] maint: Add a manifest for everything related to linux-libre.

[Leo Famulari]

* etc/kernels-manifest.scm: New file.
* Makefile.am (EXTRA_DIST): Add it.
---
 Makefile.am              |  1 +
 etc/kernels-manifest.scm | 35 +++++++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
 create mode 100644 etc/kernels-manifest.scm

diff --git a/Makefile.am b/Makefile.am
index b54288c0fc..8b026b6da6 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -689,6 +689,7 @@ EXTRA_DIST +=						\
   etc/guix-install.sh					\
   etc/historical-authorizations				\
   etc/news.scm						\
+  etc/kernels-manifest.scm				\
   etc/release-manifest.scm				\
   etc/source-manifest.scm				\
   etc/system-tests.scm					\
diff --git a/etc/kernels-manifest.scm b/etc/kernels-manifest.scm
new file mode 100644
index 0000000000..6f1b31abb3
--- /dev/null
+++ b/etc/kernels-manifest.scm
@@ -0,0 +1,35 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 Leo Famulari <leo@famulari.name>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; This file returns a manifest of packages related to linux-libre.
+;;; Specifically, it selects packages whose names begin with "linux-libre-".
+;;; It is used to assist continuous integration of the kernel packages.
+
+(use-modules (gnu packages)
+             (guix packages)
+             (guix profiles))
+
+(manifest
+  (map package->manifest-entry
+       (fold-packages
+         (lambda (package lst)
+           (if (string-prefix? "linux-libre-"
+                               (package-name package))
+             (cons package lst)
+             lst))
+         '())))
-- 
2.38.1

[bug#60373] [PATCH] hydra: cuirass: Add 'kernel-updates' jobset.

[Leo Famulari]

Can the build field actually contain this kind of list?

Should it contain system-tests? Normally they only run on x86_64, but
this would run them for all systems.

* hydra/modules/sysadmin/services.scm (cuirass-specs): Add
'kernel-updates' jobset.
---
 hydra/modules/sysadmin/services.scm | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/hydra/modules/sysadmin/services.scm b/hydra/modules/sysadmin/services.scm
index d0c5b24..8d143ad 100644
--- a/hydra/modules/sysadmin/services.scm
+++ b/hydra/modules/sysadmin/services.scm
@@ -2,6 +2,7 @@
 ;;;
 ;;; Copyright © 2016-2022 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2017, 2018, 2020, 2022 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2022 Leo Famulari <leo@famulari.name>
 ;;;
 ;;; This program is free software: you can redistribute it and/or modify
 ;;; it under the terms of the GNU General Public License as published by
@@ -347,6 +348,14 @@
                         (branch "core-updates"))))
                 (priority 4)
                 (systems '#$systems)))
+            #~())
+     #$@(if (member "kernel-updates" branches)
+            #~((specification
+                (name "kernel-updates")
+                (build '(images system-tests (manifests "etc/kernels-manifest.scm")))
+                (period 7200)
+                (priority 2)
+                (systems '#$systems)))
             #~())))
 
 \f
-- 
2.38.1

[bug#60373] Kernel job for Cuirass

[Mathieu Othacehe]

Hey Leo,

Good initiative!

> Can the build field actually contain this kind of list?

No sadly it cannot right now. That would require to edit a bit the (gnu
ci) module in Guix so that it allows a list of things to be built
instead of individual subsets. Then a thing or two would need to be
adjusted in Cuirass as well.

> Should it contain system-tests? Normally they only run on x86_64, but
> this would run them for all systems.

What you could do instead is something like:

--8<---------------cut here---------------start------------->8---
((specification
   (name "kernel-updates")
   (build '(manifests "etc/kernels-manifest.scm"))
   (period 7200)
   (priority 2)
   (systems '#$systems))
 (specification
   (name "kernel-updates-images")
   (build 'images)
   (period 7200)
   (priority 2)
   (systems '#$systems))
 (specification
   (name "kernel-updates-tests")
   (build 'system-tests)
   (period 7200)
   (priority 2)
   (systems '("x86_64-linux"))))
--8<---------------cut here---------------end--------------->8---

Then we could think about supporting multiple build subsets.

WDYT?

Thanks,

Mathieu

[bug#60373] Kernel job for Cuirass

[Leo Famulari]

[-- Attachment #1.1: Type: text/plain, Size: 1097 bytes --]

On Wed, Dec 28, 2022 at 03:35:18PM +0100, Mathieu Othacehe wrote:
> What you could do instead is something like:
> 
> --8<---------------cut here---------------start------------->8---
> ((specification
>    (name "kernel-updates")
>    (build '(manifests "etc/kernels-manifest.scm"))
>    (period 7200)
>    (priority 2)
>    (systems '#$systems))
>  (specification
>    (name "kernel-updates-images")
>    (build 'images)
>    (period 7200)
>    (priority 2)
>    (systems '#$systems))
>  (specification
>    (name "kernel-updates-tests")
>    (build 'system-tests)
>    (period 7200)
>    (priority 2)
>    (systems '("x86_64-linux"))))
> --8<---------------cut here---------------end--------------->8---

That's perfect! I've attached an updated patch.

> Then we could think about supporting multiple build subsets.

Well, your suggested workaround is great, so it's not necessary to add
the functionality.

What's the procedure for making these changes? I can push to guix.git
and maintenance.git, but I'll need assistance putting the changes into
practice.

[-- Attachment #1.2: v2-0001-hydra-cuirass-Add-jobsets-for-testing-kernel-upda.patch --]
[-- Type: text/plain, Size: 2020 bytes --]

From 299039cf429210392b979a57e898c4a00ca4a6c3 Mon Sep 17 00:00:00 2001
From: Leo Famulari <leo@famulari.name>
Date: Tue, 27 Dec 2022 21:26:16 -0500
Subject: [PATCH v2] hydra: cuirass: Add jobsets for testing kernel updates.

* hydra/modules/sysadmin/services.scm (cuirass-specs): Add 'kernel-updates',
'kernel-updates-images', and 'kernel-updates-system-tests' jobsets.
---
 hydra/modules/sysadmin/services.scm | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/hydra/modules/sysadmin/services.scm b/hydra/modules/sysadmin/services.scm
index d0c5b24..66bf231 100644
--- a/hydra/modules/sysadmin/services.scm
+++ b/hydra/modules/sysadmin/services.scm
@@ -2,6 +2,7 @@
 ;;;
 ;;; Copyright © 2016-2022 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2017, 2018, 2020, 2022 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2022 Leo Famulari <leo@famulari.name>
 ;;;
 ;;; This program is free software: you can redistribute it and/or modify
 ;;; it under the terms of the GNU General Public License as published by
@@ -347,6 +348,26 @@
                         (branch "core-updates"))))
                 (priority 4)
                 (systems '#$systems)))
+            #~())
+     #$@(if (member "kernel-updates" branches)
+            #~((specification
+                (name "kernel-updates")
+                (build '(manifests "etc/kernels-manifests.scm"))
+                (period 7200)
+                (priority 2)
+                (systems '#$systems))
+               (specification
+                (name "kernel-updates-images")
+                (build 'images)
+                (period 7200)
+                (priority 2)
+                (systems '#$systems))
+               (specification
+                (name "kernel-updates-system-tests")
+                (build 'system-tests)
+                (period 7200)
+                (priority 2)
+                (systems '("x86_64-linux"))))
             #~())))
 
 \f
-- 
2.38.1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#60373] [PATCH v2] maint: Add a manifest for everything related to linux-libre.

[Leo Famulari]

* etc/kernels-manifest.scm: New file.
* Makefile.am (EXTRA_DIST): Add it.
---
 Makefile.am              |  1 +
 etc/kernels-manifest.scm | 35 +++++++++++++++++++++++++++++++++++
 2 files changed, 36 insertions(+)
 create mode 100644 etc/kernels-manifest.scm

diff --git a/Makefile.am b/Makefile.am
index b54288c0fc..8b026b6da6 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -689,6 +689,7 @@ EXTRA_DIST +=						\
   etc/guix-install.sh					\
   etc/historical-authorizations				\
   etc/news.scm						\
+  etc/kernels-manifest.scm				\
   etc/release-manifest.scm				\
   etc/source-manifest.scm				\
   etc/system-tests.scm					\
diff --git a/etc/kernels-manifest.scm b/etc/kernels-manifest.scm
new file mode 100644
index 0000000000..5fc4f52f2b
--- /dev/null
+++ b/etc/kernels-manifest.scm
@@ -0,0 +1,35 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2022 Leo Famulari <leo@famulari.name>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+;;; This file returns a manifest of packages related to linux-libre.
+;;; Simplistically, it selects packages whose names begin with "linux-libre".
+;;; It is used to assist continuous integration of the kernel packages.
+
+(use-modules (gnu packages)
+             (guix packages)
+             (guix profiles))
+
+(manifest
+  (map package->manifest-entry
+       (fold-packages
+         (lambda (package lst)
+           (if (string-prefix? "linux-libre"
+                               (package-name package))
+             (cons package lst)
+             lst))
+         '())))
-- 
2.38.1

[bug#60373] Kernel job for Cuirass

[Mathieu Othacehe]

Hey,

> What's the procedure for making these changes? I can push to guix.git
> and maintenance.git, but I'll need assistance putting the changes into
> practice.

You can go ahead and I'll take care of reconfiguring Berlin. If you
prefer to do it by yourself, the procedure is the following:

- Log in on Berlin
- Checkout the latest maintenance repository in your home
- Run guix pull
- Run sudo -E guix system reconfigure ~/maintenance/hydra/berlin.scm -L
~/maintenance/hydra/modules/
- Notify sysadmins about the reconfiguration

Thanks,

Mathieu

[bug#60373] Kernel job for Cuirass

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 316 bytes --]

On Thu, Dec 29, 2022 at 11:33:37AM +0100, Mathieu Othacehe wrote:
> You can go ahead and I'll take care of reconfiguring Berlin. If you
> prefer to do it by yourself, the procedure is the following:

Thanks, I pushed to guix.git and maintenance.git. Can you do the rest?
I'm not free to spend much time on it today.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#60373] Kernel job for Cuirass

[Leo Famulari]

On Thu, Dec 29, 2022 at 11:33:37AM +0100, Mathieu Othacehe wrote:
> You can go ahead and I'll take care of reconfiguring Berlin. If you
> prefer to do it by yourself, the procedure is the following:
> 
> - Log in on Berlin
> - Checkout the latest maintenance repository in your home
> - Run guix pull
> - Run sudo -E guix system reconfigure ~/maintenance/hydra/berlin.scm -L
> ~/maintenance/hydra/modules/
> - Notify sysadmins about the reconfiguration

Okay, I've gone ahead and reconfigured berlin based on maintenance.git
commit 97cbea05bfc53e8d3fba1d2e1455dda8237eb3c0.

It's not clear to me if I need to restart any services now in order to
make my new Cuirass jobsets effective. A few services were restarted
automatically:

------
[...]
shepherd: Service host-name has been started.
shepherd: Service user-homes has been started.
shepherd: Service sysctl has been started.
shepherd: Service postgres-roles has been started.
[...]
------

[bug#60373] Kernel job for Cuirass

[Mathieu Othacehe]

Hello Leo,

> Okay, I've gone ahead and reconfigured berlin based on maintenance.git
> commit 97cbea05bfc53e8d3fba1d2e1455dda8237eb3c0.

Thanks for taking care of it. I added
0fad0d2cc4e7e440e80bcbcc519184a7a9111347 to the maintenance repository
because "kernel-updates" was not part of the default branches and
reconfigured.

I then restarted the "cuirass" service which is required to install the
new specifications. They are now listed in the web interface. Lets see
if they can be evaluated now ;)

Mathieu

[bug#60373] Kernel job for Cuirass

[Leo Famulari]

On Thu, Jan 05, 2023 at 06:37:40PM +0100, Mathieu Othacehe wrote:
> Thanks for taking care of it. I added
> 0fad0d2cc4e7e440e80bcbcc519184a7a9111347 to the maintenance repository
> because "kernel-updates" was not part of the default branches and
> reconfigured.

Thanks for your help!

> I then restarted the "cuirass" service which is required to install the
> new specifications. They are now listed in the web interface. Lets see
> if they can be evaluated now ;)

I just pushed to the kernel-updates branch. Now watching the Cuirass web
interface to see how it goes :)

[bug#60373] Kernel job for Cuirass

[Leo Famulari]

On Thu, Jan 05, 2023 at 08:19:41PM -0500, Leo Famulari wrote:
> I just pushed to the kernel-updates branch. Now watching the Cuirass web
> interface to see how it goes :)

It doesn't look like the kernel-updates job is working right:

https://ci.guix.gnu.org/jobset/kernel-updates

I confirmed the substitutes aren't available on the server.

The images seem to be failing often, but not exclusively on the
kernel-updates branch:

https://ci.guix.gnu.org/jobset/kernel-updates-images
https://ci.guix.gnu.org/jobset/images

[bug#60373] Kernel job for Cuirass

[Leo Famulari]

On Thu, Jan 05, 2023 at 10:00:18PM -0500, Leo Famulari wrote:
> It doesn't look like the kernel-updates job is working right:
> 
> https://ci.guix.gnu.org/jobset/kernel-updates

I think some modules were missing from kernels-manifest.scm. I added
them in commit 8be0a97a8139fdd4d196092008b98668f1a8b2bb, reconfigured,
and restarted Cuirass.

[bug#60373] Kernel job for Cuirass

[Leo Famulari]

On Fri, Jan 06, 2023 at 12:33:39AM -0500, Leo Famulari wrote:
> I think some modules were missing from kernels-manifest.scm. I added
> them in commit 8be0a97a8139fdd4d196092008b98668f1a8b2bb, reconfigured,
> and restarted Cuirass.

It didn't help, unfortunately.

[bug#60373] Kernel job for Cuirass

[Mathieu Othacehe]

Hello,

> It doesn't look like the kernel-updates job is working right:
>
> https://ci.guix.gnu.org/jobset/kernel-updates

That's because of a typo fixed on the maintenance repository with:
fa9b904b7b7cc9ccacb4ff1bfcfe92d6266b5f05.

Is it not advertised anywhere but there is a way to test that. You can
modify the following file:

--8<---------------cut here---------------start------------->8---
--- a/build-aux/cuirass/evaluate.scm
+++ b/build-aux/cuirass/evaluate.scm
@@ -96,7 +96,9 @@ (define derivation
                             inferior store
                             `(lambda (store)
                                (cuirass-jobs store
-                                             '((subset . all)
+                                             '((subset
+                                                . (manifests
+                                                   "etc/kernels-manifest.scm"))
--8<---------------cut here---------------end--------------->8---

then run `make cuirass-jobs` and the computed derivations will be
printed in the cuirass-jobs/jobs-<system>.scm files.

> The images seem to be failing often, but not exclusively on the
> kernel-updates branch:
>
> https://ci.guix.gnu.org/jobset/kernel-updates-images
> https://ci.guix.gnu.org/jobset/images

Yes, same for the tests, it is very likely of this issue:
https://issues.guix.gnu.org/48468, which is under investigation.

Thanks,

Mathieu

[bug#60373] Kernel job for Cuirass

[Mathieu Othacehe]

Hey Leo,

> I think some modules were missing from kernels-manifest.scm. I added
> them in commit 8be0a97a8139fdd4d196092008b98668f1a8b2bb, reconfigured,
> and restarted Cuirass.

Note that in that specific case, i.e modifying a manifest that is part
of Guix, it is not necessary to reconfigure Berlin.

The reason is that Cuirass role is to checkout the latest Guix then
evaluate the etc/kernel-manifest.scm file in an inferior, and build the
resulting derivations.

In that case, pushing the edited manifest to Guix's git then waiting for
the next evaluation of the specification is enough :)

Thanks,

Mathieu

[bug#60373] Kernel job for Cuirass

[Leo Famulari]

On Sat, Jan 07, 2023 at 03:47:08PM +0100, Mathieu Othacehe wrote:
> That's because of a typo fixed on the maintenance repository with:
> fa9b904b7b7cc9ccacb4ff1bfcfe92d6266b5f05.

🤦 Thank you Mathieu!

> 
> Is it not advertised anywhere but there is a way to test that. You can
> modify the following file:
> 
> --8<---------------cut here---------------start------------->8---
> --- a/build-aux/cuirass/evaluate.scm
> +++ b/build-aux/cuirass/evaluate.scm
> @@ -96,7 +96,9 @@ (define derivation
>                              inferior store
>                              `(lambda (store)
>                                 (cuirass-jobs store
> -                                             '((subset . all)
> +                                             '((subset
> +                                                . (manifests
> +                                                   "etc/kernels-manifest.scm"))
> --8<---------------cut here---------------end--------------->8---
> 
> then run `make cuirass-jobs` and the computed derivations will be
> printed in the cuirass-jobs/jobs-<system>.scm files.

Awesome, I will try it out.

> > The images seem to be failing often, but not exclusively on the
> > kernel-updates branch:
> >
> > https://ci.guix.gnu.org/jobset/kernel-updates-images
> > https://ci.guix.gnu.org/jobset/images
> 
> Yes, same for the tests, it is very likely of this issue:
> https://issues.guix.gnu.org/48468, which is under investigation.

Yes, I found these issues later. I hope we can solve them soon!