[bug#75203] Fix rootless podman system tests and support I/O delegation

[bug#75203] Fix rootless podman system tests and support I/O delegation

[paul via Guix-patches via]

Hi Guix,

since rootless Podman tests are failing on CI ( 
https://ci.guix.gnu.org/build/7694600/details and past jobs), but are 
working on my machine ( :) ), I'm sending a patch adding a delay to 
allow services to start also on slower machines.

I'm using this occasion also to introduce a minor change (implement 
rootless I/O controller delegation) required for correct cgroups v2 
setup, according to [0].

Thank you for all your work,


giacomo


[0]: https://rootlesscontaine.rs/getting-started/common/cgroup2/

[bug#75203] [PATCH 1/2] services: tests: Add delay for rootless Podman system test.

[Giacomo Leidi via Guix-patches via]

* gnu/tests/containers.scm (run-rootless-podman-test): Add 60 seconds
long delay before tests are actually run.

Change-Id: Ifcf70f7258f9e0886bf829884d7daedc9803352b
---
 gnu/tests/containers.scm | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/gnu/tests/containers.scm b/gnu/tests/containers.scm
index ba2fb22df6..047010037e 100644
--- a/gnu/tests/containers.scm
+++ b/gnu/tests/containers.scm
@@ -109,6 +109,9 @@ (define (run-rootless-podman-test oci-tarball)
                   (('service response-parts ...) #t)))
              marionette))
 
+          ;; Allow services to start on slower machines
+          (sleep 60)
+
           (test-equal "/sys/fs/cgroup/cgroup.subtree_control content is sound"
             (list "cpu" "cpuset" "memory" "pids")
             (marionette-eval

base-commit: 18463019a24a7c5acc9c2f3ddf3c0ba04a36db96
-- 
2.47.1

[bug#75203] [PATCH 2/2] services: rootless-podman: Enable I/O delegation.

[Giacomo Leidi via Guix-patches via]

Based on
https://rootlesscontaine.rs/getting-started/common/cgroup2/#enabling-cpu-cpuset-and-io-delegation
, this patch enables I/O delegation for cgroups v2 enabled users.

* gnu/services/containers.scm (cgroups-limits-entrypoint): Enable I/O
controller delegation.
* gnu/tests/containers.scm: Test it.

Change-Id: I7caba33695f11830bea477c4ab3afb89cfaa2fa5
---
 gnu/services/containers.scm | 2 +-
 gnu/tests/containers.scm    | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index d8f533f44c..cb4b617e4b 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -169,7 +169,7 @@ (define cgroups-limits-entrypoint
                 #~(system*
                    (string-append #+bash-minimal "/bin/bash") "-c"
                    (string-append "echo Setting cgroups v2 limits && "
-                                  "echo +cpu +cpuset +memory +pids"
+                                  "echo +cpu +cpuset +io +memory +pids"
                                   " >> /sys/fs/cgroup/cgroup.subtree_control"))))
 
 (define (rootless-podman-cgroups-limits-service config)
diff --git a/gnu/tests/containers.scm b/gnu/tests/containers.scm
index 047010037e..ec8a9ad440 100644
--- a/gnu/tests/containers.scm
+++ b/gnu/tests/containers.scm
@@ -113,7 +113,7 @@ (define (run-rootless-podman-test oci-tarball)
           (sleep 60)
 
           (test-equal "/sys/fs/cgroup/cgroup.subtree_control content is sound"
-            (list "cpu" "cpuset" "memory" "pids")
+            (list "cpu" "cpuset" "io" "memory" "pids")
             (marionette-eval
              `(begin
                 (use-modules (srfi srfi-1)
-- 
2.47.1

[bug#75203] [PATCH 1/2] services: tests: Add delay for rootless Podman system test.

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 1309 bytes --]

Giacomo Leidi via Guix-patches via <guix-patches@gnu.org> writes:

> * gnu/tests/containers.scm (run-rootless-podman-test): Add 60 seconds
> long delay before tests are actually run.
>
> Change-Id: Ifcf70f7258f9e0886bf829884d7daedc9803352b
> ---
>  gnu/tests/containers.scm | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/gnu/tests/containers.scm b/gnu/tests/containers.scm
> index ba2fb22df6..047010037e 100644
> --- a/gnu/tests/containers.scm
> +++ b/gnu/tests/containers.scm
> @@ -109,6 +109,9 @@ (define (run-rootless-podman-test oci-tarball)
>                    (('service response-parts ...) #t)))
>               marionette))
>  
> +          ;; Allow services to start on slower machines
> +          (sleep 60)

Would it be possible to detect whether the services started?  This seems
like needless test run time penalty for fast systems.  Even a busy loop
with (sleep 1) would be much better in my opinion.

> +
>            (test-equal "/sys/fs/cgroup/cgroup.subtree_control content is sound"
>              (list "cpu" "cpuset" "memory" "pids")
>              (marionette-eval
>
> base-commit: 18463019a24a7c5acc9c2f3ddf3c0ba04a36db96

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

[bug#75203] [PATCH 1/2] services: tests: Add delay for rootless Podman system test.

[paul via Guix-patches via]

Hi Tomas,

good point, thank you for bringing that up! I didn't do it in the first 
place as I was not able to easily test the completion status of one shot 
Shepherd services. I hope revision 2 fixes tests also on CI.


cheers,

giacomo

[bug#75203] [PATCH v2 1/2] services: tests: Add delay for rootless Podman system test.

[Giacomo Leidi via Guix-patches via]

* gnu/tests/containers.scm (run-rootless-podman-test): Add 60 seconds
long delay before tests are actually run.

Change-Id: Ifcf70f7258f9e0886bf829884d7daedc9803352b
---
 gnu/tests/containers.scm | 113 +++++++++++++++++++++------------------
 1 file changed, 60 insertions(+), 53 deletions(-)

diff --git a/gnu/tests/containers.scm b/gnu/tests/containers.scm
index ba2fb22df6..69cd311c82 100644
--- a/gnu/tests/containers.scm
+++ b/gnu/tests/containers.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2024 Giacomo Leidi <goodoldpaul@autistici.org>
+;;; Copyright © 2024, 2025 Giacomo Leidi <goodoldpaul@autistici.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -97,17 +97,65 @@ (define (run-rootless-podman-test oci-tarball)
 
           (test-runner-current (system-test-runner #$output))
           (test-begin "rootless-podman")
-
-          (test-assert "service started"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd))
-                (match (start-service 'cgroups2-fs-owner)
-                  (#f #f)
-                  ;; herd returns (running #f), likely because of one shot,
-                  ;; so consider any non-error a success.
-                  (('service response-parts ...) #t)))
-             marionette))
+          (marionette-eval
+           '(begin
+              (use-modules (gnu services herd))
+              (wait-for-service 'file-system-/sys/fs/cgroup))
+           marionette)
+
+          (test-assert "services started successfully and /sys/fs/cgroup has correct permissions"
+            (begin
+              (define (run-test)
+                (marionette-eval
+                 `(begin
+                    (use-modules (ice-9 popen)
+                                 (ice-9 match)
+                                 (ice-9 rdelim))
+
+                    (define (read-lines file-or-port)
+                      (define (loop-lines port)
+                        (let loop ((lines '()))
+                          (match (read-line port)
+                            ((? eof-object?)
+                             (reverse lines))
+                            (line
+                             (loop (cons line lines))))))
+
+                      (if (port? file-or-port)
+                          (loop-lines file-or-port)
+                          (call-with-input-file file-or-port
+                            loop-lines)))
+
+                    (define slurp
+                      (lambda args
+                        (let* ((port (apply open-pipe* OPEN_READ args))
+                               (output (read-lines port))
+                               (status (close-pipe port)))
+                          output)))
+                    (let* ((bash
+                            ,(string-append #$bash "/bin/bash"))
+                           (response1
+                            (slurp bash "-c"
+                                   (string-append "ls -la /sys/fs/cgroup | "
+                                                  "grep -E ' \\./?$' | awk '{ print $4 }'")))
+                           (response2 (slurp bash "-c"
+                                             (string-append "ls -l /sys/fs/cgroup/cgroup"
+                                                            ".{procs,subtree_control,threads} | "
+                                                            "awk '{ print $4 }' | sort -u"))))
+                      (list (string-join response1 "\n") (string-join response2 "\n"))))
+                 marionette))
+              ;; Allow services to come up on slower machines
+              (let loop ((attempts 0))
+                (if (= attempts 60)
+                    (error "Services didn't come up after more than 60 seconds")
+                    (if (equal? '("cgroup" "cgroup")
+                                (run-test))
+                        #t
+                        (begin
+                          (sleep 1)
+                          (format #t "Services didn't come up yet, retrying with attempt ~a~%"
+                                  (+ 1 attempts))
+                          (loop (+ 1 attempts))))))))
 
           (test-equal "/sys/fs/cgroup/cgroup.subtree_control content is sound"
             (list "cpu" "cpuset" "memory" "pids")
@@ -144,47 +192,6 @@ (define (run-rootless-podman-test oci-tarball)
                   (sort-list (string-split (first response1) #\space) string<?)))
              marionette))
 
-          (test-equal "/sys/fs/cgroup has correct permissions"
-            '("cgroup" "cgroup")
-            (marionette-eval
-             `(begin
-                (use-modules (ice-9 popen)
-                             (ice-9 match)
-                             (ice-9 rdelim))
-
-                (define (read-lines file-or-port)
-                  (define (loop-lines port)
-                    (let loop ((lines '()))
-                      (match (read-line port)
-                        ((? eof-object?)
-                         (reverse lines))
-                        (line
-                         (loop (cons line lines))))))
-
-                  (if (port? file-or-port)
-                      (loop-lines file-or-port)
-                      (call-with-input-file file-or-port
-                        loop-lines)))
-
-                (define slurp
-                  (lambda args
-                    (let* ((port (apply open-pipe* OPEN_READ args))
-                           (output (read-lines port))
-                           (status (close-pipe port)))
-                      output)))
-                (let* ((bash
-                        ,(string-append #$bash "/bin/bash"))
-                       (response1
-                        (slurp bash "-c"
-                               (string-append "ls -la /sys/fs/cgroup | "
-                                              "grep -E ' \\./?$' | awk '{ print $4 }'")))
-                       (response2 (slurp bash "-c"
-                                         (string-append "ls -l /sys/fs/cgroup/cgroup"
-                                                        ".{procs,subtree_control,threads} | "
-                                                        "awk '{ print $4 }' | sort -u"))))
-                  (list (string-join response1 "\n") (string-join response2 "\n"))))
-             marionette))
-
           (test-equal "Load oci image and run it (unprivileged)"
             '("hello world" "hi!" "JSON!" #o1777)
             (marionette-eval

base-commit: ee3673217b82d50e97434ae85145b8d68f077446
-- 
2.47.1

[bug#75203] [PATCH v2 2/2] services: rootless-podman: Enable I/O delegation.

[Giacomo Leidi via Guix-patches via]

Based on
https://rootlesscontaine.rs/getting-started/common/cgroup2/#enabling-cpu-cpuset-and-io-delegation
, this patch enables I/O delegation for cgroups v2 enabled users.

* gnu/services/containers.scm (cgroups-limits-entrypoint): Enable I/O
controller delegation.
* gnu/tests/containers.scm: Test it.

Change-Id: I7caba33695f11830bea477c4ab3afb89cfaa2fa5
---
 gnu/services/containers.scm | 2 +-
 gnu/tests/containers.scm    | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/services/containers.scm b/gnu/services/containers.scm
index a82fb64db3..19d35ccbcb 100644
--- a/gnu/services/containers.scm
+++ b/gnu/services/containers.scm
@@ -174,7 +174,7 @@ (define cgroups-limits-entrypoint
                 #~(system*
                    (string-append #+bash-minimal "/bin/bash") "-c"
                    (string-append "echo Setting cgroups v2 limits && "
-                                  "echo +cpu +cpuset +memory +pids"
+                                  "echo +cpu +cpuset +io +memory +pids"
                                   " >> /sys/fs/cgroup/cgroup.subtree_control"))))
 
 (define (rootless-podman-cgroups-limits-service config)
diff --git a/gnu/tests/containers.scm b/gnu/tests/containers.scm
index 69cd311c82..0ecc8ddb12 100644
--- a/gnu/tests/containers.scm
+++ b/gnu/tests/containers.scm
@@ -158,7 +158,7 @@ (define (run-rootless-podman-test oci-tarball)
                           (loop (+ 1 attempts))))))))
 
           (test-equal "/sys/fs/cgroup/cgroup.subtree_control content is sound"
-            (list "cpu" "cpuset" "memory" "pids")
+            (list "cpu" "cpuset" "io" "memory" "pids")
             (marionette-eval
              `(begin
                 (use-modules (srfi srfi-1)
-- 
2.47.1