[bug#52174] [PATCH] gnu: Add podman

[bug#52174] [PATCH] gnu: Add podman

[guix-patches--- via]

> For podman to work, the user needs to run
> `sudo mount -t cgroup2 none /sys/fs/cgroup`

I mounted /sys/fs/cgroup as v2 but podman info still shows

host:
  ...
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1

and (expectedly) podman run fails:

Error: OCI runtime error: cgroups in hybrid mode not supported,
drop all controllers from cgroupv2

How do I look for a fix?

[bug#52174] [PATCH] gnu: Add podman

[Timmy Douglas via Guix-patches via]

* gnu/packages/containers.scm (crun, conmon, libslirp, slirp4netns,
cni-plugins, podman): Add podman and dependencies.
---

I was going to try running some docker containers on my Guix system
today, but I noticed docker was an old version. I decided to try
packaging podman as the daemonless aspect seems appealing and was able
to run a basic alpine image (rootless) after a couple of hours of
putting this together. This is one of my first packages for Guix, so
please give me some feedback--I'm also interested if anyone else would
like to work together on this, because I'm fairly new to Guix and I
haven't used podman before either. This probably needs some more
testing since I only tried a basic scenario.

For podman to work, I needed to run this:
`sudo mount -t cgroup2 none /sys/fs/cgroup`


gnu/packages/containers.scm | 304 ++++++++++++++++++++++++++++++++++++
 1 file changed, 304 insertions(+)
 create mode 100644 gnu/packages/containers.scm

diff --git a/gnu/packages/containers.scm b/gnu/packages/containers.scm
new file mode 100644
index 0000000000..1c83698c2e
--- /dev/null
+++ b/gnu/packages/containers.scm
@@ -0,0 +1,304 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 Timmy Douglas <mail@timmydouglas.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages containers)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (gnu packages)
+  #:use-module (guix packages)
+  #:use-module (guix download)
+  #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
+  #:use-module (guix build-system go)
+  #:use-module (guix build-system meson)
+  #:use-module (guix utils)
+  #:use-module (gnu packages autotools)
+  #:use-module (gnu packages base)
+  #:use-module (gnu packages check)
+  #:use-module (gnu packages compression)
+  #:use-module (gnu packages glib)
+  #:use-module (gnu packages gnupg)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages python)
+  #:use-module (gnu packages networking)
+  #:use-module (gnu packages pkg-config)
+  #:use-module (gnu packages selinux)
+  #:use-module (gnu packages version-control)
+  #:use-module (gnu packages virtualization)
+  #:use-module (gnu packages web))
+
+;; For podman to work, the user needs to run
+;; `sudo mount -t cgroup2 none /sys/fs/cgroup`
+
+(define-public crun
+  (package
+    (name "crun")
+    (version "1.3")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/containers/crun")
+             (commit "8e5757a4e68590326dafe8a8b1b4a584b10a1370") ; 1.3
+             (recursive? #t)))
+       (sha256
+        (base32 "01yiss2d57kwlxb7zlqzjwlg9fyaf19yjngd1mw9n4hxls3dfj3k"))
+       (file-name (git-file-name name version))))
+    
+    (build-system gnu-build-system)
+    (arguments
+     '(#:tests? #f
+                #:configure-flags '("--disable-systemd")
+                #:phases
+                (modify-phases %standard-phases
+                  (add-after 'unpack 'do-not-depend-on-git
+                    (lambda _
+                      (substitute* "autogen.sh"
+                        (("^git submodule update.*")
+                         ""))
+                      (with-output-to-file "git-version.h"
+                        (lambda ()
+                          (display (string-append
+                                    "/* autogenerated.  */\n#ifndef GIT_VERSION\n# define GIT_VERSION \""
+                                    "8e5757a4e68590326dafe8a8b1b4a584b10a1370" ; refactor this
+                                    "\"\n#endif\n"))))
+                      #t
+                      )))))
+    (inputs
+     `(("libcap" ,libcap)
+       ("libseccomp" ,libseccomp)
+       ("libyajl" ,libyajl)))
+    (native-inputs
+     `(("automake" ,automake)
+       ("autoreconf" ,autoconf)
+       ("git" ,git)
+       ("libtool" ,libtool)
+       ("pkg-config" ,pkg-config)
+       ("python-3" ,python-3)))
+    (home-page "https://github.com/containers/crun")
+    (synopsis "OCI Container runtime")
+    (description
+     "crun is a fast and low-memory footprint OCI Container Runtime fully written in C.")
+    (license license:gpl2+)))
+
+(define-public conmon
+  (package
+    (name "conmon")
+    (version "v2.0.30")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/containers/conmon")
+             (commit version)))
+       (sha256
+        (base32 "1sxpbm01g4xak4kqwvk45gmzr6n9bjzlfp1j85wyz8rj2hg2x4rm"))
+       (file-name (git-file-name name version))))
+    
+    (build-system gnu-build-system)
+    (arguments
+     `(#:make-flags (list ,(string-append "CC=" (cc-for-target))
+                          (string-append "PREFIX=" %output))
+                    #:tests? #f ; currently broken as go tries to use network
+                    #:phases (modify-phases %standard-phases
+                               (delete 'configure)
+                               (add-after 'unpack 'set-env
+                                 (lambda* (#:key inputs #:allow-other-keys)
+                                   ;; when running go, things fail because
+                                   ;; HOME=/homeless-shelter.
+                                   (setenv "HOME" "/tmp")))
+                               (replace 'check
+                                 (lambda* (#:key tests? #:allow-other-keys)
+                                   (when tests?
+                                     (invoke "make" "test")))))))
+    (inputs
+     `(("glib" ,glib)
+       ("glibc" ,glibc)
+       ("libseccomp" ,libseccomp)
+       ("crun" ,crun)))
+    (native-inputs
+     `(("git" ,git)
+       ("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (home-page "https://github.com/containers/conmon")
+    (synopsis "Monitoring and communication tool between container manager and OCI runtime")
+    (description
+     "Conmon is a monitoring program and communication tool between a container
+manager (like Podman or CRI-O) and an OCI runtime (like runc or crun) for a
+single container.")
+    (license license:asl2.0)))
+
+(define-public libslirp
+  (package
+    (name "libslirp")
+    (version "v4.6.1")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://gitlab.freedesktop.org/slirp/libslirp")
+             (commit version)))
+       (sha256
+        (base32 "1b4cn51xvzbrxd63g6w1033prvbxfxsnsn1l0fa5i311xv28vkh0"))
+       (file-name (git-file-name name version))))
+    
+    (build-system meson-build-system)
+    (arguments '(#:tests? #f))
+    (inputs
+     `(("glib" ,glib)))
+    (native-inputs
+     `(("pkg-config" ,pkg-config)))
+    (home-page "https://gitlab.freedesktop.org/slirp/libslirp")
+    (synopsis "User-mode networking library")
+    (description
+     "libslirp is a user-mode networking library used by virtual machines,
+containers or various tools.")
+    (license license:non-copyleft))) ;fixme what is this?
+
+(define-public slirp4netns
+  (package
+    (name "slirp4netns")
+    (version "v1.1.12")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/rootless-containers/slirp4netns")
+             (commit version)))
+       (sha256
+        (base32 "03llv4dlf7qqxwz4zdyk926g4bigfj2gb50glm70ciflpvzs8081"))
+       (file-name (git-file-name name version))))
+    
+    (build-system gnu-build-system)
+    (arguments '(#:tests? #f))
+    (inputs
+     `(("glib" ,glib)
+       ("libcap" ,libcap)
+       ("libseccomp" ,libseccomp)
+       ("libslirp" ,libslirp)))
+    (native-inputs
+     `(("automake" ,automake)
+       ("autoreconf" ,autoconf)
+       ("pkg-config" ,pkg-config)))
+    (home-page "https://github.com/rootless-containers/slirp4netns")
+    (synopsis "User-mode networking for unprivileged network namespaces")
+    (description
+     "slirp4netns provides user-mode networking (\"slirp\") for unprivileged network namespaces.")
+    (license license:gpl2+)))
+
+(define-public cni-plugins
+  (package
+    (name "cni-plugins")
+    (version "v1.0.1")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/containernetworking/plugins")
+             (commit version)))
+       (sha256
+        (base32 "1j91in0mg4nblpdccyq63ncbnn2pc2zzjp1fh3jy0bsndllgv0nc"))
+       (file-name (git-file-name name version))))
+    
+    (build-system go-build-system)
+    (arguments
+     `(#:unpack-path "github.com/containernetworking/plugins"
+                     #:tests? #f
+                     #:phases (modify-phases %standard-phases
+                                (replace 'build
+                                  (lambda _
+                                    (with-directory-excursion "src/github.com/containernetworking/plugins"
+                                      (invoke "./build_linux.sh"))))
+                                (replace 'install
+                                  (lambda* (#:key outputs #:allow-other-keys)
+                                    (copy-recursively "src/github.com/containernetworking/plugins/bin"
+                                                      (string-append (assoc-ref outputs "out") "/bin"))
+                                    #t)))))
+    (home-page "https://github.com/containernetworking/plugins")
+    (synopsis "CNI network plugins")
+    (description
+     "Some CNI network plugins, maintained by the containernetworking team.")
+    (license license:asl2.0)))
+
+(define-public podman
+  (package
+    (name "podman")
+    (version "v3.4.2")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/containers/podman")
+             (commit version)))
+       (sha256
+        (base32 "0v1xpd1q6ym9ibaj6242v4mp0wwdmj4dd9l7zfyydbxrx6a8ahjn"))
+       (file-name (git-file-name name version))))
+    
+    (build-system gnu-build-system)
+    (arguments
+     `(#:make-flags (list ,(string-append "CC=" (cc-for-target))
+                          (string-append "PREFIX=" %output))
+                    #:tests? #f ; need to setup ginkgo
+                    #:phases (modify-phases %standard-phases
+                               (delete 'configure)
+                               (add-after 'unpack 'set-env
+                                 (lambda* (#:key inputs #:allow-other-keys)
+                                   ;; when running go, things fail because
+                                   ;; HOME=/homeless-shelter.
+                                   (setenv "HOME" "/tmp")))
+                               (add-after 'unpack 'fix-hardcoded-paths
+                                 (lambda _
+                                   (substitute* (find-files "libpod" "\\.go")
+                                     (("exec.LookPath[(][\"]slirp4netns[\"][)]")
+                                      (string-append "exec.LookPath(\"" (which "slirp4netns") "\")")))
+                                   (substitute* "vendor/github.com/containers/common/pkg/config/config_linux.go"
+                                     (("/usr/local/libexec/podman")
+                                      (string-append (assoc-ref %outputs "out") "/bin")))
+                                   (substitute* "vendor/github.com/containers/common/pkg/config/default.go"
+                                     (("/usr/libexec/podman/conmon") (which "conmon"))
+                                     (("/usr/local/libexec/cni")
+                                      (string-append (assoc-ref %build-inputs "cni-plugins") "/bin"))
+                                     (("/usr/bin/crun") (which "crun")))
+                                   #true))
+                               (replace 'check
+                                 (lambda* (#:key tests? #:allow-other-keys)
+                                   (when tests?
+                                     (invoke "make" "test")))))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("cni-plugins" ,cni-plugins)
+       ("conmon" ,conmon)
+       ("gpgme" ,gpgme)
+       ("go-md2man" ,go-github-com-go-md2man)
+       ("iptables" ,iptables) ; fixme not sure if podman will call this using $PATH
+       ("libassuan" ,libassuan)
+       ("libseccomp" ,libseccomp)
+       ("libselinux" ,libselinux)
+       ("slirp4netns" ,slirp4netns)
+       ("crun" ,crun)))
+    (native-inputs
+     `(("git" ,git)
+       ("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (home-page "https://podman.io")
+    (synopsis "Manage containers, images, pods, and their volumes")
+    (description
+     "Podman (the POD MANager) is a tool for managing containers and images,
+volumes mounted into those containers, and pods made from groups of
+containers.")
+    (license license:asl2.0)))
-- 
2.33.1

[bug#52174] [PATCH] gnu: Add podman

[Ludovic Courtès]

Hi Timmy,

Timmy Douglas <mail@timmydouglas.com> skribis:

> * gnu/packages/containers.scm (crun, conmon, libslirp, slirp4netns,
> cni-plugins, podman): Add podman and dependencies.
> ---
>
> I was going to try running some docker containers on my Guix system
> today, but I noticed docker was an old version. I decided to try
> packaging podman as the daemonless aspect seems appealing and was able
> to run a basic alpine image (rootless) after a couple of hours of
> putting this together. This is one of my first packages for Guix, so
> please give me some feedback--I'm also interested if anyone else would
> like to work together on this, because I'm fairly new to Guix and I
> haven't used podman before either. This probably needs some more
> testing since I only tried a basic scenario.
>
> For podman to work, I needed to run this:
> `sudo mount -t cgroup2 none /sys/fs/cgroup`

Woow, that’s a great start, and it’ll certainly be useful to many.

Overall it LGTM.  Here are some minor issues that would be nice
addressing:

> gnu/packages/containers.scm | 304 ++++++++++++++++++++++++++++++++++++
>  1 file changed, 304 insertions(+)
>  create mode 100644 gnu/packages/containers.scm

We usually have one patch per new package; IWBN if you could split this
patch accordingly.

Make sure to add ‘containers.scm’ to ‘gnu/local.mk’ too.

> +    (arguments
> +     '(#:tests? #f
> +                #:configure-flags '("--disable-systemd")
> +                #:phases

Please align keywords, as in:

  '(#:tests? #f
    #:configure-flags …
    #:phases …)

> +                          (display (string-append
> +                                    "/* autogenerated.  */\n#ifndef GIT_VERSION\n# define GIT_VERSION \""
> +                                    "8e5757a4e68590326dafe8a8b1b4a584b10a1370" ; refactor this

You can write “,commit” (read: “unquote commit”), assuming there’s a
‘commit’ local variable above.

> +    (inputs
> +     `(("libcap" ,libcap)
> +       ("libseccomp" ,libseccomp)
> +       ("libyajl" ,libyajl)))
> +    (native-inputs
> +     `(("automake" ,automake)
> +       ("autoreconf" ,autoconf)
> +       ("git" ,git)
> +       ("libtool" ,libtool)
> +       ("pkg-config" ,pkg-config)
> +       ("python-3" ,python-3)))

IWBN if you could run packages through ‘guix style’:

  https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-style.html

> +    (name "conmon")
> +    (version "v2.0.30")

Please remove “v” from the version number (here and elsewhere), and…

> +       (uri (git-reference
> +             (url "https://github.com/containers/conmon")
> +             (commit version)))
                        ^
… add it here, with (string-append "v" version).

> +    (home-page "https://gitlab.freedesktop.org/slirp/libslirp")
> +    (synopsis "User-mode networking library")
> +    (description
> +     "libslirp is a user-mode networking library used by virtual machines,
> +containers or various tools.")
> +    (license license:non-copyleft))) ;fixme what is this?

It’s ‘license:bsd-3’, per
<https://gitlab.freedesktop.org/slirp/libslirp/-/blob/master/COPYRIGHT>.

That’s it.

Could you send updated patches?

Thank you, and apologies for the delay!

Ludo’.

[bug#52174] [PATCH v2 1/6] gnu: add crun

[Timmy Douglas via Guix-patches via]

* gnu/packages/containers.scm: new file
* gnu/local.mk: add containers.scm
---
 gnu/local.mk                |  1 +
 gnu/packages/containers.scm | 87 +++++++++++++++++++++++++++++++++++++
 2 files changed, 88 insertions(+)
 create mode 100644 gnu/packages/containers.scm

diff --git a/gnu/local.mk b/gnu/local.mk
index 33e108c752..8ffcf662ce 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -157,6 +157,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/packages/configuration-management.scm	\
   %D%/packages/conky.scm			\
   %D%/packages/connman.scm			\
+  %D%/packages/containers.scm			\
   %D%/packages/convmv.scm			\
   %D%/packages/cook.scm				\
   %D%/packages/coq.scm				\
diff --git a/gnu/packages/containers.scm b/gnu/packages/containers.scm
new file mode 100644
index 0000000000..7ffe6e95d0
--- /dev/null
+++ b/gnu/packages/containers.scm
@@ -0,0 +1,87 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 Timmy Douglas <mail@timmydouglas.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages containers)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (gnu packages)
+  #:use-module (guix packages)
+  #:use-module (guix download)
+  #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
+  #:use-module (guix build-system go)
+  #:use-module (guix build-system meson)
+  #:use-module (guix utils)
+  #:use-module (gnu packages autotools)
+  #:use-module (gnu packages base)
+  #:use-module (gnu packages check)
+  #:use-module (gnu packages compression)
+  #:use-module (gnu packages glib)
+  #:use-module (gnu packages gnupg)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages python)
+  #:use-module (gnu packages networking)
+  #:use-module (gnu packages pkg-config)
+  #:use-module (gnu packages selinux)
+  #:use-module (gnu packages version-control)
+  #:use-module (gnu packages virtualization)
+  #:use-module (gnu packages web))
+
+(define-public crun
+  (let ((commit "8e5757a4e68590326dafe8a8b1b4a584b10a1370"))
+    (package
+      (name "crun")
+      (version "1.3")
+      (source
+       (origin
+         (method git-fetch)
+         (uri (git-reference
+               (url "https://github.com/containers/crun")
+               (commit commit)
+               (recursive? #t)))
+         (sha256
+          (base32 "01yiss2d57kwlxb7zlqzjwlg9fyaf19yjngd1mw9n4hxls3dfj3k"))
+         (file-name (git-file-name name version))))
+
+      (build-system gnu-build-system)
+      (arguments
+       `(#:tests? #f
+         #:configure-flags '("--disable-systemd")
+         #:phases
+         (modify-phases %standard-phases
+           (add-after 'unpack 'do-not-depend-on-git
+             (lambda _
+               (substitute* "autogen.sh"
+                 (("^git submodule update.*")
+                  ""))
+               (with-output-to-file "git-version.h"
+                 (lambda ()
+                   (display (string-append
+                             "/* autogenerated.  */\n#ifndef GIT_VERSION\n# define GIT_VERSION \""
+                             ,commit
+                             "\"\n#endif\n"))))
+               #t)))))
+      (inputs
+       (list libcap libseccomp libyajl))
+      (native-inputs
+       (list automake autoconf git libtool pkg-config python-3))
+      (home-page "https://github.com/containers/crun")
+      (synopsis "OCI Container runtime")
+      (description
+       "crun is a fast and low-memory footprint OCI Container Runtime fully written in C.")
+      (license license:gpl2+))))
-- 
2.34.0

[bug#52174] [PATCH v2 1/6] gnu: add crun

[Timmy Douglas via Guix-patches via]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Timmy Douglas <mail@timmydouglas.com> skribis:
>
>> * gnu/packages/containers.scm: new file
>> * gnu/local.mk: add containers.scm
>
> [...]
>
>> +      (arguments
>> +       `(#:tests? #f
>
> Sorry for not noticing earlier: we try hard to run test suites, and only
> disable it as a last resort, with a comment explaining why.  Often, we
> disable specific tests that cannot run in the build environment (for
> instance because they assume network access) and run everything else.
>
> I had a quick go at running crun’s test suite, and there’s a number of
> failures, which is not reassuring.
>
> Could you investigate a bit with the goal of fixing any actual issues
> the test suite reveals and/or skipping individual tests that cannot be
> run?

are there some docs I could refer to on how Guix sandboxes the build?
I tried
https://guix.gnu.org/manual/en/html_node/Debugging-Build-Failures.html#Debugging-Build-Failures
but I think I need to go further than sourcing env vars.


I tried modifying `make check` phase to invoking `mount` to see if I
could poke around (which didn't exactly work in my quick attempt...)
because I'm not sure how to debug some of the issues. I suspect some are
due to the fs layout/mounts. Probably could figure it out with some more
time but thought it might be best to ask first.

With these changes I could get them to pass by manually `make check`ing
in the --keep-failed directory:

           (add-after 'unpack 'fix-tests
             (lambda _
               (substitute* (find-files "tests" "\\.(c|py)")
                 (("/bin/true") (which "true"))
                 (("/bin/false") (which "false"))
                 ; relies on sd_notify which requires systemd?
                 (("\"sd-notify\" : test_sd_notify,") "")
                 (("\"sd-notify-file\" : test_sd_notify_file,") "")))))))


from guix build:

============================================================================
Testsuite summary for crun UNKNOWN
============================================================================
# TOTAL: 73
# PASS:  9
# SKIP:  28
# XFAIL: 0
# FAIL:  36
# XPASS: 0
# ERROR: 0


from cd /tmp/guix-build-crun-1.3.drv-5/source && make check

============================================================================
Testsuite summary for crun UNKNOWN
============================================================================
# TOTAL: 73
# PASS:  40
# SKIP:  33
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0


> Nitpick: You can omit the trailing #t now (yay!).

great!

[bug#52174] [PATCH v2 1/6] gnu: add crun

[Ludovic Courtès]

Hello,

Timmy Douglas <mail@timmydouglas.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> Sorry for not noticing earlier: we try hard to run test suites, and only
>> disable it as a last resort, with a comment explaining why.  Often, we
>> disable specific tests that cannot run in the build environment (for
>> instance because they assume network access) and run everything else.
>>
>> I had a quick go at running crun’s test suite, and there’s a number of
>> failures, which is not reassuring.
>>
>> Could you investigate a bit with the goal of fixing any actual issues
>> the test suite reveals and/or skipping individual tests that cannot be
>> run?
>
> are there some docs I could refer to on how Guix sandboxes the build?
> I tried
> https://guix.gnu.org/manual/en/html_node/Debugging-Build-Failures.html#Debugging-Build-Failures
> but I think I need to go further than sourcing env vars.

Pretty much all the tricks are there.

There’s also info on what’s visible in the build environment here:

  https://guix.gnu.org/manual/en/html_node/Invoking-guix_002ddaemon.html

> I tried modifying `make check` phase to invoking `mount` to see if I
> could poke around (which didn't exactly work in my quick attempt...)
> because I'm not sure how to debug some of the issues. I suspect some are
> due to the fs layout/mounts. Probably could figure it out with some more
> time but thought it might be best to ask first.
>
> With these changes I could get them to pass by manually `make check`ing
> in the --keep-failed directory:
>
>            (add-after 'unpack 'fix-tests
>              (lambda _
>                (substitute* (find-files "tests" "\\.(c|py)")
>                  (("/bin/true") (which "true"))
>                  (("/bin/false") (which "false"))
>                  ; relies on sd_notify which requires systemd?
>                  (("\"sd-notify\" : test_sd_notify,") "")
>                  (("\"sd-notify-file\" : test_sd_notify_file,") "")))))))
>
>
> from guix build:
>
> ============================================================================
> Testsuite summary for crun UNKNOWN
> ============================================================================
> # TOTAL: 73
> # PASS:  9
> # SKIP:  28
> # XFAIL: 0
> # FAIL:  36
> # XPASS: 0
> # ERROR: 0

There are 36 failures though; does ‘test-suite.log’ show more details?

Thanks,
Ludo’.

[bug#52174] [PATCH v2 1/6] gnu: add crun

[Timmy Douglas via Guix-patches via]

Ludovic Courtès <ludo@gnu.org> writes:

>> from guix build:
>>
>> ============================================================================
>> Testsuite summary for crun UNKNOWN
>> ============================================================================
>> # TOTAL: 73
>> # PASS:  9
>> # SKIP:  28
>> # XFAIL: 0
>> # FAIL:  36
>> # XPASS: 0
>> # ERROR: 0
>
> There are 36 failures though; does ‘test-suite.log’ show more details?

Maybe my original reply wasn't clear--they all pass when I cd into the
tmp (--keep-failed) directory and run them manually. They fail when the builder runs
them. I think I put the reasons in the v3 patch. For crun, it's because
it needs the /sys/fs/cgroup mount which doesn't seem to be there for the
builder. For the others, it's a similar situation--off the top of my
head, they were either missing that mount or a /dev device.

bug#52174: [PATCH] gnu: Add podman

[Ludovic Courtès]

Hi,

Timmy Douglas <mail@timmydouglas.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>>> from guix build:
>>>
>>> ============================================================================
>>> Testsuite summary for crun UNKNOWN
>>> ============================================================================
>>> # TOTAL: 73
>>> # PASS:  9
>>> # SKIP:  28
>>> # XFAIL: 0
>>> # FAIL:  36
>>> # XPASS: 0
>>> # ERROR: 0
>>
>> There are 36 failures though; does ‘test-suite.log’ show more details?
>
> Maybe my original reply wasn't clear--they all pass when I cd into the
> tmp (--keep-failed) directory and run them manually. They fail when the builder runs
> them. I think I put the reasons in the v3 patch. For crun, it's because
> it needs the /sys/fs/cgroup mount which doesn't seem to be there for the
> builder. For the others, it's a similar situation--off the top of my
> head, they were either missing that mount or a /dev device.

Oh I see.  I pushed v3 as 637dec9d45db4df2a3e6aa565fa2c5cf6bb77768 with
minor tweaks (long lines and one or two synopses/descriptions tweaked.)

IWBN to see if we can still run those tests somehow, or at least the
subset of them that doesn’t rely on /sys/fs/cgroup.  I’d argue that the
test harness should automatically skip tests that cannot be run; perhaps
worth raising upstream?

Thanks!

Ludo’.

[bug#52174] [PATCH] gnu: Add podman

[Timmy Douglas via Guix-patches via]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
>> Timmy Douglas <mail@timmydouglas.com> skribis:
>>
>> Maybe my original reply wasn't clear--they all pass when I cd into the
>> tmp (--keep-failed) directory and run them manually. They fail when the builder runs
>> them. I think I put the reasons in the v3 patch. For crun, it's because
>> it needs the /sys/fs/cgroup mount which doesn't seem to be there for the
>> builder. For the others, it's a similar situation--off the top of my
>> head, they were either missing that mount or a /dev device.
>
> Oh I see.  I pushed v3 as 637dec9d45db4df2a3e6aa565fa2c5cf6bb77768 with
> minor tweaks (long lines and one or two synopses/descriptions tweaked.)

thanks for the help Ludo!

> IWBN to see if we can still run those tests somehow, or at least the
> subset of them that doesn’t rely on /sys/fs/cgroup.  I’d argue that the
> test harness should automatically skip tests that cannot be run; perhaps
> worth raising upstream?

I'd like to get the tests to run also, but the builder sandbox appears
to be blocking some pretty major functionality that the tests would rely
on.

I think pretty much all of the container/crun ones would rely on the
cgroup mount because that's the kernel interface into the container
APIs... Is there some way that guix and the builder could eventually
expose those by default? I don't know how receptive upstream would be
towards an ask to run container tests with the container interface
disabled?

For the networking ones that fail, they try to use /dev/net/tun. Like
the cgroup one, I assume this is a kernel interface needed to perform
network operations. I guess the builders disable this as a part of the
network disabling stuff because the tests pass outside of the builder
sandbox.

The cni-plugins (cni=container network interface) use /var/run to mount
network namespaces. /var/run is present on my machine but I don't think
it exists inside the builder sandbox. The actual directory used can be
set with XDG_RUNTIME_DIR, but it appears the code still checks the
ownership of /var/run to see if it's running in a user namespace:
https://github.com/containernetworking/plugins/blob/2c46a726805bcf13e2f78580c57b21e9de107285/pkg/testutils/netns_linux.go

[bug#52174] [PATCH] gnu: Add podman

[Ludovic Courtès]

Hello,

Timmy Douglas <mail@timmydouglas.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> IWBN to see if we can still run those tests somehow, or at least the
>> subset of them that doesn’t rely on /sys/fs/cgroup.  I’d argue that the
>> test harness should automatically skip tests that cannot be run; perhaps
>> worth raising upstream?
>
> I'd like to get the tests to run also, but the builder sandbox appears
> to be blocking some pretty major functionality that the tests would rely
> on.
>
> I think pretty much all of the container/crun ones would rely on the
> cgroup mount because that's the kernel interface into the container
> APIs... Is there some way that guix and the builder could eventually
> expose those by default? I don't know how receptive upstream would be
> towards an ask to run container tests with the container interface
> disabled?

The daemon probably won’t expose those; we’re rather conservative into
what to expose and how to change it because changes could break
bit-reproducible builds in unexpected ways.

I understand many/most tests require cgroups, I’m just wondering if we
can run at least those that don’t require it.  Perhaps we’re talking
about a very limited number of tests, in which case it’s moot, I don’t
know.

> For the networking ones that fail, they try to use /dev/net/tun. Like
> the cgroup one, I assume this is a kernel interface needed to perform
> network operations. I guess the builders disable this as a part of the
> network disabling stuff because the tests pass outside of the builder
> sandbox.

Yeah.

> The cni-plugins (cni=container network interface) use /var/run to mount
> network namespaces. /var/run is present on my machine but I don't think
> it exists inside the builder sandbox. The actual directory used can be
> set with XDG_RUNTIME_DIR, but it appears the code still checks the
> ownership of /var/run to see if it's running in a user namespace:
> https://github.com/containernetworking/plugins/blob/2c46a726805bcf13e2f78580c57b21e9de107285/pkg/testutils/netns_linux.go

Hmm OK.  So yeah, maybe there’s nothing we can do here.

Thanks for your feedback,
Ludo’.