ZFS on Guix

ZFS on Guix

[raid5atemyhomework]

Hi guix-developers,

I'm trying to get ZFS working on a test VM running Guix System.

Attempting to do a simple `guix install zfs` does not work, as I reported in https://issues.guix.gnu.org/45401 , ZFS 0.8.5 is only up to kernel 5.9 but latest guix has kernel 5.10.

I created a procedure to modify the base `zfs` package and compile it for a different Linux kernel:

```scheme
(define (make-zfs-package kernel)
  (package
    (inherit zfs)
    (name (string-append "zfs-for-"
                         (package-name kernel)
                         "-"
                         (package-version kernel)
                         "-version"))
    (arguments
      (cons* #:linux kernel (package-arguments zfs)))))
```

Then in my system configuration `.scm`, I selected a specific Linux and had its module added to the boot and had it loaded in a Shepherd service:

```scheme
(define system-kernel linux-libre-5.4)
(define system-zfs (make-zfs-package system-kernel))

;; ...

(operating-system
  (kernel system-kernel)
  (kernel-loadable-modules (list (list system-zfs "module")))
  ;; ...
  (packages
    (cons* system-zfs
           ; ...
           %base-packages))
  ;; ...

  (services
    (cons* (service kernel-module-loader-service-type
                    '("zfs"))
           ; ...
           %base-services))
  ;; ...
  )
```

As I documented here: https://issues.guix.gnu.org/45592#1

Now ZFS is loaded at boot (`lsmod | grep zfs` shows ZFS loaded), and `zfs version` and `zpool list` don't error out.

However, when I created some extra space in the QEMU image and created a partition `vda3`, I get an error in `zpool create tank vda3`.
The error is `cannot mount 'tank': Input/output error`.

I've also tried adding a new device to the QEMU boot as well, also the same occurs.

Looking at a partition editor, the partition does get formatted as zfs, it just won't mount.



Has anyone ever managed to get ZFS working on Guix?  Any tips?  Is it enough to `modprobe` it in a system service or does it have to somehow be added to the Linux command line?  Maybe it only works on even older Linux-libre versions than 5.4?

Thanks
raid5atemyhomework

Re: ZFS on Guix

[raid5atemyhomework]

Hi guix-developers,

I just did a an `strace -f zfs mount tank`, and got this interesting bit in a child process:

    execve("/gnu/store/a45p39mgqvfd8kjwibyr0q42klmw7gmf-util-linux-2.35.1-lib/bin/mount", ["/gnu/store/a45p39mgqvfd8kjwibyr0"..., "--no-canonicalize", "-t", "zfs", "-o", "defaults,atime,strictatime,dev,e"..., "tank", "/tank"], 0x7ffee6ad7bf8 /* 29 vars */) = -1 ENOENT (No such file or directory)

The `/gnu/store` directory indeed does not contain a `/bin/mount` executable, it only contains `include` `lib` and `share` dirs.  So it looks to me that maybe the `zfs` package needs some more modifications to get it working on guix, since apparently ZFS userspace tools still go through `mount` anyway.

Trying to `mount` directly makes ZFS complain about requiring a "legacy" mountpoint (there might be an option somewhere not shown by the `strace` that suppress that complaint), and setting a legacy mountpoint makes `mount` error with `filesystem 'tank' cannot be mounted at '/tank' due to canonicalization error 2.` even though I give `--no-canonicalize`, though.

Thanks
raid5atemyhomework

Re: ZFS on Guix

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]

Hi,

the reason is that our "zfs" package uses

  ("util-linux" ,util-linux "lib")

and then does

               (substitute* "lib/libzfs/libzfs_mount.c"
                 (("/bin/mount") (string-append util-linux "/bin/mount"))
                 (("/bin/umount") (string-append util-linux "/bin/umount")))

.

That can't work.

zfs interna that will be patched by the Guix package are:

./lib/libzfs/libzfs_mount.c

do_mount(const char *src, const char *mntpt, char *opts)

        char *argv[9] = {
            "/bin/mount",
            "--no-canonicalize",
            "-t", MNTTYPE_ZFS,
            "-o", opts,
            (char *)src,
            (char *)mntpt,
            (char *)NULL };

Easy fix would be to change our package to have both

  ("util-linux:lib" ,util-linux "lib")

and

  ("util-linux" ,util-linux)

.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: ZFS on Guix

[raid5atemyhomework]

Yes, I found that out after a little more digging.

I have a preliminary patch here: https://issues.guix.gnu.org/45592
However I will make a patchset based on this in order to support /home on ZFS and managing ZFS the "legacy" way by (file-system ...) declaration, please wait.



> Hi,
>
> the reason is that our "zfs" package uses
>
> ("util-linux" ,util-linux "lib")
>
> and then does
>
> (substitute* "lib/libzfs/libzfs_mount.c"
> (("/bin/mount") (string-append util-linux "/bin/mount"))
> (("/bin/umount") (string-append util-linux "/bin/umount")))
>
> .
>
> That can't work.
>
> zfs interna that will be patched by the Guix package are:
>
> ./lib/libzfs/libzfs_mount.c
>
> do_mount(const char *src, const char *mntpt, char *opts)
>
> char *argv[9] = {
> "/bin/mount",
> "--no-canonicalize",
> "-t", MNTTYPE_ZFS,
> "-o", opts,
> (char *)src,
> (char *)mntpt,
> (char *)NULL };
>
> Easy fix would be to change our package to have both
>
> ("util-linux:lib" ,util-linux "lib")
>
> and
>
> ("util-linux" ,util-linux)
>
> .

Re: ZFS on Guix

[raid5atemyhomework]

Available here: https://issues.guix.gnu.org/45643

Please review! Hopefully it gets merged!

> Yes, I found that out after a little more digging.
>
> I have a preliminary patch here: https://issues.guix.gnu.org/45592
> However I will make a patchset based on this in order to support /home on ZFS and managing ZFS the "legacy" way by (file-system ...) declaration, please wait.
>
> > Hi,
> > the reason is that our "zfs" package uses
> > ("util-linux" ,util-linux "lib")
> > and then does
> > (substitute* "lib/libzfs/libzfs_mount.c"
> > (("/bin/mount") (string-append util-linux "/bin/mount"))
> > (("/bin/umount") (string-append util-linux "/bin/umount")))
> > .
> > That can't work.
> > zfs interna that will be patched by the Guix package are:
> > ./lib/libzfs/libzfs_mount.c
> > do_mount(const char *src, const char *mntpt, char *opts)
> > char *argv[9] = {
> > "/bin/mount",
> > "--no-canonicalize",
> > "-t", MNTTYPE_ZFS,
> > "-o", opts,
> > (char *)src,
> > (char *)mntpt,
> > (char *)NULL };
> > Easy fix would be to change our package to have both
> > ("util-linux:lib" ,util-linux "lib")
> > and
> > ("util-linux" ,util-linux)
> > .

Re: ZFS on Guix

[raid5atemyhomework]

Hi guix-developers,


In https://lists.gnu.org/archive/html/guix-devel/2021-01/msg00053.html Carlo mentioned to instead use the `service` system to modify all parts needed in the operating system to get ZFS installed, and noted as well that the only thing missing in the `service` system is the ability to push kernel modules into the operating system.

Here's a sketch of an attempt to add that feature so we can conveniently add ZFS on Guix by just adding a `(service zfs-service-type...)`.

First, we need a new `kernel-loadable-module-service-type` in `gnu/services.scm`:

```scheme
(define kernel-loadable-module-service-type
  ;; A service to add the kernel modules of a package.
  (service-type (name 'kernel-loadable-module)
                (extensions '())
                (compose concatenate)
                (extend append)
                (description "Register kernel modules that will be placed in a
loadable place where the kernel can find them.")
                (default-value '())))
```

Absolutely no idea if a `'()` extensions makes sense but *shrug*.

Then, in `gnu/system.scm`, we create this new function:

```scheme
(define (operating-system-all-kernel-loadable-modules os)
  ;; Gathers the kernel-loadable modules in the KERNEL-LOADABLE-FIELD
  ;; field of the OS, as well as those registered by services.
  (let ((service (fold-services (cons (service kernel-loadable-module-service-type '())
                                      (operating-system-user-services os))
                                #:target-type kernel-loadable-module-service-type)))
    (append (service-value service)
            (operating-system-kernel-loadable-modules))))
```

Does that make sense? Am I misusing `fold-services` and `service-value` here?

Onward, we modify as well the function `operating-system-directory-base`:

```scheme
(define* (operating-system-directory-base-entries os)
  "Return the basic entries of the 'system' directory of OS for use as the
value of the SYSTEM-SERVICE-TYPE service."
  (let* ((locale  (operating-system-locale-directory os))
         (kernel  (operating-system-kernel os))
         (hurd    (operating-system-hurd os))
         (modules (operating-system-all-kernel-loadable-modules os))
;...
```

The above is the reason why we need to `cons` a synthetic `kernel-loadable-module-service-type` service, the function `operating-system-directory-base-entries` is used do create the default for `operating-system-essential-services`, and the `operating-sstem-services` is just the concatenation of the user and essential services.  I think.

Would this work?

Thanks
raid5atemyhomework

Re: ZFS on Guix

[raid5atemyhomework]

Hi Carlo and guix-developers,

I have this patch below for creating a new `kernel-loadable-module-service-type`, which can be extended by another service to add kernel-loadable modules provided by packages, hope for a review.


From 984602faba1e18b9eb64e62970147aab653d997f Mon Sep 17 00:00:00 2001
From: raid5atemyhomework <raid5atemyhomework@protonmail.com>
Date: Tue, 5 Jan 2021 22:27:56 +0800
Subject: [PATCH] gnu: Add 'kernel-loadable-module-service-type' for services
 to extend with kernel-loadable modules.

---
 doc/guix.texi    |  6 +++++
 gnu/services.scm | 68 ++++++++++++++++++++++++++++++++++++++++++++++++
 gnu/system.scm   | 36 +++++++------------------
 3 files changed, 83 insertions(+), 27 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 0f6e95a65a..78770151e3 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -32409,6 +32409,12 @@ configuration when you use @command{guix system reconfigure},
 @command{guix system init}, or @command{guix deploy}.
 @end defvr

+@defvr {Scheme Variable} kernel-loadable-module-service-type
+Type of the service that collects lists of packages containing
+kernel-loadable modules, and adds them to the set of kernel-loadable
+modules.
+@end defvr
+
 @node Shepherd Services
 @subsection Shepherd Services

diff --git a/gnu/services.scm b/gnu/services.scm
index 13259dfaee..2c0bbf9725 100644
--- a/gnu/services.scm
+++ b/gnu/services.scm
@@ -33,6 +33,8 @@
   #:use-module (guix diagnostics)
   #:autoload   (guix openpgp) (openpgp-format-fingerprint)
   #:use-module (guix modules)
+  #:use-module (guix packages)
+  #:use-module (guix utils)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages hurd)
@@ -75,6 +77,7 @@
             service-back-edges
             instantiate-missing-services
             fold-services
+            kernel-loadable-module-service

             service-error?
             missing-value-service-error?
@@ -106,6 +109,7 @@
             profile-service-type
             firmware-service-type
             gc-root-service-type
+            kernel-loadable-module-service-type

             %boot-service
             %activation-service
@@ -864,6 +868,70 @@ as Wifi cards.")))
 will not be reclaimed by the garbage collector.")
                 (default-value '())))

+;; Configuration for the kernel builder.
+;; Only used by the KERNEL-LOADABLE-MODULE-SERVICE-TYPE.
+(define-record-type* <kernel-builder-configuration> kernel-builder-configuration
+  make-kernel-builder-configuration
+  kernel-builder-configuration?
+  this-kernel-builder-configuration
+
+  (kernel   kernel-builder-configuration-kernel   (default #f))
+  (hurd     kernel-builder-configuration-hurd     (default #f))
+  (modules  kernel-builder-configuration-modules  (default '())))
+
+(define (package-for-kernel target-kernel module-package)
+  "Return a package like MODULE-PACKAGE, adapted for TARGET-KERNEL, if
+possible (that is if there's a LINUX keyword argument in the build system)."
+  (package
+    (inherit module-package)
+    (arguments
+     (substitute-keyword-arguments (package-arguments module-package)
+       ((#:linux kernel #f)
+        target-kernel)))))
+
+(define (kernel-builder-configuration->system-entry config)
+  "Return the kernel and hurd entries of the 'system' directory of OS."
+  (mbegin %store-monad
+    (let* ((kernel  (kernel-builder-configuration-kernel config))
+           (hurd    (kernel-builder-configuration-hurd config))
+           (modules (kernel-builder-configuration-modules config))
+           (kernel  (if hurd
+                        kernel
+                        (profile
+                         (content (packages->manifest
+                                   (cons kernel
+                                         (map (lambda (module)
+                                                (if (package? module)
+                                                    (package-for-kernel kernel module)
+                                                    module))
+                                              modules))))
+                         (hooks (list linux-module-database))))))
+      (return `(("kernel" ,kernel)
+                ,@(if hurd `(("hurd" ,hurd)) '()))))))
+(define (kernel-builder-configuration-add-modules config modules)
+  "Constructs a kernel builder configuration that has its modules extended."
+  (kernel-builder-configuration
+    (inherit config)
+    (modules (append (kernel-builder-configuration-modules config) modules))))
+
+(define kernel-loadable-module-service-type
+  (service-type (name 'kernel-loadable-modules)
+                (extensions
+                 (list (service-extension system-service-type
+                                          kernel-builder-configuration->system-entry)))
+                (compose concatenate)
+                (extend kernel-builder-configuration-add-modules)
+                (description
+                 "Register packages containing kernel-loadable modules and adds them
+to the system.")))
+(define (kernel-loadable-module-service kernel hurd modules)
+  "Constructs the service that sets up kernel loadable modules."
+  (service kernel-loadable-module-service-type
+    (kernel-builder-configuration
+      (kernel kernel)
+      (hurd hurd)
+      (modules modules))))
+
 \f
 ;;;
 ;;; Service folding.
diff --git a/gnu/system.scm b/gnu/system.scm
index c284a18379..a7f2365754 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -600,16 +600,6 @@ OS."
       (file-append (operating-system-kernel os)
                       "/" (system-linux-image-file-name))))

-(define (package-for-kernel target-kernel module-package)
-  "Return a package like MODULE-PACKAGE, adapted for TARGET-KERNEL, if
-possible (that is if there's a LINUX keyword argument in the build system)."
-  (package
-    (inherit module-package)
-    (arguments
-     (substitute-keyword-arguments (package-arguments module-package)
-       ((#:linux kernel #f)
-        target-kernel)))))
-
 (define %default-modprobe-blacklist
   ;; List of kernel modules to blacklist by default.
   '("usbmouse" ;races with bcm5974, see <https://bugs.gnu.org/35574>
@@ -625,26 +615,10 @@ possible (that is if there's a LINUX keyword argument in the build system)."
   "Return the basic entries of the 'system' directory of OS for use as the
 value of the SYSTEM-SERVICE-TYPE service."
   (let* ((locale  (operating-system-locale-directory os))
-         (kernel  (operating-system-kernel os))
          (hurd    (operating-system-hurd os))
-         (modules (operating-system-kernel-loadable-modules os))
-         (kernel  (if hurd
-                      kernel
-                      (profile
-                       (content (packages->manifest
-                                 (cons kernel
-                                       (map (lambda (module)
-                                              (if (package? module)
-                                                  (package-for-kernel kernel
-                                                                      module)
-                                                  module))
-                                            modules))))
-                       (hooks (list linux-module-database)))))
          (initrd  (and (not hurd) (operating-system-initrd-file os)))
          (params  (operating-system-boot-parameters-file os)))
-    `(("kernel" ,kernel)
-      ,@(if hurd `(("hurd" ,hurd)) '())
-      ("parameters" ,params)
+    `(("parameters" ,params)
       ,@(if initrd `(("initrd" ,initrd)) '())
       ("locale" ,locale))))   ;used by libc

@@ -663,6 +637,10 @@ bookkeeping."
          (host-name (host-name-service (operating-system-host-name os)))
          (entries   (operating-system-directory-base-entries os)))
     (cons* (service system-service-type entries)
+           (kernel-loadable-module-service
+             (operating-system-kernel os)
+             (operating-system-hurd os)
+             (operating-system-kernel-loadable-modules os))
            %boot-service

            ;; %SHEPHERD-ROOT-SERVICE must come last so that the gexp that
@@ -699,6 +677,10 @@ bookkeeping."
 (define (hurd-default-essential-services os)
   (let ((entries (operating-system-directory-base-entries os)))
     (list (service system-service-type entries)
+          (kernel-loadable-module-service
+            (operating-system-kernel os)
+            (operating-system-hurd os)
+            (operating-system-kernel-loadable-modules os))
           %boot-service
           %hurd-startup-service
           %activation-service
--
2.29.2

Re: ZFS on Guix

[Carlo Zancanaro]

Hi raid5atemyhomework,

On Wed, Jan 06 2021, raid5atemyhomework wrote:
> I have this patch below for creating a new 
> `kernel-loadable-module-service-type`, which can be extended by 
> another service to add kernel-loadable modules provided by 
> packages, hope for a review.

I tried out building a VM using this patch and it seemed to work 
properly. I was able to add a kernel module by extending 
kernel-loadable-module-service-type, and then load it in the 
running VM with modprobe.

I only have superficial comments about the patch itself. Hopefully 
someone else will provide a better review than I can.

> From 984602faba1e18b9eb64e62970147aab653d997f Mon Sep 17 
> 00:00:00 2001
> From: raid5atemyhomework <raid5atemyhomework@protonmail.com>
> Date: Tue, 5 Jan 2021 22:27:56 +0800
> Subject: [PATCH] gnu: Add 'kernel-loadable-module-service-type' 
> for services
>  to extend with kernel-loadable modules.

This will need a proper commit message before being committed. 
Guix generally follows the GNU Change Log style[1]. You can see 
examples of what commit messages look like by looking at other 
commits in the repository.

> +;; Only used by the KERNEL-LOADABLE-MODULE-SERVICE-TYPE.

I would remove this comment. This is a description of the current 
state, rather than a restriction on how it should be used. As a 
description, it might get out of date (if we use 
<kernel-builder-configuration> values elsewhere), and it doesn't 
add much value (because it can be easily verified, and the record 
type isn't exported). If you indent this to be normative, the 
comment should explain why this type should not be used elsewhere.

> +      (return `(("kernel" ,kernel)
> +                ,@(if hurd `(("hurd" ,hurd)) '()))))))
> +(define (kernel-builder-configuration-add-modules config 
> modules)
> +  "Constructs a kernel builder configuration that has its 
> modules extended."

Put empty lines between definitions here ...

> +                 "Register packages containing kernel-loadable 
> modules and adds them
> +to the system.")))
> +(define (kernel-loadable-module-service kernel hurd modules)
> +  "Constructs the service that sets up kernel loadable 
> modules."

... and here.

It would also be worth adding a test to 
gnu/tests/linux-modules.scm to test loading modules through this 
service.

Other than that, it looks good to me and it seems to work in my 
limited testing. 👍 Good job!

Carlo

[1]: https://www.gnu.org/prep/standards/html_node/Change-Logs.html

Re: ZFS on Guix

[raid5atemyhomework]

Hi Carlo,

Thanks, I modified patch as below.

I also can't get `guix system vm` to run in my hacking env, so I couldn't make a VM for testing like you did.  I got this:

```
Formatting '/gnu/store/imdlq0cay61d2cw3199g04z9bp16qx8j-qemu-image', fmt=qcow2 cluster_size=65536 compression_type=zlib size=73400320 lazy_refcounts=off refcount_bits=16
Could not access KVM kernel module: Permission denied
qemu-system-x86_64: failed to initialize kvm: Permission denied
```

Any tips on getting `guix system vm` working?  I've never used VMs before.  My dev env is a foreign distro with Guix installed on top.

The above issue also means I haven't actually run the tests I made in `gnu/tests/linux-modules.scm`, because it requires building a VM as well.

However, I *did* do a bunch of `guix system build` tests:

* With the same `configuration.scm` that uses the legacy `(kernel-loadable-modules ...)` declaration in `operating-system`, I tried both with and without the patch.
  * Without patch and with patch created builds with a different hash, however, they linked to the same directories. `diff -r` did not find any differences.
* With the patch, I made two variants of `configuration.scm`, one that used the legacy `(kernel-loadable-modules...)` and another that used a service that extended `kernel-laodable-module-service-type`.
  * Builds resulted in the exact same system with exact same hash.



From 4beb73c62995cf236b402dad8e1c36016027c781 Mon Sep 17 00:00:00 2001
From: raid5atemyhomework <raid5atemyhomework@protonmail.com>
Date: Tue, 5 Jan 2021 22:27:56 +0800
Subject: [PATCH] gnu: Allow services to install kernel-loadable modules.

* gnu/system.scm (operating-system-directory-base-entries): Remove code
to handle generation of "kernel" and "hurd".
(operating-system-default-essential-services): Instantiate
kernel-loadable-module-service.
(hurd-default-essential-services): Instantiate
kernel-loadable-module-service.
(package-for-kernel): Move ...
* gnu/services.scm: ... to here.
(kernel-loadable-module-service-type): New variable.
(kernel-loadable-module-service): New procedure.
* gnu/tests/linux-modules.scm (run-loadable-kernel-modules-test): Move
code to ...
(run-loadable-kernel-modules-test-base): ... new procedure here.
(run-loadable-kernel-modules-service-test): New procedure.
(%test-loadable-kernel-modules-service-0): New variable.
(%test-loadable-kernel-modules-service-1): New variable.
(%test-loadable-kernel-modules-service-2): New variable.
* doc/guix.texi: Document kernel-loadable-module-service-type.
---
 doc/guix.texi               |  6 +++
 gnu/services.scm            | 70 ++++++++++++++++++++++++++++++++
 gnu/system.scm              | 37 +++++------------
 gnu/tests/linux-modules.scm | 81 ++++++++++++++++++++++++++++++++-----
 4 files changed, 157 insertions(+), 37 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 0f6e95a65a..78770151e3 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -32409,6 +32409,12 @@ configuration when you use @command{guix system reconfigure},
 @command{guix system init}, or @command{guix deploy}.
 @end defvr

+@defvr {Scheme Variable} kernel-loadable-module-service-type
+Type of the service that collects lists of packages containing
+kernel-loadable modules, and adds them to the set of kernel-loadable
+modules.
+@end defvr
+
 @node Shepherd Services
 @subsection Shepherd Services

diff --git a/gnu/services.scm b/gnu/services.scm
index 13259dfaee..d7332a46b2 100644
--- a/gnu/services.scm
+++ b/gnu/services.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2016 Chris Marusich <cmmarusich@gmail.com>
 ;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -33,6 +34,8 @@
   #:use-module (guix diagnostics)
   #:autoload   (guix openpgp) (openpgp-format-fingerprint)
   #:use-module (guix modules)
+  #:use-module (guix packages)
+  #:use-module (guix utils)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages hurd)
@@ -75,6 +78,7 @@
             service-back-edges
             instantiate-missing-services
             fold-services
+            kernel-loadable-module-service

             service-error?
             missing-value-service-error?
@@ -106,6 +110,7 @@
             profile-service-type
             firmware-service-type
             gc-root-service-type
+            kernel-loadable-module-service-type

             %boot-service
             %activation-service
@@ -864,6 +869,71 @@ as Wifi cards.")))
 will not be reclaimed by the garbage collector.")
                 (default-value '())))

+;; Configuration for the kernel builder.
+(define-record-type* <kernel-builder-configuration> kernel-builder-configuration
+  make-kernel-builder-configuration
+  kernel-builder-configuration?
+  this-kernel-builder-configuration
+
+  (kernel   kernel-builder-configuration-kernel   (default #f))
+  (hurd     kernel-builder-configuration-hurd     (default #f))
+  (modules  kernel-builder-configuration-modules  (default '())))
+
+(define (package-for-kernel target-kernel module-package)
+  "Return a package like MODULE-PACKAGE, adapted for TARGET-KERNEL, if
+possible (that is if there's a LINUX keyword argument in the build system)."
+  (package
+    (inherit module-package)
+    (arguments
+     (substitute-keyword-arguments (package-arguments module-package)
+       ((#:linux kernel #f)
+        target-kernel)))))
+
+(define (kernel-builder-configuration->system-entry config)
+  "Return the kernel and hurd entries of the 'system' directory."
+  (mbegin %store-monad
+    (let* ((kernel  (kernel-builder-configuration-kernel config))
+           (hurd    (kernel-builder-configuration-hurd config))
+           (modules (kernel-builder-configuration-modules config))
+           (kernel  (if hurd
+                        kernel
+                        (profile
+                         (content (packages->manifest
+                                   (cons kernel
+                                         (map (lambda (module)
+                                                (if (package? module)
+                                                    (package-for-kernel kernel module)
+                                                    module))
+                                              modules))))
+                         (hooks (list linux-module-database))))))
+      (return `(("kernel" ,kernel)
+                ,@(if hurd `(("hurd" ,hurd)) '()))))))
+
+(define (kernel-builder-configuration-add-modules config modules)
+  "Constructs a kernel builder configuration that has its modules extended."
+  (kernel-builder-configuration
+    (inherit config)
+    (modules (append (kernel-builder-configuration-modules config) modules))))
+
+(define kernel-loadable-module-service-type
+  (service-type (name 'kernel-loadable-modules)
+                (extensions
+                 (list (service-extension system-service-type
+                                          kernel-builder-configuration->system-entry)))
+                (compose concatenate)
+                (extend kernel-builder-configuration-add-modules)
+                (description
+                 "Register packages containing kernel-loadable modules and adds them
+to the system.")))
+
+(define (kernel-loadable-module-service kernel hurd modules)
+  "Constructs the service that sets up kernel loadable modules."
+  (service kernel-loadable-module-service-type
+    (kernel-builder-configuration
+      (kernel kernel)
+      (hurd hurd)
+      (modules modules))))
+
 \f
 ;;;
 ;;; Service folding.
diff --git a/gnu/system.scm b/gnu/system.scm
index c284a18379..5c530f176e 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -12,6 +12,7 @@
 ;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <jannek@gnu.org>
 ;;; Copyright © 2020 Efraim Flashner <efraim@flashner.co.il>
+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -600,16 +601,6 @@ OS."
       (file-append (operating-system-kernel os)
                       "/" (system-linux-image-file-name))))

-(define (package-for-kernel target-kernel module-package)
-  "Return a package like MODULE-PACKAGE, adapted for TARGET-KERNEL, if
-possible (that is if there's a LINUX keyword argument in the build system)."
-  (package
-    (inherit module-package)
-    (arguments
-     (substitute-keyword-arguments (package-arguments module-package)
-       ((#:linux kernel #f)
-        target-kernel)))))
-
 (define %default-modprobe-blacklist
   ;; List of kernel modules to blacklist by default.
   '("usbmouse" ;races with bcm5974, see <https://bugs.gnu.org/35574>
@@ -625,26 +616,10 @@ possible (that is if there's a LINUX keyword argument in the build system)."
   "Return the basic entries of the 'system' directory of OS for use as the
 value of the SYSTEM-SERVICE-TYPE service."
   (let* ((locale  (operating-system-locale-directory os))
-         (kernel  (operating-system-kernel os))
          (hurd    (operating-system-hurd os))
-         (modules (operating-system-kernel-loadable-modules os))
-         (kernel  (if hurd
-                      kernel
-                      (profile
-                       (content (packages->manifest
-                                 (cons kernel
-                                       (map (lambda (module)
-                                              (if (package? module)
-                                                  (package-for-kernel kernel
-                                                                      module)
-                                                  module))
-                                            modules))))
-                       (hooks (list linux-module-database)))))
          (initrd  (and (not hurd) (operating-system-initrd-file os)))
          (params  (operating-system-boot-parameters-file os)))
-    `(("kernel" ,kernel)
-      ,@(if hurd `(("hurd" ,hurd)) '())
-      ("parameters" ,params)
+    `(("parameters" ,params)
       ,@(if initrd `(("initrd" ,initrd)) '())
       ("locale" ,locale))))   ;used by libc

@@ -663,6 +638,10 @@ bookkeeping."
          (host-name (host-name-service (operating-system-host-name os)))
          (entries   (operating-system-directory-base-entries os)))
     (cons* (service system-service-type entries)
+           (kernel-loadable-module-service
+             (operating-system-kernel os)
+             (operating-system-hurd os)
+             (operating-system-kernel-loadable-modules os))
            %boot-service

            ;; %SHEPHERD-ROOT-SERVICE must come last so that the gexp that
@@ -699,6 +678,10 @@ bookkeeping."
 (define (hurd-default-essential-services os)
   (let ((entries (operating-system-directory-base-entries os)))
     (list (service system-service-type entries)
+          (kernel-loadable-module-service
+            (operating-system-kernel os)
+            (operating-system-hurd os)
+            (operating-system-kernel-loadable-modules os))
           %boot-service
           %hurd-startup-service
           %activation-service
diff --git a/gnu/tests/linux-modules.scm b/gnu/tests/linux-modules.scm
index 953b132ef7..9739e4124d 100644
--- a/gnu/tests/linux-modules.scm
+++ b/gnu/tests/linux-modules.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2019 Jakob L. Kreuze <zerodaysfordays@sdf.org>
 ;;; Copyright © 2020 Danny Milosavljevic <dannym@scratchpost.org>
 ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re>
+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -34,7 +35,10 @@
   #:use-module (guix utils)
   #:export (%test-loadable-kernel-modules-0
             %test-loadable-kernel-modules-1
-            %test-loadable-kernel-modules-2))
+            %test-loadable-kernel-modules-2
+            %test-loadable-kernel-modules-service-0
+            %test-loadable-kernel-modules-service-1
+            %test-loadable-kernel-modules-service-2))

 ;;; Commentary:
 ;;;
@@ -66,17 +70,11 @@ that MODULES are actually loaded."
                        (member module modules string=?))
                      '#$modules))))))

-(define* (run-loadable-kernel-modules-test module-packages module-names)
-  "Run a test of an OS having MODULE-PACKAGES, and verify that MODULE-NAMES
-are loaded in memory."
+(define* (run-loadable-kernel-modules-test-base base-os module-names)
+  "Run a test of BASE-OS, verifying that MODULE-NAMES are loaded in memory."
   (define os
     (marionette-operating-system
-     (operating-system
-      (inherit (simple-operating-system))
-      (services (cons (service kernel-module-loader-service-type module-names)
-                      (operating-system-user-services
-                       (simple-operating-system))))
-      (kernel-loadable-modules module-packages))
+     base-os
      #:imported-modules '((guix combinators))))
   (define vm (virtual-machine os))
   (define (test script)
@@ -98,6 +96,37 @@ are loaded in memory."
   (gexp->derivation "loadable-kernel-modules"
                     (test (modules-loaded?-program os module-names))))

+(define* (run-loadable-kernel-modules-test module-packages module-names)
+  "Run a test of an OS having MODULE-PACKAGES, and verify that MODULE-NAMES
+are loaded in memory."
+  (run-loadable-kernel-modules-test-base
+    (operating-system
+      (inherit (simple-operating-system))
+      (services (cons (service kernel-module-loader-service-type module-names)
+                      (operating-system-user-services
+                       (simple-operating-system))))
+      (kernel-loadable-modules module-packages))
+    module-names))
+
+(define* (run-loadable-kernel-modules-service-test module-packages module-names)
+  "Run a test of an OS having MODULE-PACKAGES, which are loaded by creating a
+service that extends KERNEL-LOADABLE-MODULE-SERVICE-TYPE. Then verify that
+MODULE-NAMES are loaded in memory."
+  (define module-installing-service-type
+    (service-type
+      (name 'module-installing-service)
+      (extensions (list (service-extension kernel-loadable-module-service-type
+                                           (const module-packages))))
+      (default-value #f)))
+  (run-loadable-kernel-modules-test-base
+    (operating-system
+      (inherit (simple-operating-system))
+      (services (cons* (service kernel-module-loader-service-type module-names)
+                       (service module-installing-service-type)
+                       (operating-system-user-services
+                        (simple-operating-system)))))
+    module-names))
+
 (define %test-loadable-kernel-modules-0
   (system-test
    (name "loadable-kernel-modules-0")
@@ -129,3 +158,35 @@ with two extra modules.")
                                                  (package-arguments
                                                   ddcci-driver-linux))))))
            '("acpi_call" "ddcci")))))
+
+(define %test-loadable-kernel-modules-service-0
+  (system-test
+   (name "loadable-kernel-modules-service-0")
+   (description "Tests loadable kernel modules extensible service with no
+extra modules.")
+   (value (run-loadable-kernel-modules-service-test '() '()))))
+
+(define %test-loadable-kernel-modules-service-1
+  (system-test
+   (name "loadable-kernel-modules-service-1")
+   (description "Tests loadable kernel modules extensible service with one
+extra module.")
+   (value (run-loadable-kernel-modules-service-test
+           (list ddcci-driver-linux)
+           '("ddcci")))))
+
+(define %test-loadable-kernel-modules-service-2
+  (system-test
+   (name "loadable-kernel-modules-service-2")
+   (description "Tests loadable kernel modules extensible service with two
+extra modules.")
+   (value (run-loadable-kernel-modules-service-test
+           (list acpi-call-linux-module
+                 (package
+                   (inherit ddcci-driver-linux)
+                   (arguments
+                    `(#:linux #f
+                      ,@(strip-keyword-arguments '(#:linux)
+                                                 (package-arguments
+                                                  ddcci-driver-linux))))))
+           '("acpi_call" "ddcci")))))
--
2.29.2

Re: ZFS on Guix

[Carlo Zancanaro]

Apologies for the short reply, but that permission issue can be solved in the short term by changing the permissions on /dev/kvm (I set it to 777 to force it when necessary, because I don't know what the right permissions are).

The proper solution is probably to add your user to a kvm group on your distribution, which should give you the appropriate access.

Re: ZFS on Guix

[raid5atemyhomework]

Thank you Carlo!

I've now tested the new tests I added in `gnu/tests/linux-modules.scm`, and the existing tests as well, and they all pass.

Hope to get some review on that patch!

Thanks
raid5atemyhomework

Re: ZFS on Guix

[raid5atemyhomework]

Hi guix-developers,

Another issue here is that ZFS prefers to not be managed via `/etc/fstab`/`mount`/`umount`.  Instead, at startup ZFS magically imports ZFS pools and mounts ZFS datasets in the correct place, as configured in the `mountpoint` properties of the dataset.  This magic is actually implemented by executing `zpool import -a` at startup after ZFS module loading.  Note that `zpool import -a` has to be executed always so that ZFS can detect its pools, though the automatic mounting can be suppressed with `zpool import -a -N`.

(`systemd`-based distros have a number of `systemd` services that handle ZFS importation by use of a `/etc/zfs/zpool.cache` file that ZFS maintains, and then does `zfs mount -a` to do automounting, but I'll leave that for later.)

Now, properly speaking, the shepherd `file-systems` service should not be started until we have actually performed ZFS automounting.  This means that `file-systems` has to have as a requirement the ZFS shepherd service that implements the automounting.  And of course it should *not* require that if ZFS isn't installed in the system.

So, I made another patch that makes the `file-systems` shepherd service have an extensible set of requirements, like `user-processes` does.  It's below.  Actual `file-system`s declared in the operating system then make themselves requirements of `file-systems`.  Then, a ZFS automounting shepherd service can be installed by the `zfs-service-type` and added as a requirement of the `file-systems` shepherd service.

This is important since one possible use of ZFS is to have it provide the `/home` filesystem.  And the `/home` filesystem has to be mounted before `user-homes` shepherd service starts.  So the ZFS automounting shepherd service has to be a requirement of `file-systems`, which is enabled by the below.

Please review!


From 792a8f8efc95e4fe9a94d42f839ddcfb034b8540 Mon Sep 17 00:00:00 2001
From: raid5atemyhomework <raid5atemyhomework@protonmail.com>
Date: Wed, 6 Jan 2021 08:15:54 +0800
Subject: [PATCH] gnu: Make file-systems target extensible by services.

* gnu/services/base.scm (file-system-shepherd-services): Move file-systems
shepherd service to ...
(file-systems-target-shepherd-services): ... new procedure here.
(file-systems-target-service-type): New variable.
(file-system-service-type): Extend file-systems-target service to add each
file-system as a requirement.
* gnu/system.scm (operating-system-default-essential-services): Instantiate
file-systems-target-service-type.
(hurd-default-essential-services): Instantiate file-systems-target-service-type.
---
 gnu/services/base.scm | 37 +++++++++++++++++++++++++++----------
 gnu/system.scm        |  2 ++
 2 files changed, 29 insertions(+), 10 deletions(-)

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 945b546607..13cfb6a8a2 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -13,6 +13,7 @@
 ;;; Copyright © 2019 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
 ;;; Copyright © 2020 Florian Pelz <pelzflorian@pelzflorian.de>
 ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re>
+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -67,6 +68,7 @@
   #:export (fstab-service-type
             root-file-system-service
             file-system-service-type
+            file-systems-target-service-type
             swap-service
             host-name-service
             console-keymap-service
@@ -362,18 +364,29 @@ FILE-SYSTEM."
                        (gnu system file-systems)
                        ,@%default-modules)))))))

+(define (file-systems-target-shepherd-services requirements)
+  (list
+    (shepherd-service
+      (provision '(file-systems))
+      (requirement (cons* 'root-file-system 'user-file-systems requirements))
+      (documentation "Target for all the initially-mounted file systems")
+      (start #~(const #t))
+      (stop #~(const #t)))))
+(define file-systems-target-service-type
+  (service-type
+    (name 'file-systems)
+    (extensions (list (service-extension shepherd-root-service-type
+                                         file-systems-target-shepherd-services)))
+    (compose concatenate)
+    (extend append)
+    ;; Extensions can add new values to this list.
+    (default-value '())
+    (description "The @code{file-systems} service is the target that is started
+when all file systems have been mounted.")))
+
 (define (file-system-shepherd-services file-systems)
   "Return the list of Shepherd services for FILE-SYSTEMS."
   (let* ((file-systems (filter file-system-mount? file-systems)))
-    (define sink
-      (shepherd-service
-       (provision '(file-systems))
-       (requirement (cons* 'root-file-system 'user-file-systems
-                           (map file-system->shepherd-service-name
-                                file-systems)))
-       (documentation "Target for all the initially-mounted file systems")
-       (start #~(const #t))
-       (stop #~(const #f))))

     (define known-mount-points
       (map file-system-mount-point file-systems))
@@ -403,7 +416,7 @@ FILE-SYSTEM."
                            (filter (negate known?) (mount-points)))
                  #f))))

-    (cons* sink user-unmount
+    (cons* user-unmount
            (map file-system-shepherd-service file-systems))))

 (define (file-system-fstab-entries file-systems)
@@ -431,6 +444,10 @@ FILE-SYSTEM."
                        (service-extension fstab-service-type
                                           file-system-fstab-entries)

+                       ;; Have 'file-systems' depend on each declared file system.
+                       (service-extension file-systems-target-service-type
+                                          (cut map file-system->shepherd-service-name <>))
+
                        ;; Have 'user-processes' depend on 'file-systems'.
                        (service-extension user-processes-service-type
                                           (const '(file-systems)))))
diff --git a/gnu/system.scm b/gnu/system.scm
index 5c530f176e..6987641ee8 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -667,6 +667,7 @@ bookkeeping."
                     (operating-system-setuid-programs os))
            (service profile-service-type
                     (operating-system-packages os))
+           (service file-systems-target-service-type)
            other-fs
            (append mappings swaps

@@ -691,6 +692,7 @@ bookkeeping."
                                    (operating-system-groups os))
                            (operating-system-skeletons os))
           (root-file-system-service)
+          (service file-systems-target-service-type)
           (service file-system-service-type '())
           (service fstab-service-type
                    (filter file-system-needed-for-boot?
--
2.29.2

Re: ZFS on Guix

[raid5atemyhomework]

Latest patchset: https://issues.guix.gnu.org/45692

Re: ZFS on Guix

[raid5atemyhomework]

Hi guix-developers,

I just found out that ZFS on Linux maintains a file `/etc/zfs/zpool.cache` which contains information on ZPOOLs (i.e. ZFS-managed RAID arrays).  I just didn't encounter this file before on Guix because the file is created if and only if the directory `/etc/zfs/` exists, and that directory is not created by the ZFS installation process on Guix (because I'm the one making the ZFS installation process on Guix right now and I didn't know about this directory).

Now, my understanding is that `/etc/` directory is recreated at each `guix system reconfigure`.  Thus, if ZFS maintains information in `/etc/zfs/` then on a reconfigure the information is lost.

If so --- for Guix, what should I use instead?

As it happens, ZFS uses this directory for at least these things:

* `/etc/zfs/zpool.cache` - a binary file containing information about what ZPOOLs were created on this system.  The `zpool` command updates this file!
* `/etc/zfs/zpool.d/` - a directory of scripts that can be used to extend the `zpool` command; the ZFS release normally installs a bunch of files it has in those directories.
* `/etc/zfs/vdev_id.conf` - a sysad-managed configuration file that is used to indicate the paths.
* `/etc/zfs/zed.d/` - a directory of scripts that are executed when particular events are detected by the ZFS Event Daemon (which I didn't know about, and which should also be installed and started as a Shepherd service as well).  Sysads are supposed to link or copy files from `/libexec/zfs/zed.d/` (which are provided by the installation) to enable/disable particular zedlets, and can add other events here. The ZFS release normally links some of the files in `/libexec/zfs/zed.d/`.
* `/etc/zfs/zfs-functions` - a shell file containing functions that some bits of ZFS `init` scripts / `initramfs` scripts use, and maybe more. I think Guix can safely ignore this, though somebody (most likely me) will have to read it through to understand how ZFS actually implements some of its magical abilities.

Of these, many are stuff that we might plausibly generate by adding more fields in `zfs-configuration`, though the ZED and its zedlet system would bring in the possibility of writing Guile GEXP to create scripts for particular events.

However, I'm not so sure about `zpool.cache` file.  For one, it doesn't contain text data so it's not easy to generate it ourselves.

On the other hand I've got ZFS working without it, by just using `zpool import -a` to have ZFS scan for all devices.  The problem with this technique is if Guix is used on a system with several dozen or hundred devices in various ZPOOLs; this would slow down boot while ZFS is checking all arrays and figuring out which device goes into which array, whereas a `zpool.cache` file would let ZFS know about all pools created or imported on the system and would not need to scan their labels and reassemble their arrays and so on.  Another is that for a complex enough setup, a storage device might be connected to the hardware by hostile parties (for example, by network-addressable block devices, or USB) that contains a ZPOOL with a `mountpoint=/gnu/store` (or other sensitive directory) property, which, if auto-imported via `zpool import -a`, could let particular subdirectories of the filesystem to be subverted.

How best do I handle this?


Thanks
raid5atemyhomework

Re: ZFS on Guix

[raid5atemyhomework]

Hi guix-developers,

> Now, my understanding is that `/etc/` directory is recreated at each `guix system reconfigure`. Thus, if ZFS maintains information in `/etc/zfs/` then on a reconfigure the information is lost.
>
> If so --- for Guix, what should I use instead?
>


Okay, a `guix system reconfigure` doesn't remove files in `/etc`, at best I **think** it just copies/overwrites data, but if the `etc` directory of the built system doesn't have a file, it will not be overwritten if something else writes to `/etc`.  Is my understanding correct?

On the other hand --- is it "properly Guix" for a system component to write to a file in the `/etc` directory? Where should I put this kind of not-quite-configuration cache file?



Thanks
raid5atemyhomework

Re: ZFS on Guix

[raid5atemyhomework]

Greetings guix-developers and other people trying to use Guix,

The patchset currently dying on issues.guix.gnu.org would provide a nice simple single-step way to enable *very basic* ZFS support on your Guix system.  Until it gets merged, however, you can still enable *very very basic* ZFS support on your Guix system by following the below minimal guide.

First, select a Linux kernel that the latest ZFS package on Guix supports.  Current on Guix is ZFS 2.0.1.  That supports up to Linux 5.10.  So I suggest using a `linux-libre-5.10` kernel.  Don't go with the default `linux-libre` or `linux-libre-lts` since those could be updated to past what the ZFS on Guix supports, be explicit with what kernel version you have.  Just remember to update to a later LTS version if ZFS on Guix ever gets updated to a later version.

Then, you need to create a ZFS package, by adding some declarations before the `operating-system` form:

```scheme
(use-modules #;...
             (guix packages))
(use-package-modules #;... linux file-systems)

(define my-kernel linux-libre-5.10)
(define my-zfs
  (package
    (inherit zfs)
    (arguments (cons* #:linux my-kernel (package-arguments zfs)))))
```

Then in the `operating-system` form, you need to add the following bits:

* Add ZFS to the kernel-loadable modules so that the installed kernel knows how to read and write ZFS disks.
* Add ZFS to the system profile packages so that you can easily manage ZFS disks.

So you have to modify a number of places:

```scheme
(operating-system
  (kernel my-kernel)
  (kernel-loadable-modules (list (list my-zfs "module")))
  #;...
  (packages
    (append (map specification->package (list "nss-certs"))
            (list my-zfs)
            %base-packages))
  #;...)
```

With the above you get a ***ridiculously minimal*** ZFS support.

* You have to `sudo modprobe zfs` explicitly at startup.
* You have to `sudo zpool import -a` explicitly at startup.

To do the above automatically at startup you need to add a Shepherd service to do the `zpool import`, and add a `kernel-module-loader` extension.  This requires a bit more code to be added to your `configuration.scm`.

Here's what I got in my `configuration.scm`:

```scheme
(define zfs-shepherd-services
  (let ((zpool            (file-append my-zfs "/sbin/zpool"))
        (zfs              (file-append my-zfs "/sbin/zfs"))
        (scheme-modules   `((srfi srfi-1)
                            (srfi srfi-34)
                            (srfi srfi-35)
                            (rnrs io ports)
                            ,@%default-modules)))
    (define zfs-scan
      (shepherd-service
        (provision '(zfs-scan))
        (documentation "Scans for ZFS pools.")
        (requirement '(kernel-module-loader udev))
        (modules scheme-modules)
        (start #~(lambda _
                   (invoke/quiet #$zpool "import" "-a" "-N")))
        (stop #~(const #f))))
    (define zfs-automount
      (shepherd-service
        (provision '(zfs-automount))
        (documentation "Automounts ZFS data sets.")
        (requirement '(zfs-scan))
        (modules scheme-modules)
        (start #~(lambda _
                   (with-output-to-port
                     (current-error-port)
                     (lambda ()
                       (invoke #$zfs "mount" "-a" "-l")))))
        (stop #~(lambda _
                  (chdir "/")
                  (invoke/quiet #$zfs "unmount" "-a" "-f")
                  #f))))
    (list zfs-scan
          zfs-automount)))
```

Then, add some `simple-service`s to your `operating-system`:

```scheme
(use-modules
  #;...
  (gnu services))
(use-service-modules linux shepherd #;...)

#;...

(operating-system
  #;...
  (services
    (append
      (list
        #;...
        (simple-service 'zfs-loader
                        kernel-module-loader-service-type
                        '("zfs"))
        (simple-service 'zfs-shepherd-services
                        shepherd-root-service-type
                        zfs-shepherd-services)
        (simple-service 'zfs-shepherd-services-user-processes
                        user-processes-service-type
                        '(zfs-automount))
        #;...)
      #;...
      %desktop-services))
  #;...)
```

The above lets you mount ZFS pools automatically at startup.  Encrypted pools with `keylocation=prompt` will prompt at the console on bootup.

Caveats:

* You can't have a `/home` on ZFS mount.  ZFS has to be mounted before `file-systems` target starts, otherwise Guix will fill up the root-mounr's `/home` directory and ZFS will refuse to mount over that.  No `/` or `/boot` on ZFS either.  Probably no good way to put `/gnu/store` on ZFS either, because Guix inherently assumes it's on the `/` mount.
* The above setup does not maintain a `/etc/zfs/zpool.cache` file, because I'm not really certain whether it's kosher in Guix to have a file maintained by ZFS in the `/etc` dir hierarchy.  This has a number of consequences:
  * `zdb` doesn't work because it looks for `/etc/zfs/zpool.cache`.  Good luck trying to figure out why your pool is slow.
  * You can't practically have more than a few dozen disks in your system, because the above uses `zpool import -a` which will cause ZFS to scan all the disks at bootup which would probably be slow if you have lots of disks.
* You can't practically use `zvol`s to back other filesystems in such a way that you can put them in a `file-system` declaration.  Though why use any file-system other than ZFS amirite.  You can still use `zvol`s to e.g. back a virtual machine that you launch manually when the system boot is finished, or from a Shepherd service that explicitly lists `user-processes` as a requirement.  Though hmmm the above doesn't use `zvol_wait` anywhere... sigh.
* There's no ZED.  No automatic replacement of failing drives with a hot spare.  No monitoring.  You can probably try launching it in its own Shepherd service, but you need to figure out how to populate `/etc/zfs/zed/` yourself.  If you do, you probably will not be doing it from the `configuration.scm` file meaning it'll be hard to replicate the setup elsewhere.


Thanks
raid5atemyhomework

Re: ZFS on Guix

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 321 bytes --]

I just wanted to say that I'm not ignoring your patch, I'm just not qualified
to review it.  I hope someone steps up to it--otherwise I can't really tell
whether (mbegin %state-monad...) inside a random service procedure is a good
idea.

Then again, provenance-service-type does it and there it seems to be fine...

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: ZFS on Guix

[Joshua Branson]

raid5atemyhomework <raid5atemyhomework@protonmail.com> writes:

> The patchset currently dying on issues.guix.gnu.org would provide a nice simple single-step way to enable *very basic* ZFS support on your Guix system.  Until it gets merged, however, you can still enable *very very basic* ZFS support on your Guix system by following the below minimal guide.

If this patch does not get merged into guix, then you could always add
it to the cookbook.  :)

-- 
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar
  

Re: ZFS on Guix

[raid5atemyhomework]

Hi Joshua,

> raid5atemyhomework raid5atemyhomework@protonmail.com writes:
>
> > The patchset currently dying on issues.guix.gnu.org would provide a nice simple single-step way to enable very basic ZFS support on your Guix system. Until it gets merged, however, you can still enable very very basic ZFS support on your Guix system by following the below minimal guide.
>
> If this patch does not get merged into guix, then you could always add
> it to the cookbook. :)

It includes changes to how Guix builds the system, and requires code changes to Guix itself, not just to the `configuration.scm`.

The code changes enable:

* `/home` on ZFS.
* Allowing you to put any filesystem inside a `zvol` and declare it in `file-system`.
* Assurance that by the time `file-systems` is started, even ZFS automounts have been mounted; important if you are starting a Shepherd service that has been configured to access filesystems (i.e. all the important ones).

The modifications in the previous email work only in `configuration.scm` but do not allow the above.  I think `/home` on ZFS is an important target to have (it's what I wanted for my server) at least, but requires changes to how other services in Guix System are built anyway.

Thanks
raid5atemyhomework

Re: ZFS on Guix

[raid5atemyhomework]

Hi guix-developers and users,

Here are some notes I made about how to get `/` on ZFS, maybe someone else can think about it.

-------

Most importantly, it seems for this style we need to consider first `/boot`
***not*** on ZFS, and have `/` on ZFS.  I presume grub has some way to read
ZFS pools in order to get at the `/boot` if `/boot` is on ZFS, since
`/`-on-ZFS for Ubuntu is able to put even `/boot` on ZFS, however note that
even there the `/boot` is on a different pool from the `/`.

When this works (*cough*) it should be possible to create a ZFS pool (with
`mountpoint=legacy`) from a ZFS-enabled boot of Guix, then create a
`configuration.scm` that we would then `sudo guix system init` onto a
temporary mountpoint.  I would strongly suggest putting `/boot` elsewhere
though.

Here are a bunch of things we need for `/` on ZFS:

* ZFS module installing into `initrd`.
* ZFS module loading while in `initrd` before the pivot to the "real" `/`.
* ZFS `import` of the pool containing `/` mount.

## `initrd` module installation

We need a facility to add modules to the `initrd` that are not
part of the kernel.  Currently `raw-initrd`/`base-initrd` will
only get `.ko` files from the given `linux` package.  We need
to modify the `initrd` interface to add say a `#:module-packages`
list of packages whose `.ko` files we will also add to the `initrd`.

This translates to modifying `flat-linux-module-directory` in
`gnu/system/linux-initrd.scm` to additionally accept a list of packages
all of whose `.ko` and/or `.ko.gz` files will be added to the module
directory.  Then somewhere over in `raw-initrd` we would:

    (define kodir
      (flat-linux-module-directory linux linux-modules module-packages))

We also need an extensible service type that will eventually lead to
adding new entries in the `#:module-packages`.

We need a "root" `initrd` extensible service type that will construct the `initrd`
via the `operating-system-initrd` field.  This service type will accept
additional arguments to pass to the `initrd` function.  Then the
`initrd-kernel-loadable-module-service-type` would accept lists of package
specifications, then provide a `#:module-packages` argument to the root
`initrd` service type.

Rough sketch:

    (define initrd-arguments-service-type (service-type #;...))
    (define initrd-kernel-loadable-module-service-type
      (service-type
        (name 'initrd-kernel-loadable-module-service-type)
        (extensions (list (service-extension
                            initrd-arguments-service-type
                            (lambda (module-packages)
                              (if (null? module-packages)
                                  '()
                                  (list #:module-packages module-packages))))))
        (compose concatenate)
        (extend append)
        (default-value '())))

## `initrd` module explicit loading

Normally modules are loaded "as needed" but ZFS needs to be explicitly loaded.

The `base-initrd` already accepts a list of `linux-modules` to load, we just
need some way to hook into adding `#:linux-modules`.  This probably means
modifying how `operating-system-initrd-file` in `(gnu system)` works (which
would probably be needed by the `initrd-arguments-service-type` anyway).

We could try hooking into the currently-deprecated `extra-modules` instead,
here's a sketch:


    (define initrd-kernel-module-loader-service-type
      (service-type
        (name 'initrd-kernel-module-loader-service-type)
        (extensions (list (service-extension
                            initrd-arguments-service-type
                            (lambda (modules)
                              (if (null? modules)
                                  '()
                                  (list #:extra-modules modules))))))
        (compose concatenate)
        (extend append)
        (default-value '())))

## `initrd` additional pre-mount actions

Before a `/` on ZFS can be mounted, ZFS has to be told to scan for the
ZFS pool containing the `/`.

We could add yet another argument to `raw-initrd`/`base-initrd`, `#:premount-actions`,
a list of gexpressions.
Then the `#:pre-mount` argument to `boot-system` would become something like:

        #:pre-mount (lambda ()
                      (and #$@device-mapping-commands
                           #$@premount-actions))

As usual a service type can be created which extends `initrd-arguments-service-type`.

For root-on-ZFS specifically we would need to know the root pool and execute something
like this:

    #~(begin
        (invoke/quiet #$(file-append zfs/static "/sbin/zpool") "-a" "-N" #$root-pool))

Then, "normal" root specification would be done, with the root pool name given as the
device name of the `/` mountpoint, and with the ZFS mountpoint set to `legacy`.

Re: ZFS on Guix

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 445 bytes --]

Hi raid5atemyhomework,

I can't help you with ZFS, but I think I've found some small issues
with the configuration file:

* the shepherd services defined in `configuration.scm`
  seem one-shot services to me, so maybe add '(one-shot? #t)'.

* in the 'stop' of zfs-automount, the code changes the working
  directory.  As it isn't restored afterwards, this doesn't seem
  kosher to me.  Or maybe this isn't an issue at all.

Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: ZFS on Guix

[raid5atemyhomework]

> * the shepherd services defined in `configuration.scm`
>   seem one-shot services to me, so maybe add '(one-shot? #t)'.

I was wary of making the `zfs-scan` one-shot, since there is a dependent service on top of it.  Not to mention restarting `zfs-scan` could be useful if for example you were doing hotswapping of devices.  Maybe.


> * in the 'stop' of zfs-automount, the code changes the working
>   directory.  As it isn't restored afterwards, this doesn't seem
>   kosher to me.  Or maybe this isn't an issue at all.

The code is modeled after `file-system-shepherd-service` procedure in `gnu/services/base.scm`, which includes a `(chdir "/")` as well with the comment "Make sure PID 1 doesn't keep TARGET busy.".  Since this also does filesystem mounting/unmounting, I judged it best to imitate the existing filesystem mounting/unmounting service.

Thanks
raid5atemyhomework

Re: ZFS on Guix

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 984 bytes --]

On Mon, 2021-02-08 at 09:32 +0000, raid5atemyhomework wrote:
> > * the shepherd services defined in `configuration.scm`
> >   seem one-shot services to me, so maybe add '(one-shot? #t)'.
> 
> I was wary of making the `zfs-scan` one-shot, since there is a dependent service on top of it.  Not to mention restarting `zfs-scan` could be useful if for example you were doing hotswapping of devices.  Maybe.
> 
> 
> > * in the 'stop' of zfs-automount, the code changes the working
> >   directory.  As it isn't restored afterwards, this doesn't seem
> >   kosher to me.  Or maybe this isn't an issue at all.
> 
> The code is modeled after `file-system-shepherd-service` procedure in `gnu/services/base.scm`, which includes a `(chdir "/")` as well with the comment "Make sure PID 1 doesn't keep TARGET busy.".  Since this also does filesystem mounting/unmounting, I judged it best to imitate the existing filesystem mounting/unmounting service.

Ok, seems reasonable to me!


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: ZFS on Guix

[raid5atemyhomework]

Hello Danny,

> I just wanted to say that I'm not ignoring your patch, I'm just not qualified
> to review it. I hope someone steps up to it--otherwise I can't really tell
> whether (mbegin %state-monad...) inside a random service procedure is a good
> idea.
>
> Then again, provenance-service-type does it and there it seems to be fine...


For ***this*** very specific case it is because of a random weirdness of `system-service-type`.

Specifically, the *value* of that service-type is an association list of filenames and their contained store values.  However, an ***extension*** of that service-type must be a monadic action that results in an association list of filename-contents.


Here is relevant code in `gnu/services`:

```scheme
(define (system-derivation entries mextensions)
  "Return as a monadic value the derivation of the 'system' directory
containing the given entries."
  (mlet %store-monad ((extensions (mapm/accumulate-builds identity
                                                          mextensions)))
    (lower-object
     (file-union "system"
                 (append entries (concatenate extensions))))))

(define system-service-type
  ;; This is the ultimate service type, the root of the service DAG.  The
  ;; service of this type is extended by monadic name/item pairs.  These items
  ;; end up in the "system directory" as returned by
  ;; 'operating-system-derivation'.
  (service-type (name 'system)
                (extensions '())
                (compose identity)
                (extend system-derivation)
                (description
                 "Build the operating system top-level directory, which in
turn refers to everything the operating system needs: its kernel, initrd,
system profile, boot script, and so on.")))
```

So *extensions* must be monads (due to `mapm/accumulate-builds` on the `mextensions`) but the raw value must be a simple non-monadic assoc list.

The patch moves some generated files ("kernel" and "hurd") from the value of the `system-service-type` to an extension of `system-service-type`, thus the extra `mbegin %store-monad`. It needs to be `%store-monad` since that is the monad used by the `system-derivation` function.

See:

```scheme
(define (kernel-builder-configuration->system-entry config)
  "Return the kernel and hurd entries of the 'system' directory."
  (mbegin %store-monad
    #;...))
#;...
(define kernel-loadable-module-service-type
  (service-type (name 'kernel-loadable-modules)
                (extensions
                 (list (service-extension system-service-type
                                          kernel-builder-configuration->system-entry))) ;; <-- OVER HERE
                (compose concatenate)
                (extend kernel-builder-configuration-add-modules)
                (description
                 "Register packages containing kernel-loadable modules and adds them+to the system.")))
```

So it is not just some "random service procedure", it is because that is the interface exposed by `system-service-type` for its extensions, extensions of `system-service-type` have to yield a monadic action.  `mbegin` is one of the simpler monadic actions.  It is also in the "correct place" as best as I can tell, since only service types in `gnu/services.scm` dare to extend `system-service-type`.

`provenance-service-type` does this as well because it *also* extends `system-service-type`.  This is basically done here simply because that is what `system-service-type` expects.

Thanks
raid5atemyhomework

Re: ZFS on Guix

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1545 bytes --]

On Mon, Feb 08, 2021 at 09:32:27AM +0000, raid5atemyhomework wrote:
> > * the shepherd services defined in `configuration.scm`
> >   seem one-shot services to me, so maybe add '(one-shot? #t)'.
> 
> I was wary of making the `zfs-scan` one-shot, since there is a dependent service on top of it.  Not to mention restarting `zfs-scan` could be useful if for example you were doing hotswapping of devices.  Maybe.
> 

After a one-shot service 'happens' it is effectively in a 'stopped'
state. So if something else comes along and requires it it will start it
again. So if running 'zfs scan' can be considered a one-off action and
it's ok if it's run when other services need it then making it a
one-shot service is OK.

> 
> > * in the 'stop' of zfs-automount, the code changes the working
> >   directory.  As it isn't restored afterwards, this doesn't seem
> >   kosher to me.  Or maybe this isn't an issue at all.
> 
> The code is modeled after `file-system-shepherd-service` procedure in `gnu/services/base.scm`, which includes a `(chdir "/")` as well with the comment "Make sure PID 1 doesn't keep TARGET busy.".  Since this also does filesystem mounting/unmounting, I judged it best to imitate the existing filesystem mounting/unmounting service.

Makes sense to me.

> 
> Thanks
> raid5atemyhomework
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]