[bug#45692] [PATCH 4/4] gnu: Add ZFS service.

[raid5atemyhomework via Guix-patches via <guix-patches@gnu.org>]

New version again.  Change: Fix documentation `doc/guix.texi` to remove mentions of `options` and add documentation about the new `dependencies` features.


From dfe9ad7512d348933beb5b42041965ff604422b5 Mon Sep 17 00:00:00 2001
From: raid5atemyhomework <raid5atemyhomework@protonmail.com>
Date: Wed, 6 Jan 2021 09:24:20 +0800
Subject: [PATCH 1/2] gnu: Add service to install ZFS.

* gnu/services/file-systems.scm: New file.
(zfs-service-type): New variable.
(<zfs-configuration>): New type.
(%zfs-zvol-dependency): New variable.
* gnu/local.mk: Add gnu/services/file-systems.scm.
* gnu/services/base.scm (dependency->shepherd-service-name): Export.
* doc/guix.texi (ZFS file system): New subsection.
---
 doc/guix.texi                 | 129 +++++++++++++++++++++++++
 gnu/local.mk                  |   1 +
 gnu/services/base.scm         |   4 +-
 gnu/services/file-systems.scm | 174 ++++++++++++++++++++++++++++++++++
 4 files changed, 307 insertions(+), 1 deletion(-)
 create mode 100644 gnu/services/file-systems.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index a31d355780..7004efe3c4 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -13886,6 +13886,135 @@ a file system declaration such as:
 compress-force=zstd,space_cache=v2"))
 @end lisp

+@node ZFS file system
+@subsection ZFS file system
+
+The ZFS file system has a license incompatible with the Linux kernel,
+and thus cannot be distributed with the kernel. However, as a user
+you have the right to do whatever you want on your own hardware,
+including download the ZFS source code, compile it, link it to your
+own private copy of Linux, and run it. You can even use the Guix build
+system to automate this.
+
+As a large and complex kernel module, ZFS on Linux has to be compiled
+with a specific version of the kernel. Often even the latest ZFS
+package available cannot be compiled with the latest Linux kernel
+package provided by Guix System. Thus, installing the @code{zfs}
+package is likely to fail.
+
+Instead, you have to use an older long-term-support Linux kernel.
+Do not use @code{linux-libre-lts}, since the latest long-term-support
+kernel might be too new for the ZFS package; instead, explicitly
+select the version number, like @code{linux-libre-5.4}, and upgrade
+it manually later when you have verified that the ZFS version
+available on Guix can be compiled with a later LTS kernel.
+
+Then, you have to modify your system configuration file and use the
+selected older kernel, and add the @code{zfs-service-type} service.
+
+@lisp
+(use-modules (gnu))
+(use-package-modules
+   ; @dots{}
+   linux)
+(use-service-modules
+   ; @dots{}
+   file-systems)
+
+(define my-kernel linux-libre-5.4)
+
+(operating-system
+  (kernel my-kernel)
+  ;; @dots{}
+  (services
+    (cons* (service zfs-service-type
+             (zfs-configuration
+               (kernel my-kernel)))
+           ; @dots{}
+           %desktop-services))
+  ;; @dots{}
+  )
+@end lisp
+
+@defvr {Scheme Variable} zfs-service-type
+This is the type of the service to compile and install OpenZFS to
+your operating system. It loads the ZFS module at startup, imports
+pools, mounts automounted ZFS datasets, and installs the ZFS command
+line tools.  Its value must be a @code{zfs-configuration} record
+(see below).
+
+Here is an example use:
+
+@lisp
+(service zfs-service-type
+  (zfs-configuration
+    (kernel linux-libre-5.4)))
+@end lisp
+@end defvr
+
+@deftp {Data Type} zfs-configuration
+This data type represents the configuration of the ZFS service.
+The available fields are:
+
+@table @asis
+@item @code{kernel}
+The package of the Linux kernel to compile ZFS for. Required. It
+@emph{must} be the same kernel you use in your operating system.
+
+@item @code{base-zfs} (default: @code{zfs})
+The ZFS package to use. It will be modified to use the indicated
+kernel.
+
+@item @code{dependencies} (default: @code{'()})
+A list of @code{<file-system>}s or @code{<mapped-device>}s that
+must be mounted or opened before ZFS scans for pools to import.
+For example, you might have LUKS containers as the leaf VDEVs of
+a ZFS pool.
+@end table
+@end deftp
+
+Once your system has been configured to include the ZFS service
+and you have restarted the system, you can manage ZFS pools and
+datasets with @code{zpool} and @code{zfs} commands.
+
+ZFS datasets with an appropriate @code{mountpoint} property will
+be automounted at startup after the root file system is started.
+Encrypted datasets that are automounted will cause boot to pause
+and prompt for the password to be provided on the console.
+
+It's possible to have a ZFS dataset as your @code{/home} by simply
+setting the @code{mountpoint} ZFS property. However, note that ZFS
+will refuse to mount over a non-empty directory, so if your root
+filesystem already has a non-empty @code{/home}, you should remove
+it and its contents, then restart the system.
+
+ZFS features @dfn{ZVOL}s, which are block devices that ZFS exposes
+to the system. You can put any file system inside a ZVOL. In order
+to mount such a filesystem at boot, you can declare it as dependent
+on @code{%zfs-zvol-dependency}.
+
+@defvr {Scheme Variable} %zfs-zvol-dependency
+This is an artificial @code{<mapped-device>} object which tells
+the file mounting service to wait for ZFS to provide ZVOLs before
+mounting the file system that is dependent on it. For example:
+
+@lisp
+(file-system
+  (device "/dev/zvol/pool-name/zvol-name")
+  (mount-point "/ext4-on-zfs")
+  (type "ext4")
+  (dependencies (list %zfs-zvol-dependency)))
+@end lisp
+
+Do @emph{not} add @code{%zfs-zvol-dependency} to your
+@code{mapped-devices} declaration, In addition, only use it as a
+@code{file-system} dependency if you instantiate a
+@code{zfs-service-type} service in your operating system.
+@end defvr
+
+Having ZFS as a root filesystem or as @code{/boot} is not supported
+yet.
+
 @node Mapped Devices
 @section Mapped Devices

diff --git a/gnu/local.mk b/gnu/local.mk
index 1151d4642e..5aeb45c4c2 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -602,6 +602,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/dict.scm				\
   %D%/services/dns.scm				\
   %D%/services/docker.scm			\
+  %D%/services/file-systems.scm			\
   %D%/services/authentication.scm		\
   %D%/services/games.scm			\
   %D%/services/ganeti.scm			\
diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 13cfb6a8a2..ef3680583b 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -188,7 +188,9 @@

             references-file

-            %base-services))
+            %base-services
+
+            dependency->shepherd-service-name))

 ;;; Commentary:
 ;;;
diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm
new file mode 100644
index 0000000000..9061ab9582
--- /dev/null
+++ b/gnu/services/file-systems.scm
@@ -0,0 +1,174 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services file-systems)
+  #:use-module (gnu packages file-systems)
+  #:use-module (gnu services)
+  #:use-module (gnu services base)
+  #:use-module (gnu services linux)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system mapped-devices)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+  #:use-module (guix records)
+  #:export (zfs-service-type
+
+            zfs-configuration
+            zfs-configuration?
+            zfs-configuration-kernel
+            zfs-configuration-base-zfs
+            zfs-configuration-dependencies
+
+            %zfs-zvol-dependency))
+
+(define-record-type* <zfs-configuration>
+  zfs-configuration make-zfs-configuration zfs-configuration?
+  ; kernel you want to compile the base-zfs module for.
+  (kernel         zfs-configuration-kernel)
+  ; base package that will be compiled for the kernel
+  (base-zfs       zfs-configuration-base-zfs
+                  (default zfs))
+  ; list of <mapped-device> | <file-system> we should wait for,
+  ; before scanning for ZFS pools.
+  (dependencies   zfs-configuration-dependencies
+                  (default '())))
+
+;; This is a synthetic and unusable MAPPED-DEVICE; its only use
+;; is to be added as a (dependency ...) of some FILE-SYSTEM.
+(define %zfs-zvol-dependency
+  (mapped-device
+    (source '())
+    ;; The /* prevents naming conflict with non-ZFS device mappings,
+    ;; since it is not a valid name for mapped devices, and also
+    ;; implies "all zvols" in terms of globs.
+    (targets '("zvol/*"))
+    (type #f)))
+
+(define (make-zfs-package conf)
+  (let ((base-zfs (zfs-configuration-base-zfs conf))
+        (kernel   (zfs-configuration-kernel conf)))
+    (package
+      (inherit base-zfs)
+      (name (string-join (list (package-name base-zfs)
+                               "for"
+                               (package-name kernel)
+                               (package-version kernel)
+                               "version")
+                         "-"))
+      (arguments (cons* #:linux kernel (package-arguments base-zfs))))))
+
+(define (zfs-loadable-module conf)
+  (list (list (make-zfs-package conf) "module")))
+
+(define (zfs-shepherd-services conf)
+  (let* ((zfs-package     (make-zfs-package conf))
+         (zpool           (file-append zfs-package "/sbin/zpool"))
+         (zfs             (file-append zfs-package "/sbin/zfs"))
+         (zvol_wait       (file-append zfs-package "/bin/zvol_wait"))
+         (scheme-modules  `((srfi srfi-1)
+                            (srfi srfi-34)
+                            (srfi srfi-35)
+                            (rnrs io ports)
+                            ,@%default-modules)))
+
+    (define zfs-scan
+      (shepherd-service
+        (provision '(zfs-scan))
+        (documentation "Scans for ZFS pools.")
+        (requirement `(kernel-module-loader
+                       root-file-system
+                       ,@(map dependency->shepherd-service-name
+                              (zfs-configuration-dependencies conf))))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error importing zpools: ~a~%"
+                                      (condition-message c))
+                              #f))
+                     ; TODO: optionally use a cachefile, for systems with dozens or
+                     ; hundreds of devices.
+                     (invoke/quiet #$zpool "import" "-a" "-N"))))
+        (stop #~(const #t))))
+
+    (define device-mapping-zvol/*
+      (shepherd-service
+        (provision '(device-mapping-zvol/*))
+        (documentation "Waits for ZFS ZVOL devices to appear.")
+        (requirement '(zfs-scan))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error waiting for zvols: ~a~%"
+                                      (condition-message c))
+                              #f))
+                     (invoke/quiet #$zvol_wait))))
+        (stop #~(const #t))))
+
+    (define zfs-automount
+      (shepherd-service
+        (provision '(zfs-automount))
+        (documentation "Automounts ZFS datasets.")
+        (requirement '(zfs-scan))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error automounting zfs: ~a~$")
+                              #f))
+                     ; (current-output-port) is typically connected to /dev/klog,
+                     ; so redirect it to (current-error-port) so that user can see
+                     ; prompts for passphrases on console
+                     (with-output-to-port (current-error-port)
+                       (lambda ()
+                         (invoke #$zfs "mount" "-a" "-l"))))))
+        (stop #~(lambda _
+                  ;; make sure we don't keep any ZFS mountpoints busy.
+                  (chdir "/")
+                  ;; unmount everything.
+                  (invoke/quiet #$zfs "unmount" "-a" "-f")))))
+
+    (list zfs-scan
+          device-mapping-zvol/*
+          zfs-automount)))
+
+(define zfs-service-type
+  (service-type (name 'zfs)
+                (extensions
+                  (list
+                    ; install the kernel module
+                    (service-extension kernel-loadable-module-service-type
+                                       zfs-loadable-module)
+                    ; load the kernel module
+                    (service-extension kernel-module-loader-service-type
+                                       (const '("zfs")))
+                    ; scan ZFS pools, automount filesystem, wait for zvols.
+                    (service-extension shepherd-root-service-type
+                                       zfs-shepherd-services)
+                    ; make sure automount occurs before file-systems target is reached
+                    (service-extension file-systems-target-service-type
+                                       (const '(zfs-automount)))
+                    ; install ZFS management tools
+                    (service-extension profile-service-type
+                                       (compose list make-zfs-package))
+                    ; install ZFS udev rules
+                    (service-extension udev-service-type
+                                       (compose list make-zfs-package))))
+                (description
+                  "Install ZFS, an advanced filesystem and volume manager.")))
--
2.30.0