unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
From: raid5atemyhomework via Guix-patches via <guix-patches@gnu.org>
To: "45692@debbugs.gnu.org" <45692@debbugs.gnu.org>
Subject: [bug#45692] [PATCH 4/4] gnu: Add ZFS service.
Date: Sat, 09 Jan 2021 08:31:37 +0000	[thread overview]
Message-ID: <4XtBUBZvLlaRj3Qw8d44cVUm2QWePWKlxR4G7yH_bwXA62SoedCr5VHkch5VkOXfEd5frGaddsTLEEyiVQC0kQmearhfiiveI626gGbVjts=@protonmail.com> (raw)
In-Reply-To: <LmlpVtErWRRqKI9LIqHvswFA3ig_rjwIk9u8S96Bp6A9VG_l0bAnQdhIQsxf_fB5CAJxMnoClNXDObuPloDX7JDDOl4ZuFGn6u54FmJ5K2o=@protonmail.com>

New version again.  Change: Fix documentation `doc/guix.texi` to remove mentions of `options` and add documentation about the new `dependencies` features.


From dfe9ad7512d348933beb5b42041965ff604422b5 Mon Sep 17 00:00:00 2001
From: raid5atemyhomework <raid5atemyhomework@protonmail.com>
Date: Wed, 6 Jan 2021 09:24:20 +0800
Subject: [PATCH 1/2] gnu: Add service to install ZFS.

* gnu/services/file-systems.scm: New file.
(zfs-service-type): New variable.
(<zfs-configuration>): New type.
(%zfs-zvol-dependency): New variable.
* gnu/local.mk: Add gnu/services/file-systems.scm.
* gnu/services/base.scm (dependency->shepherd-service-name): Export.
* doc/guix.texi (ZFS file system): New subsection.
---
 doc/guix.texi                 | 129 +++++++++++++++++++++++++
 gnu/local.mk                  |   1 +
 gnu/services/base.scm         |   4 +-
 gnu/services/file-systems.scm | 174 ++++++++++++++++++++++++++++++++++
 4 files changed, 307 insertions(+), 1 deletion(-)
 create mode 100644 gnu/services/file-systems.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index a31d355780..7004efe3c4 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -13886,6 +13886,135 @@ a file system declaration such as:
 compress-force=zstd,space_cache=v2"))
 @end lisp

+@node ZFS file system
+@subsection ZFS file system
+
+The ZFS file system has a license incompatible with the Linux kernel,
+and thus cannot be distributed with the kernel. However, as a user
+you have the right to do whatever you want on your own hardware,
+including download the ZFS source code, compile it, link it to your
+own private copy of Linux, and run it. You can even use the Guix build
+system to automate this.
+
+As a large and complex kernel module, ZFS on Linux has to be compiled
+with a specific version of the kernel. Often even the latest ZFS
+package available cannot be compiled with the latest Linux kernel
+package provided by Guix System. Thus, installing the @code{zfs}
+package is likely to fail.
+
+Instead, you have to use an older long-term-support Linux kernel.
+Do not use @code{linux-libre-lts}, since the latest long-term-support
+kernel might be too new for the ZFS package; instead, explicitly
+select the version number, like @code{linux-libre-5.4}, and upgrade
+it manually later when you have verified that the ZFS version
+available on Guix can be compiled with a later LTS kernel.
+
+Then, you have to modify your system configuration file and use the
+selected older kernel, and add the @code{zfs-service-type} service.
+
+@lisp
+(use-modules (gnu))
+(use-package-modules
+   ; @dots{}
+   linux)
+(use-service-modules
+   ; @dots{}
+   file-systems)
+
+(define my-kernel linux-libre-5.4)
+
+(operating-system
+  (kernel my-kernel)
+  ;; @dots{}
+  (services
+    (cons* (service zfs-service-type
+             (zfs-configuration
+               (kernel my-kernel)))
+           ; @dots{}
+           %desktop-services))
+  ;; @dots{}
+  )
+@end lisp
+
+@defvr {Scheme Variable} zfs-service-type
+This is the type of the service to compile and install OpenZFS to
+your operating system. It loads the ZFS module at startup, imports
+pools, mounts automounted ZFS datasets, and installs the ZFS command
+line tools.  Its value must be a @code{zfs-configuration} record
+(see below).
+
+Here is an example use:
+
+@lisp
+(service zfs-service-type
+  (zfs-configuration
+    (kernel linux-libre-5.4)))
+@end lisp
+@end defvr
+
+@deftp {Data Type} zfs-configuration
+This data type represents the configuration of the ZFS service.
+The available fields are:
+
+@table @asis
+@item @code{kernel}
+The package of the Linux kernel to compile ZFS for. Required. It
+@emph{must} be the same kernel you use in your operating system.
+
+@item @code{base-zfs} (default: @code{zfs})
+The ZFS package to use. It will be modified to use the indicated
+kernel.
+
+@item @code{dependencies} (default: @code{'()})
+A list of @code{<file-system>}s or @code{<mapped-device>}s that
+must be mounted or opened before ZFS scans for pools to import.
+For example, you might have LUKS containers as the leaf VDEVs of
+a ZFS pool.
+@end table
+@end deftp
+
+Once your system has been configured to include the ZFS service
+and you have restarted the system, you can manage ZFS pools and
+datasets with @code{zpool} and @code{zfs} commands.
+
+ZFS datasets with an appropriate @code{mountpoint} property will
+be automounted at startup after the root file system is started.
+Encrypted datasets that are automounted will cause boot to pause
+and prompt for the password to be provided on the console.
+
+It's possible to have a ZFS dataset as your @code{/home} by simply
+setting the @code{mountpoint} ZFS property. However, note that ZFS
+will refuse to mount over a non-empty directory, so if your root
+filesystem already has a non-empty @code{/home}, you should remove
+it and its contents, then restart the system.
+
+ZFS features @dfn{ZVOL}s, which are block devices that ZFS exposes
+to the system. You can put any file system inside a ZVOL. In order
+to mount such a filesystem at boot, you can declare it as dependent
+on @code{%zfs-zvol-dependency}.
+
+@defvr {Scheme Variable} %zfs-zvol-dependency
+This is an artificial @code{<mapped-device>} object which tells
+the file mounting service to wait for ZFS to provide ZVOLs before
+mounting the file system that is dependent on it. For example:
+
+@lisp
+(file-system
+  (device "/dev/zvol/pool-name/zvol-name")
+  (mount-point "/ext4-on-zfs")
+  (type "ext4")
+  (dependencies (list %zfs-zvol-dependency)))
+@end lisp
+
+Do @emph{not} add @code{%zfs-zvol-dependency} to your
+@code{mapped-devices} declaration, In addition, only use it as a
+@code{file-system} dependency if you instantiate a
+@code{zfs-service-type} service in your operating system.
+@end defvr
+
+Having ZFS as a root filesystem or as @code{/boot} is not supported
+yet.
+
 @node Mapped Devices
 @section Mapped Devices

diff --git a/gnu/local.mk b/gnu/local.mk
index 1151d4642e..5aeb45c4c2 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -602,6 +602,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/dict.scm				\
   %D%/services/dns.scm				\
   %D%/services/docker.scm			\
+  %D%/services/file-systems.scm			\
   %D%/services/authentication.scm		\
   %D%/services/games.scm			\
   %D%/services/ganeti.scm			\
diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 13cfb6a8a2..ef3680583b 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -188,7 +188,9 @@

             references-file

-            %base-services))
+            %base-services
+
+            dependency->shepherd-service-name))

 ;;; Commentary:
 ;;;
diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm
new file mode 100644
index 0000000000..9061ab9582
--- /dev/null
+++ b/gnu/services/file-systems.scm
@@ -0,0 +1,174 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services file-systems)
+  #:use-module (gnu packages file-systems)
+  #:use-module (gnu services)
+  #:use-module (gnu services base)
+  #:use-module (gnu services linux)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system mapped-devices)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+  #:use-module (guix records)
+  #:export (zfs-service-type
+
+            zfs-configuration
+            zfs-configuration?
+            zfs-configuration-kernel
+            zfs-configuration-base-zfs
+            zfs-configuration-dependencies
+
+            %zfs-zvol-dependency))
+
+(define-record-type* <zfs-configuration>
+  zfs-configuration make-zfs-configuration zfs-configuration?
+  ; kernel you want to compile the base-zfs module for.
+  (kernel         zfs-configuration-kernel)
+  ; base package that will be compiled for the kernel
+  (base-zfs       zfs-configuration-base-zfs
+                  (default zfs))
+  ; list of <mapped-device> | <file-system> we should wait for,
+  ; before scanning for ZFS pools.
+  (dependencies   zfs-configuration-dependencies
+                  (default '())))
+
+;; This is a synthetic and unusable MAPPED-DEVICE; its only use
+;; is to be added as a (dependency ...) of some FILE-SYSTEM.
+(define %zfs-zvol-dependency
+  (mapped-device
+    (source '())
+    ;; The /* prevents naming conflict with non-ZFS device mappings,
+    ;; since it is not a valid name for mapped devices, and also
+    ;; implies "all zvols" in terms of globs.
+    (targets '("zvol/*"))
+    (type #f)))
+
+(define (make-zfs-package conf)
+  (let ((base-zfs (zfs-configuration-base-zfs conf))
+        (kernel   (zfs-configuration-kernel conf)))
+    (package
+      (inherit base-zfs)
+      (name (string-join (list (package-name base-zfs)
+                               "for"
+                               (package-name kernel)
+                               (package-version kernel)
+                               "version")
+                         "-"))
+      (arguments (cons* #:linux kernel (package-arguments base-zfs))))))
+
+(define (zfs-loadable-module conf)
+  (list (list (make-zfs-package conf) "module")))
+
+(define (zfs-shepherd-services conf)
+  (let* ((zfs-package     (make-zfs-package conf))
+         (zpool           (file-append zfs-package "/sbin/zpool"))
+         (zfs             (file-append zfs-package "/sbin/zfs"))
+         (zvol_wait       (file-append zfs-package "/bin/zvol_wait"))
+         (scheme-modules  `((srfi srfi-1)
+                            (srfi srfi-34)
+                            (srfi srfi-35)
+                            (rnrs io ports)
+                            ,@%default-modules)))
+
+    (define zfs-scan
+      (shepherd-service
+        (provision '(zfs-scan))
+        (documentation "Scans for ZFS pools.")
+        (requirement `(kernel-module-loader
+                       root-file-system
+                       ,@(map dependency->shepherd-service-name
+                              (zfs-configuration-dependencies conf))))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error importing zpools: ~a~%"
+                                      (condition-message c))
+                              #f))
+                     ; TODO: optionally use a cachefile, for systems with dozens or
+                     ; hundreds of devices.
+                     (invoke/quiet #$zpool "import" "-a" "-N"))))
+        (stop #~(const #t))))
+
+    (define device-mapping-zvol/*
+      (shepherd-service
+        (provision '(device-mapping-zvol/*))
+        (documentation "Waits for ZFS ZVOL devices to appear.")
+        (requirement '(zfs-scan))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error waiting for zvols: ~a~%"
+                                      (condition-message c))
+                              #f))
+                     (invoke/quiet #$zvol_wait))))
+        (stop #~(const #t))))
+
+    (define zfs-automount
+      (shepherd-service
+        (provision '(zfs-automount))
+        (documentation "Automounts ZFS datasets.")
+        (requirement '(zfs-scan))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error automounting zfs: ~a~$")
+                              #f))
+                     ; (current-output-port) is typically connected to /dev/klog,
+                     ; so redirect it to (current-error-port) so that user can see
+                     ; prompts for passphrases on console
+                     (with-output-to-port (current-error-port)
+                       (lambda ()
+                         (invoke #$zfs "mount" "-a" "-l"))))))
+        (stop #~(lambda _
+                  ;; make sure we don't keep any ZFS mountpoints busy.
+                  (chdir "/")
+                  ;; unmount everything.
+                  (invoke/quiet #$zfs "unmount" "-a" "-f")))))
+
+    (list zfs-scan
+          device-mapping-zvol/*
+          zfs-automount)))
+
+(define zfs-service-type
+  (service-type (name 'zfs)
+                (extensions
+                  (list
+                    ; install the kernel module
+                    (service-extension kernel-loadable-module-service-type
+                                       zfs-loadable-module)
+                    ; load the kernel module
+                    (service-extension kernel-module-loader-service-type
+                                       (const '("zfs")))
+                    ; scan ZFS pools, automount filesystem, wait for zvols.
+                    (service-extension shepherd-root-service-type
+                                       zfs-shepherd-services)
+                    ; make sure automount occurs before file-systems target is reached
+                    (service-extension file-systems-target-service-type
+                                       (const '(zfs-automount)))
+                    ; install ZFS management tools
+                    (service-extension profile-service-type
+                                       (compose list make-zfs-package))
+                    ; install ZFS udev rules
+                    (service-extension udev-service-type
+                                       (compose list make-zfs-package))))
+                (description
+                  "Install ZFS, an advanced filesystem and volume manager.")))
--
2.30.0




  reply	other threads:[~2021-01-09  8:32 UTC|newest]

Thread overview: 81+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-06 15:52 [bug#45692] [PATCH 0/4] Even Better ZFS Support on Guix raid5atemyhomework via Guix-patches via
2021-01-06 15:54 ` [bug#45692] [PATCH 1/4] gnu: Allow services to install kernel-loadable modules raid5atemyhomework via Guix-patches via
2021-01-08 16:16   ` raid5atemyhomework via Guix-patches via
2021-02-10 14:13   ` [bug#45692] [PATCH 0/4] Even Better ZFS Support on Guix Ludovic Courtès
2021-02-10 15:44   ` Ludovic Courtès
2021-02-10 16:49     ` raid5atemyhomework via Guix-patches via
2021-01-06 15:55 ` [bug#45692] [PATCH 2/4] gnu: Make file-systems target extensible by services raid5atemyhomework via Guix-patches via
2021-01-23 13:05   ` 宋文武
2021-01-25  0:18     ` guix-patches--- via
2021-02-10 14:17       ` [bug#45692] [PATCH 0/4] Even Better ZFS Support on Guix Ludovic Courtès
2021-02-10 14:46         ` raid5atemyhomework via Guix-patches via
2021-01-06 15:56 ` [bug#45692] [PATCH 3/4] gnu: Fix ZFS package raid5atemyhomework via Guix-patches via
2021-01-07  8:23   ` Danny Milosavljevic
2021-01-06 15:57 ` [bug#45692] [PATCH 4/4] gnu: Add ZFS service raid5atemyhomework via Guix-patches via
2021-01-06 19:41   ` [bug#45703] kernel-module-configuration-service for configuring kernel parameters Danny Milosavljevic
2021-01-07  0:04     ` raid5atemyhomework via Guix-patches via
2021-01-07  5:38       ` [bug#45692] " raid5atemyhomework via Guix-patches via
2021-01-07  9:16         ` [bug#42193] " raid5atemyhomework via Guix-patches via
2021-01-08 15:02   ` [bug#45692] [PATCH 4/4] gnu: Add ZFS service raid5atemyhomework via Guix-patches via
2021-01-09  8:31     ` raid5atemyhomework via Guix-patches via [this message]
2021-02-08  3:31       ` Danny Milosavljevic
2021-02-08  6:25         ` raid5atemyhomework via Guix-patches via
2021-02-10 14:27     ` [bug#45692] [PATCH 0/4] Even Better ZFS Support on Guix Ludovic Courtès
2021-02-10 14:32       ` raid5atemyhomework via Guix-patches via
2021-02-13  1:49       ` raid5atemyhomework via Guix-patches via
2021-03-22 14:33 ` [bug#45692] [PATCH v3 0/3] New patch series for " raid5atemyhomework via Guix-patches via
2021-03-28 12:55   ` Léo Le Bouter via Guix-patches via
2021-03-29  4:39     ` raid5atemyhomework via Guix-patches via
2021-07-23 15:11       ` raid5atemyhomework via Guix-patches via
2021-03-22 14:33 ` [bug#45692] [PATCH v3 1/3] gnu: Allow services to install kernel-loadable modules raid5atemyhomework via Guix-patches via
2021-05-11 14:17   ` Danny Milosavljevic
2021-03-22 14:34 ` [bug#45692] [PATCH v3 2/3] gnu: Add zfs-auto-snapshot raid5atemyhomework via Guix-patches via
2021-05-11 14:05   ` Danny Milosavljevic
2021-05-13  1:21     ` raid5atemyhomework via Guix-patches via
2021-05-13 13:08       ` Danny Milosavljevic
2021-03-22 14:35 ` [bug#45692] [PATCH v3 3/3] gnu: Add ZFS service type raid5atemyhomework via Guix-patches via
2021-07-25 14:03   ` raid5atemyhomework via Guix-patches via
2021-07-25 14:31 ` [bug#45692] [PATCH v4 " raid5atemyhomework via Guix-patches via
2021-08-01  9:41   ` raid5atemyhomework via Guix-patches via
2021-08-10 11:43     ` raid5atemyhomework via Guix-patches via
2021-08-31  0:48       ` raid5atemyhomework via Guix-patches via
2021-09-02 20:57   ` Maxime Devos
2021-09-02 22:22     ` Maxime Devos
2021-09-03 12:41       ` raid5atemyhomework via Guix-patches via
2021-09-04 18:58     ` raid5atemyhomework via Guix-patches via
2021-09-06  8:08     ` zimoun
2021-09-06 10:40       ` Maxime Devos
2021-09-06 11:08         ` raid5atemyhomework via Guix-patches via
2021-09-06 17:17         ` zimoun
2021-09-07  9:54           ` Maxime Devos
2021-09-08  1:23             ` raid5atemyhomework via Guix-patches via
2021-09-15 14:04               ` raid5atemyhomework via Guix-patches via
2021-09-21  9:42                 ` zimoun
2021-09-04 21:19   ` Xinglu Chen
2021-09-06 10:52     ` raid5atemyhomework via Guix-patches via
2021-09-06 14:22       ` Xinglu Chen
2021-09-02 21:24 ` [bug#45692] Gaslighting Mason Loring Bliss
2021-09-03 12:22   ` Maxime Devos
2021-09-06  7:59   ` [bug#45692] zimoun
2021-09-30 14:56 ` [bug#45692] [PATCH v5 3/3] gnu: Add ZFS service type raid5atemyhomework via Guix-patches via
2021-10-19 13:18   ` raid5atemyhomework via Guix-patches via
2021-10-27  7:30     ` raid5atemyhomework via Guix-patches via
2021-10-27 16:38       ` pelzflorian (Florian Pelz)
2021-11-30 15:26         ` raid5atemyhomework via Guix-patches via
2021-12-12 13:32           ` raid5atemyhomework via Guix-patches via
2021-12-21 21:15             ` [bug#45643] [PATCH 0/3] Better Support for ZFS on Guix Brice Waegeneire
2022-01-01 11:59               ` [bug#45692] bug#45643: " raid5atemyhomework via Guix-patches via
2022-01-19 14:24                 ` raid5atemyhomework via Guix-patches via
2022-01-07  4:21 ` [bug#45692] " raid5atemyhomework via Guix-patches via
2022-02-14 14:10   ` raid5atemyhomework via Guix-patches via
2022-02-18  7:13     ` raid5atemyhomework via Guix-patches via
2022-03-16 23:44       ` raid5atemyhomework via Guix-patches via
2022-03-17  8:24       ` Liliana Marie Prikler
2022-03-17 17:22         ` Maxime Devos
2022-03-17 18:38           ` zimoun
2022-03-17 19:10             ` Maxime Devos
2022-03-19 14:24           ` raid5atemyhomework via Guix-patches via
2022-03-20  4:42             ` Maxim Cournoyer
2022-03-19 14:09         ` raid5atemyhomework via Guix-patches via
2022-03-19 16:22           ` Leo Famulari
2022-03-19 14:25 ` [bug#45692] (No Subject) raid5atemyhomework via Guix-patches via

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='4XtBUBZvLlaRj3Qw8d44cVUm2QWePWKlxR4G7yH_bwXA62SoedCr5VHkch5VkOXfEd5frGaddsTLEEyiVQC0kQmearhfiiveI626gGbVjts=@protonmail.com' \
    --to=guix-patches@gnu.org \
    --cc=45692@debbugs.gnu.org \
    --cc=raid5atemyhomework@protonmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).