From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id XGVeIAtq+V8hFgAA0tVLHw (envelope-from ) for ; Sat, 09 Jan 2021 08:32:11 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id qEyaGwtq+V8rOAAA1q6Kng (envelope-from ) for ; Sat, 09 Jan 2021 08:32:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C12BD94043A for ; Sat, 9 Jan 2021 08:32:10 +0000 (UTC) Received: from localhost ([::1]:36392 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ky9fN-0001qn-LM for larch@yhetil.org; Sat, 09 Jan 2021 03:32:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:44398) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ky9fG-0001qg-40 for guix-patches@gnu.org; Sat, 09 Jan 2021 03:32:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:39642) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ky9fF-0004UQ-S7 for guix-patches@gnu.org; Sat, 09 Jan 2021 03:32:01 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ky9fF-0004CE-OW for guix-patches@gnu.org; Sat, 09 Jan 2021 03:32:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45692] [PATCH 4/4] gnu: Add ZFS service. Resent-From: raid5atemyhomework Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 09 Jan 2021 08:32:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45692 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: "45692@debbugs.gnu.org" <45692@debbugs.gnu.org> Received: via spool by 45692-submit@debbugs.gnu.org id=B45692.161018111416117 (code B ref 45692); Sat, 09 Jan 2021 08:32:01 +0000 Received: (at 45692) by debbugs.gnu.org; 9 Jan 2021 08:31:54 +0000 Received: from localhost ([127.0.0.1]:51188 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ky9f7-0004Bs-IY for submit@debbugs.gnu.org; Sat, 09 Jan 2021 03:31:54 -0500 Received: from mail-40137.protonmail.ch ([185.70.40.137]:14300) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ky9f5-0004Bd-88 for 45692@debbugs.gnu.org; Sat, 09 Jan 2021 03:31:52 -0500 Date: Sat, 09 Jan 2021 08:31:37 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail; t=1610181104; bh=d643oRIfvk9MlprRc+g5Ld1PuPEGjmuzbCYSI31hQxU=; h=Date:To:From:Reply-To:Subject:In-Reply-To:References:From; b=xTqxNg5DjyvvJ/fYyrupt553KYdrXF8mISq9uK9Ys43BY0umRb7MlhA8szeHkjEE/ QPCoV4HeLa3rcCflAHBdR5h1TyL8aU9z+WXtmMdCfCKDTqUljO4Le/4knDMHP0Byd2 GI3WPBN1lKR4UOFG90mvyi5lIrXI0i9t2IUacEZA= Message-ID: <4XtBUBZvLlaRj3Qw8d44cVUm2QWePWKlxR4G7yH_bwXA62SoedCr5VHkch5VkOXfEd5frGaddsTLEEyiVQC0kQmearhfiiveI626gGbVjts=@protonmail.com> In-Reply-To: References: <6wemXB-PfHUqbuVr5-XRf0-tY4cKGGtKiUqrZPrIZYXoBw17L3xRuZrGOJQfTo5PKfFNCM8KyRTllidoc7asPE2x98BTiJSPVR7OSjxCuw8=@protonmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" Reply-to: raid5atemyhomework , raid5atemyhomework via Guix-patches From: raid5atemyhomework via Guix-patches via X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.85 Authentication-Results: aspmx1.migadu.com; dkim=fail (headers rsa verify failed) header.d=protonmail.com header.s=protonmail header.b=xTqxNg5D; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: C12BD94043A X-Spam-Score: -2.85 X-Migadu-Scanner: scn0.migadu.com X-TUID: wx5CBqaH/lN8 New version again. Change: Fix documentation `doc/guix.texi` to remove men= tions of `options` and add documentation about the new `dependencies` featu= res. >From dfe9ad7512d348933beb5b42041965ff604422b5 Mon Sep 17 00:00:00 2001 From: raid5atemyhomework Date: Wed, 6 Jan 2021 09:24:20 +0800 Subject: [PATCH 1/2] gnu: Add service to install ZFS. * gnu/services/file-systems.scm: New file. (zfs-service-type): New variable. (): New type. (%zfs-zvol-dependency): New variable. * gnu/local.mk: Add gnu/services/file-systems.scm. * gnu/services/base.scm (dependency->shepherd-service-name): Export. * doc/guix.texi (ZFS file system): New subsection. --- doc/guix.texi | 129 +++++++++++++++++++++++++ gnu/local.mk | 1 + gnu/services/base.scm | 4 +- gnu/services/file-systems.scm | 174 ++++++++++++++++++++++++++++++++++ 4 files changed, 307 insertions(+), 1 deletion(-) create mode 100644 gnu/services/file-systems.scm diff --git a/doc/guix.texi b/doc/guix.texi index a31d355780..7004efe3c4 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -13886,6 +13886,135 @@ a file system declaration such as: compress-force=3Dzstd,space_cache=3Dv2")) @end lisp +@node ZFS file system +@subsection ZFS file system + +The ZFS file system has a license incompatible with the Linux kernel, +and thus cannot be distributed with the kernel. However, as a user +you have the right to do whatever you want on your own hardware, +including download the ZFS source code, compile it, link it to your +own private copy of Linux, and run it. You can even use the Guix build +system to automate this. + +As a large and complex kernel module, ZFS on Linux has to be compiled +with a specific version of the kernel. Often even the latest ZFS +package available cannot be compiled with the latest Linux kernel +package provided by Guix System. Thus, installing the @code{zfs} +package is likely to fail. + +Instead, you have to use an older long-term-support Linux kernel. +Do not use @code{linux-libre-lts}, since the latest long-term-support +kernel might be too new for the ZFS package; instead, explicitly +select the version number, like @code{linux-libre-5.4}, and upgrade +it manually later when you have verified that the ZFS version +available on Guix can be compiled with a later LTS kernel. + +Then, you have to modify your system configuration file and use the +selected older kernel, and add the @code{zfs-service-type} service. + +@lisp +(use-modules (gnu)) +(use-package-modules + ; @dots{} + linux) +(use-service-modules + ; @dots{} + file-systems) + +(define my-kernel linux-libre-5.4) + +(operating-system + (kernel my-kernel) + ;; @dots{} + (services + (cons* (service zfs-service-type + (zfs-configuration + (kernel my-kernel))) + ; @dots{} + %desktop-services)) + ;; @dots{} + ) +@end lisp + +@defvr {Scheme Variable} zfs-service-type +This is the type of the service to compile and install OpenZFS to +your operating system. It loads the ZFS module at startup, imports +pools, mounts automounted ZFS datasets, and installs the ZFS command +line tools. Its value must be a @code{zfs-configuration} record +(see below). + +Here is an example use: + +@lisp +(service zfs-service-type + (zfs-configuration + (kernel linux-libre-5.4))) +@end lisp +@end defvr + +@deftp {Data Type} zfs-configuration +This data type represents the configuration of the ZFS service. +The available fields are: + +@table @asis +@item @code{kernel} +The package of the Linux kernel to compile ZFS for. Required. It +@emph{must} be the same kernel you use in your operating system. + +@item @code{base-zfs} (default: @code{zfs}) +The ZFS package to use. It will be modified to use the indicated +kernel. + +@item @code{dependencies} (default: @code{'()}) +A list of @code{}s or @code{}s that +must be mounted or opened before ZFS scans for pools to import. +For example, you might have LUKS containers as the leaf VDEVs of +a ZFS pool. +@end table +@end deftp + +Once your system has been configured to include the ZFS service +and you have restarted the system, you can manage ZFS pools and +datasets with @code{zpool} and @code{zfs} commands. + +ZFS datasets with an appropriate @code{mountpoint} property will +be automounted at startup after the root file system is started. +Encrypted datasets that are automounted will cause boot to pause +and prompt for the password to be provided on the console. + +It's possible to have a ZFS dataset as your @code{/home} by simply +setting the @code{mountpoint} ZFS property. However, note that ZFS +will refuse to mount over a non-empty directory, so if your root +filesystem already has a non-empty @code{/home}, you should remove +it and its contents, then restart the system. + +ZFS features @dfn{ZVOL}s, which are block devices that ZFS exposes +to the system. You can put any file system inside a ZVOL. In order +to mount such a filesystem at boot, you can declare it as dependent +on @code{%zfs-zvol-dependency}. + +@defvr {Scheme Variable} %zfs-zvol-dependency +This is an artificial @code{} object which tells +the file mounting service to wait for ZFS to provide ZVOLs before +mounting the file system that is dependent on it. For example: + +@lisp +(file-system + (device "/dev/zvol/pool-name/zvol-name") + (mount-point "/ext4-on-zfs") + (type "ext4") + (dependencies (list %zfs-zvol-dependency))) +@end lisp + +Do @emph{not} add @code{%zfs-zvol-dependency} to your +@code{mapped-devices} declaration, In addition, only use it as a +@code{file-system} dependency if you instantiate a +@code{zfs-service-type} service in your operating system. +@end defvr + +Having ZFS as a root filesystem or as @code{/boot} is not supported +yet. + @node Mapped Devices @section Mapped Devices diff --git a/gnu/local.mk b/gnu/local.mk index 1151d4642e..5aeb45c4c2 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -602,6 +602,7 @@ GNU_SYSTEM_MODULES =3D=09=09=09=09\ %D%/services/dict.scm=09=09=09=09\ %D%/services/dns.scm=09=09=09=09\ %D%/services/docker.scm=09=09=09\ + %D%/services/file-systems.scm=09=09=09\ %D%/services/authentication.scm=09=09\ %D%/services/games.scm=09=09=09\ %D%/services/ganeti.scm=09=09=09\ diff --git a/gnu/services/base.scm b/gnu/services/base.scm index 13cfb6a8a2..ef3680583b 100644 --- a/gnu/services/base.scm +++ b/gnu/services/base.scm @@ -188,7 +188,9 @@ references-file - %base-services)) + %base-services + + dependency->shepherd-service-name)) ;;; Commentary: ;;; diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm new file mode 100644 index 0000000000..9061ab9582 --- /dev/null +++ b/gnu/services/file-systems.scm @@ -0,0 +1,174 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2021 raid5atemyhomework +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu services file-systems) + #:use-module (gnu packages file-systems) + #:use-module (gnu services) + #:use-module (gnu services base) + #:use-module (gnu services linux) + #:use-module (gnu services shepherd) + #:use-module (gnu system mapped-devices) + #:use-module (guix gexp) + #:use-module (guix packages) + #:use-module (guix records) + #:export (zfs-service-type + + zfs-configuration + zfs-configuration? + zfs-configuration-kernel + zfs-configuration-base-zfs + zfs-configuration-dependencies + + %zfs-zvol-dependency)) + +(define-record-type* + zfs-configuration make-zfs-configuration zfs-configuration? + ; kernel you want to compile the base-zfs module for. + (kernel zfs-configuration-kernel) + ; base package that will be compiled for the kernel + (base-zfs zfs-configuration-base-zfs + (default zfs)) + ; list of | we should wait for, + ; before scanning for ZFS pools. + (dependencies zfs-configuration-dependencies + (default '()))) + +;; This is a synthetic and unusable MAPPED-DEVICE; its only use +;; is to be added as a (dependency ...) of some FILE-SYSTEM. +(define %zfs-zvol-dependency + (mapped-device + (source '()) + ;; The /* prevents naming conflict with non-ZFS device mappings, + ;; since it is not a valid name for mapped devices, and also + ;; implies "all zvols" in terms of globs. + (targets '("zvol/*")) + (type #f))) + +(define (make-zfs-package conf) + (let ((base-zfs (zfs-configuration-base-zfs conf)) + (kernel (zfs-configuration-kernel conf))) + (package + (inherit base-zfs) + (name (string-join (list (package-name base-zfs) + "for" + (package-name kernel) + (package-version kernel) + "version") + "-")) + (arguments (cons* #:linux kernel (package-arguments base-zfs)))))) + +(define (zfs-loadable-module conf) + (list (list (make-zfs-package conf) "module"))) + +(define (zfs-shepherd-services conf) + (let* ((zfs-package (make-zfs-package conf)) + (zpool (file-append zfs-package "/sbin/zpool")) + (zfs (file-append zfs-package "/sbin/zfs")) + (zvol_wait (file-append zfs-package "/bin/zvol_wait")) + (scheme-modules `((srfi srfi-1) + (srfi srfi-34) + (srfi srfi-35) + (rnrs io ports) + ,@%default-modules))) + + (define zfs-scan + (shepherd-service + (provision '(zfs-scan)) + (documentation "Scans for ZFS pools.") + (requirement `(kernel-module-loader + root-file-system + ,@(map dependency->shepherd-service-name + (zfs-configuration-dependencies conf)))) + (modules scheme-modules) + (start #~(lambda _ + (guard (c ((message-condition? c) + (format (current-error-port) + "error importing zpools: ~a~%" + (condition-message c)) + #f)) + ; TODO: optionally use a cachefile, for systems with = dozens or + ; hundreds of devices. + (invoke/quiet #$zpool "import" "-a" "-N")))) + (stop #~(const #t)))) + + (define device-mapping-zvol/* + (shepherd-service + (provision '(device-mapping-zvol/*)) + (documentation "Waits for ZFS ZVOL devices to appear.") + (requirement '(zfs-scan)) + (modules scheme-modules) + (start #~(lambda _ + (guard (c ((message-condition? c) + (format (current-error-port) + "error waiting for zvols: ~a~%" + (condition-message c)) + #f)) + (invoke/quiet #$zvol_wait)))) + (stop #~(const #t)))) + + (define zfs-automount + (shepherd-service + (provision '(zfs-automount)) + (documentation "Automounts ZFS datasets.") + (requirement '(zfs-scan)) + (modules scheme-modules) + (start #~(lambda _ + (guard (c ((message-condition? c) + (format (current-error-port) + "error automounting zfs: ~a~$") + #f)) + ; (current-output-port) is typically connected to /de= v/klog, + ; so redirect it to (current-error-port) so that user= can see + ; prompts for passphrases on console + (with-output-to-port (current-error-port) + (lambda () + (invoke #$zfs "mount" "-a" "-l")))))) + (stop #~(lambda _ + ;; make sure we don't keep any ZFS mountpoints busy. + (chdir "/") + ;; unmount everything. + (invoke/quiet #$zfs "unmount" "-a" "-f"))))) + + (list zfs-scan + device-mapping-zvol/* + zfs-automount))) + +(define zfs-service-type + (service-type (name 'zfs) + (extensions + (list + ; install the kernel module + (service-extension kernel-loadable-module-service-type + zfs-loadable-module) + ; load the kernel module + (service-extension kernel-module-loader-service-type + (const '("zfs"))) + ; scan ZFS pools, automount filesystem, wait for zvols= . + (service-extension shepherd-root-service-type + zfs-shepherd-services) + ; make sure automount occurs before file-systems targe= t is reached + (service-extension file-systems-target-service-type + (const '(zfs-automount))) + ; install ZFS management tools + (service-extension profile-service-type + (compose list make-zfs-package)) + ; install ZFS udev rules + (service-extension udev-service-type + (compose list make-zfs-package)))) + (description + "Install ZFS, an advanced filesystem and volume manager.= "))) -- 2.30.0