From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id mJ/yAjl0+F/pbwAA0tVLHw (envelope-from ) for ; Fri, 08 Jan 2021 15:03:21 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id qEFKOjh0+F94FAAAbx9fmQ (envelope-from ) for ; Fri, 08 Jan 2021 15:03:20 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id EF0299402AF for ; Fri, 8 Jan 2021 15:03:19 +0000 (UTC) Received: from localhost ([::1]:51766 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kxtIM-0006Nh-LP for larch@yhetil.org; Fri, 08 Jan 2021 10:03:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:37852) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kxtI6-0006Lj-BX for guix-patches@gnu.org; Fri, 08 Jan 2021 10:03:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:38794) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kxtI6-0007ov-3Y for guix-patches@gnu.org; Fri, 08 Jan 2021 10:03:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kxtI5-0001TE-WB for guix-patches@gnu.org; Fri, 08 Jan 2021 10:03:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45692] [PATCH 4/4] gnu: Add ZFS service. Resent-From: raid5atemyhomework Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 08 Jan 2021 15:03:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45692 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: "45692@debbugs.gnu.org" <45692@debbugs.gnu.org> Received: via spool by 45692-submit@debbugs.gnu.org id=B45692.16101181555615 (code B ref 45692); Fri, 08 Jan 2021 15:03:01 +0000 Received: (at 45692) by debbugs.gnu.org; 8 Jan 2021 15:02:35 +0000 Received: from localhost ([127.0.0.1]:50340 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxtHe-0001SV-5T for submit@debbugs.gnu.org; Fri, 08 Jan 2021 10:02:34 -0500 Received: from mail-40137.protonmail.ch ([185.70.40.137]:58897) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kxtHb-0001S7-Tr for 45692@debbugs.gnu.org; Fri, 08 Jan 2021 10:02:33 -0500 Date: Fri, 08 Jan 2021 15:02:21 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail; t=1610118144; bh=PVd20+nCRSa82RwW8MCxBXQjWmfcoUyW9lE6OjQqE6U=; h=Date:To:From:Reply-To:Subject:In-Reply-To:References:From; b=CfsYzv4brhNE8l7amVV3Ue/MMmbJE2P/Z6sAraMOYRbs3nJT2Ge3RGF1hwV3nYs4L wrV8Re9vrBz33YYUo8/JDgPQmyHYaJRKkAcr9wv6eg/n/OpPyvIV5IyzJB4EvhMYsX BHg03m6ZiHv46sSCtO1vUJq5C+k+CHYQyUzX7O4Y= Message-ID: In-Reply-To: <6wemXB-PfHUqbuVr5-XRf0-tY4cKGGtKiUqrZPrIZYXoBw17L3xRuZrGOJQfTo5PKfFNCM8KyRTllidoc7asPE2x98BTiJSPVR7OSjxCuw8=@protonmail.com> References: <6wemXB-PfHUqbuVr5-XRf0-tY4cKGGtKiUqrZPrIZYXoBw17L3xRuZrGOJQfTo5PKfFNCM8KyRTllidoc7asPE2x98BTiJSPVR7OSjxCuw8=@protonmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" Reply-to: raid5atemyhomework , raid5atemyhomework via Guix-patches From: raid5atemyhomework via Guix-patches via X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.85 Authentication-Results: aspmx1.migadu.com; dkim=fail (headers rsa verify failed) header.d=protonmail.com header.s=protonmail header.b=CfsYzv4b; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: EF0299402AF X-Spam-Score: -2.85 X-Migadu-Scanner: scn0.migadu.com X-TUID: TCemh6tFqpgo This new version is dependent on https://issues.guix.info/45723 and https:/= /issues.guix.info/45722 New features: * We can now delay pool importation after particular mapped devices are ope= ned or file systems are mounted. This is useful if you want to make a ZFS = pool out of LUKS containers, or mount an encrypted ZFS dataset with the enc= ryption key in another (hopefully also encrypted) filesystem. * We can now put other non-ZFS filesystems inside ZVOLs, block devices prov= ided by a ZFS pool, and have them mounted after ZFS has scanned pools and o= pened such ZVOLs for use by the ystem. It's now possible to have a bunch of LUKS containers in a ZPOOL that provid= es a ZVOL that contains an EXT4 filesystem (though why would you do that, j= ust use builtin ZFS encryption). This uses my [PATCH 1/4] instead of https://issues.guix.info/42193#4. Shou= ld I use the latter instead? What would get ZFS on Guix faster? I decided to remove ZFS parameter options, because I don't know whether I s= hould wait for https://issues.guix.info/42193 and how long that would take. >From d0e095136cd471f083d92416f12ca22b47301f31 Mon Sep 17 00:00:00 2001 From: raid5atemyhomework Date: Wed, 6 Jan 2021 09:24:20 +0800 Subject: [PATCH 4/4] gnu: Add service to install ZFS. * gnu/services/file-systems.scm: New file. (zfs-service-type): New variable. (): New type. (%zfs-zvol-dependency): New variable. * gnu/local.mk: Add gnu/services/file-systems.scm. * gnu/services/base.scm (dependency->shepherd-service-name): Export. * doc/guix.texi (ZFS file system): New subsection. --- doc/guix.texi | 107 +++++++++++++++++++++ gnu/local.mk | 1 + gnu/services/base.scm | 4 +- gnu/services/file-systems.scm | 174 ++++++++++++++++++++++++++++++++++ 4 files changed, 285 insertions(+), 1 deletion(-) create mode 100644 gnu/services/file-systems.scm diff --git a/doc/guix.texi b/doc/guix.texi index a31d355780..714274b8c9 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -13886,6 +13886,113 @@ a file system declaration such as: compress-force=3Dzstd,space_cache=3Dv2")) @end lisp +@node ZFS file system +@subsection ZFS file system + +The ZFS file system has a license incompatible with the Linux kernel, +and thus cannot be distributed with the kernel. However, as a user +you have the right to do whatever you want on your own hardware, +including download the ZFS source code, compile it, link it to your +own private copy of Linux, and run it. You can even use the Guix build +system to automate this. + +As a large and complex kernel module, ZFS on Linux has to be compiled +with a specific version of the kernel. Often even the latest ZFS +package available cannot be compiled with the latest Linux kernel +package provided by Guix System. Thus, installing the @code{zfs} +package is likely to fail. + +Instead, you have to use an older long-term-support Linux kernel. +Do not use @code{linux-libre-lts} as the latest long-term-support +kernel might be too new for the ZFS package; instead, explicitly +select the version number, like @code{linux-libre-5.4}. + +Then, you have to modify your system configuration file and use the +selected older kernel, and add the @code{zfs-service-type} service. + +@lisp +(use-modules (gnu)) +(use-package-modules + ; @dots{} + linux) +(use-service-modules + ; @dots{} + file-systems) + +(define my-kernel linux-libre-5.4) + +(operating-system + (kernel my-kernel) + ;; @dots{} + (services + (cons* (service zfs-service-type + (zfs-configuration + (kernel my-kernel))) + ; @dots{} + %desktop-services)) + ;; @dots{} + ) +@end lisp + +@defvr {Scheme Variable} zfs-service-type +This is the type of the service to compile and install OpenZFS to +your operating system. It loads the ZFS module at startup, imports +pools, mounts automounted ZFS datasets, installs the ZFS command +line tools, and can provide module options for the ZFS module. Its +value must be a @code{zfs-configuration} record (see below). + +Here is an example use: + +@lisp +(service zfs-service-type + (zfs-configuration + (kernel linux-libre-5.4) + (options '("zfs_arc_max=3D5000000000")))) +@end lisp +@end defvr + +@deftp {Data Type} zfs-configuration +This data type represents the configuration of the ZFS service. +The available fields are: + +@table @asis +@item @code{kernel} +The package of the Linux kernel to compile ZFS for. Required. + +@item @code{base-zfs} (default: @code{zfs}) +The ZFS package to use. It will be modified to use the indicated +kernel. + +@item @code{options} (default: @code{'()}) +A list of string options to pass as options to the ZFS module. +These will be put in a @file{/etc/modprobe.d/zfs.conf} file, +for example setting this to @code{'("zfs_admin_snapshot=3D1" +"zfs_trim_extent_bytes_min=3D0")} will create the following file: + +@example +options zfs zfs_admin_snapshot=3D1 zfs_trim_extent_bytes_min=3D0 +@end example +@end table +@end deftp + +Once your system has been configured to include the ZFS service +and you have restarted the system, you can manage ZFS pools and +datasets with @code{zpool} and @code{zfs} commands. + +ZFS datasets with an appropriate @code{mountpoint} property will +be automounted at startup after the root file system is started. +Encrypted datasets that are automounted will cause boot to pause +and prompt for the password to be provided on the console. + +It's possible to have a ZFS dataset as your @code{/home} by simply +setting the @code{mountpoint} property. However, note that ZFS will +refuse to mount over a non-empty directory, so if your root +filesystem already has a non-empty @code{/home}, you should remove +it and its contents, then restart the system. + +Having ZFS as a root filesystem or as @code{/boot} is not supported +yet. + @node Mapped Devices @section Mapped Devices diff --git a/gnu/local.mk b/gnu/local.mk index 1151d4642e..5aeb45c4c2 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -602,6 +602,7 @@ GNU_SYSTEM_MODULES =3D=09=09=09=09\ %D%/services/dict.scm=09=09=09=09\ %D%/services/dns.scm=09=09=09=09\ %D%/services/docker.scm=09=09=09\ + %D%/services/file-systems.scm=09=09=09\ %D%/services/authentication.scm=09=09\ %D%/services/games.scm=09=09=09\ %D%/services/ganeti.scm=09=09=09\ diff --git a/gnu/services/base.scm b/gnu/services/base.scm index 13cfb6a8a2..ef3680583b 100644 --- a/gnu/services/base.scm +++ b/gnu/services/base.scm @@ -188,7 +188,9 @@ references-file - %base-services)) + %base-services + + dependency->shepherd-service-name)) ;;; Commentary: ;;; diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm new file mode 100644 index 0000000000..9061ab9582 --- /dev/null +++ b/gnu/services/file-systems.scm @@ -0,0 +1,174 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2021 raid5atemyhomework +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu services file-systems) + #:use-module (gnu packages file-systems) + #:use-module (gnu services) + #:use-module (gnu services base) + #:use-module (gnu services linux) + #:use-module (gnu services shepherd) + #:use-module (gnu system mapped-devices) + #:use-module (guix gexp) + #:use-module (guix packages) + #:use-module (guix records) + #:export (zfs-service-type + + zfs-configuration + zfs-configuration? + zfs-configuration-kernel + zfs-configuration-base-zfs + zfs-configuration-dependencies + + %zfs-zvol-dependency)) + +(define-record-type* + zfs-configuration make-zfs-configuration zfs-configuration? + ; kernel you want to compile the base-zfs module for. + (kernel zfs-configuration-kernel) + ; base package that will be compiled for the kernel + (base-zfs zfs-configuration-base-zfs + (default zfs)) + ; list of | we should wait for, + ; before scanning for ZFS pools. + (dependencies zfs-configuration-dependencies + (default '()))) + +;; This is a synthetic and unusable MAPPED-DEVICE; its only use +;; is to be added as a (dependency ...) of some FILE-SYSTEM. +(define %zfs-zvol-dependency + (mapped-device + (source '()) + ;; The /* prevents naming conflict with non-ZFS device mappings, + ;; since it is not a valid name for mapped devices, and also + ;; implies "all zvols" in terms of globs. + (targets '("zvol/*")) + (type #f))) + +(define (make-zfs-package conf) + (let ((base-zfs (zfs-configuration-base-zfs conf)) + (kernel (zfs-configuration-kernel conf))) + (package + (inherit base-zfs) + (name (string-join (list (package-name base-zfs) + "for" + (package-name kernel) + (package-version kernel) + "version") + "-")) + (arguments (cons* #:linux kernel (package-arguments base-zfs)))))) + +(define (zfs-loadable-module conf) + (list (list (make-zfs-package conf) "module"))) + +(define (zfs-shepherd-services conf) + (let* ((zfs-package (make-zfs-package conf)) + (zpool (file-append zfs-package "/sbin/zpool")) + (zfs (file-append zfs-package "/sbin/zfs")) + (zvol_wait (file-append zfs-package "/bin/zvol_wait")) + (scheme-modules `((srfi srfi-1) + (srfi srfi-34) + (srfi srfi-35) + (rnrs io ports) + ,@%default-modules))) + + (define zfs-scan + (shepherd-service + (provision '(zfs-scan)) + (documentation "Scans for ZFS pools.") + (requirement `(kernel-module-loader + root-file-system + ,@(map dependency->shepherd-service-name + (zfs-configuration-dependencies conf)))) + (modules scheme-modules) + (start #~(lambda _ + (guard (c ((message-condition? c) + (format (current-error-port) + "error importing zpools: ~a~%" + (condition-message c)) + #f)) + ; TODO: optionally use a cachefile, for systems with = dozens or + ; hundreds of devices. + (invoke/quiet #$zpool "import" "-a" "-N")))) + (stop #~(const #t)))) + + (define device-mapping-zvol/* + (shepherd-service + (provision '(device-mapping-zvol/*)) + (documentation "Waits for ZFS ZVOL devices to appear.") + (requirement '(zfs-scan)) + (modules scheme-modules) + (start #~(lambda _ + (guard (c ((message-condition? c) + (format (current-error-port) + "error waiting for zvols: ~a~%" + (condition-message c)) + #f)) + (invoke/quiet #$zvol_wait)))) + (stop #~(const #t)))) + + (define zfs-automount + (shepherd-service + (provision '(zfs-automount)) + (documentation "Automounts ZFS datasets.") + (requirement '(zfs-scan)) + (modules scheme-modules) + (start #~(lambda _ + (guard (c ((message-condition? c) + (format (current-error-port) + "error automounting zfs: ~a~$") + #f)) + ; (current-output-port) is typically connected to /de= v/klog, + ; so redirect it to (current-error-port) so that user= can see + ; prompts for passphrases on console + (with-output-to-port (current-error-port) + (lambda () + (invoke #$zfs "mount" "-a" "-l")))))) + (stop #~(lambda _ + ;; make sure we don't keep any ZFS mountpoints busy. + (chdir "/") + ;; unmount everything. + (invoke/quiet #$zfs "unmount" "-a" "-f"))))) + + (list zfs-scan + device-mapping-zvol/* + zfs-automount))) + +(define zfs-service-type + (service-type (name 'zfs) + (extensions + (list + ; install the kernel module + (service-extension kernel-loadable-module-service-type + zfs-loadable-module) + ; load the kernel module + (service-extension kernel-module-loader-service-type + (const '("zfs"))) + ; scan ZFS pools, automount filesystem, wait for zvols= . + (service-extension shepherd-root-service-type + zfs-shepherd-services) + ; make sure automount occurs before file-systems targe= t is reached + (service-extension file-systems-target-service-type + (const '(zfs-automount))) + ; install ZFS management tools + (service-extension profile-service-type + (compose list make-zfs-package)) + ; install ZFS udev rules + (service-extension udev-service-type + (compose list make-zfs-package)))) + (description + "Install ZFS, an advanced filesystem and volume manager.= "))) -- 2.30.0