From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id kK06Ho9J8F/UeQAA0tVLHw (envelope-from ) for ; Sat, 02 Jan 2021 10:23:11 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id GKEEGo9J8F8VOgAAB5/wlQ (envelope-from ) for ; Sat, 02 Jan 2021 10:23:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 017FF9404C4 for ; Sat, 2 Jan 2021 10:23:09 +0000 (UTC) Received: from localhost ([::1]:48756 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kve3w-00044l-Ox for larch@yhetil.org; Sat, 02 Jan 2021 05:23:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:51304) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kve3q-00044G-0S for guix-patches@gnu.org; Sat, 02 Jan 2021 05:23:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:40585) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kve3p-0007WW-Pz for guix-patches@gnu.org; Sat, 02 Jan 2021 05:23:01 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kve3p-0005B8-M5 for guix-patches@gnu.org; Sat, 02 Jan 2021 05:23:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45592] [PATCH] gnu, doc: Create and document procedure to compile ZFS for specific kernel. Resent-From: raid5atemyhomework Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Jan 2021 10:23:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 45592 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: "45592@debbugs.gnu.org" <45592@debbugs.gnu.org> Received: via spool by 45592-submit@debbugs.gnu.org id=B45592.160958298119899 (code B ref 45592); Sat, 02 Jan 2021 10:23:01 +0000 Received: (at 45592) by debbugs.gnu.org; 2 Jan 2021 10:23:01 +0000 Received: from localhost ([127.0.0.1]:52127 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kve3o-0005Ap-Fs for submit@debbugs.gnu.org; Sat, 02 Jan 2021 05:23:01 -0500 Received: from mail4.protonmail.ch ([185.70.40.27]:63010) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kve3S-00059S-Ey for 45592@debbugs.gnu.org; Sat, 02 Jan 2021 05:22:39 -0500 Date: Sat, 02 Jan 2021 10:22:25 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail; t=1609582951; bh=nLXc5ft7aek0uzvX7Je+d6BWdlFNyjgXZtuzrpgAuhs=; h=Date:To:From:Reply-To:Subject:In-Reply-To:References:From; b=S8iGHa0QQzXjnTeD0RCZxGV4hNpDzvf6xi1KikyOVnQB/IrTYBPgy3YAgUPSPKreT fGyFqeGvSGlJZmdspbjyZlIEP4kh2MVYwq1REpB2Zjcv0IiCS8YxV6EZcQ0JpZAis4 frfgIBUvprjMIA6/1z4rL+vFH6jaVOAZvcEFpK3E= Message-ID: In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" Reply-to: raid5atemyhomework , raid5atemyhomework via Guix-patches From: raid5atemyhomework via Guix-patches via X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.83 Authentication-Results: aspmx1.migadu.com; dkim=fail (headers rsa verify failed) header.d=protonmail.com header.s=protonmail header.b=S8iGHa0Q; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 017FF9404C4 X-Spam-Score: -2.83 X-Migadu-Scanner: scn1.migadu.com X-TUID: +HXNyDwzfnCE Did some more testing and finally got ZFS working inside a VM! Turns out the `zfs` package definition was buggy --- it was using `util-lin= ux:lib`, but trying to execute `mount` and `umount` from it. So I modified= the package definition as well to include `util-linux:lib` as `"util-linux= -lib"` and gave it `util-linux` directly as well. I also created a new service to load ZFS kernel module *and* import all ZFS= pools at boot and mount them, which is the behavior ZFS has on other distr= os and platforms, so we should emulate it as well. ZFS as root is not supported yet. I presume it requires `initrd` hacking, p= lus it also has to be supported somehow in the installer or something as we= ll. Here's the patch! ---- >From d02b9a69ad8c261e7f7c7bf145117327d1217d1c Mon Sep 17 00:00:00 2001 From: raid5atemyhomework Date: Fri, 1 Jan 2021 20:26:42 +0800 Subject: [PATCH] gnu, doc: Create and document procedure to compile ZFS for specific kernel. --- doc/guix.texi | 98 +++++++++++++++++++++++++++++++++++ gnu/packages/file-systems.scm | 25 ++++++++- gnu/services/file-systems.scm | 65 +++++++++++++++++++++++ 3 files changed, 186 insertions(+), 2 deletions(-) create mode 100644 gnu/services/file-systems.scm diff --git a/doc/guix.texi b/doc/guix.texi index 1081ed26a3..620dc48ff5 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -13885,6 +13885,104 @@ a file system declaration such as: compress-force=3Dzstd,space_cache=3Dv2")) @end lisp +@node ZFS file system +@subsection ZFS file system + +The ZFS on Linux file system cannot legally be downloaded as part of the +Linux kernel, but you as a user can do anything on hardware you own, +including download ZFS as source code, compile ZFS as a kernel module, +and link it into Linux. + +As a large and complex kernel module, ZFS has to be compiled for a +specific version of Linux. Often the latest ZFS package available in Guix +cannot be compiled with the latest Linux kernel available in Guix, so +installing the @code{zfs} package in your system configuration file will +fail. + +Instead, you have to check the +@url{https://github.com/openzfs/zfs/releases,OpenZFS release notes} for +the specific version of ZFS that Guix has packaged to determine what +Linux kernels you can use, then check the @code{linux-libre} packages +that Guix has packaged, and select one you can use on your system. + +Then, you have to modify your system configuration file, and create a +ZFS package that compiles using the specific Linux version you chose. +Below is a sketch of how you would modify your @code{operating-system} +declaration in order to set up ZFS: + +@lisp +(use-modules (gnu)) +;; @dots{} +(use-service-modules + ; @dots{} + linux + file-systems) +(use-package-modules + ; @dots{} + linux + file-systems) + +;; @dots{} + +;;; (1) Select a specific kernel. +(define system-kernel linux-libre-5.4) +;;; (2) Define a ZFS package for your kernel. +(define system-zfs (make-zfs-package system-kernel)) + +;; @dots{} + +(operating-system + ;;; (3) Specify your selected kernel. + (kernel system-kernel) + ;;; (4) Add the "module" output of the system ZFS package to + ;;; the kernel-loadable modules. + (kernel-loadable-modules (list (list system-zfs "module"))) + + ;; @dots{} + + (packages + ;;; (5) Add the system ZFS package to global packages so that + ;;; "zfs", "zpool" etc. commands are available. + (cons* system-zfs + ; @dots{} + %base-packages)) + + ;; @dots{} + + (services + ;;; (6) Add a service that loads ZFS and all ZFS pools at boot. + (cons* (service zfs-loader-service-type + system-zfs) + ; @dots{} + %base-services)) + ;; @dots{} + ) +@end lisp + +@deffn (Scheme Procedure) make-zfs-package @var{kernel} +This procedure creates a package which, when included as a package +in your system, can be loaded as a kernel module for the specified +@var{kernel}, a Linux kernel package. +It is intended to create a system-specific ZFS kernel module for +the Linux kernel you will use in your system. +@end deffn + +@deffn (Scheme Variable) zfs-loader-service-type +This is the service that loads the ZFS kernel module and imports all +ZFS pools. The value.associated with @code{zfs-loader-servie-type} +services must be the ZFS package you defined for your system, and +whose @code{"module"} output you have included as a loadable kernel +module. +@end deffn + +When you have modified your @code{operating-system} to include ZFS +as in the above and reconfigured and rebooted, you will now be able +to create, mount, and manage ZFS pools. At each boot, ZFS will +automatically open any ZFS pools it detects in your system, and will +mount ZFS pools that have a non-@code{legacy} mountpoint. + +ZFS as root filesystem is not supported yet. + @node Mapped Devices @section Mapped Devices diff --git a/gnu/packages/file-systems.scm b/gnu/packages/file-systems.scm index 895ad069c5..e8f49a59fa 100644 --- a/gnu/packages/file-systems.scm +++ b/gnu/packages/file-systems.scm @@ -944,16 +944,37 @@ APFS.") ("openssl" ,openssl) ("python" ,python) ("python-cffi" ,python-cffi) - ("util-linux" ,util-linux "lib") + ("util-linux" ,util-linux) + ("util-linux-lib" ,util-linux "lib") ("zlib" ,zlib))) (home-page "https://zfsonlinux.org/") (synopsis "Native ZFS on Linux") (description "ZFS on Linux is an advanced file system and volume manager which was originally developed for Solaris and is now maintained by the OpenZFS -community.") +community. + +DO NOT INSTALL THIS PACKAGE. Instead, refer to the 'ZFS file system' secti= on +of the Guix info manual for how to install ZFS.") (license license:cddl1.0))) +(define-public (make-zfs-package kernel) + (package + (inherit zfs) + (name (string-append "zfs-for-" + (package-name kernel) + "-" + (package-version kernel) + "-version")) + (arguments + (cons* #:linux kernel (package-arguments zfs))) + (description + "ZFS on Linux is an advanced file system and volume manager which was +originally developed for Solaris and is now maintained by the OpenZFS +community. + +This package has been compiled for a specific Linux kernel."))) + (define-public mergerfs (package (name "mergerfs") diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm new file mode 100644 index 0000000000..97132cc53d --- /dev/null +++ b/gnu/services/file-systems.scm @@ -0,0 +1,65 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2021 raid5atemyhomework +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu services file-systems) + #:use-module (guix gexp) + #:use-module (gnu services) + #:use-module (gnu services shepherd)) + +;;; ZFS +(define (zfs-loader-shepherd-service system-zfs) + (let* ((zpool (file-append system-zfs "/sbin/zpool"))) + (list + (shepherd-service + (documentation "Load ZFS kernel module and import ZFS pools.") + (provision '(zfs)) + (requirement '(file-systems)) + (one-shot? #t) + (modules `((srfi srfi-1) + (srfi srfi-34) + (srfi srfi-35) + (rnrs io ports) + ,@%default-modules)) + (start #~(lambda _ + (if (not (file-exists? "/proc/sys/kernel/modprobe")) + (begin (format (current-error-port) "error: ~a~%" + "Kernel is missing loadable module su= pport.") + #f) + (and + (let ((modprobe (call-with-input-file + "/proc/sys/kernel/modprobe" get-= line))) + (guard (c ((message-condition? c) + (format (current-error-port) + "error loading 'zfs' kernel m= odule: ~a~%" + (condition-message c)) + #f)) + (invoke/quiet modprobe "--" "zfs"))) + (guard (c ((message-condition? c) + (format (current-error-port) + "error importing zpools: ~a~%" + (condition-message c)) + #f)) + (invoke/quiet #$zpool "import" "-a")))))))))) + +(define-public zfs-loader-service-type + (service-type + (name 'zfs-loader) + (description "Load ZFS kernel module and import ZFS pools.") + (extensions + (list (service-extension shepherd-root-service-type zfs-loader-shepher= d-service))))) + -- 2.29.2