From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:44860) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i5fWO-0006Wn-Vd for guix-patches@gnu.org; Wed, 04 Sep 2019 20:21:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i5fWK-0000nN-El for guix-patches@gnu.org; Wed, 04 Sep 2019 20:21:08 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:53930) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1i5fWI-0000m0-0q for guix-patches@gnu.org; Wed, 04 Sep 2019 20:21:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1i5fWH-00052F-Ry for guix-patches@gnu.org; Wed, 04 Sep 2019 20:21:01 -0400 Subject: [bug#37305] [PATCH] Allow booting from a Btrfs subvolume. Resent-Message-ID: Received: from eggs.gnu.org ([2001:470:142:3::10]:43319) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i5fVi-0006I9-Nr for guix-patches@gnu.org; Wed, 04 Sep 2019 20:20:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i5fVf-0008F7-8G for guix-patches@gnu.org; Wed, 04 Sep 2019 20:20:26 -0400 Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]:38839) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1i5fVZ-00086L-Dv for guix-patches@gnu.org; Wed, 04 Sep 2019 20:20:19 -0400 Received: by mail-pg1-x542.google.com with SMTP id d10so348754pgo.5 for ; Wed, 04 Sep 2019 17:20:10 -0700 (PDT) Received: from x200 ([240f:c7:38e9:1:314b:485c:9ba4:72c6]) by smtp.gmail.com with ESMTPSA id d15sm231367pfo.118.2019.09.04.17.20.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2019 17:20:07 -0700 (PDT) From: Maxim Cournoyer Date: Thu, 05 Sep 2019 09:20:02 +0900 Message-ID: <87sgpby4p9.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: 37305@debbugs.gnu.org --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain Hello! I'm sending this patch series to add support for booting off Btrfs subvolumes. There was some interested shown on #guix, so hopefully someone can test it on their system :-) Before this change, it wasn't possible to pass the required options to the Linux kernel as our init script would ignore them. I'm not including system tests yet, as this will take a bit more time and is starting to be a big change in itself. --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0001-bootloader-grub-Allow-booting-from-a-Btrfs-subvolume.patch Content-Transfer-Encoding: quoted-printable From=206858efa540d89c54ce377bfa6a6882e551cd2e56 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Sun, 14 Jul 2019 20:50:23 +0900 Subject: [PATCH 1/4] bootloader: grub: Allow booting from a Btrfs subvolume. * gnu/bootloader/grub.scm (prepend-subvol, arguments->subvol): New procedur= es. (grub-configuration-file): Use ARGUMENTS->SUBVOL to extract the subvolume n= ame from the kernel arguments, and prepend it to the kernel and initrd paths us= ing the PREPEND-SUBVOL procedure. * tests/grub.scm: Add tests for the "arguments->subvol" procedure. * doc/guix.texi (File Systems, (Bootloader Configuration): Document the use= of Btrfs subvolumes. =2D-- doc/guix.texi | 29 ++++++++++++++++++++- gnu/bootloader/grub.scm | 43 +++++++++++++++++++++++++------ gnu/build/linux-boot.scm | 7 +++-- gnu/build/linux-modules.scm | 3 ++- tests/grub.scm | 51 +++++++++++++++++++++++++++++++++++++ 5 files changed, 121 insertions(+), 12 deletions(-) create mode 100644 tests/grub.scm diff --git a/doc/guix.texi b/doc/guix.texi index 707c2ba700..cc7c91ac92 100644 =2D-- a/doc/guix.texi +++ b/doc/guix.texi @@ -48,7 +48,7 @@ Copyright @copyright{} 2017 humanitiesNerd@* Copyright @copyright{} 2017 Christopher Allan Webber@* Copyright @copyright{} 2017, 2018 Marius Bakke@* Copyright @copyright{} 2017 Hartmut Goebel@* =2DCopyright @copyright{} 2017 Maxim Cournoyer@* +Copyright @copyright{} 2017, 2019 Maxim Cournoyer@* Copyright @copyright{} 2017, 2018, 2019 Tobias Geerinckx-Rice@* Copyright @copyright{} 2017 George Clemmer@* Copyright @copyright{} 2017 Andy Wingo@* @@ -10802,6 +10802,20 @@ using the @code{file-system} form, like this: (type "ext4")) @end example =20 +@cindex Btrfs subvolume, file system +Below is a more complex example, making use of a Btrfs subvolume, named +@code{rootfs}, which parent Btrfs file system is labeled @code{my-btrfs-po= ol}, +on an encrypted device (hence the dependency on @code{mapped-devices}): + +@example +(file-system + (device (file-system-label "my-btrfs-pool")) + (mount-point "/") + (type "btrfs") + (options "defaults,subvol=3Drootfs") + (dependencies mapped-devices)) +@end example + As usual, some of the fields are mandatory---those shown in the example above---while others can be omitted. These are described below. =20 @@ -24868,6 +24882,19 @@ when you boot it on your system. @code{grub-bootloader} allows you to boot in particular Intel-based machin= es in ``legacy'' BIOS mode. =20 +@cindex rootflags, Grub +@cindex Btrfs root subvolume, Grub +To allow using a Btrfs @emph{subvolume} for the root partition, the Grub-b= ased +bootloaders can use a subvolume @emph{name} passed using the @code{rootfla= gs} +kernel argument, e.g.: + +@example +(kernel-arguments '("rootflags=3Dsubvol=3D@var{root-subvolume-name}")) +@end example + +to correctly populate the @code{kernel} and @code{initrd} fields of the Gr= ub +configuration file. + @cindex ARM, bootloaders @cindex AArch64, bootloaders Available bootloaders are described in @code{(gnu bootloader @dots{})} diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm index d984d5f5e3..dee6028dd0 100644 =2D-- a/gnu/bootloader/grub.scm +++ b/gnu/bootloader/grub.scm @@ -3,6 +3,7 @@ ;;; Copyright =C2=A9 2016 Chris Marusich ;;; Copyright =C2=A9 2017 Leo Famulari ;;; Copyright =C2=A9 2017 Mathieu Othacehe +;;; Copyright =C2=A9 2019 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -25,6 +26,8 @@ #:use-module (guix gexp) #:use-module (gnu artwork) #:use-module (gnu bootloader) + #:use-module ((gnu build linux-modules) #:select (%not-comma)) + #:use-module ((gnu build linux-boot) #:select (find-long-option)) #:use-module (gnu system uuid) #:use-module (gnu system file-systems) #:use-module (gnu system keyboard) @@ -34,6 +37,7 @@ #:use-module (ice-9 match) #:use-module (ice-9 regex) #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) #:export (grub-image grub-image? grub-image-aspect-ratio @@ -73,6 +77,14 @@ denoting a file name." file)))) (#f file))) =20 +(define (prepend-subvol subvol file) + "Prepend SUBVOL from FILE, which is a gexp or other lowerable object +denoting a file name." + (match subvol + ((? string? subvol) + #~(string-append "/" #$subvol #$file)) + (#f file))) + (define-record-type* grub-image make-grub-image grub-image? @@ -313,6 +325,13 @@ code." ((or #f (? string?)) #~(format #f "search --file --set ~a" #$file))))) =20 +(define (arguments->subvol arguments) + "Return any \"subvol\" value given as an option to the \"rootflags\" +argument, or #f on failure." + (let* ((rootflags (find-long-option "rootflags" arguments)) + (rootflags-options (and=3D> rootflags (cut string-tokenize <> %no= t-comma)))) + (and=3D> rootflags-options (cut find-long-option "subvol" <>)))) + (define* (grub-configuration-file config entries #:key (system (%current-system)) @@ -324,18 +343,26 @@ entries corresponding to old generations of the syste= m." (define all-entries (append entries (bootloader-configuration-menu-entries config))) (define (menu-entry->gexp entry) =2D (let ((device (menu-entry-device entry)) =2D (device-mount-point (menu-entry-device-mount-point entry)) =2D (label (menu-entry-label entry)) =2D (kernel (menu-entry-linux entry)) =2D (arguments (menu-entry-linux-arguments entry)) =2D (initrd (menu-entry-initrd entry))) + (let* ((device (menu-entry-device entry)) + (device-mount-point (menu-entry-device-mount-point entry)) + (label (menu-entry-label entry)) + (kernel (menu-entry-linux entry)) + (arguments (menu-entry-linux-arguments entry)) + (subvol (arguments->subvol arguments)) + (initrd (menu-entry-initrd entry))) ;; Here DEVICE is the store and DEVICE-MOUNT-POINT is its mount poin= t. ;; Use the right file names for KERNEL and INITRD in case ;; DEVICE-MOUNT-POINT is not "/", meaning that the store is on a ;; separate partition. =2D (let ((kernel (strip-mount-point device-mount-point kernel)) =2D (initrd (strip-mount-point device-mount-point initrd))) + + ;; Also, in case a subvolume name could be extracted from the "subvo= l" + ;; option given to the "rootflags" argument of the kernel, it is + ;; prepended to the kernel and initrd paths, to allow booting from + ;; a Btrfs subvolume. + (let ((kernel (prepend-subvol subvol (strip-mount-point + device-mount-point kernel))) + (initrd (prepend-subvol subvol (strip-mount-point + device-mount-point initrd)))) #~(format port "menuentry ~s { ~a linux ~a ~a diff --git a/gnu/build/linux-boot.scm b/gnu/build/linux-boot.scm index f273957d78..7a30ebcef1 100644 =2D-- a/gnu/build/linux-boot.scm +++ b/gnu/build/linux-boot.scm @@ -92,9 +92,12 @@ =20 (define (find-long-option option arguments) "Find OPTION among ARGUMENTS, where OPTION is something like \"--load\". =2DReturn the value associated with OPTION, or #f on failure." +Return the value associated with OPTION, or #f on failure. Any non-string +arguments are ignored." (let ((opt (string-append option "=3D"))) =2D (and=3D> (find (cut string-prefix? opt <>) + (and=3D> (find (lambda (arg) + (and (string? arg) + (string-prefix? opt arg))) arguments) (lambda (arg) (substring arg (+ 1 (string-index arg #\=3D))))))) diff --git a/gnu/build/linux-modules.scm b/gnu/build/linux-modules.scm index a149eff329..5f2efe7484 100644 =2D-- a/gnu/build/linux-modules.scm +++ b/gnu/build/linux-modules.scm @@ -32,7 +32,8 @@ #:use-module (ice-9 match) #:use-module (ice-9 rdelim) #:autoload (ice-9 pretty-print) (pretty-print) =2D #:export (dot-ko + #:export (%not-comma + dot-ko ensure-dot-ko module-formal-name module-aliases diff --git a/tests/grub.scm b/tests/grub.scm new file mode 100644 index 0000000000..81453b00ab =2D-- /dev/null +++ b/tests/grub.scm @@ -0,0 +1,51 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2019 Maxim Cournoyer +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (test-grub) + #:use-module (gnu bootloader grub) + #:use-module (srfi srfi-64)) + +;;; Local bindings to private procedures that are to be tested. +(define arguments->subvol (@@ (gnu bootloader grub) arguments->subvol)) + + +(test-begin "grub") + +(test-equal "Get subvolume name from arguments, multiple options" + "@" + (arguments->subvol '("ro" "debug" + "rootflags=3Dfoo=3Dbar,subvol=3D@,bar=3Dbaz"))) + +(test-equal "No subvolume from arguments" + #f + (arguments->subvol '("rootflags=3Dfoo=3Dbar"))) + +(test-equal "No rootflags argument" + #f + (arguments->subvol '("ro" "debug"))) + +(test-equal "Empty arguments list" + #f + (arguments->subvol '())) + +;;; The other types would typically be gexp objects. +(test-equal "Argument list may contain other types than string" + "rootfs" + (arguments->subvol '(#f "rootflags=3Dsubvol=3Drootfs"))) + +(test-end "grub") =2D-=20 2.23.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0002-build-initrd-Fix-write-cpio-archive-return-value.patch Content-Transfer-Encoding: quoted-printable From=203a628d1e731b2857a4c964484213cce980cb596f Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Tue, 16 Jul 2019 18:09:38 +0900 Subject: [PATCH 2/4] build: initrd: Fix "write-cpio-archive" return value. * gnu/build/linux-initrd.scm (write-cpio-archive): Really return OUTPUT on success, even when compression is disabled. =2D-- gnu/build/linux-initrd.scm | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/gnu/build/linux-initrd.scm b/gnu/build/linux-initrd.scm index 3aaa06d3a0..ea7de58553 100644 =2D-- a/gnu/build/linux-initrd.scm +++ b/gnu/build/linux-initrd.scm @@ -71,8 +71,7 @@ COMPRESS? is true, compress it using GZIP. On success, r= eturn OUTPUT." (cpio:write-cpio-archive files port #:file->header cpio:file->cpio-header*))) =20 =2D (or (not compress?) =2D + (if compress? ;; Gzip insists on adding a '.gz' suffix and does nothing if the inp= ut ;; file already has that suffix. Shuffle files around to placate it. (let* ((gz-suffix? (string-suffix? ".gz" output)) @@ -88,7 +87,6 @@ COMPRESS? is true, compress it using GZIP. On success, r= eturn OUTPUT." (unless gz-suffix? (rename-file (string-append output ".gz") output)) output))) =2D output)) =20 (define (cache-compiled-file-name file) =2D-=20 2.23.0 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0003-linux-boot-Fix-typo.patch Content-Transfer-Encoding: quoted-printable From=2049ffe9a2645252bb708995169a9f1749f3982385 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Thu, 18 Jul 2019 07:23:48 +0900 Subject: [PATCH 3/4] linux-boot: Fix typo. * gnu/build/linux-boot.scm (mount-root-file-system): Fix typo. =2D-- gnu/build/linux-boot.scm | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/gnu/build/linux-boot.scm b/gnu/build/linux-boot.scm index 7a30ebcef1..b4e6421b27 100644 =2D-- a/gnu/build/linux-boot.scm +++ b/gnu/build/linux-boot.scm @@ -362,8 +362,9 @@ the last argument of `mknod'." (define* (mount-root-file-system root type #:key volatile-root?) "Mount the root file system of type TYPE at device ROOT. If VOLATILE-RO= OT? =2Dis true, mount ROOT read-only and make it a overlay with a writable tmpfs =2Dusing the kernel build-in overlayfs." +is true, mount ROOT read-only and make it an overlay with a writable tmpfs +using the kernel built-in overlayfs." + (if volatile-root? (begin (mkdir-p "/real-root") =2D-=20 2.23.0 --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0004-linux-boot-Honor-rootflags-kernel-argument.patch Content-Transfer-Encoding: quoted-printable From=20b56aea9c62b015c8a8b48827f9587b1578c83af3 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Thu, 18 Jul 2019 04:59:25 +0900 Subject: [PATCH 4/4] linux-boot: Honor "rootflags" kernel argument. * gnu/build/linux-boot.scm (mount-root-file-system): Add the optional FLAGS and OPTIONS arguments; and document them. Pass those to the `mount' calls. (boot-system): Parse the "rootflags" kernel argument, and use it when calli= ng `mount-root-file-system'. * doc/guix.texi (Initial RAM Disk): Document the use of the "rootflags" argument. =2D-- doc/guix.texi | 19 +++++++++++++++++++ gnu/build/linux-boot.scm | 22 +++++++++++++--------- 2 files changed, 32 insertions(+), 9 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index cc7c91ac92..1e093b38a0 100644 =2D-- a/doc/guix.texi +++ b/doc/guix.texi @@ -24761,6 +24761,25 @@ Instruct the initial RAM disk as well as the @comm= and{modprobe} command must be a comma-separated list of module names---e.g., @code{usbkbd,9pnet}. =20 +@item rootflags=3D@var{options}@dots{} +@cindex mount options, passed to initrd +@cindex rootflags, initrd +This argument allows passing one or multiple file system specific mount +options to the @code{mount} procedure used by the init script. @var{optio= ns} +must be a comma-separated list of option names or option-value pairs. The +following example instructs the initial RAM disk to mount the Btrfs subvol= ume +named ``rootfs'' as the root file system, and to enable automatic file +defragmentation: + +@example +rootflags=3Dsubvol=3Drootfs,autodefrag +@end example + +Specifying the subvolume to mount by its name, as shown above, is also use= d in +Guix to produce a working Grub configuration for the Grub-based bootloaders +when using a Btrfs subvolume for the root file system (@xref{Bootloader +Configuration}). + @item --repl Start a read-eval-print loop (REPL) from the initial RAM disk before it tries to load kernel modules and to mount the root file system. Our diff --git a/gnu/build/linux-boot.scm b/gnu/build/linux-boot.scm index b4e6421b27..b2d8f74a71 100644 =2D-- a/gnu/build/linux-boot.scm +++ b/gnu/build/linux-boot.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2013, 2014, 2015, 2016, 2017, 2018, 2019 Ludovic Cour= t=C3=A8s ;;; Copyright =C2=A9 2017 Mathieu Othacehe +;;; Copyright =C2=A9 2019 Maxim Cournoyer ;;; ;;; This file is part of GNU Guix. ;;; @@ -360,15 +361,16 @@ the last argument of `mknod'." (filter-map string->number (scandir "/proc"))))) =20 (define* (mount-root-file-system root type + #:optional (flags 0) options #:key volatile-root?) =2D "Mount the root file system of type TYPE at device ROOT. If VOLATILE-= ROOT? =2Dis true, mount ROOT read-only and make it an overlay with a writable tmp= fs =2Dusing the kernel built-in overlayfs." =2D + "Mount the root file system of type TYPE at device ROOT. The optional F= LAGS +and OPTIONS arguments behave the same as for the `mount' procedure. If +VOLATILE-ROOT? is true, mount ROOT read-only and make it an overlay with a +writable tmpfs using the kernel built-in overlayfs." (if volatile-root? (begin (mkdir-p "/real-root") =2D (mount root "/real-root" type MS_RDONLY) + (mount root "/real-root" type (logior MS_RDONLY flags) options) (mkdir-p "/rw-root") (mount "none" "/rw-root" "tmpfs") =20 @@ -385,11 +387,11 @@ using the kernel built-in overlayfs." "lowerdir=3D/real-root,upperdir=3D/rw-root/upper,workdir=3D= /rw-root/work")) (begin (check-file-system root type) =2D (mount root "/root" type))) + (mount root "/root" type flags options))) =20 ;; Make sure /root/etc/mtab is a symlink to /proc/self/mounts. (false-if-exception =2D (delete-file "/root/etc/mtab")) + (delete-file "/root/etc/mtab")) (mkdir-p "/root/etc") (symlink "/proc/self/mounts" "/root/etc/mtab")) =20 @@ -483,7 +485,8 @@ upon error." (mount-essential-file-systems) (let* ((args (linux-command-line)) (to-load (find-long-option "--load" args)) =2D (root (find-long-option "--root" args))) + (root (find-long-option "--root" args)) + (rootflags (find-long-option "rootflags" args))) =20 (when (member "--repl" args) (start-repl)) @@ -526,7 +529,8 @@ upon error." ((uuid root) =3D> identity) (else (file-system-label root))))) (mount-root-file-system (canonicalize-device-spec root) =2D root-fs-type + root-fs-type 0 + rootflags #:volatile-root? volatile-root?)) (mount "none" "/root" "tmpfs")) =20 =2D-=20 2.23.0 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEJ9WGpPiQCFQyn/CfEmDkZILmNWIFAl1wVLIACgkQEmDkZILm NWLKPhAAixwXUlhjtluKkuXgMqow/tjWQf2O0fpWJB+UrmZjJvVDWkpIsR6fOjpn +/RdR7cDp4SqlhsNr15CwWIt0NYbNcSZGwgcVD/59Hj5YER2Nl/gBdgw8wV1jBbQ nkW45+PhCko5WBahxOB4F53VWdPJ5YzO5mLRNSbAvH7+mRqdyctLZmqkSRfGmbb3 4/ndo3399GMzhgS1h4hc+t+jsBwFfj9SmaVNZbAyXB/5gmyCSvmcwYFMrfYaCHGT 7ZOSw/khhCKzGuzXyE8fsYsybOHuWkD2fDrOHLW9lv4F9+VyIoZgY1gAdoddAmPy At0kxr+Lpeqf9ujlJu494A8hG+DpmgeMP2zp4pCVPcTN5S/LNqvBhzlifyf7bynj cI5Lv15C6YNibT8Su/v27XSo4heZm8GsCtpVLkjzVJAFeQnoedFelIgot/nLAuHU 2CSEe0ZUC0IC5ltmB16T/5POVBF0dqWBgKa0xXRVyY202f6AyhY3OE3z929kgiD4 w8oBHNed2Oxq7ZTc2WdTsgJ5C1IJod+sUr6R6BtdVTZKXgH6ijA7hf35rBUSKSh5 6YgzG1J0/KoaWSICfGLUgJ8DRX7s+vDFjBCxem68ZLhwvp0vvQAyhDSEGxdeOFeB nXbrtg0YbM4R2VJr8Ic5bcKovzlhtuwINu2Ku+Weu/6BSegyVes= =rD4K -----END PGP SIGNATURE----- --==-=-=--