From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id mPjANRBpX2W02AAA9RJhRA:P1 (envelope-from ) for ; Thu, 23 Nov 2023 16:00:32 +0100 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id mPjANRBpX2W02AAA9RJhRA (envelope-from ) for ; Thu, 23 Nov 2023 16:00:32 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id AA5325DF12 for ; Thu, 23 Nov 2023 16:00:32 +0100 (CET) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=lease-up.com header.s=2017 header.b=Cp9FF8rF; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1700751632; a=rsa-sha256; cv=none; b=md9XhIjIiZYGEGAOb8zY5fvOz0UXb4sn6o16LXMcnZNav/S5JBHsabRzEuk2gKOqbql0L/ nMbmKm/1di23HqJUSaCFlEyaeuB5OhcK9fRXJJlKa5oun36u03PupC/yZMtWxY9Mb317sF QzYIJJU2CAQLmwKIEOeA3+qim9wpvHMizxuo2vhqHBmtM6A20FWvEuE/k/ImiXfiqovXTr uMr9ulxMsaHXtPjrbnFZMURnKJw44fIxfMqw+KzKb7QB+MGrZGH6I8KmkGgsOGxH6xTzA5 gqbYF0MdQdnwY7uCmecM+eiAXEg2w7Yq+2JGp1OuAs8CPPYjenChMCXcBUiDNw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=lease-up.com header.s=2017 header.b=Cp9FF8rF; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1700751632; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=Zb8AWY/rXET0K9VWEKMTGamARJmUDpVMOwi2T1Ob2+w=; b=Hnii8pU199VU6fQPfyFBE6eVHDwaaDXngnDdjWReXoagk5yo+jIlp1f74gcXIVI9aqRYRN eSC0zETpzM9zmnz/mgr/014gx8C6uz39fFCu5iTkhL1Qkyj4Bio9F2dS7pYqeMno/iqJh0 UEsrBSYYOG7MLvL1jy5mCuCqfYPyVcTiL/qXb7+i1rqOo0/f24TGT6TnNwTWwV8wR2uWEE rk/PHUmseHhrxyITDLYZuy7LbMfMJhmuwk5+H5bPTqHcYUdkg2fZ2zJe7EmSCs/kX6rG9M oDBjcJTq4jVqfXT+1nZA+eJBeOc+M2h1RNKfFiYflL8/XoY3luRpXapomlJQpw== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r6BBQ-0007yc-KL; Thu, 23 Nov 2023 10:00:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r6BBP-0007y0-4Q for guix-patches@gnu.org; Thu, 23 Nov 2023 09:59:59 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r6BBO-0003kG-Rb for guix-patches@gnu.org; Thu, 23 Nov 2023 09:59:58 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1r6BBS-0007ic-N4 for guix-patches@gnu.org; Thu, 23 Nov 2023 10:00:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#64259] [PATCH v2 2/2] Provide md-array-device-mapping to start MD arrays via UUID or name. Resent-From: Felix Lechner Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 23 Nov 2023 15:00:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 64259 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 64259@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= , Felix Lechner Received: via spool by 64259-submit@debbugs.gnu.org id=B64259.170075155429574 (code B ref 64259); Thu, 23 Nov 2023 15:00:02 +0000 Received: (at 64259) by debbugs.gnu.org; 23 Nov 2023 14:59:14 +0000 Received: from localhost ([127.0.0.1]:34586 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r6BAf-0007gv-Il for submit@debbugs.gnu.org; Thu, 23 Nov 2023 09:59:14 -0500 Received: from sail-ipv4.us-core.com ([208.82.101.137]:34494) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1r6BAd-0007gY-RE for 64259@debbugs.gnu.org; Thu, 23 Nov 2023 09:59:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; s=2017; bh=xhxXrXJXWOOqNfw dKiRHiICfjw8Abf+iKozsMFeWXCM=; h=references:in-reply-to:date:subject: cc:to:from; d=lease-up.com; b=Cp9FF8rF/BQ35H6swBrQgKQfjmTmng9rAHcGVC2Z km7Nu90MKXz/eXiXsoK6QAsUfPJ6IMuYgq5h/fdrIX/NymdXjOilCB9P7b2z49KDP6z/aX zpQXCyf2G+ZTHYZ/7ByJyQ6cn/hKvdQe1tZO4gmQ7lJ0+h+eSkdOgFWHFfUlM= Received: by sail-ipv4.us-core.com (OpenSMTPD) with ESMTPSA id e4d87c6b (TLSv1.3:TLS_CHACHA20_POLY1305_SHA256:256:NO); Thu, 23 Nov 2023 14:59:06 +0000 (UTC) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 216e76ac; Thu, 23 Nov 2023 14:59:06 +0000 (UTC) Date: Thu, 23 Nov 2023 06:57:00 -0800 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: <9b4c88707c00531fa2a43e5172d1fc0c4f4af3d9.1700751420.git.felix.lechner@lease-up.com> References: <9b4c88707c00531fa2a43e5172d1fc0c4f4af3d9.1700751420.git.felix.lechner@lease-up.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Felix Lechner X-ACL-Warn: , Felix Lechner via Guix-patches From: Felix Lechner via Guix-patches via Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Queue-Id: AA5325DF12 X-Migadu-Scanner: mx12.migadu.com X-Migadu-Spam-Score: -6.12 X-Spam-Score: -6.12 X-TUID: wSJcuc4RS8m2 This commit cures the most precipitous danger for users of MD arrays in GNU Guix, namely that their equipment may not boot after a drive failure. That behavior likely contradicts their primary expectation for having such a disk arrangments. In order to facilitate a smooth transition from raid-device-mapping to md-array-device-mapping, this commit introduces a new mapping rather than repurpose the old one. The new mapping here is also incompatible with raid-device-mapping in the sense that a plain string is now interpreted as the array name from the MD superblock. For details, please consult the mdadm manual page. Personally, the author prefers UUIDs over array names when identifying array components, but either will work. The system test uses the name. The name for the new device mapping was chosen instead of the traditional RAID to account for the fact that some modern technologies (like SSDs) and some array configurations, such as striping, are neither redundant nor inexpensive. Adjusts the documentation by erasing any mention of the obsolete raid-device-mapping. No one should use that any longer. Ideally, users would be a deprecation warning, but I was unable to adapt 'define-deprecated' to this use case. Please feel free to make further changes. This commit includes an updated system test for the root file system on an-md-array. More details for the motivation of these changes may be available here: https://lists.gnu.org/archive/html/guix-devel/2023-04/msg00010.html The author of this commit used to maintain mdadm in Debian. Please feel free to insert better changelog messages. I had some difficulty meeting the likely expectations of any reviewer. Please also feel free to make any other adjustments as needed without checking with me. Thanks! * gnu/system/mapped-devices.scm: New variable md-array-device-mapping. * doc/guix.texi: Mention md-array-device-mapping in the documentation.. * gnu/tests/install.scm: Adjust test for root-on-md-array. --- Hi Ludo' With this updated patch series, I hope to address all your questions and concerns. Thanks! Kind regards Felix doc/guix.texi | 28 ++++++------ gnu/system/mapped-devices.scm | 36 +++++++++++++++ gnu/tests/install.scm | 84 +++++++++++++++++++++++++++++++++++ 3 files changed, 135 insertions(+), 13 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 94903fb5e2..7676a58d99 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -17762,18 +17762,19 @@ the system boots up. @table @code @item source -This is either a string specifying the name of the block device to be mapped, -such as @code{"/dev/sda3"}, or a list of such strings when several devices -need to be assembled for creating a new one. In case of LVM this is a -string specifying name of the volume group to be mapped. +This is either a string specifying the name of the block device to be +mapped, such as @code{"/dev/sda3"}. For MD array devices it is either +the UUID of the array or a string that is interpreted as the array name +(see mdadm.conf(5) in the manual). In case of LVM it is a string +specifying name of the volume group to be mapped. @item target This string specifies the name of the resulting mapped device. For kernel mappers such as encrypted devices of type @code{luks-device-mapping}, specifying @code{"my-partition"} leads to the creation of the @code{"/dev/mapper/my-partition"} device. -For RAID devices of type @code{raid-device-mapping}, the full device name -such as @code{"/dev/md0"} needs to be given. +For MD array devices of type @code{md-array-device-mapping}, the full device +name such as @code{"/dev/md18"} needs to be given. LVM logical volumes of type @code{lvm-device-mapping} need to be specified as @code{"VGNAME-LVNAME"}. @@ -17793,11 +17794,12 @@ command from the package with the same name. It relies on the @code{dm-crypt} Linux kernel module. @end defvar -@defvar raid-device-mapping +@defvar md-array-device-mapping This defines a RAID device, which is assembled using the @code{mdadm} -command from the package with the same name. It requires a Linux kernel -module for the appropriate RAID level to be loaded, such as @code{raid456} -for RAID-4, RAID-5 or RAID-6, or @code{raid10} for RAID-10. +command from the package with the same name. It requires the Linux kernel +module for the appropriate RAID level to be loaded, such as @code{raid1} +for mirroring, @code{raid456} for the checksum-based RAID levels 4, 5 or 6, +or @code{raid10} for RAID-10. @end defvar @cindex LVM, logical volume manager @@ -17855,9 +17857,9 @@ may be declared as follows: @lisp (mapped-device - (source (list "/dev/sda1" "/dev/sdb1")) - (target "/dev/md0") - (type raid-device-mapping)) + (source (uuid "33cf3e31:8e33d75b:517d64b9:0a8f7623" 'mdadm)) + (target "/dev/md17") + (type md-array-device-mapping)) @end lisp The @file{/dev/md0} device can then be used as the @code{device} of a diff --git a/gnu/system/mapped-devices.scm b/gnu/system/mapped-devices.scm index e6b8970c12..e6635b531d 100644 --- a/gnu/system/mapped-devices.scm +++ b/gnu/system/mapped-devices.scm @@ -64,6 +64,7 @@ (define-module (gnu system mapped-devices) check-device-initrd-modules ;XXX: needs a better place luks-device-mapping + md-array-device-mapping raid-device-mapping lvm-device-mapping)) @@ -276,6 +277,39 @@ (define luks-device-mapping (close close-luks-device) (check check-luks-device))) +(define (open-md-array-device source targets) + "Return a gexp that assembles SOURCE to the MD device +TARGET (e.g., \"/dev/md0\"), using 'mdadm'." + (let ((array-selector + (match source + ((? uuid?) + (string-append "--uuid=" (uuid->string source))) + ((? string?) + (string-append "--name=" source)))) + (md-device + (match targets + ((target) + target)))) + ;; Use 'mdadm-static' rather than 'mdadm' to avoid pulling its whole + ;; closure (80 MiB) in the initrd when an MD device is needed for boot. + #~(zero? (system* #$(file-append mdadm-static "/sbin/mdadm") + "--assemble" #$md-device + "--run" + #$array-selector)))) + +(define (close-md-array-device source targets) + "Return a gexp that stops the MD device TARGET." + (match targets + ((target) + #~(zero? (system* #$(file-append mdadm-static "/sbin/mdadm") + "--stop" #$target))))) + +(define md-array-device-mapping + ;; The type of MD mapped device. + (mapped-device-kind + (open open-md-array-device) + (close close-md-array-device))) + (define (open-raid-device sources targets) "Return a gexp that assembles SOURCES (a list of devices) to the RAID device TARGET (e.g., \"/dev/md0\"), using 'mdadm'." @@ -317,6 +351,8 @@ (define raid-device-mapping (open open-raid-device) (close close-raid-device))) +(define-deprecated raid-device-mapping md-array-device-mapping) + (define (open-lvm-device source targets) #~(and (zero? (system* #$(file-append lvm2-static "/sbin/lvm") diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm index daa4647299..9e80b55f84 100644 --- a/gnu/tests/install.scm +++ b/gnu/tests/install.scm @@ -64,6 +64,7 @@ (define-module (gnu tests install) %test-iso-image-installer %test-separate-store-os %test-separate-home-os + %test-md-array-root-os %test-raid-root-os %test-encrypted-root-os %test-encrypted-home-os @@ -610,6 +611,89 @@ (define %test-separate-store-os (command (qemu-command* images))) (run-basic-test %separate-store-os command "separate-store-os"))))) + +;;; +;;; MD root device. +;;; + +(define-os-with-source (%md-array-root-os %md-array-root-os-source) + ;; An OS whose root partition is a MD partition. + (use-modules (gnu) (gnu tests)) + + (operating-system + (host-name "raidified") + (timezone "Europe/Paris") + (locale "en_US.utf8") + + (bootloader (bootloader-configuration + (bootloader grub-bootloader) + (targets (list "/dev/vdb")))) + (kernel-arguments '("console=ttyS0")) + + ;; Add a kernel module for RAID-1 (aka. "mirror"). + (initrd-modules (cons "raid1" %base-initrd-modules)) + + (mapped-devices (list (mapped-device + (source "marionette:mirror0") + (target "/dev/md0") + (type md-array-device-mapping)))) + (file-systems (cons (file-system + (device (file-system-label "root-fs")) + (mount-point "/") + (type "ext4") + (dependencies mapped-devices)) + %base-file-systems)) + (users %base-user-accounts) + (services (cons (service marionette-service-type + (marionette-configuration + (imported-modules '((gnu services herd) + (guix combinators))))) + %base-services)))) + +(define %md-array-root-installation-script + ;; Installation with a separate /gnu partition. See + ;; for more on RAID and + ;; mdadm. + "\ +. /etc/profile +set -e -x +guix --version + +export GUIX_BUILD_OPTIONS=--no-grafts +parted --script /dev/vdb mklabel gpt \\ + mkpart primary ext2 1M 3M \\ + mkpart primary ext2 3M 1.6G \\ + mkpart primary ext2 1.6G 3.2G \\ + set 1 boot on \\ + set 1 bios_grub on +yes | mdadm --create /dev/md0 --verbose --homehost=marionette --name=mirror0 \\ + --level=mirror --raid-devices=2 /dev/vdb2 /dev/vdb3 +mkfs.ext4 -L root-fs /dev/md0 +mount /dev/md0 /mnt +df -h /mnt +herd start cow-store /mnt +mkdir /mnt/etc +cp /etc/target-config.scm /mnt/etc/config.scm +guix system init /mnt/etc/config.scm /mnt --no-substitutes +sync +reboot\n") + +(define %test-md-array-root-os + (system-test + (name "md-array-root-os") + (description + "Test functionality of an OS installed with a RAID root partition managed +by 'mdadm'.") + (value + (mlet* %store-monad ((images (run-install %md-array-root-os + %md-array-root-os-source + #:script + %md-array-root-installation-script + #:target-size (* 3200 MiB))) + (command (qemu-command* images))) + (run-basic-test %md-array-root-os + `(,@command) "md-array-root-os"))))) + ;;; ;;; RAID root device. -- 2.41.0