From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andreas Enge Subject: [PATCH} Add RAID devices. Date: Sat, 23 Jul 2016 16:07:01 +0200 Message-ID: <20160723140701.GB6873@solar> References: <20160714131306.GA11566@solar> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="VS++wcV0S1rZb1Fb" Content-Transfer-Encoding: 8bit Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:40914) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bQxaE-0006Vf-C6 for guix-devel@gnu.org; Sat, 23 Jul 2016 10:07:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bQxa8-0004Ne-8v for guix-devel@gnu.org; Sat, 23 Jul 2016 10:07:13 -0400 Received: from mailrelay1.public.one.com ([91.198.169.124]:22948) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bQxa7-0004NY-Ny for guix-devel@gnu.org; Sat, 23 Jul 2016 10:07:08 -0400 Content-Disposition: inline In-Reply-To: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: myglc2 Cc: guix-devel@gnu.org --VS++wcV0S1rZb1Fb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello, On Fri, Jul 15, 2016 at 10:25:47AM -0400, myglc2 wrote: > How do you plan to address mdadm and grub config? > Will you be booting from RAID? the attached patch adds RAID support to our mapped device mechanism. To give more explanation: Like for LUKS devices or file systems, the RAID itself needs to be created during installation, after booting from the USB key: mdadm --create /dev/md0 --level=raid10 --layout=f2 \ --raid-devices=2 /dev/sda2 /dev/sdb2 (wait 12 hours or so for 4 TB disks) mkfs.ext4 /dev/md0 Then one adds a mapped-device of type raid-device-mapping, adds the raid10 kernel module to the initrd and can boot from the RAID device. The mapped-device mechanism calls "mdadm --assemble" and "mdadm --stop". I might write a more detailed blog post about this; there is a little subtlety with the non-automatic determination of dependencies between devices, so one needs to make sure that the partitions to be assembled are present before the mdadm command is executed. Looking forward to comments on the patch, Andreas --VS++wcV0S1rZb1Fb Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: attachment; filename="0001-system-Add-mapped-devices-for-RAID.patch" Content-Transfer-Encoding: 8bit >From fc2d8dc30c04677ebf553b02227dc10b0be49665 Mon Sep 17 00:00:00 2001 From: Andreas Enge Date: Thu, 14 Jul 2016 15:51:59 +0200 Subject: [PATCH] system: Add mapped devices for RAID. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * gnu/system/mapped-devices.scm (raid-device-mapping, open-raid-device, close-raid-device): New variables. * doc/guix.texi (Mapped Devices): Add documentation for RAID devices, reorganize documentation for LUKS devices. Co-authored-by: Ludovic Courtès --- doc/guix.texi | 113 +++++++++++++++++++++++++++--------------- gnu/system/mapped-devices.scm | 29 ++++++++++- 2 files changed, 102 insertions(+), 40 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 393efab..ddeeb71 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -6946,6 +6946,7 @@ and unmount user-space FUSE file systems. This requires the @cindex mapped devices The Linux kernel has a notion of @dfn{device mapping}: a block device, such as a hard disk partition, can be @dfn{mapped} into another device, +usually in @code{/dev/mapper/}, with additional processing over the data that flows through it@footnote{Note that the GNU@tie{}Hurd makes no difference between the concept of a ``mapped device'' and that of a file system: both boil down @@ -6955,42 +6956,14 @@ devices, like file systems, using the generic @dfn{translator} mechanism (@pxref{Translators,,, hurd, The GNU Hurd Reference Manual}).}. A typical example is encryption device mapping: all writes to the mapped device are encrypted, and all reads are deciphered, transparently. +Guix extends this notion by considering any device or set of devices that +are @dfn{transformed} in some way to create a new device; for instance, +RAID devices are obtained by @dfn{assembling} several other devices, such +as hard disks or partitions, into a new one that behaves as one partition. +Other examples, not yet implemented, are LVM logical volumes. -Mapped devices are declared using the @code{mapped-device} form: - -@example -(mapped-device - (source "/dev/sda3") - (target "home") - (type luks-device-mapping)) -@end example - -Or, better yet, like this: - -@example -(mapped-device - (source (uuid "cb67fc72-0d54-4c88-9d4b-b225f30b0f44")) - (target "home") - (type luks-device-mapping)) -@end example - -@cindex disk encryption -@cindex LUKS -This example specifies a mapping from @file{/dev/sda3} to -@file{/dev/mapper/home} using LUKS---the -@url{http://code.google.com/p/cryptsetup,Linux Unified Key Setup}, a -standard mechanism for disk encryption. In the second example, the UUID -(unique identifier) is the LUKS UUID returned for the device by a -command like: - -@example -cryptsetup luksUUID /dev/sdx9 -@end example - -The @file{/dev/mapper/home} -device can then be used as the @code{device} of a @code{file-system} -declaration (@pxref{File Systems}). The @code{mapped-device} form is -detailed below. +Mapped devices are declared using the @code{mapped-device} form, +defined as follows; for examples, see below. @deftp {Data Type} mapped-device Objects of this type represent device mappings that will be made when @@ -6998,13 +6971,17 @@ the system boots up. @table @code @item source -This string specifies the name of the block device to be mapped, such as -@code{"/dev/sda3"}. +This is either a string specifying the name of the block device to be mapped, +such as @code{"/dev/sda3"}, or a list of such strings when several devices +need to be assembled for creating a new one. @item target -This string specifies the name of the mapping to be established. For -example, specifying @code{"my-partition"} will lead to the creation of +This string specifies the name of the resulting mapped device. For +kernel mappers such as encrypted devices of type @code{luks-device-mapping}, +specifying @code{"my-partition"} leads to the creation of the @code{"/dev/mapper/my-partition"} device. +For RAID devices of type @code{raid-device-mapping}, the full device name +such as @code{"/dev/md0"} needs to be given. @item type This must be a @code{mapped-device-kind} object, which specifies how @@ -7018,6 +6995,64 @@ command from the package with the same name. It relies on the @code{dm-crypt} Linux kernel module. @end defvr +@defvr {Scheme Variable} raid-device-mapping +This defines a RAID device, which is assembled using the @code{mdadm} +command from the package with the same name. It requires a Linux kernel +module for the appropriate RAID level to be loaded, such as @code{raid456} +for RAID-4, RAID-5 or RAID-6, or @code{raid10} for RAID-10. +@end defvr + +@cindex disk encryption +@cindex LUKS +The following example specifies a mapping from @file{/dev/sda3} to +@file{/dev/mapper/home} using LUKS---the +@url{http://code.google.com/p/cryptsetup,Linux Unified Key Setup}, a +standard mechanism for disk encryption. +The @file{/dev/mapper/home} +device can then be used as the @code{device} of a @code{file-system} +declaration (@pxref{File Systems}). + +@example +(mapped-device + (source "/dev/sda3") + (target "home") + (type luks-device-mapping)) +@end example + +Alternatively, to become independent of device numbering, one may obtain +the LUKS UUID (@dfn{unique identifier}) of the source device by a +command like: + +@example +cryptsetup luksUUID /dev/sda3 +@end example + +and use it as follows: + +@example +(mapped-device + (source (uuid "cb67fc72-0d54-4c88-9d4b-b225f30b0f44")) + (target "home") + (type luks-device-mapping)) +@end example + +A RAID device formed of the partitions @file{/dev/sda1} and @file{/dev/sdb1} +may be declared as follows: + +@example +(mapped-device + (source (list "/dev/sda1" "/dev/sdb1")) + (target "/dev/md0") + (type raid-device-mapping)) +@end example + +The @file{/dev/md0} device can then be used as the @code{device} of a +@code{file-system} declaration (@pxref{File Systems}). +Note that the RAID level need not be given; it is chosen during the +initial creation and formatting of the RAID device and is determined +automatically later. + + @node User Accounts @subsection User Accounts diff --git a/gnu/system/mapped-devices.scm b/gnu/system/mapped-devices.scm index 732f73c..d0a9f02 100644 --- a/gnu/system/mapped-devices.scm +++ b/gnu/system/mapped-devices.scm @@ -1,5 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2014, 2015, 2016 Ludovic Courtès +;;; Copyright © 2016 Andreas Enge ;;; ;;; This file is part of GNU Guix. ;;; @@ -22,6 +23,7 @@ #:use-module (gnu services) #:use-module (gnu services shepherd) #:autoload (gnu packages cryptsetup) (cryptsetup) + #:autoload (gnu packages linux) (mdadm) #:use-module (srfi srfi-1) #:use-module (ice-9 match) #:export (mapped-device @@ -38,7 +40,8 @@ device-mapping-service-type device-mapping-service - luks-device-mapping)) + luks-device-mapping + raid-device-mapping)) ;;; Commentary: ;;; @@ -127,4 +130,28 @@ (open open-luks-device) (close close-luks-device))) +(define (open-raid-device source target) + "Return a gexp that assembles SOURCE (a list of devices) to the RAID device +TARGET, using 'mdadm'." + #~(let ((every (@ (srfi srfi-1) every))) + (let loop () + (unless (every file-exists? '#$source) + (format #t "waiting a bit...~%") + (sleep 1) + (loop))) + (zero? (system* (string-append #$mdadm "/sbin/mdadm") + "--assemble" #$target + #$@source)))) + +(define (close-raid-device source target) + "Return a gexp that stops the RAID device TARGET." + #~(zero? (system* (string-append #$mdadm "/sbin/mdadm") + "--stop" #$target))) + +(define raid-device-mapping + ;; The type of RAID mapped devices. + (mapped-device-kind + (open open-raid-device) + (close close-raid-device))) + ;;; mapped-devices.scm ends here -- 2.9.0 --VS++wcV0S1rZb1Fb--