[PATCH] Add mdadm to the installation image.

[PATCH] Add mdadm to the installation image.

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 352 bytes --]

Hello,

the hydra replacement comes with two hard disks, which we would like to set
up as a raid10. For this, one needs mdadm. While it can be pulled in during
installation via "guix package -I mdadm", I think it is preferable that it
already be around, the same as cryptsetup, for instance. The attached patch
does this.

What do you think?

Andreas


[-- Attachment #2: 0001-install-Add-mdadm-to-the-image.patch --]
[-- Type: text/plain, Size: 1194 bytes --]

From eda9d6e5814c0e53f481599d0efd09d5237444d6 Mon Sep 17 00:00:00 2001
From: Andreas Enge <andreas@enge.fr>
Date: Thu, 14 Jul 2016 15:07:49 +0200
Subject: [PATCH] install: Add mdadm to the image.

* gnu/system/install.scm (installation-os)[packages]: Add mdadm.
---
 gnu/system/install.scm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gnu/system/install.scm b/gnu/system/install.scm
index 329c7ab..734a361 100644
--- a/gnu/system/install.scm
+++ b/gnu/system/install.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2014, 2015, 2016 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2015 Mark H Weaver <mhw@netris.org>
+;;; Copyright © 2016 Andreas Enge <andreas@enge.fr>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -392,6 +393,7 @@ Use Alt-F2 for documentation.
                      parted ddrescue
                      grub                  ;mostly so xrefs to its manual work
                      cryptsetup
+                     mdadm
                      btrfs-progs
                      wireless-tools iw wpa-supplicant-minimal iproute
                      ;; XXX: We used to have GNU fdisk here, but as of version
-- 
2.8.4

Re: [PATCH] Add mdadm to the installation image.

[Mathieu Lirzin]

Hello,

Andreas Enge <andreas@enge.fr> writes:

> the hydra replacement comes with two hard disks, which we would like to set
> up as a raid10. For this, one needs mdadm. While it can be pulled in during
> installation via "guix package -I mdadm", I think it is preferable that it
> already be around, the same as cryptsetup, for instance. The attached patch
> does this.
>
> What do you think?

I fully agree that mdadm should be directly available.

> From eda9d6e5814c0e53f481599d0efd09d5237444d6 Mon Sep 17 00:00:00 2001
> From: Andreas Enge <andreas@enge.fr>
> Date: Thu, 14 Jul 2016 15:07:49 +0200
> Subject: [PATCH] install: Add mdadm to the image.
>
> * gnu/system/install.scm (installation-os)[packages]: Add mdadm.
> ---
>  gnu/system/install.scm | 2 ++
>  1 file changed, 2 insertions(+)
>

-- 
Mathieu Lirzin

Re: [PATCH] Add mdadm to the installation image.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 868 bytes --]

On Thu, Jul 14, 2016 at 04:53:06PM +0200, Mathieu Lirzin wrote:
> Hello,
> 
> Andreas Enge <andreas@enge.fr> writes:
> 
> > the hydra replacement comes with two hard disks, which we would like to set
> > up as a raid10. For this, one needs mdadm. While it can be pulled in during
> > installation via "guix package -I mdadm", I think it is preferable that it
> > already be around, the same as cryptsetup, for instance. The attached patch
> > does this.
> >
> > What do you think?
> 
> I fully agree that mdadm should be directly available.
> 
> -- 
> Mathieu Lirzin
> 

How much does it add to the size?

also, raid10 with 2 disks?

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] Add mdadm to the installation image.

[Andreas Enge]

On Thu, Jul 14, 2016 at 10:21:25PM +0300, Efraim Flashner wrote:
> How much does it add to the size?

$ guix size mdadm
store item                                                       total    self
/gnu/store/k2hlwyzfs1v5xj1v6475n52jljd9q89x-mdadm-3.4               79.7     1.1   1.4%
/gnu/store/lpvkjic9dhj55plc29jhq8l39irvqm4f-coreutils-8.25          78.6    14.3  17.9%
/gnu/store/g6704rf6gk8w1czvj24d5sbh9zvhf50i-gmp-6.1.0               63.5     2.6   3.2%
/gnu/store/cny25x3kx0z7c93dsnd9vsxasjgln76d-acl-2.2.52              61.6     0.4   0.5%
/gnu/store/m92xmqp7skc233s867bg3gab860sikwl-libcap-2.24             61.3     0.1   0.2%
/gnu/store/zv5x1g1p4kail830r9aw6n49kw0gk92f-attr-2.4.47             61.2     0.2   0.3%
/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib           60.9    22.9  28.7%
/gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23              38.1    36.7  46.1%
/gnu/store/0010wvgs40kdq8chzsh403qm7la9jxq7-bash-static-4.3.42       1.4     1.4   1.7%
total: 79.7 MiB

But I think that most of these are already there anyway, and only the "self"
part of 1.1MB actually comes extra.

> also, raid10 with 2 disks?

Yes. With the "far" layout, the second disk is a mirror of the first one,
but with the blocks in a different order. So the second disk creates a
backup, and large sequential reads should become close to twice as fast.

Andreas

Re: [PATCH] Add mdadm to the installation image.

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> On Thu, Jul 14, 2016 at 10:21:25PM +0300, Efraim Flashner wrote:
>> How much does it add to the size?
>
> $ guix size mdadm

You should check the size of the whole system:

  guix size $(guix system build gnu/system/install.scm)

Ludo’.

Re: [PATCH] Add mdadm to the installation image.

[Andreas Enge]

On Fri, Jul 15, 2016 at 03:27:29PM +0200, Ludovic Courtès wrote:
> You should check the size of the whole system:
>   guix size $(guix system build gnu/system/install.scm)

Good point! Here we are. Before:
/gnu/store/q62mdc418pr2wm9160a7qp3dhc8ii1d8-system                 824.8     0.0   0.0%

After:
/gnu/store/inv57icbw0dp7s4d33r37l1yjdv3sp2n-system                 825.9     0.0   0.0%

:-)


For good measure, here the size after adding also lm-sensors:
/gnu/store/v6nlhanbhcabk2vvqmi4xsgividq1060-system                 904.0     0.0   0.0%

:-|
Not great, but not as bad as I had feared; probably because perl is
unavoidable anyway.

Andreas

Re: [PATCH] Add mdadm to the installation image.

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> On Fri, Jul 15, 2016 at 03:27:29PM +0200, Ludovic Courtès wrote:
>> You should check the size of the whole system:
>>   guix size $(guix system build gnu/system/install.scm)
>
> Good point! Here we are. Before:
> /gnu/store/q62mdc418pr2wm9160a7qp3dhc8ii1d8-system                 824.8     0.0   0.0%
>
> After:
> /gnu/store/inv57icbw0dp7s4d33r37l1yjdv3sp2n-system                 825.9     0.0   0.0%
>
> :-)

Perfect.  :-)

> For good measure, here the size after adding also lm-sensors:
> /gnu/store/v6nlhanbhcabk2vvqmi4xsgividq1060-system                 904.0     0.0   0.0%

Then we should avoid it IMO.  My understanding is that not having
lm-sensors is at worst annoying, but not critical, right?  WDYT?

Thanks,
Ludo’.

Re: [PATCH] Add mdadm to the installation image.

[Andreas Enge]

On Sat, Jul 16, 2016 at 12:50:41PM +0200, Ludovic Courtès wrote:
> > For good measure, here the size after adding also lm-sensors:
> > /gnu/store/v6nlhanbhcabk2vvqmi4xsgividq1060-system                 904.0     0.0   0.0%
> Then we should avoid it IMO.  My understanding is that not having
> lm-sensors is at worst annoying, but not critical, right?  WDYT?

The comment was just a follow-up to my previous messages about lm-sensors
and my complaint about its size. Indeed I do not mean to imply we should
add it.

Andreas

Re: [PATCH] Add mdadm to the installation image.

[myglc2]

Andreas Enge <andreas@enge.fr> writes:

> Hello,
>
> the hydra replacement comes with two hard disks, which we would like to set
> up as a raid10. For this, one needs mdadm. While it can be pulled in during
> installation via "guix package -I mdadm", I think it is preferable that it
> already be around, the same as cryptsetup, for instance. The attached patch
> does this.
>
> What do you think?
>
> Andreas

I want to do this on my own servers, so I am wondering ...

How do you plan to address mdadm and grub config?

Will you be booting from RAID?

TIA - George

[PATCH} Add RAID devices.

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]

Hello,

On Fri, Jul 15, 2016 at 10:25:47AM -0400, myglc2 wrote:
> How do you plan to address mdadm and grub config?
> Will you be booting from RAID?

the attached patch adds RAID support to our mapped device mechanism.

To give more explanation:
Like for LUKS devices or file systems, the RAID itself needs to be created
during installation, after booting from the USB key:
   mdadm --create /dev/md0 --level=raid10 --layout=f2 \
      --raid-devices=2 /dev/sda2 /dev/sdb2
   (wait 12 hours or so for 4 TB disks)
   mkfs.ext4 /dev/md0

Then one adds a mapped-device of type raid-device-mapping, adds the
raid10 kernel module to the initrd and can boot from the RAID device.
The mapped-device mechanism calls "mdadm --assemble" and "mdadm --stop".

I might write a more detailed blog post about this; there is a little
subtlety with the non-automatic determination of dependencies between
devices, so one needs to make sure that the partitions to be assembled
are present before the mdadm command is executed.

Looking forward to comments on the patch,

Andreas


[-- Attachment #2: 0001-system-Add-mapped-devices-for-RAID.patch --]
[-- Type: text/plain, Size: 8094 bytes --]

From fc2d8dc30c04677ebf553b02227dc10b0be49665 Mon Sep 17 00:00:00 2001
From: Andreas Enge <andreas@enge.fr>
Date: Thu, 14 Jul 2016 15:51:59 +0200
Subject: [PATCH] system: Add mapped devices for RAID.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* gnu/system/mapped-devices.scm (raid-device-mapping, open-raid-device,
close-raid-device): New variables.
* doc/guix.texi (Mapped Devices): Add documentation for RAID devices,
reorganize documentation for LUKS devices.

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
---
 doc/guix.texi                 | 113 +++++++++++++++++++++++++++---------------
 gnu/system/mapped-devices.scm |  29 ++++++++++-
 2 files changed, 102 insertions(+), 40 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 393efab..ddeeb71 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -6946,6 +6946,7 @@ and unmount user-space FUSE file systems.  This requires the
 @cindex mapped devices
 The Linux kernel has a notion of @dfn{device mapping}: a block device,
 such as a hard disk partition, can be @dfn{mapped} into another device,
+usually in @code{/dev/mapper/},
 with additional processing over the data that flows through
 it@footnote{Note that the GNU@tie{}Hurd makes no difference between the
 concept of a ``mapped device'' and that of a file system: both boil down
@@ -6955,42 +6956,14 @@ devices, like file systems, using the generic @dfn{translator} mechanism
 (@pxref{Translators,,, hurd, The GNU Hurd Reference Manual}).}.  A
 typical example is encryption device mapping: all writes to the mapped
 device are encrypted, and all reads are deciphered, transparently.
+Guix extends this notion by considering any device or set of devices that
+are @dfn{transformed} in some way to create a new device; for instance,
+RAID devices are obtained by @dfn{assembling} several other devices, such
+as hard disks or partitions, into a new one that behaves as one partition.
+Other examples, not yet implemented, are LVM logical volumes.
 
-Mapped devices are declared using the @code{mapped-device} form:
-
-@example
-(mapped-device
-  (source "/dev/sda3")
-  (target "home")
-  (type luks-device-mapping))
-@end example
-
-Or, better yet, like this:
-
-@example
-(mapped-device
-  (source (uuid "cb67fc72-0d54-4c88-9d4b-b225f30b0f44"))
-  (target "home")
-  (type luks-device-mapping))
-@end example
-
-@cindex disk encryption
-@cindex LUKS
-This example specifies a mapping from @file{/dev/sda3} to
-@file{/dev/mapper/home} using LUKS---the
-@url{http://code.google.com/p/cryptsetup,Linux Unified Key Setup}, a
-standard mechanism for disk encryption.  In the second example, the UUID
-(unique identifier) is the LUKS UUID returned for the device by a
-command like:
-
-@example
-cryptsetup luksUUID /dev/sdx9
-@end example
-
-The @file{/dev/mapper/home}
-device can then be used as the @code{device} of a @code{file-system}
-declaration (@pxref{File Systems}).  The @code{mapped-device} form is
-detailed below.
+Mapped devices are declared using the @code{mapped-device} form,
+defined as follows; for examples, see below.
 
 @deftp {Data Type} mapped-device
 Objects of this type represent device mappings that will be made when
@@ -6998,13 +6971,17 @@ the system boots up.
 
 @table @code
 @item source
-This string specifies the name of the block device to be mapped, such as
-@code{"/dev/sda3"}.
+This is either a string specifying the name of the block device to be mapped,
+such as @code{"/dev/sda3"}, or a list of such strings when several devices
+need to be assembled for creating a new one.
 
 @item target
-This string specifies the name of the mapping to be established.  For
-example, specifying @code{"my-partition"} will lead to the creation of
+This string specifies the name of the resulting mapped device.  For
+kernel mappers such as encrypted devices of type @code{luks-device-mapping},
+specifying @code{"my-partition"} leads to the creation of
 the @code{"/dev/mapper/my-partition"} device.
+For RAID devices of type @code{raid-device-mapping}, the full device name
+such as @code{"/dev/md0"} needs to be given.
 
 @item type
 This must be a @code{mapped-device-kind} object, which specifies how
@@ -7018,6 +6995,64 @@ command from the package with the same name.  It relies on the
 @code{dm-crypt} Linux kernel module.
 @end defvr
 
+@defvr {Scheme Variable} raid-device-mapping
+This defines a RAID device, which is assembled using the @code{mdadm}
+command from the package with the same name.  It requires a Linux kernel
+module for the appropriate RAID level to be loaded, such as @code{raid456}
+for RAID-4, RAID-5 or RAID-6, or @code{raid10} for RAID-10.
+@end defvr
+
+@cindex disk encryption
+@cindex LUKS
+The following example specifies a mapping from @file{/dev/sda3} to
+@file{/dev/mapper/home} using LUKS---the
+@url{http://code.google.com/p/cryptsetup,Linux Unified Key Setup}, a
+standard mechanism for disk encryption.
+The @file{/dev/mapper/home}
+device can then be used as the @code{device} of a @code{file-system}
+declaration (@pxref{File Systems}).
+
+@example
+(mapped-device
+  (source "/dev/sda3")
+  (target "home")
+  (type luks-device-mapping))
+@end example
+
+Alternatively, to become independent of device numbering, one may obtain
+the LUKS UUID (@dfn{unique identifier}) of the source device by a
+command like:
+
+@example
+cryptsetup luksUUID /dev/sda3
+@end example
+
+and use it as follows:
+
+@example
+(mapped-device
+  (source (uuid "cb67fc72-0d54-4c88-9d4b-b225f30b0f44"))
+  (target "home")
+  (type luks-device-mapping))
+@end example
+
+A RAID device formed of the partitions @file{/dev/sda1} and @file{/dev/sdb1}
+may be declared as follows:
+
+@example
+(mapped-device
+  (source (list "/dev/sda1" "/dev/sdb1"))
+  (target "/dev/md0")
+  (type raid-device-mapping))
+@end example
+
+The @file{/dev/md0} device can then be used as the @code{device} of a
+@code{file-system} declaration (@pxref{File Systems}).
+Note that the RAID level need not be given; it is chosen during the
+initial creation and formatting of the RAID device and is determined
+automatically later.
+
+
 @node User Accounts
 @subsection User Accounts
 
diff --git a/gnu/system/mapped-devices.scm b/gnu/system/mapped-devices.scm
index 732f73c..d0a9f02 100644
--- a/gnu/system/mapped-devices.scm
+++ b/gnu/system/mapped-devices.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2014, 2015, 2016 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2016 Andreas Enge <andreas@enge.fr>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -22,6 +23,7 @@
   #:use-module (gnu services)
   #:use-module (gnu services shepherd)
   #:autoload   (gnu packages cryptsetup) (cryptsetup)
+  #:autoload   (gnu packages linux) (mdadm)
   #:use-module (srfi srfi-1)
   #:use-module (ice-9 match)
   #:export (mapped-device
@@ -38,7 +40,8 @@
             device-mapping-service-type
             device-mapping-service
 
-            luks-device-mapping))
+            luks-device-mapping
+            raid-device-mapping))
 
 ;;; Commentary:
 ;;;
@@ -127,4 +130,28 @@
    (open open-luks-device)
    (close close-luks-device)))
 
+(define (open-raid-device source target)
+  "Return a gexp that assembles SOURCE (a list of devices) to the RAID device
+TARGET, using 'mdadm'."
+  #~(let ((every (@ (srfi srfi-1) every)))
+      (let loop ()
+        (unless (every file-exists? '#$source)
+          (format #t "waiting a bit...~%")
+          (sleep 1)
+          (loop)))
+       (zero? (system* (string-append #$mdadm "/sbin/mdadm")
+                                      "--assemble" #$target
+                                      #$@source))))
+
+(define (close-raid-device source target)
+  "Return a gexp that stops the RAID device TARGET."
+  #~(zero? (system* (string-append #$mdadm "/sbin/mdadm")
+                    "--stop" #$target)))
+
+(define raid-device-mapping
+  ;; The type of RAID mapped devices.
+  (mapped-device-kind
+   (open open-raid-device)
+   (close close-raid-device)))
+
 ;;; mapped-devices.scm ends here
-- 
2.9.0

Re: [PATCH} Add RAID devices.

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2055 bytes --]

Andreas Enge <andreas@enge.fr> writes:

>> How do you plan to address mdadm and grub config?
>> Will you be booting from RAID?
>
> the attached patch adds RAID support to our mapped device mechanism.

Cool!  Is it possible to use them in combination?  Using the example
From the documentation, would it possible to use LUKS to create an
encrypted /dev/mapper/home which uses /dev/md0 instead of /dev/sda3?

> +Alternatively, to become independent of device numbering, one may obtain
> +the LUKS UUID (@dfn{unique identifier}) of the source device by a
> +command like:

How difficult would it be to add support for specifying the sources
using UUIDs?  I know we could use use the device files in
/dev/disk/by-uuid/, but "(guix) File Systems" (footnote 2) recommends
against that: "Note that, while it is tempting to use
‘/dev/disk/by-uuid’ and similar device names to achieve the same result,
this is not recommended: These special device nodes are created by the
udev daemon and may be unavailable at the time the device is mounted."

> +The @file{/dev/md0} device can then be used as the @code{device} of a
> +@code{file-system} declaration (@pxref{File Systems}).
> +Note that the RAID level need not be given; it is chosen during the
> +initial creation and formatting of the RAID device and is determined
> +automatically later.

I understand that in Linux, the device file names (e.g., /dev/sda3) can
sometimes change unexpectedly.  If they change later on, will anything
bad happen?

> +  #~(let ((every (@ (srfi srfi-1) every)))

Can't you just use "every" on its own?  It looks like you've imported
the srfi-1 module earlier on.

> +        (unless (every file-exists? '#$source)
> +          (format #t "waiting a bit...~%")
> +          (sleep 1)
> +          (loop)))

Does the code in this gexp get invoked every time the system starts up?
Why is a loop better here than an error?  What if the source device
files never show up?

Also, will that string be properly localized?

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: [PATCH} Add RAID devices.

[Ludovic Courtès]

Hello,

Chris Marusich <cmmarusich@gmail.com> skribis:

> Andreas Enge <andreas@enge.fr> writes:

[...]

>> +  #~(let ((every (@ (srfi srfi-1) every)))
>
> Can't you just use "every" on its own?  It looks like you've imported
> the srfi-1 module earlier on.

I’m the one who suggested it as a “temporary hack”, as we call such
things.  ;-)

The story is that this expression here gets stages in non-top-level
position, where it cannot directly do ‘use-modules’, hence this hack.

This should be fixed eventually, possibly in gexp themselves.

>> +        (unless (every file-exists? '#$source)
>> +          (format #t "waiting a bit...~%")
>> +          (sleep 1)
>> +          (loop)))
>
> Does the code in this gexp get invoked every time the system starts up?
> Why is a loop better here than an error?  What if the source device
> files never show up?

Right, another super-temporary hack.  The right thing would be to do
like ‘canonicalize-device-spec’ in (gnu build file-systems) does, which
is to error out after a few iterations, with the effect of spawning an
emergency REPL.

The mechanism to wait for devices should be factorized.

> Also, will that string be properly localized?

No it won’t, indeed.  Currently message catalogs and locales are
unavailable in the initrd, and it would probably make the initrd pretty
big to add them, so I’d be tempted to ignore i18n for early boot
messages that hopefully few people will notice.

Thanks,
Ludo’.

Re: [PATCH} Add RAID devices.

[Andreas Enge]

Hello!

On Sat, Jul 23, 2016 at 10:43:58PM -0700, Chris Marusich wrote:
> Cool!  Is it possible to use them in combination?  Using the example
> From the documentation, would it possible to use LUKS to create an
> encrypted /dev/mapper/home which uses /dev/md0 instead of /dev/sda3?

unfortunately not. This is an area where some work should be invested.
It would require to do things properly in order also. LVM support also
comes to mind. I think these devices can be staged in an arbitrarily
complex way, no? Encrypting a RAID device, or creating a RAID device
from encrypted partitions, for instance (of which the former sounds more
reasonable to me).

> I understand that in Linux, the device file names (e.g., /dev/sda3) can
> sometimes change unexpectedly.  If they change later on, will anything
> bad happen?

Yes, the RAID could not be assembled, and then the machine could not boot
up. But I have never experienced this kind of problem for hard disks.

> Does the code in this gexp get invoked every time the system starts up?
> Why is a loop better here than an error?

Yes. An error is too harsh: When I tried things out, my RAID assembling
was (tried to be) carried out before the hard disks were visible, so this
failed. Waiting a little solved the problem.

Andreas

Re: [PATCH} Add RAID devices.

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2286 bytes --]

Hi Andreas,

Ludo's response clarified a lot of things for me.  The only remaining
feedback I have is that (1) to aid the reader, you should consider
adding a cross-reference from "(guix) Mapped Devices" (in the part where
you mention that certain modules must be added) to "(guix) Initial RAM
Disk", and (2) you might want to look into using the "Auto Assembly"
feature of mdadm (see below).

If you've tested the changes, though, and it works, I see no reason not
to commit this and enable everyone to enjoy the use of RAID arrays! :)

Andreas Enge <andreas@enge.fr> writes:

> On Sat, Jul 23, 2016 at 10:43:58PM -0700, Chris Marusich wrote:
>> Cool!  Is it possible to use them in combination?  Using the example
>> From the documentation, would it possible to use LUKS to create an
>> encrypted /dev/mapper/home which uses /dev/md0 instead of /dev/sda3?
>
> unfortunately not. This is an area where some work should be invested.
> It would require to do things properly in order also. LVM support also
> comes to mind. I think these devices can be staged in an arbitrarily
> complex way, no? Encrypting a RAID device, or creating a RAID device
> from encrypted partitions, for instance (of which the former sounds more
> reasonable to me).

I agree it would be a nice feature, but if your patch is working right
now to enable the use of RAID, then I think it would be fine to submit
your patch now and add such a feature later.

>> I understand that in Linux, the device file names (e.g., /dev/sda3) can
>> sometimes change unexpectedly.  If they change later on, will anything
>> bad happen?
>
> Yes, the RAID could not be assembled, and then the machine could not boot
> up. But I have never experienced this kind of problem for hard disks.

It occurs occasionally [1], but the fewer disks you have, the less
likely you are to observe it.  I wonder if you can use mdadm's "Auto
Assembly" feature (see "man 8 mdadm" for details) to avoid this issue
entirely?  It sounds like you might not even need to specify a source
device list, if the description of "Auto Assembly" is to be believed.

[1] For example: https://serverfault.com/questions/140071/hard-drive-device-names-are-different-from-one-reboot-to-another-in-ubuntu

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: [PATCH} Add RAID devices.

[myglc2]

[-- Attachment #1: Type: text/plain, Size: 2105 bytes --]

Andreas Enge <andreas@enge.fr> writes:

> the attached patch adds RAID support to our mapped device mechanism.
>
> To give more explanation:
> Like for LUKS devices or file systems, the RAID itself needs to be created
> during installation, after booting from the USB key:
>    mdadm --create /dev/md0 --level=raid10 --layout=f2 \
>       --raid-devices=2 /dev/sda2 /dev/sdb2
>    (wait 12 hours or so for 4 TB disks)
>    mkfs.ext4 /dev/md0
>
> Then one adds a mapped-device of type raid-device-mapping, adds the
> raid10 kernel module to the initrd and can boot from the RAID device.
> The mapped-device mechanism calls "mdadm --assemble" and "mdadm --stop".
>
> I might write a more detailed blog post about this; there is a little
> subtlety with the non-automatic determination of dependencies between
> devices, so one needs to make sure that the partitions to be assembled
> are present before the mdadm command is executed.

Thanks. Starting with baby steps, I am running GuixSD system from a
separate drive and adding raid1 array with 2 new disks. I created an
array that will assemble like so...

g1@g1 ~$ sudo mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1
mdadm: /dev/md0 has been started with 2 drives.

g1@g1 ~$ cat /proc/mdstat
Personalities : [raid1] 
md0 : active raid1 sdc1[0] sdb1[1]
      239809536 blocks super 1.2 [2/2] [UU]
      bitmap: 0/2 pages [0KB], 65536KB chunk

I added this to my config (system25.scm attached) ...

(operating-system
[...]
  (initrd (lambda (fs . args)
	    (apply base-initrd fs
		   #:extra-modules '("raid1")
		   #:mapped-devices '((mapped-device
				       (source (list "/dev/sdb1" "/dev/sdc1"))
				       (target "/dev/md0")
				       (type raid-device-mapping)))				      
		   args)))
[...]

I rebooted and the array is not assembled ;-(

g1@g1 ~$ cat /proc/mdstat
Personalities : [raid1] 
unused devices: <none>

/var/log/messages contains ...
[...]
Jul 30 18:09:59 localhost vmunix: [    1.872801] md: raid1 personality registered for level 1
[...]

Any suggestions, corrections or a sample working config would be most welcome.

TIA - George


[-- Attachment #2: system25.scm --]
[-- Type: application/octet-stream, Size: 2038 bytes --]

;;; g1 system config
(use-modules (gnu))
(use-service-modules networking ssh)
(use-package-modules
 base
 admin
 disk
 linux                 ; mdadm
 package-management    ; guix
 screen
 ghostscript           ; gs-fonts
 fonts                 ; font-dejavu font-gnu-freefont-ttf
 curl                  ; lpaste
 ssh
 xorg certs
 rsync
 wget
 version-control       ; git
 aspell
 emacs
 cups
 graphviz
 qemu
 )
(operating-system
  (host-name "g1")
  (timezone "America/New_York")
  (locale "en_US.utf8")
  (bootloader (grub-configuration (device "/dev/sda")))
  (initrd (lambda (fs . args)
	    (apply base-initrd fs
		   #:extra-modules '("raid1")
		   #:mapped-devices '((mapped-device
				       (source (list "/dev/sdb1" "/dev/sdc1"))
				       (target "/dev/md0")
				       (type raid-device-mapping)))				      
		   args)))
  (file-systems (cons (file-system
			(device "ssd-root")
			(title 'label)
			(mount-point "/")
			(type "ext4"))
		      %base-file-systems))
  (users (cons* (user-account
		 (name "glc")
		 (group "users")
		 (supplementary-groups '("wheel"))
		 (home-directory "/home/glc"))
		(user-account
		 (name "g1")
		 (group "users")
		 (supplementary-groups '("wheel" "kvm"))
		 (home-directory "/home/g1"))
		(user-account
		 (name "g1x")
		 (group "users")
		 (supplementary-groups '("wheel" "kvm"))
		 (home-directory "/home/g1x"))
		(user-account
		 (name "gx")
		 (group "users")
		 (supplementary-groups '("wheel" "kvm"))
		 (home-directory "/home/gx"))
		%base-user-accounts))
  (packages
   (cons*
    glibc-utf8-locales
    parted
    mdadm
    qemu
    guix
    screen
    openssh nss-certs xauth
    rsync wget
    curl                                                     ;lpaste
    git git-manpages
    gs-fonts font-dejavu font-gnu-freefont-ttf
    aspell
    emacs
    flycheck paredit magit
    emacs-zenburn-theme emacs-markdown-mode emacs-web-mode
    cups
    graphviz
    %base-packages))
  (services (cons* (dhcp-client-service)
		   (lsh-service #:port-number 22)
		   %base-services)))

Re: [PATCH} Add RAID devices.

[Andreas Enge]

On Sat, Jul 30, 2016 at 07:05:25PM -0400, myglc2 wrote:
> > I might write a more detailed blog post about this; there is a little
> > subtlety with the non-automatic determination of dependencies between
> > devices, so one needs to make sure that the partitions to be assembled
> > are present before the mdadm command is executed.
> I rebooted and the array is not assembled ;-(

Strange! But I will also tell you the subtlety ;-)  Here is a trick to use
(thanks to Ludovic):
(define md0
  (mapped-device
    (source (list "/dev/sda2" "/dev/sdb2"))
      (target "/dev/md0")
      (type raid-device-mapping)))
(operating-system
...
  (mapped-devices (list md0))
  (file-systems (cons (file-system
                        (title 'device)
                        (device "/dev/md0")
                        (dependencies (list md0))
                        (mount-point "/")
                        (type "ext4"))
                      %base-file-systems))

The "dependencies" field makes sure that the file system is only mounted
after the array is assembled; I am not sure that this is your problem,
but you might want to give it a try.

In the long run, this should be reprogrammed: Devices and file systems
should wait until all their "inputs" are present, or at least wait for
a reasonable time.


Andreas

Re: [PATCH} Add RAID devices.

[myglc2]

[-- Attachment #1: Type: text/plain, Size: 1612 bytes --]

Andreas Enge <andreas@enge.fr> writes:

> On Sat, Jul 30, 2016 at 07:05:25PM -0400, myglc2 wrote:
>> > I might write a more detailed blog post about this; there is a little
>> > subtlety with the non-automatic determination of dependencies between
>> > devices, so one needs to make sure that the partitions to be assembled
>> > are present before the mdadm command is executed.
>> I rebooted and the array is not assembled ;-(
>
> Strange! But I will also tell you the subtlety ;-)  Here is a trick to use
> (thanks to Ludovic):
> (define md0
>   (mapped-device
>     (source (list "/dev/sda2" "/dev/sdb2"))
>       (target "/dev/md0")
>       (type raid-device-mapping)))
> (operating-system
> ...
>   (mapped-devices (list md0))
>   (file-systems (cons (file-system
>                         (title 'device)
>                         (device "/dev/md0")
>                         (dependencies (list md0))
>                         (mount-point "/")
>                         (type "ext4"))
>                       %base-file-systems))
>
> The "dependencies" field makes sure that the file system is only mounted
> after the array is assembled; I am not sure that this is your problem,
> but you might want to give it a try.
>
> In the long run, this should be reprogrammed: Devices and file systems
> should wait until all their "inputs" are present, or at least wait for
> a reasonable time.

Thanks, I tried that. the 'guix system reconfigure' succeeds and starts
the raid array (please see system35.scm & system35.log, attached).

But the reboot hangs at:

[...] clocksource: Switched to clocksource tsc


[-- Attachment #2: system35.log --]
[-- Type: application/octet-stream, Size: 1418 bytes --]

root@g1 ~# cat /proc/mdstat
Personalities : [raid1] 
unused devices: <none>
root@g1 ~# rm -fr /mnt/md0
root@g1 ~# ls /mnt
root@g1 ~# guix system reconfigure system35.scm
substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%
The following derivation will be built:
   /gnu/store/rd6m57pga9ddf9lsvfja6jybb1vnrj7d-grub.cfg.drv
/gnu/store/l1mpwaxlf2nhm18vzxir9npdi2inis7j-system
/gnu/store/s6x0j9cy0vgfnzc6xybrgcx0k327brrf-grub.cfg
/gnu/store/7p3pc22s5ap4smm6habggjcgj83aaidv-grub-2.02beta3
activating system...
setting up setuid programs in '/run/setuid-programs'...
populating /etc from /gnu/store/7v9p4s8hkx8k58vsz6zk1q58hws9dmbz-etc...
usermod: no changes
usermod: no changes
usermod: no changes
making '/gnu/store/l1mpwaxlf2nhm18vzxir9npdi2inis7j-system' the current system...
guix system: loading new services: device-mapping-/dev/md0 file-system-/mnt/md0...
shepherd: Evaluating user expression (register-services (primitive-load "/gn...") #).
shepherd: Service device-mapping-/dev/md0 has been started.
shepherd: Service file-system-/mnt/md0 has been started.
Installing for i386-pc platform.
Installation finished. No error reported.
root@g1 ~# cat /proc/mdstat
Personalities : [raid1] 
md0 : active raid1 sdc1[0] sdb1[1]
      239809536 blocks super 1.2 [2/2] [UU]
      bitmap: 1/2 pages [4KB], 65536KB chunk

unused devices: <none>
root@g1 ~# exit
exit

Process shell finished

[-- Attachment #3: system35.scm --]
[-- Type: application/octet-stream, Size: 2375 bytes --]

;;; g1 system config
(use-modules (gnu))
(use-service-modules networking ssh)
(use-package-modules
 base
 admin
 disk
 linux                 ; mdadm
 package-management    ; guix
 screen
 ghostscript           ; gs-fonts
 fonts                 ; font-dejavu font-gnu-freefont-ttf
 curl                  ; lpaste
 ssh
 xorg certs
 rsync
 wget
 version-control       ; git
 aspell
 emacs
 cups
 graphviz
 qemu
 )
(define md0
  (mapped-device
   (source (list "/dev/sdb1" "/dev/sdc1"))
   (target "/dev/md0")
   (type raid-device-mapping)))
(operating-system
  (host-name "g1")
  (timezone "America/New_York")
  (locale "en_US.utf8")
  (bootloader (grub-configuration (device "/dev/sda")))
  (mapped-devices (list md0))
  (initrd (lambda (fs . args)
	    (apply base-initrd fs
		   #:extra-modules '("raid1")
		   #:mapped-devices '((mapped-device
				       (source (list "/dev/sdb1" "/dev/sdc1"))
				       (target "/dev/md0")
				       (type raid-device-mapping)))				      
		   args)))
  (file-systems (cons*
		 (file-system
		   (device "ssd-root")
		   (title 'label)
		   (mount-point "/")
		   (type "ext4"))
		 (file-system
		   (title 'device)
		   (device "/dev/md0")
		   (dependencies (list md0))
		   (mount-point "/mnt/md0")
		   (create-mount-point? #t)
		   (type "ext4"))
		 %base-file-systems))
  (users (cons* (user-account
		 (name "glc")
		 (group "users")
		 (supplementary-groups '("wheel"))
		 (home-directory "/home/glc"))
		(user-account
		 (name "g1")
		 (group "users")
		 (supplementary-groups '("wheel" "kvm"))
		 (home-directory "/home/g1"))
		(user-account
		 (name "g1x")
		 (group "users")
		 (supplementary-groups '("wheel" "kvm"))
		 (home-directory "/home/g1x"))
		(user-account
		 (name "gx")
		 (group "users")
		 (supplementary-groups '("wheel" "kvm"))
		 (home-directory "/home/gx"))
		%base-user-accounts))
  (packages
   (cons*
    glibc-utf8-locales
    parted
    mdadm
    qemu
    guix
    screen
    openssh nss-certs xauth
    rsync wget
    curl                                                     ;lpaste
    git git-manpages
    gs-fonts font-dejavu font-gnu-freefont-ttf
    aspell
    emacs
    flycheck paredit magit
    emacs-zenburn-theme emacs-markdown-mode emacs-web-mode
    cups
    graphviz
    %base-packages))
  (services (cons* (dhcp-client-service)
		   (lsh-service #:port-number 22)
		   %base-services)))

Re: [PATCH} Add RAID devices.

[Andreas Enge]

On Sun, Jul 31, 2016 at 12:12:02PM -0400, myglc2 wrote:
> Thanks, I tried that. the 'guix system reconfigure' succeeds and starts
> the raid array (please see system35.scm & system35.log, attached).
> 
> But the reboot hangs at:
> 
> [...] clocksource: Switched to clocksource tsc

No idea. The only difference I saw with my setup is that you have the
#:mapped-devices clause in the initrd; I am just using this:

  ;; Add a kernel module for RAID-10.
  (initrd (lambda (file-systems . rest)
            (apply base-initrd file-systems
                   #:extra-modules '("raid10")
                                   rest)))

Andreas

Re: [PATCH} Add RAID devices.

[myglc2]

Andreas Enge <andreas@enge.fr> writes:

>
> No idea. The only difference I saw with my setup is that you have the
> #:mapped-devices clause in the initrd; I am just using this:
>
>   ;; Add a kernel module for RAID-10.
>   (initrd (lambda (file-systems . rest)
>             (apply base-initrd file-systems
>                    #:extra-modules '("raid10")
>                                    rest)))
>
> Andreas

Unfortunately w/ this change the boot still hangs, so ... I will post it
to bug-guix. Thanks again! - George