[PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[Jookia]

DISCLMAIMER: This commit isn't meant for merging, so donut merge it.
It's meant for people to use until we get something better. There's
also code I haven't fully checked is needed (particularly mknodes)
so there's duplicates. Use this at the risk of having to ask me
to fix it and possibly have me say no.

So I've come up with the following hack commit that effectively
stops any sort of dependency management and adds some new targets
for LVM and LUKS with a keyfile.

Here's my current setup, take note that order of mapped devices
matter since there's no dependency management:

  (mapped-devices (list (mapped-device
                          (source "/dev/sda")
                          (target "hdd")
                          (type (luks-device-keyfile-mapping
                                  (local-file "/root/keyfile"))))
                        (mapped-device
                          (source "/dev/mapper/hdd")
                          (target "matrix")
                          (type lvm-device-mapping))))

  (file-systems (cons (file-system
                        (device "/dev/mapper/matrix-root")
                        (title 'device)
                        (mount-point "/")
                        (type "ext4"))
                      %base-file-systems))

  (swap-devices '("/dev/mapper/matrix-swap"))

This shouldn't break existing installs, but don't rely on this
behaviour or API unless you're willing to maintain it.

You'll note that I'm using a keyfile. It gets copied to initramfs,
but you generate it like so:

  dd bs=512 count=4 if=/dev/urandom of=/root/keyfile iflag=fullblock

Then you add it to your cryptsetup device like:

  cryptsetup luksAddKey /dev/sda /root/keyfile

I'm assuming you're using /dev/sda. But you might not be? Anyways
that means you only have the enter the password to decrypt root
once (at GRUB) instead of twice.

BE WARNED THAT YOUR DRIVE CAN BE DECRYPTED BY USING THE COPY OF YOUR
KEYFILE IN /GNU/STORE OR YOUR INITRAMFS IN /GNU/STORE.

  root@t400-apparent-situation ~# ls /gnu/store | grep keyfile
  rfwrwxpcvqqw8az8c6k37bqzqvgzrh34-keyfile

IF YOU ARE LOOKING FOR SECURITY IT IS NOT HERE. ANY APPLICATION YOU
RUN CAN READ /GNU/STORE. YOU HAVE BEEN WARNED.

Also you can do something like this to autologin, making it only one
password to get in to your system (GRUB):

(services (modify-services %desktop-services
  (slim-service-type config =>
		     (slim-configuration
		       (inherit config)
		       (auto-login? #t)
		       (default-user "jookia")
		       (auto-login-session
			#~(string-append #$xfce "/bin/startxfce4"))))))

Cheers,
Jookia.
---
 gnu/services/base.scm       |  8 +----
 gnu/system.scm              | 73 ++++++++++++++++++++++++++++++++++++---------
 gnu/system/linux-initrd.scm | 15 +++++++++-
 3 files changed, 74 insertions(+), 22 deletions(-)

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 9b3dc73..cb248fc 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -1200,15 +1200,9 @@ gexp, to open it, and evaluate @var{close} to close it."
   (shepherd-service-type
    'swap
    (lambda (device)
-     (define requirement
-       (if (string-prefix? "/dev/mapper/" device)
-           (list (symbol-append 'device-mapping-
-                                (string->symbol (basename device))))
-           '()))
-
      (shepherd-service
       (provision (list (symbol-append 'swap- (string->symbol device))))
-      (requirement `(udev ,@requirement))
+      (requirement `(udev root-file-system))
       (documentation "Enable the given swap device.")
       (start #~(lambda ()
                  (restart-on-EINTR (swapon #$device))
diff --git a/gnu/system.scm b/gnu/system.scm
index 5be24ba..922e1f0 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -44,6 +44,7 @@
   #:use-module (gnu packages compression)
   #:use-module (gnu packages firmware)
   #:autoload   (gnu packages cryptsetup) (cryptsetup)
+  #:autoload   (gnu packages linux) (lvm2)
   #:use-module (gnu services)
   #:use-module (gnu services shepherd)
   #:use-module (gnu services base)
@@ -103,7 +104,9 @@
             %base-packages
             %base-firmware
 
-            luks-device-mapping))
+            luks-device-mapping
+            luks-device-keyfile-mapping
+            lvm-device-mapping))
 
 ;;; Commentary:
 ;;;
@@ -194,6 +197,46 @@
    (open open-luks-device)
    (close close-luks-device)))
 
+;;; HACK HACK HACCKK
+
+(define (open-luks-device-keyfile key-file)
+  "Return a gexp that maps SOURCE to TARGET as a LUKS device, using
+'cryptsetup'."
+  (lambda (source target)
+    #~(zero? (system* (string-append #$cryptsetup "/sbin/cryptsetup")
+                        "open" "--type" "luks"
+                        (string-append "--key-file=" #$key-file)
+                        #$source #$target))))
+
+(define (luks-device-keyfile-mapping key-file)
+  ;; The type of LUKS mapped devices.
+  (mapped-device-kind
+   (open (open-luks-device-keyfile key-file))
+   (close close-luks-device)))
+
+(define (open-lvm-device source target)
+  "Return a gexp that opens the TARGET logical volume on the SOURCE device,
+using 'cryptsetup'." ;; TODO: fix
+  #~(and (zero? (system* (string-append #$lvm2 "/sbin/dmsetup")
+                    "mknodes"))
+         (zero? (system* (string-append #$lvm2 "/sbin/vgchange")
+                    "-ay" #$target "--verbose"))
+         (zero? (system* (string-append #$lvm2 "/sbin/vgscan")
+                    "--mknodes" "--verbose"))))
+
+(define (close-lvm-device source target)
+  "Return a gexp that closes the TARGET logical volume."
+  #~(zero? (system* (string-append #$lvm2 "/sbin/lvchange")
+                    "-an" #$target)))
+
+(define lvm-device-mapping
+  ;; The type of LUKS mapped devices.
+  (mapped-device-kind
+   (open open-lvm-device)
+   (close close-lvm-device)))
+
+;;; HACK HACK HACCKK
+
 (define (other-file-system-services os)
   "Return file system services for the file systems of OS that are not marked
 as 'needed-for-boot'."
@@ -233,23 +276,25 @@ as 'needed-for-boot'."
 (define (operating-system-user-mapped-devices os)
   "Return the subset of mapped devices that can be installed in
 user-land--i.e., those not needed during boot."
-  (let ((devices      (operating-system-mapped-devices os))
-        (file-systems (operating-system-file-systems os)))
-   (filter (lambda (md)
-             (let ((user (mapped-device-user md file-systems)))
-               (or (not user)
-                   (not (file-system-needed-for-boot? user)))))
-           devices)))
+  '())
+  ;(let ((devices      (operating-system-mapped-devices os))
+  ;      (file-systems (operating-system-file-systems os)))
+  ; (filter (lambda (md)
+  ;           (let ((user (mapped-device-user md file-systems)))
+  ;             (or (not user)
+  ;                 (not (file-system-needed-for-boot? user)))))
+  ;         devices)))
 
 (define (operating-system-boot-mapped-devices os)
   "Return the subset of mapped devices that must be installed during boot,
 from the initrd."
-  (let ((devices      (operating-system-mapped-devices os))
-        (file-systems (operating-system-file-systems os)))
-   (filter (lambda (md)
-             (let ((user (mapped-device-user md file-systems)))
-               (and user (file-system-needed-for-boot? user))))
-           devices)))
+  (operating-system-mapped-devices os))
+  ;(let ((devices      (operating-system-mapped-devices os))
+  ;      (file-systems (operating-system-file-systems os)))
+  ; (filter (lambda (md)
+  ;           (let ((user (mapped-device-user md file-systems)))
+  ;             (and user (file-system-needed-for-boot? user))))
+  ;         devices)))
 
 (define (device-mapping-services os)
   "Return the list of device-mapping services for OS as a list."
diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm
index 8ca7410..70a2e4a 100644
--- a/gnu/system/linux-initrd.scm
+++ b/gnu/system/linux-initrd.scm
@@ -207,6 +207,9 @@ loaded at boot time in the order in which they appear."
                   file-systems)
             (list e2fsck/static)
             '())
+      ,@(if #t ;(lvm-mapping-used? mapped-devices)
+            (list lvm2)
+            '())
       ,@(if volatile-root?
             (list unionfs-fuse/static)
             '())))
@@ -237,7 +240,17 @@ loaded at boot time in the order in which they appear."
 
          (boot-system #:mounts '#$(map file-system->spec file-systems)
                       #:pre-mount (lambda ()
-                                    (and #$@device-mapping-commands))
+                                    (and #$@device-mapping-commands
+                                    ;; If we activated any volume group, we
+                                    ;; need to ensure that device nodes are
+                                    ;; created.  Add code here to call it
+                                    ;; once for all activations.
+                                    #$(when #t ;(lvm-mapping-used? mapped-devices)
+                                        #~(zero?
+                                           (system* (string-append
+                                                     #$lvm2
+                                                     "/sbin/vgscan")
+                                                     "--mknodes")))))
                       #:linux-modules '#$linux-modules
                       #:linux-module-directory '#$kodir
                       #:qemu-guest-networking? #$qemu-networking?
-- 
2.7.0

[PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[Jookia]

DISCLMAIMER: This commit isn't meant for merging, so donut merge it.
It's meant for people to use until we get something better. There's
also code I haven't fully checked is needed (particularly mknodes)
so there's duplicates. Use this at the risk of having to ask me
to fix it and possibly have me say no.

So I've come up with the following hack commit that effectively
stops any sort of dependency management and adds some new targets
for LVM and LUKS with a keyfile.

Here's my current setup, take note that order of mapped devices
matter since there's no dependency management:

  (mapped-devices (list (mapped-device
                          (source "/dev/sda")
                          (target "hdd")
                          (type (luks-device-keyfile-mapping
                                  (local-file "/root/keyfile"))))
                        (mapped-device
                          (source "/dev/mapper/hdd")
                          (target "matrix")
                          (type lvm-device-mapping))))

  (file-systems (cons (file-system
                        (device "/dev/mapper/matrix-root")
                        (title 'device)
                        (mount-point "/")
                        (type "ext4"))
                      %base-file-systems))

  (swap-devices '("/dev/mapper/matrix-swap"))

This shouldn't break existing installs, but don't rely on this
behaviour or API unless you're willing to maintain it.

You'll note that I'm using a keyfile. It gets copied to initramfs,
but you generate it like so:

  dd bs=512 count=4 if=/dev/urandom of=/root/keyfile iflag=fullblock

Then you add it to your cryptsetup device like:

  cryptsetup luksAddKey /dev/sda /root/keyfile

I'm assuming you're using /dev/sda. But you might not be? Anyways
that means you only have the enter the password to decrypt root
once (at GRUB) instead of twice.

BE WARNED THAT YOUR DRIVE CAN BE DECRYPTED BY USING THE COPY OF YOUR
KEYFILE IN /GNU/STORE OR YOUR INITRAMFS IN /GNU/STORE.

  root@t400-apparent-situation ~# ls /gnu/store | grep keyfile
  rfwrwxpcvqqw8az8c6k37bqzqvgzrh34-keyfile

IF YOU ARE LOOKING FOR SECURITY IT IS NOT HERE. ANY APPLICATION YOU
RUN CAN READ /GNU/STORE. YOU HAVE BEEN WARNED.

Also you can do something like this to autologin, making it only one
password to get in to your system (GRUB):

(services (modify-services %desktop-services
  (slim-service-type config =>
		     (slim-configuration
		       (inherit config)
		       (auto-login? #t)
		       (default-user "jookia")
		       (auto-login-session
			#~(string-append #$xfce "/bin/startxfce4"))))))

Cheers,
Jookia.
---
 gnu/services/base.scm       |  8 +----
 gnu/system.scm              | 73 ++++++++++++++++++++++++++++++++++++---------
 gnu/system/linux-initrd.scm | 15 +++++++++-
 3 files changed, 74 insertions(+), 22 deletions(-)

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 9b3dc73..cb248fc 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -1200,15 +1200,9 @@ gexp, to open it, and evaluate @var{close} to close it."
   (shepherd-service-type
    'swap
    (lambda (device)
-     (define requirement
-       (if (string-prefix? "/dev/mapper/" device)
-           (list (symbol-append 'device-mapping-
-                                (string->symbol (basename device))))
-           '()))
-
      (shepherd-service
       (provision (list (symbol-append 'swap- (string->symbol device))))
-      (requirement `(udev ,@requirement))
+      (requirement `(udev root-file-system))
       (documentation "Enable the given swap device.")
       (start #~(lambda ()
                  (restart-on-EINTR (swapon #$device))
diff --git a/gnu/system.scm b/gnu/system.scm
index 5be24ba..922e1f0 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -44,6 +44,7 @@
   #:use-module (gnu packages compression)
   #:use-module (gnu packages firmware)
   #:autoload   (gnu packages cryptsetup) (cryptsetup)
+  #:autoload   (gnu packages linux) (lvm2)
   #:use-module (gnu services)
   #:use-module (gnu services shepherd)
   #:use-module (gnu services base)
@@ -103,7 +104,9 @@
             %base-packages
             %base-firmware
 
-            luks-device-mapping))
+            luks-device-mapping
+            luks-device-keyfile-mapping
+            lvm-device-mapping))
 
 ;;; Commentary:
 ;;;
@@ -194,6 +197,46 @@
    (open open-luks-device)
    (close close-luks-device)))
 
+;;; HACK HACK HACCKK
+
+(define (open-luks-device-keyfile key-file)
+  "Return a gexp that maps SOURCE to TARGET as a LUKS device, using
+'cryptsetup'."
+  (lambda (source target)
+    #~(zero? (system* (string-append #$cryptsetup "/sbin/cryptsetup")
+                        "open" "--type" "luks"
+                        (string-append "--key-file=" #$key-file)
+                        #$source #$target))))
+
+(define (luks-device-keyfile-mapping key-file)
+  ;; The type of LUKS mapped devices.
+  (mapped-device-kind
+   (open (open-luks-device-keyfile key-file))
+   (close close-luks-device)))
+
+(define (open-lvm-device source target)
+  "Return a gexp that opens the TARGET logical volume on the SOURCE device,
+using 'cryptsetup'." ;; TODO: fix
+  #~(and (zero? (system* (string-append #$lvm2 "/sbin/dmsetup")
+                    "mknodes"))
+         (zero? (system* (string-append #$lvm2 "/sbin/vgchange")
+                    "-ay" #$target "--verbose"))
+         (zero? (system* (string-append #$lvm2 "/sbin/vgscan")
+                    "--mknodes" "--verbose"))))
+
+(define (close-lvm-device source target)
+  "Return a gexp that closes the TARGET logical volume."
+  #~(zero? (system* (string-append #$lvm2 "/sbin/lvchange")
+                    "-an" #$target)))
+
+(define lvm-device-mapping
+  ;; The type of LUKS mapped devices.
+  (mapped-device-kind
+   (open open-lvm-device)
+   (close close-lvm-device)))
+
+;;; HACK HACK HACCKK
+
 (define (other-file-system-services os)
   "Return file system services for the file systems of OS that are not marked
 as 'needed-for-boot'."
@@ -233,23 +276,25 @@ as 'needed-for-boot'."
 (define (operating-system-user-mapped-devices os)
   "Return the subset of mapped devices that can be installed in
 user-land--i.e., those not needed during boot."
-  (let ((devices      (operating-system-mapped-devices os))
-        (file-systems (operating-system-file-systems os)))
-   (filter (lambda (md)
-             (let ((user (mapped-device-user md file-systems)))
-               (or (not user)
-                   (not (file-system-needed-for-boot? user)))))
-           devices)))
+  '())
+  ;(let ((devices      (operating-system-mapped-devices os))
+  ;      (file-systems (operating-system-file-systems os)))
+  ; (filter (lambda (md)
+  ;           (let ((user (mapped-device-user md file-systems)))
+  ;             (or (not user)
+  ;                 (not (file-system-needed-for-boot? user)))))
+  ;         devices)))
 
 (define (operating-system-boot-mapped-devices os)
   "Return the subset of mapped devices that must be installed during boot,
 from the initrd."
-  (let ((devices      (operating-system-mapped-devices os))
-        (file-systems (operating-system-file-systems os)))
-   (filter (lambda (md)
-             (let ((user (mapped-device-user md file-systems)))
-               (and user (file-system-needed-for-boot? user))))
-           devices)))
+  (operating-system-mapped-devices os))
+  ;(let ((devices      (operating-system-mapped-devices os))
+  ;      (file-systems (operating-system-file-systems os)))
+  ; (filter (lambda (md)
+  ;           (let ((user (mapped-device-user md file-systems)))
+  ;             (and user (file-system-needed-for-boot? user))))
+  ;         devices)))
 
 (define (device-mapping-services os)
   "Return the list of device-mapping services for OS as a list."
diff --git a/gnu/system/linux-initrd.scm b/gnu/system/linux-initrd.scm
index 8ca7410..70a2e4a 100644
--- a/gnu/system/linux-initrd.scm
+++ b/gnu/system/linux-initrd.scm
@@ -207,6 +207,9 @@ loaded at boot time in the order in which they appear."
                   file-systems)
             (list e2fsck/static)
             '())
+      ,@(if #t ;(lvm-mapping-used? mapped-devices)
+            (list lvm2)
+            '())
       ,@(if volatile-root?
             (list unionfs-fuse/static)
             '())))
@@ -237,7 +240,17 @@ loaded at boot time in the order in which they appear."
 
          (boot-system #:mounts '#$(map file-system->spec file-systems)
                       #:pre-mount (lambda ()
-                                    (and #$@device-mapping-commands))
+                                    (and #$@device-mapping-commands
+                                    ;; If we activated any volume group, we
+                                    ;; need to ensure that device nodes are
+                                    ;; created.  Add code here to call it
+                                    ;; once for all activations.
+                                    #$(when #t ;(lvm-mapping-used? mapped-devices)
+                                        #~(zero?
+                                           (system* (string-append
+                                                     #$lvm2
+                                                     "/sbin/vgscan")
+                                                     "--mknodes")))))
                       #:linux-modules '#$linux-modules
                       #:linux-module-directory '#$kodir
                       #:qemu-guest-networking? #$qemu-networking?
-- 
2.7.0

Re: [PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[Taylan Ulrich Bayırlı/Kammer]

FYI this landed in my Gmail spam folder.  Maybe because of the
all-caps. :-)

Taylan

Re: [PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[Jookia]

On Thu, Mar 10, 2016 at 08:48:25AM +0100, Taylan Ulrich Bayırlı/Kammer wrote:
> FYI this landed in my Gmail spam folder.  Maybe because of the
> all-caps. :-)

Haha wow, I hope nobody else who uses gmail cares about this patch.

> Taylan

Jookia.

Re: [PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[Ludovic Courtès]

Jookia <166291@gmail.com> skribis:

> So I've come up with the following hack commit that effectively
> stops any sort of dependency management and adds some new targets
> for LVM and LUKS with a keyfile.
>
> Here's my current setup, take note that order of mapped devices
> matter since there's no dependency management:
>
>   (mapped-devices (list (mapped-device
>                           (source "/dev/sda")
>                           (target "hdd")
>                           (type (luks-device-keyfile-mapping
>                                   (local-file "/root/keyfile"))))
>                         (mapped-device
>                           (source "/dev/mapper/hdd")
>                           (target "matrix")
>                           (type lvm-device-mapping))))
>
>   (file-systems (cons (file-system
>                         (device "/dev/mapper/matrix-root")
>                         (title 'device)
>                         (mount-point "/")
>                         (type "ext4"))
>                       %base-file-systems))
>
>   (swap-devices '("/dev/mapper/matrix-swap"))

As you note, and as discussed on IRC, this is not OK because the private
key ends up being stored world-readable in the store.  :-/

Am I missing the part you wanted to discuss?

Thanks,
Ludo’.

PS: I still intend to look at the patch series you sent ;-), just
    prioritizing things that relate to 0.9.1.

Re: [PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[Jookia]

On Thu, Mar 10, 2016 at 05:10:09PM +0100, Ludovic Courtès wrote:
> Jookia <166291@gmail.com> skribis:
> 
> > So I've come up with the following hack commit that effectively
> > stops any sort of dependency management and adds some new targets
> > for LVM and LUKS with a keyfile.
> >
> > Here's my current setup, take note that order of mapped devices
> > matter since there's no dependency management:
> >
> >   (mapped-devices (list (mapped-device
> >                           (source "/dev/sda")
> >                           (target "hdd")
> >                           (type (luks-device-keyfile-mapping
> >                                   (local-file "/root/keyfile"))))
> >                         (mapped-device
> >                           (source "/dev/mapper/hdd")
> >                           (target "matrix")
> >                           (type lvm-device-mapping))))
> >
> >   (file-systems (cons (file-system
> >                         (device "/dev/mapper/matrix-root")
> >                         (title 'device)
> >                         (mount-point "/")
> >                         (type "ext4"))
> >                       %base-file-systems))
> >
> >   (swap-devices '("/dev/mapper/matrix-swap"))
> 
> As you note, and as discussed on IRC, this is not OK because the private
> key ends up being stored world-readable in the store.  :-/

That's one thing to talk about- Store permissions and what to do about them. I
also have another situation where I want to run a container with an OpenVPN
service, but I'd have to pass credentials to them somehow. It's tricky to do
this on NixOS because I'd have to edit the container files which means I now
have state not only in /etc but in my containers too!

Setting permissions to just 'root' might be a bit bad if container's 'root' also
get to read it, or containers can read each other's 'root' values.

> Am I missing the part you wanted to discuss?

I'd really like to discuss how much I needed to break to get the mapped-devices,
file-systems and swap-devices to just 'work'. I even had to make a function to
return a mapped-device type, and have swap-devices not do dependency tests since
I technically don't use a device I've defined.

It'd be much much better if I could do something like this in my services:

  (devices (list (file-system
                   (uses '("/dev/matrix/root"))
                   (creates '("/"))
                   (device "/dev/matrix/root")
                   (mount-point "/")
                   (type "ext4"))
                 (swap-device
                   (uses '("/dev/mapper/matrix-swap"))
                   (creates '()))
                   (device "/dev/mapper/matrix-swap")
                 (lvm-device
                   (uses '("/dev/mapper/hdd" "/dev/sdb"))
                   (creates '("/dev/matrix/"
                              "/dev/mapper/matrix-swap"))
                   (devices '("/dev/mapper/hdd" "/dev/sdb")))
                 (luks-device
                   (uses '("UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb"))
                   (creates '("/dev/mapper/hdd"))
                   (device "UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb")
                   (name "hdd")
                   (key-file "..."))))

The issue is that it has a lot of duplicate information as I'm not sure
uses/creates could always map to device/mount-point, like LUKS names. But this
should satisfy most dependency issues automatically, I hope.

> Thanks,
> Ludo’.
> 
> PS: I still intend to look at the patch series you sent ;-), just
>     prioritizing things that relate to 0.9.1.

That's fine. :)
Jookia.

Re: [PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[Ludovic Courtès]

Jookia <166291@gmail.com> skribis:

> I'd really like to discuss how much I needed to break to get the mapped-devices,
> file-systems and swap-devices to just 'work'. I even had to make a function to
> return a mapped-device type, and have swap-devices not do dependency tests since
> I technically don't use a device I've defined.
>
> It'd be much much better if I could do something like this in my services:
>
>   (devices (list (file-system
>                    (uses '("/dev/matrix/root"))
>                    (creates '("/"))
>                    (device "/dev/matrix/root")
>                    (mount-point "/")
>                    (type "ext4"))
>                  (swap-device
>                    (uses '("/dev/mapper/matrix-swap"))
>                    (creates '()))
>                    (device "/dev/mapper/matrix-swap")
>                  (lvm-device
>                    (uses '("/dev/mapper/hdd" "/dev/sdb"))
>                    (creates '("/dev/matrix/"
>                               "/dev/mapper/matrix-swap"))
>                    (devices '("/dev/mapper/hdd" "/dev/sdb")))
>                  (luks-device
>                    (uses '("UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb"))
>                    (creates '("/dev/mapper/hdd"))
>                    (device "UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb")
>                    (name "hdd")
>                    (key-file "..."))))
>
> The issue is that it has a lot of duplicate information as I'm not sure
> uses/creates could always map to device/mount-point, like LUKS names. But this
> should satisfy most dependency issues automatically, I hope.

There are several issues being addressed here, IIUC:

  1. How to refer to block devices (in the Unix sense) using UUIDs,
     labels, or /dev file names in general, and not just for
     ‘file-system’.

  2. How to determine dependencies among all these things.

  3. How to handle mapped devices that lead to several /dev nodes, as is
     the case with LVM.

For #1, I would like to have a general ‘device’ type, so one could
write:

  (operating-system
    ;; …
    (file-systems (list (file-system
                          (source (device (title 'label)
                                          (name "my-root")))
                          (mount-point "/"))))
    (swap (list (device
                  (title 'uuid)
                  (name (uuid "4dab5feb-d176-45de-b287-9b0a6e4c01cb")))))

For that to work, we ideally need code to recognize swap signatures,
similar to what we do for ext2 in (gnu build file-systems).

For #3, what about changing the ‘target’ field of ‘mapped-device’ such
that it can be a list of /dev file names?

For #2, I’m not sure we need to change anything, but let’s discuss it
later.  :-)

Ludo’.

Re: [PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[Jookia]

On Fri, Mar 11, 2016 at 03:30:10PM +0100, Ludovic Courtès wrote:
> Jookia <166291@gmail.com> skribis:
> 
> > I'd really like to discuss how much I needed to break to get the mapped-devices,
> > file-systems and swap-devices to just 'work'. I even had to make a function to
> > return a mapped-device type, and have swap-devices not do dependency tests since
> > I technically don't use a device I've defined.
> >
> > It'd be much much better if I could do something like this in my services:
> >
> >   (devices (list (file-system
> >                    (uses '("/dev/matrix/root"))
> >                    (creates '("/"))
> >                    (device "/dev/matrix/root")
> >                    (mount-point "/")
> >                    (type "ext4"))
> >                  (swap-device
> >                    (uses '("/dev/mapper/matrix-swap"))
> >                    (creates '()))
> >                    (device "/dev/mapper/matrix-swap")
> >                  (lvm-device
> >                    (uses '("/dev/mapper/hdd" "/dev/sdb"))
> >                    (creates '("/dev/matrix/"
> >                               "/dev/mapper/matrix-swap"))
> >                    (devices '("/dev/mapper/hdd" "/dev/sdb")))
> >                  (luks-device
> >                    (uses '("UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb"))
> >                    (creates '("/dev/mapper/hdd"))
> >                    (device "UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb")
> >                    (name "hdd")
> >                    (key-file "..."))))
> >
> > The issue is that it has a lot of duplicate information as I'm not sure
> > uses/creates could always map to device/mount-point, like LUKS names. But this
> > should satisfy most dependency issues automatically, I hope.
> 
> There are several issues being addressed here, IIUC:
> 
>   1. How to refer to block devices (in the Unix sense) using UUIDs,
>      labels, or /dev file names in general, and not just for
>      ‘file-system’.

Yes, this is one of the major problems.

>   2. How to determine dependencies among all these things.

Shouldn't be too hard if we have inputs and outputs for each.

>   3. How to handle mapped devices that lead to several /dev nodes, as is
>      the case with LVM.

Well, it's a bit more complex than that. We end up with filesystems that use
multiple devices too, like Btrfs.

I was also showing another point: It'd be nicer to have file-system,
swap-device, lvm-device, luks-device as functions rather than data structures.
I was a bit tired when writing and realized later on that the functions could
automatically create the uses/creates stuff as output. I'll show another
hypothetical, though I see interest in keeping the existing way.

> For #1, I would like to have a general ‘device’ type, so one could
> write:
> 
>   (operating-system
>     ;; …
>     (file-systems (list (file-system
>                           (source (device (title 'label)
>                                           (name "my-root")))
>                           (mount-point "/"))))
>     (swap (list (device
>                   (title 'uuid)
>                   (name (uuid "4dab5feb-d176-45de-b287-9b0a6e4c01cb")))))
> 
> For that to work, we ideally need code to recognize swap signatures,
> similar to what we do for ext2 in (gnu build file-systems).

This would work, but not all file-systems use devices. Bind/union mounts, for
one, which is why having them as paths would be perhaps easier to resolve. I'm
skeptical we could have a single file-system data structure that could account
for all this, which is why I'd much rather like functions that output stuff:

  (devices (list (file-system
                   #:device "/dev/matrix/root"
                   #:mount-point "/"
                   #:type "ext4")
                 (swap-device
                   #:device "/dev/mapper/matrix-swap")
                 (lvm-device
                   #:targets '("/dev/matrix/"
                              "/dev/mapper/matrix-swap")
                   #:devices (list "/dev/mapper/hdd" "/dev/sdb"))
                 (luks-device
                   #:device "UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb"
                   #:name "hdd"
                   #:key-file "...")))

which could output this tree:

  (devices (list (device
                   (uses '("/dev/matrix/root"))
                   (creates '("/"))
                   (init-function ...)
                   (destroy-function ...))
                 (swap-device
                   (uses '("/dev/mapper/matrix-swap"))
                   (creates '()))
                   (init-function ...)
                   (destroy-function ...))
                 (lvm-device
                   (uses '("/dev/mapper/hdd" "/dev/sdb"))
                   (creates '("/dev/matrix/"
                              "/dev/mapper/matrix-swap"))
                   (init-function ...)
                   (destroy-function ...))
                 (luks-device
                   (uses '("UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb"))
                   (creates '("/dev/mapper/hdd"))
                   (init-function ...)
                   (destroy-function ...)))

Though this is a bit of a dream in my case.

> For #2, I’m not sure we need to change anything, but let’s discuss it
> later.  :-)

Hmm, okay.

There also a small issue where mapped devices need more data in special cases,
like LUKS keyfiles which returns a function to create the type.

> Ludo’.

Jookia.

Re: [PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[Jean Louis]

Let me give some tips when making this:

The system must be able to encrypt:

- if possible whole / though I personally don't prefer it like that
- to encrypt /home on a device
- to encrypt /tmp (very important)
- to encrypt swap

On Fri, Mar 11, 2016 at 03:30:10PM +0100, Ludovic Courtès wrote:
> Jookia <166291@gmail.com> skribis:
> 
> > I'd really like to discuss how much I needed to break to get the mapped-devices,
> > file-systems and swap-devices to just 'work'. I even had to make a function to
> > return a mapped-device type, and have swap-devices not do dependency tests since
> > I technically don't use a device I've defined.
> >
> > It'd be much much better if I could do something like this in my services:
> >
> >   (devices (list (file-system
> >                    (uses '("/dev/matrix/root"))
> >                    (creates '("/"))
> >                    (device "/dev/matrix/root")
> >                    (mount-point "/")
> >                    (type "ext4"))
> >                  (swap-device
> >                    (uses '("/dev/mapper/matrix-swap"))
> >                    (creates '()))
> >                    (device "/dev/mapper/matrix-swap")
> >                  (lvm-device
> >                    (uses '("/dev/mapper/hdd" "/dev/sdb"))
> >                    (creates '("/dev/matrix/"
> >                               "/dev/mapper/matrix-swap"))
> >                    (devices '("/dev/mapper/hdd" "/dev/sdb")))
> >                  (luks-device
> >                    (uses '("UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb"))
> >                    (creates '("/dev/mapper/hdd"))
> >                    (device "UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb")
> >                    (name "hdd")
> >                    (key-file "..."))))
> >
> > The issue is that it has a lot of duplicate information as I'm not sure
> > uses/creates could always map to device/mount-point, like LUKS names. But this
> > should satisfy most dependency issues automatically, I hope.
> 
> There are several issues being addressed here, IIUC:
> 
>   1. How to refer to block devices (in the Unix sense) using UUIDs,
>      labels, or /dev file names in general, and not just for
>      ‘file-system’.
> 
>   2. How to determine dependencies among all these things.
> 
>   3. How to handle mapped devices that lead to several /dev nodes, as is
>      the case with LVM.
> 
> For #1, I would like to have a general ‘device’ type, so one could
> write:
> 
>   (operating-system
>     ;; …
>     (file-systems (list (file-system
>                           (source (device (title 'label)
>                                           (name "my-root")))
>                           (mount-point "/"))))
>     (swap (list (device
>                   (title 'uuid)
>                   (name (uuid "4dab5feb-d176-45de-b287-9b0a6e4c01cb")))))
> 
> For that to work, we ideally need code to recognize swap signatures,
> similar to what we do for ext2 in (gnu build file-systems).
> 
> For #3, what about changing the ‘target’ field of ‘mapped-device’ such
> that it can be a list of /dev file names?
> 
> For #2, I’m not sure we need to change anything, but let’s discuss it
> later.  :-)
> 
> Ludo’.
> 

Re: [PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[Ludovic Courtès]

Jookia <166291@gmail.com> skribis:

> On Fri, Mar 11, 2016 at 03:30:10PM +0100, Ludovic Courtès wrote:
>> Jookia <166291@gmail.com> skribis:
>> 
>> > I'd really like to discuss how much I needed to break to get the mapped-devices,
>> > file-systems and swap-devices to just 'work'. I even had to make a function to
>> > return a mapped-device type, and have swap-devices not do dependency tests since
>> > I technically don't use a device I've defined.
>> >
>> > It'd be much much better if I could do something like this in my services:
>> >
>> >   (devices (list (file-system
>> >                    (uses '("/dev/matrix/root"))
>> >                    (creates '("/"))
>> >                    (device "/dev/matrix/root")
>> >                    (mount-point "/")
>> >                    (type "ext4"))
>> >                  (swap-device
>> >                    (uses '("/dev/mapper/matrix-swap"))
>> >                    (creates '()))
>> >                    (device "/dev/mapper/matrix-swap")
>> >                  (lvm-device
>> >                    (uses '("/dev/mapper/hdd" "/dev/sdb"))
>> >                    (creates '("/dev/matrix/"
>> >                               "/dev/mapper/matrix-swap"))
>> >                    (devices '("/dev/mapper/hdd" "/dev/sdb")))
>> >                  (luks-device
>> >                    (uses '("UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb"))
>> >                    (creates '("/dev/mapper/hdd"))
>> >                    (device "UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb")
>> >                    (name "hdd")
>> >                    (key-file "..."))))
>> >
>> > The issue is that it has a lot of duplicate information as I'm not sure
>> > uses/creates could always map to device/mount-point, like LUKS names. But this
>> > should satisfy most dependency issues automatically, I hope.
>> 
>> There are several issues being addressed here, IIUC:
>> 
>>   1. How to refer to block devices (in the Unix sense) using UUIDs,
>>      labels, or /dev file names in general, and not just for
>>      ‘file-system’.
>
> Yes, this is one of the major problems.
>
>>   2. How to determine dependencies among all these things.
>
> Shouldn't be too hard if we have inputs and outputs for each.
>
>>   3. How to handle mapped devices that lead to several /dev nodes, as is
>>      the case with LVM.
>
> Well, it's a bit more complex than that. We end up with filesystems that use
> multiple devices too, like Btrfs.

OK.

Would a ‘mapped-device’ type where both ‘source’ and ‘target’ are lists
adequately model Linux’s notion of mapped devices?

> I was also showing another point: It'd be nicer to have file-system,
> swap-device, lvm-device, luks-device as functions rather than data structures.
> I was a bit tired when writing and realized later on that the functions could
> automatically create the uses/creates stuff as output. I'll show another
> hypothetical, though I see interest in keeping the existing way.

Keeping thing purely declarative, with high-level data structures such
as ‘file-system’ and ‘mapped-device’ is pretty nice IMO.  It allows
users to easily inspect the config, map over the various bits, etc.

>> For #1, I would like to have a general ‘device’ type, so one could
>> write:
>> 
>>   (operating-system
>>     ;; …
>>     (file-systems (list (file-system
>>                           (source (device (title 'label)
>>                                           (name "my-root")))
>>                           (mount-point "/"))))
>>     (swap (list (device
>>                   (title 'uuid)
>>                   (name (uuid "4dab5feb-d176-45de-b287-9b0a6e4c01cb")))))
>> 
>> For that to work, we ideally need code to recognize swap signatures,
>> similar to what we do for ext2 in (gnu build file-systems).
>
> This would work, but not all file-systems use devices. Bind/union mounts, for
> one, which is why having them as paths would be perhaps easier to resolve. I'm
> skeptical we could have a single file-system data structure that could account
> for all this,

Note that it already handles bind mounts and other pseudo file systems
(see (gnu system file-systems)).  Basically, ‘file-system’ directly
corresponds to the ‘mount’ system call.

> which is why I'd much rather like functions that output stuff:
>
>   (devices (list (file-system
>                    #:device "/dev/matrix/root"
>                    #:mount-point "/"
>                    #:type "ext4")
>                  (swap-device
>                    #:device "/dev/mapper/matrix-swap")
>                  (lvm-device
>                    #:targets '("/dev/matrix/"
>                               "/dev/mapper/matrix-swap")
>                    #:devices (list "/dev/mapper/hdd" "/dev/sdb"))
>                  (luks-device
>                    #:device "UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb"
>                    #:name "hdd"
>                    #:key-file "...")))
>
> which could output this tree:
>
>   (devices (list (device
>                    (uses '("/dev/matrix/root"))
>                    (creates '("/"))
>                    (init-function ...)
>                    (destroy-function ...))
>                  (swap-device
>                    (uses '("/dev/mapper/matrix-swap"))
>                    (creates '()))
>                    (init-function ...)
>                    (destroy-function ...))
>                  (lvm-device
>                    (uses '("/dev/mapper/hdd" "/dev/sdb"))
>                    (creates '("/dev/matrix/"
>                               "/dev/mapper/matrix-swap"))
>                    (init-function ...)
>                    (destroy-function ...))
>                  (luks-device
>                    (uses '("UUID=4dab5feb-d176-45de-b287-9b0a6e4c01cb"))
>                    (creates '("/dev/mapper/hdd"))
>                    (init-function ...)
>                    (destroy-function ...)))
>
> Though this is a bit of a dream in my case.

Hmm it seems to me that these are roughly to different ways to write the
same thing (with the 2nd one making dependencies explicit.)

I’m not sure there’s an intermediate representation that file systems,
swap devices, LVM devices, etc. could all be “compiled” to.  I feel that
we should stick to the abstractions of the Linux kernel, where device
mapping is entirely different from file systems, and so on.

However, we must definitely unify device naming (the /dev vs. UUID
vs. label thing.)

> There also a small issue where mapped devices need more data in special cases,
> like LUKS keyfiles which returns a function to create the type.

What?  :-)

Thanks for your insightful comments!

Ludo’.

Re: [PATCH] DISCUSSION: Jookia's Libreboot+LUKS+LVM FDE patch.

[Jookia]

On Tue, Mar 15, 2016 at 03:40:46PM +0100, Ludovic Courtès wrote:
> Would a ‘mapped-device’ type where both ‘source’ and ‘target’ are lists
> adequately model Linux’s notion of mapped devices?

I don't know, the problem is things like LVM create multiple mapped devices
rather than just one. So it doesn't model the notion of mapped devices, but
systems where multiple devices.

> Keeping thing purely declarative, with high-level data structures such
> as ‘file-system’ and ‘mapped-device’ is pretty nice IMO.  It allows
> users to easily inspect the config, map over the various bits, etc.

This is true, but 'mapped-device' isn't a good abstraction I don't think since
we're not often trying to map devices but instead use tools that automatically
map them for us and take arguments, such as a key-file. In my patch you can see
I have to make the mapped-device type field call a function to generate a type
based on some LUKS-specific input. Perhaps mapped-device needs to be rethought?

> Note that it already handles bind mounts and other pseudo file systems
> (see (gnu system file-systems)).  Basically, ‘file-system’ directly
> corresponds to the ‘mount’ system call.

It does, but I don't think it allows multiple devices in the dependency graph.

> Hmm it seems to me that these are roughly to different ways to write the
> same thing (with the 2nd one making dependencies explicit.)

Oh, I typo'd a bit. In the second one device/swap-device/lvm-device/luks-device
should all just be 'device', since it's unified.

> I’m not sure there’s an intermediate representation that file systems,
> swap devices, LVM devices, etc. could all be “compiled” to.  I feel that
> we should stick to the abstractions of the Linux kernel, where device
> mapping is entirely different from file systems, and so on.

I don't think this is compiling to a representation of the devices themselves,
but more a dependency graph. But yeah, you're probably right on that. I thought
about it a bit above, but perhaps mapped-device needs to be expanded from just
'type' to things like lvm-mapped-device where we can have some more control.
Mirroring this, most mapped devices aren't directly called with something like
'mount', but instead userspace utilities.

> However, we must definitely unify device naming (the /dev vs. UUID
> vs. label thing.)

Yes, definitely.

> What?  :-)

I mentioned it above, but mapped-device doesn't support per-type arguments. Like
specifying a key-file for LUKS or volume group for LVM. mapped-device also
doesn't allow specifying output devices so we can't really handle dependencies
between them properly either.

> Thanks for your insightful comments!

Yours too, I hope we figure out something that solves both our issues. :)

> Ludo’.

Jookia.