bug#70826: luks-device-mapping-with-options breaks bootloader

bug#70826: luks-device-mapping-with-options breaks bootloader

[Tadhg McDonald-Jensen]

[-- Attachment #1: Type: text/plain, Size: 827 bytes --]

using the `luks-device-mapping-with-options` mapped device type defined in
(gnu system mapped-devices) causes grub or other bootloaders to not
properly attempt to mount the encrypted drive. This is caused by the
commit 39a9404 which identifies luks mapped devices by checking if the type
is equal to `luks-device-mapping`, so by using a different routine that is
a proxy to that one it doesn't forward it to grub in the
store-crypto-devices list.

For anyone who finds this before it is fixed, you can boot your device by
hitting 'c' in grub and typing these commands:
grub> insmod luks
grub> insmod luks2
grub> cryptomount (XXX)
grub> set root=(crypto)
grub> configfile (YYY)/grub/grub.cfg

Where (XXX) is the encrypted partition and (YYY) is the boot partition with
the grub config, these can be found by doing `ls` command.

[-- Attachment #2: Type: text/html, Size: 976 bytes --]

bug#70826: luks-device-mapping-with-options breaks bootloader

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 561 bytes --]

Hi,

Tadhg McDonald-Jensen <tadhgmister@gmail.com> skribis:

> using the `luks-device-mapping-with-options` mapped device type defined in
> (gnu system mapped-devices) causes grub or other bootloaders to not
> properly attempt to mount the encrypted drive. This is caused by the
> commit 39a9404 which identifies luks mapped devices by checking if the type
> is equal to `luks-device-mapping`, so by using a different routine that is
> a proxy to that one it doesn't forward it to grub in the
> store-crypto-devices list.

Ouch, indeed.  The immediate fix is:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1058 bytes --]

diff --git a/gnu/system.scm b/gnu/system.scm
index c76f4d7c502..bb851b1b75f 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -667,10 +667,13 @@ (define (operating-system-boot-mapped-devices os)
 (define operating-system-bootloader-crypto-devices
   (mlambdaq (os)                        ;to avoid duplicated output
     "Return the sources of the LUKS mapped devices specified by UUID."
+    (define (luks-device? m)
+      (memq (mapped-device-type m)
+            (list luks-device-mapping-with-options
+                  luks-device-mapping)))
+
     ;; XXX: Device ordering is important, we trust the returned one.
-    (let* ((luks-devices (filter (lambda (m)
-                                   (eq? luks-device-mapping
-                                        (mapped-device-type m)))
+    (let* ((luks-devices (filter luks-device?
                                  (operating-system-boot-mapped-devices os)))
            (uuid-crypto-devices non-uuid-crypto-devices
                                 (partition (compose uuid? mapped-device-source)

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]


Not ideal, but it fixes the problem.

I’ll go ahead with this patch if there are no objections.

Thanks!

Ludo’.

bug#70826: luks-device-mapping-with-options breaks bootloader

[Tadhg McDonald-Jensen]

That unfortunately doesn't fix the problem, 
`luks-device-mapping-with-options` is a routine that returns the 
`mapped-device-kind` so it won't check by equality.

A possible solution is to check whether the `mapped-device-kind-close` 
routines are the same as these are shared.


diff --git a/gnu/system.scm b/gnu/system.scm
index cb6e719ca6..b564bf3788 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -661,10 +661,12 @@ (define (operating-system-boot-mapped-devices os)
  (define operating-system-bootloader-crypto-devices
    (mlambdaq (os)                        ;to avoid duplicated output
      "Return the sources of the LUKS mapped devices specified by UUID."
+    (define (luks-device? m)
+      (eq? (mapped-device-kind-close (mapped-device-type m))
+           (mapped-device-kind-close luks-device-mapping)))
+
      ;; XXX: Device ordering is important, we trust the returned one.
-    (let* ((luks-devices (filter (lambda (m)
-                                   (eq? luks-device-mapping
-                                        (mapped-device-type m)))
+    (let* ((luks-devices (filter luks-device?
                                   (operating-system-boot-mapped-devices 
os)))
             (uuid-crypto-devices non-uuid-crypto-devices
                                  (partition (compose uuid? 
mapped-device-source)



(I apologize if my email client is adding line wraps to the diffs, I 
will look into it after sending this)

I tried to implement this initially but it didn't work on my previous 
attempt so I abandoned trying to submit a patch, but this version does 
do the trick even if it seems inelegant.

On 2024-05-25 5:47 a.m., Ludovic Courtès wrote:
> Hi,
> 
> Tadhg McDonald-Jensen <tadhgmister@gmail.com> skribis:
> 
>> using the `luks-device-mapping-with-options` mapped device type defined in
>> (gnu system mapped-devices) causes grub or other bootloaders to not
>> properly attempt to mount the encrypted drive. This is caused by the
>> commit 39a9404 which identifies luks mapped devices by checking if the type
>> is equal to `luks-device-mapping`, so by using a different routine that is
>> a proxy to that one it doesn't forward it to grub in the
>> store-crypto-devices list.
> 
> Ouch, indeed.  The immediate fix is:
> 
> 
> diff --git a/gnu/system.scm b/gnu/system.scm
> index c76f4d7c502..bb851b1b75f 100644
> --- a/gnu/system.scm
> +++ b/gnu/system.scm
> @@ -667,10 +667,13 @@ (define (operating-system-boot-mapped-devices os)
>   (define operating-system-bootloader-crypto-devices
>     (mlambdaq (os)                        ;to avoid duplicated output
>       "Return the sources of the LUKS mapped devices specified by UUID."
> +    (define (luks-device? m)
> +      (memq (mapped-device-type m)
> +            (list luks-device-mapping-with-options
> +                  luks-device-mapping)))
> +
>       ;; XXX: Device ordering is important, we trust the returned one.
> -    (let* ((luks-devices (filter (lambda (m)
> -                                   (eq? luks-device-mapping
> -                                        (mapped-device-type m)))
> +    (let* ((luks-devices (filter luks-device?
>                                    (operating-system-boot-mapped-devices os)))
>              (uuid-crypto-devices non-uuid-crypto-devices
>                                   (partition (compose uuid? mapped-device-source)
> 
> 
> 
> Not ideal, but it fixes the problem.
> 
> I’ll go ahead with this patch if there are no objections.
> 
> Thanks!
> 
> Ludo’.

bug#70826: luks-device-mapping-with-options breaks bootloader

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 862 bytes --]

On 2024-05-25 10:30:49 -0400, Tadhg McDonald-Jensen wrote:
> That unfortunately doesn't fix the problem,
> `luks-device-mapping-with-options` is a routine that returns the
> `mapped-device-kind` so it won't check by equality.
>
> A possible solution is to check whether the `mapped-device-kind-close`
> routines are the same as these are shared.

What I find interesting is that I too am using luks-device-mapping-with-options
and my system boots just fine.  So I wonder what the difference is.  Could you
share your system configuration please?  Or at least the relevant parts (I
assume at least bootloader, file-systems and mapped-devices fields)?

I would like to properly understand the problem here and why it works for me.

Thanks,
Tomas Volf

--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#70826: luks-device-mapping-with-options breaks bootloader

[Tadhg McDonald-Jensen]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]

I have attached a config I just did `sudo guix system reconfigure`
and confirmed it was missing the `insmod luks` in /boot/grub/grub.cfg

Sorry for the delay,
Tadhg McD-J

On 2024-07-23 2:19 p.m., Tomas Volf wrote:
> On 2024-05-25 10:30:49 -0400, Tadhg McDonald-Jensen wrote:
>> That unfortunately doesn't fix the problem,
>> `luks-device-mapping-with-options` is a routine that returns the
>> `mapped-device-kind` so it won't check by equality.
>>
>> A possible solution is to check whether the `mapped-device-kind-close`
>> routines are the same as these are shared.
> 
> What I find interesting is that I too am using luks-device-mapping-with-options
> and my system boots just fine.  So I wonder what the difference is.  Could you
> share your system configuration please?  Or at least the relevant parts (I
> assume at least bootloader, file-systems and mapped-devices fields)?
> 
> I would like to properly understand the problem here and why it works for me.
> 
> Thanks,
> Tomas Volf
> 
> --
> There are only two hard things in Computer Science:
> cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: os.tmp.scm --]
[-- Type: text/x-scheme, Size: 6192 bytes --]


(use-modules
 (gnu)
 ((guix packages) #:select (origin base32 modify-inputs package-source package-inputs package))
 ((guix download) #:select (url-fetch))
 ((guix gexp) #:select(file-append))
 ((gnu packages freedesktop) #:select(fprintd))
 ((gnu packages suckless) #:select(slock))
 ((gnu packages games) #:select (steam-devices-udev-rules))
 ((gnu packages linux) #:select (brightnessctl))
 ((gnu packages wm) #:select (swaylock))
 ((gnu packages cups) #:select (cups cups-filters epson-inkjet-printer-escpr hplip-minimal))
 ((gnu services cups) #:select (cups-service-type cups-configuration))
 ((gnu services nfs) #:select (nfs-service-type nfs-configuration))
 ((gnu services desktop) #:select (sane-service-type bluetooth-service-type %desktop-services elogind-service-type elogind-configuration))
 ;;((gnu services docker) #:select(docker-service-type))
 ((gnu services virtualization) #:select(qemu-binfmt-service-type qemu-binfmt-configuration lookup-qemu-platforms libvirt-service-type))
 ((gnu services nix) #:select (nix-service-type))
 ((gnu services networking) #:select (ipfs-service-type ipfs-configuration))
 ((gnu services syncthing) #:select (syncthing-service-type syncthing-configuration))
 ((gnu services sound) #:select (pulseaudio-service-type pulseaudio-configuration))
 ((gnu services audio) #:select (mpd-service-type mpd-configuration))
 ((gnu services xorg) #:select (xorg-server-service-type gdm-service-type screen-locker-service screen-locker-service-type xorg-configuration set-xorg-configuration))
 ;;((gnu services authentication) #:select (fprintd-service-type))
 ((gnu services file-sharing) #:select (transmission-daemon-service-type transmission-daemon-configuration))
 ((gnu services pm) #:select (tlp-service-type tlp-configuration thermald-service-type))
 )

(define username "tadhg")
;; commit 39a9404 in guix broke this, a function in the os checks for equality with luks-device-mapping as the type and only puts the
;; needed commands into grub.cfg if it identifies it that way, so this makes grub just not try to mount the encrypted device which
;; obviously causes it to fail. I will need to submit a bug report and get it properly fixed but for now I will just need to
;; continue to type my decryption password twice.
(define cryptroot-type (luks-device-mapping-with-options
				 ;; NOTE: when specified as a string this is a path relative to the initrd internal filesystem
				 ;; which is populated by the cpio file passed as 'extra-initrd' to grub.
				 ;; if it was (local-file "/crypto_keyfile.bin") it would copy the file on the local filesystem
				 ;; to the initrd, but it would also put a copy of it in the guix store which is globally readable
				 ;; (it'd also be readable from the initrd which is also in the guix store so even if it
				 ;;   wasn't copied in there'd be a problem)
				 ;; if this file ever needs to be recaptured use the command `cpio -i /crypto_keyfile.bin < /crypto_keyfile.cpio` run as root and it will restore this file to the root directory.
				 #:key-file "/crypto_keyfile.bin"))
(operating-system
  (locale "en_CA.utf8")
  (timezone "America/Toronto")
  (keyboard-layout (keyboard-layout "us"))
  (host-name "framework")

  ;; The list of user accounts ('root' is implicit).
  (users (cons*
	  (user-account
                  (name username)
                  (comment "Tadhg McDonald-Jensen")
                  (group "users")
                  (home-directory "/home/tadhg")
                  (supplementary-groups '("wheel" ;; for sudo access
					  "netdev" ;; TODO: what is this for?
					  "audio" ;; to be able to use alsamixer etc
					  "video"  ;; think this is to control brightness
					  "scanner" ;; for scanning
					  "input" ;; to control caps lock light
					  )))
                %base-user-accounts))
  (bootloader (bootloader-configuration
                (bootloader grub-efi-bootloader)
                (targets (list "/boot"))
                (keyboard-layout keyboard-layout)
		(extra-initrd "/crypto_keyfile.cpio")
		))
  (mapped-devices (list (mapped-device
                          (source (uuid
                                   "c0010d06-0bd1-4ae2-93e6-f2f89a3a670b"))
                          (target "cryptroot")
			  (type cryptroot-type))))
			  ;;(type luks-device-mapping))))
  
  (swap-devices (list (swap-space
                       (target "/swapfile")
		       ;; TODO: see example about btrfs mounting in docs about swap, just depending on mapped-devices isn't sufficient to guarentee the root partition is mounted.
		       (dependencies mapped-devices))))

  ;; The list of file systems that get "mounted".  The unique
  ;; file system identifiers there ("UUIDs") can be obtained
  ;; by running 'blkid' in a terminal.
  (file-systems (cons* (file-system
                         (mount-point "/boot")
                         (device (uuid "5190-E840" 'fat32))
                         (type "vfat"))
                       (file-system
                         (mount-point "/")
                         (device "/dev/mapper/cryptroot")
                         (type "btrfs")
                         (flags '(lazy-time))
                         (options
                          (alist->file-system-options
                           '(("compress" . "lzo"))))
                         (dependencies mapped-devices)) 
                         %base-file-systems))
  (packages (append
	     (list) ;;os-packages
             %base-packages))

  ;; Below is the list of system services.  To search for available
  ;; services, run 'guix system search KEYWORD' in a terminal.
  (services
   (cons*
    
    (service xorg-server-service-type) ;; needed for display (kind of important)
    
    (modify-services
        %desktop-services
	;;(guix-service-type config => (tadhg:substitutes config))
	
	(elogind-service-type
	 config =>
	 (elogind-configuration
          (inherit config)
	  (handle-power-key 'hibernate)
	  ;;(idle-action 'suspend)
          ;;(handle-lid-switch 'ignore)
	  ))
	(delete gdm-service-type)
	)))
  ;; allow using .local with mdns resolution, used for printer in particular
  (name-service-switch %mdns-host-lookup-nss)
  )

bug#70826: luks-device-mapping-with-options breaks bootloader

[Tadhg McDonald-Jensen]

[-- Attachment #1: Type: text/plain, Size: 1896 bytes --]

In case it is relevant my disk is using GPT partition table with this
layout:

$ lsblk --output="NAME,MAJ:MIN,TYPE,MOUNTPOINTS,UUID"
NAME MAJ:MIN TYPE MOUNTPOINTS UUID
nvme0n1 259:0 disk
├─nvme0n1p1 259:1 part /boot 5190-E840
└─nvme0n1p2 259:2 part c0010d06-0bd1-4ae2-93e6-f2f89a3a670b
└─cryptroot 253:0 crypt /gnu/store
/

Only the main partition is encrypted with LUKS and grub is located on
its own partition not in the in between space in an MBR drive.

It is grub that is being responsible for decrypting the partition
not my UEFI decrypting the whole drive.

Tadhg

On Sun, Aug 11, 2024 at 6:33 PM Tadhg McDonald-Jensen <tadhgmister@gmail.com>
wrote:

> I have attached a config I just did `sudo guix system reconfigure`
> and confirmed it was missing the `insmod luks` in /boot/grub/grub.cfg
>
> Sorry for the delay,
> Tadhg McD-J
>
> On 2024-07-23 2:19 p.m., Tomas Volf wrote:
> > On 2024-05-25 10:30:49 -0400, Tadhg McDonald-Jensen wrote:
> >> That unfortunately doesn't fix the problem,
> >> `luks-device-mapping-with-options` is a routine that returns the
> >> `mapped-device-kind` so it won't check by equality.
> >>
> >> A possible solution is to check whether the `mapped-device-kind-close`
> >> routines are the same as these are shared.
> >
> > What I find interesting is that I too am using
> luks-device-mapping-with-options
> > and my system boots just fine.  So I wonder what the difference is.
> Could you
> > share your system configuration please?  Or at least the relevant parts
> (I
> > assume at least bootloader, file-systems and mapped-devices fields)?
> >
> > I would like to properly understand the problem here and why it works
> for me.
> >
> > Thanks,
> > Tomas Volf
> >
> > --
> > There are only two hard things in Computer Science:
> > cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: Type: text/html, Size: 2432 bytes --]