installation on LVM on LUKS

installation on LVM on LUKS

[Emmanuel Beffara]

[-- Attachment #1: Type: text/plain, Size: 1759 bytes --]

Hello,

I am currently trying to install Guix System on my laptop and I am facing an
issue with the bootloader configuration.

I use full-disk encryption with a single encrypted partition, split into
several logical volumes using LVM, plus an extra non-encrypted partition for
EFI boot material:

	nvme0n1              259:0    0 953,9G  0 disk
	├─nvme0n1p1          259:1    0 953,4G  0 part
	│ └─manivelle        254:0    0 953,4G  0 crypt
	│   ├─storage-swap   254:1    0    32G  0 lvm   [SWAP]
	│   │ [...]
	│   └─storage-guix   254:5    0    64G  0 lvm   /
	└─nvme0n1p2          259:2    0   487M  0 part  /boot

I attach the system configuration, which I derived from the desktop template.

Everything installed fine EXCEPT that Grub fails to load its LVM volume, hence
the root partition is not found. Indeed, the produced grub.cfg has no mention
of lvm anywhere. If I adjust it by inserting "insmod lvm" somewhere, either at
the start or in a menuentry, or if I do that by hand in the Grub shell, then
the system starts and works fine.

Did I miss something in the configuration ?


As a related point, this setup requires entering the decryption password
twice: once so that Grub can load the kernel from the store, and once so that
the kernel can open the volume itself. I understand the situation is known,
but it could be avoided, for instance, by copying the kernel and initrd
somewhere in the EFI partition so that they could be loaded directly. Besides,
for some reason, Grub is extremely slow at opening the partition: it takes
around 30 seconds to start after I correctly enter the password, whereas the
kernel boots in just a few seconds after I enter the password for it.

Any thoughts on this ?

-- 
Emmanuel

[-- Attachment #2: config.scm --]
[-- Type: text/plain, Size: 2290 bytes --]

(use-modules (gnu) (gnu system nss) (guix utils))
(use-service-modules desktop sddm xorg)
(use-package-modules certs gnome)

(operating-system
  (host-name "manivelle")
  (timezone "Europe/Paris")
  (locale "fr_FR.utf8")

  (keyboard-layout (keyboard-layout "fr"))

  (bootloader (bootloader-configuration
                (bootloader grub-efi-bootloader)
                (targets '("/boot"))
                (keyboard-layout keyboard-layout)
                (terminal-outputs '(vga_text))
                ))

  (mapped-devices
   (list (mapped-device
          (source (uuid "8c8e1490-378f-4e1c-b4c6-25f44070afce"))
          (target "storage")
          (type luks-device-mapping))
         (mapped-device
          (source "storage")
          (targets (list "storage-guix" "storage-swap"))
          (type lvm-device-mapping))))

  (file-systems (append
                 (list (file-system
                         (device (file-system-label "guix"))
                         (mount-point "/")
                         (type "ext4")
                         (dependencies mapped-devices))
                       (file-system
                         (device (file-system-label "boot"))
                         (mount-point "/boot")
                         (type "vfat")))
                 %base-file-systems))

  (swap-devices (list (swap-space
                       (target (file-system-label "swap"))
                       (dependencies mapped-devices))))

  (users (cons (user-account
                (name "manu")
                (comment "Manu")
                (home-directory "/home/manu")
                (group "users")
                (supplementary-groups '("wheel" "netdev"
                                        "audio" "video")))
               %base-user-accounts))

  (packages (append (list
                     nss-certs
                     gvfs)
                    %base-packages))

  (services (append (list (service gnome-desktop-service-type)
                          (service xfce-desktop-service-type)
                          (set-xorg-configuration
                            (xorg-configuration
                              (keyboard-layout keyboard-layout))))
                    %desktop-services))

  (name-service-switch %mdns-host-lookup-nss))

Re: installation on LVM on LUKS

[Raffael Mancini]

Hi Emmanuel,

You might want to look into adding missing modules to the initramfs (https://guix.gnu.org/manual/en/html_node/Initial-RAM-Disk.html). I had to do this so I could boot a system with Linux MD RAID.

Best regards,

Raffael 

> Emmanuel Beffara <manu@beffara.org> hat am 02.03.2023 11:25 CET geschrieben:
> 
>  
> Hello,
> 
> I am currently trying to install Guix System on my laptop and I am facing an
> issue with the bootloader configuration.
> 
> I use full-disk encryption with a single encrypted partition, split into
> several logical volumes using LVM, plus an extra non-encrypted partition for
> EFI boot material:
> 
> 	nvme0n1              259:0    0 953,9G  0 disk
> 	├─nvme0n1p1          259:1    0 953,4G  0 part
> 	│ └─manivelle        254:0    0 953,4G  0 crypt
> 	│   ├─storage-swap   254:1    0    32G  0 lvm   [SWAP]
> 	│   │ [...]
> 	│   └─storage-guix   254:5    0    64G  0 lvm   /
> 	└─nvme0n1p2          259:2    0   487M  0 part  /boot
> 
> I attach the system configuration, which I derived from the desktop template.
> 
> Everything installed fine EXCEPT that Grub fails to load its LVM volume, hence
> the root partition is not found. Indeed, the produced grub.cfg has no mention
> of lvm anywhere. If I adjust it by inserting "insmod lvm" somewhere, either at
> the start or in a menuentry, or if I do that by hand in the Grub shell, then
> the system starts and works fine.
> 
> Did I miss something in the configuration ?
> 
> 
> As a related point, this setup requires entering the decryption password
> twice: once so that Grub can load the kernel from the store, and once so that
> the kernel can open the volume itself. I understand the situation is known,
> but it could be avoided, for instance, by copying the kernel and initrd
> somewhere in the EFI partition so that they could be loaded directly. Besides,
> for some reason, Grub is extremely slow at opening the partition: it takes
> around 30 seconds to start after I correctly enter the password, whereas the
> kernel boots in just a few seconds after I enter the password for it.
> 
> Any thoughts on this ?
> 
> -- 
> Emmanuel

Re: installation on LVM on LUKS

[Roman Scherer]

[-- Attachment #1: Type: text/plain, Size: 2577 bytes --]


Hi Emmanuel,

did you add the cryptsetup-static and lvm2-static packages to the
packages field of your operating system?

Apart from that, I think you also need to add the dm-crypt module to the
initrd-modules field of the of the operating system.

I'm not sure about your other question, but from what I understand the
reason why the kernel and the initrd live in the store and not in the
EFI partition might be that you actually would need to put the kernel
and the initrd for *each* system generation onto the EFI partition, so
you can boot different system generations. And that would fill up the
EFI partition pretty quickly.

I hope that helps.

Roman

Emmanuel Beffara <manu@beffara.org> writes:

> Hello,
>
> I am currently trying to install Guix System on my laptop and I am facing an
> issue with the bootloader configuration.
>
> I use full-disk encryption with a single encrypted partition, split into
> several logical volumes using LVM, plus an extra non-encrypted partition for
> EFI boot material:
>
> 	nvme0n1              259:0    0 953,9G  0 disk
> 	├─nvme0n1p1          259:1    0 953,4G  0 part
> 	│ └─manivelle        254:0    0 953,4G  0 crypt
> 	│   ├─storage-swap   254:1    0    32G  0 lvm   [SWAP]
> 	│   │ [...]
> 	│   └─storage-guix   254:5    0    64G  0 lvm   /
> 	└─nvme0n1p2          259:2    0   487M  0 part  /boot
>
> I attach the system configuration, which I derived from the desktop template.
>
> Everything installed fine EXCEPT that Grub fails to load its LVM volume, hence
> the root partition is not found. Indeed, the produced grub.cfg has no mention
> of lvm anywhere. If I adjust it by inserting "insmod lvm" somewhere, either at
> the start or in a menuentry, or if I do that by hand in the Grub shell, then
> the system starts and works fine.
>
> Did I miss something in the configuration ?
>
>
> As a related point, this setup requires entering the decryption password
> twice: once so that Grub can load the kernel from the store, and once so that
> the kernel can open the volume itself. I understand the situation is known,
> but it could be avoided, for instance, by copying the kernel and initrd
> somewhere in the EFI partition so that they could be loaded directly. Besides,
> for some reason, Grub is extremely slow at opening the partition: it takes
> around 30 seconds to start after I correctly enter the password, whereas the
> kernel boots in just a few seconds after I enter the password for it.
>
> Any thoughts on this ?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 528 bytes --]

Re: installation on LVM on LUKS

[Emmanuel Beffara]

Hi Roman,

Thanks for the suggestions.

De Roman Scherer le 03/03/2023 à 16:05:
> did you add the cryptsetup-static and lvm2-static packages to the
> packages field of your operating system?

I had not, but I just tried adding them and nothing changed.

> Apart from that, I think you also need to add the dm-crypt module to the
> initrd-modules field of the of the operating system.

Unless I am missing something, tinkering with initrd modules has nothing to do
with my issue. The missing “insmod lvm” is in grub.cfg, it is related to Grub
modules, not kernel modules. The required modules for Grub are properly
installed in /boot/grub (I mount the EFI partition as /boot), it is just that
the generated configuration file does not load enough of them.

> I'm not sure about your other question, but from what I understand the
> reason why the kernel and the initrd live in the store and not in the
> EFI partition might be that you actually would need to put the kernel
> and the initrd for *each* system generation onto the EFI partition, so
> you can boot different system generations. And that would fill up the
> EFI partition pretty quickly.

Indeed, it would require some space, but it would solve the double-passphrase
issue, among other things.

Besides, storing kernels and initrds in the EFI boot partition is how NixOS
proceeds on my system (although it is set up to use systemd-boot and not Grub,
in case it makes a difference). Filling up the EFI partition has never been a
problem in a few years of use, because the partition is large enough to hold a
few generations (512Mib) and I drop old generations often (as soon as the last
one is checked to be functional, essentially).

-- 
Emmanuel

Re: installation on LVM on LUKS

[wolf]

[-- Attachment #1: Type: text/plain, Size: 2760 bytes --]

On 2023-03-03 18:03:39 +0100, Emmanuel Beffara wrote:
> Hi Roman,
> 
> Thanks for the suggestions.
> 
> De Roman Scherer le 03/03/2023 à 16:05:
> > did you add the cryptsetup-static and lvm2-static packages to the
> > packages field of your operating system?
> 
> I had not, but I just tried adding them and nothing changed.
> 
> > Apart from that, I think you also need to add the dm-crypt module to the
> > initrd-modules field of the of the operating system.
> 
> Unless I am missing something, tinkering with initrd modules has nothing to do
> with my issue. The missing “insmod lvm” is in grub.cfg, it is related to Grub
> modules, not kernel modules. The required modules for Grub are properly
> installed in /boot/grub (I mount the EFI partition as /boot), it is just that
> the generated configuration file does not load enough of them.

Maybe that is the problem? For me it works out of the box, but I have EFI
mounted as /boot/efi. Could you maybe either try to do that as well, or
(untested idea I just had) provide (dependencies mapped-devices) for the /boot
mount point as well (I know, it is not technically required)?

> 
> > I'm not sure about your other question, but from what I understand the
> > reason why the kernel and the initrd live in the store and not in the
> > EFI partition might be that you actually would need to put the kernel
> > and the initrd for *each* system generation onto the EFI partition, so
> > you can boot different system generations. And that would fill up the
> > EFI partition pretty quickly.
> 
> Indeed, it would require some space, but it would solve the double-passphrase
> issue, among other things.
> 
> Besides, storing kernels and initrds in the EFI boot partition is how NixOS
> proceeds on my system (although it is set up to use systemd-boot and not Grub,
> in case it makes a difference). Filling up the EFI partition has never been a
> problem in a few years of use, because the partition is large enough to hold a
> few generations (512Mib) and I drop old generations often (as soon as the last
> one is checked to be functional, essentially).

I am using 64M EFI partition, so I could imagine that filling up, especially if
one does not clean up the generations very often. Since increasing the EFI size
could be impossible without reinstall, if this is done, it should likely be
opt-in only.

In the future I plan to look into eliminating the second password prompt using a
key file, following similar approach other distributions are using (for example
cryptkey= argument on alpine).

> 
> -- 
> Emmanuel
> 

W.

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: installation on LVM on LUKS

[Emmanuel Beffara]

De wolf le 04/03/2023 à 00:42:
> On 2023-03-03 18:03:39 +0100, Emmanuel Beffara wrote:
> > Unless I am missing something, tinkering with initrd modules has nothing to do
> > with my issue. The missing “insmod lvm” is in grub.cfg, it is related to Grub
> > modules, not kernel modules. The required modules for Grub are properly
> > installed in /boot/grub (I mount the EFI partition as /boot), it is just that
> > the generated configuration file does not load enough of them.
> 
> Maybe that is the problem? For me it works out of the box, but I have EFI
> mounted as /boot/efi. Could you maybe either try to do that as well,

I just tried that: mounting the EFI partition in /boot/efi instead of /boot
and adujsting the config.scm accordingly (the entry for /boot/efi in
file-systems and the target for grub). I also cleaned everything there that
was installed by Guix (the Guix folder, the grub folder). Turns out that this
way, the system just cannot start. In the EFI partition, there is
EFI/Guix/grubx64.efi but running this leads to an error:

	error: disk 'lvmid/(the UUID of the root fs)' not found.
	Entering rescue mode...

I cannot do much in this recue mode. Of course Grub cannot find the partition
since its modules for LUKS and LVM are now stored in the system's root
partition under /boot, encrypted…

I'm surprised that it works for you out of the box given what I observe. Where
are Grub's modules stored ? Is the installed EFI binary somehow able to
decrypt the partition and its LVM contents without loading modules for that ?

> or (untested idea I just had) provide (dependencies mapped-devices) for the
> /boot mount point as well (I know, it is not technically required)?

I also tried that and it does not change anything.

> I am using 64M EFI partition, so I could imagine that filling up, especially if
> one does not clean up the generations very often. Since increasing the EFI size
> could be impossible without reinstall, if this is done, it should likely be
> opt-in only.

Certainly.

-- 
Emmanuel