How to use custom grub?

How to use custom grub?

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 936 bytes --]

Greetings Guix,

I have a working UEFI GuixSD system, by adding FAT32 support to
base-initrd and installing the initial UEFI bootloader from a non-Guix
live CD.

Now that it's booted, I want to use a custom 'grub-efi' package so that
`guix system reconfigure` works without passing --no-grub and manually
copying grub.cfg.

The "grub-configuration" object takes a 'grub' argument which defaults
to (@ (gnu packages grub) grub). However setting that to 'grub-efi' in
the operating-system declaration or even directly in gnu/system/grub.scm
has no effect.

Looking at guix/scripts/system.scm, grub is defined as
(package->derivation grub). I tried exporting grub-configuration-grub
and using that in the (package->derivation) procedure, but that broke
the world :)

What is the correct approach here?

Additionally the "grub-install" arguments are slightly different for EFI
systems. Tips for properly handling that appreciated.

Thanks!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]

Re: How to use custom grub?

[Danny Milosavljevic]

Hi Marius,

yeah, I've noticed that problem, too.

David and I got some steps in the direction of fixing it (for U-Boot support):

The U-Boot support patch generalized this, see my E-Mail to this list, subject "[WIP PATCH] gnu: add U-Boot support to operating-system" on 09 Sep 2016 at 20:09.

The relevant part is:
- guix/scripts/system.scm: The "install" routine is changed to also accept the bootloader package as argument (variable is called "grub" to keep the number of trivial patch chunks low).
- guix/scripts/system.scm: The new "install-bootloader*" routine is introduced instead of "install-grub*" to also accept the bootloader package as argument.

If you want you can cherry-pick these changes. I've reconfigured and booted with that patch. It works.

I'm not sure how the build side picks up which package's grub-install is invoked (gnu/build/install.scm just invokes "grub-install" without any further path specification).

I've asked about it before (multiple times now) and apparently nobody knows. If someone finds out, gnu/build/install.scm install-grub after the patch accepts the grub-name and could just invoke grub-install correctly. I *think* it's fine to just (string-append grub-name "/bin/grub-install") or something - but that's something I don't want to guess.

David knows more about the VM side - the patch also addresses this by carring around the bootloader als variable "grub" (which is not necessarily actually grub).

I hope that helps.

Re: How to use custom grub?

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1219 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> Hi Marius,
>
> yeah, I've noticed that problem, too.
>
> David and I got some steps in the direction of fixing it (for U-Boot support):
>
> The U-Boot support patch generalized this, see my E-Mail to this list, subject "[WIP PATCH] gnu: add U-Boot support to operating-system" on 09 Sep 2016 at 20:09.
>
> The relevant part is:
> - guix/scripts/system.scm: The "install" routine is changed to also accept the bootloader package as argument (variable is called "grub" to keep the number of trivial patch chunks low).
> - guix/scripts/system.scm: The new "install-bootloader*" routine is introduced instead of "install-grub*" to also accept the bootloader package as argument.
>
> If you want you can cherry-pick these changes. I've reconfigured and booted with that patch. It works.

Thanks a lot for the fast reply! That patch was really helpful.

I'll try to extract the parts that are relevant for grub-efi support
while keeping other bootloaders in mind. Hoping to largely piggy-back on
the regular grub stuff for grub-efi, as it only needs two extra lines in
grub.cfg and adding "--bootloader-id=GNU" "--efi-directory=<boot-dir>"
to the grub-install command.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]

Re: How to use custom grub?

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1219 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> Hi Marius,
>
> yeah, I've noticed that problem, too.
>
> David and I got some steps in the direction of fixing it (for U-Boot support):
>
> The U-Boot support patch generalized this, see my E-Mail to this list, subject "[WIP PATCH] gnu: add U-Boot support to operating-system" on 09 Sep 2016 at 20:09.
>
> The relevant part is:
> - guix/scripts/system.scm: The "install" routine is changed to also accept the bootloader package as argument (variable is called "grub" to keep the number of trivial patch chunks low).
> - guix/scripts/system.scm: The new "install-bootloader*" routine is introduced instead of "install-grub*" to also accept the bootloader package as argument.
>
> If you want you can cherry-pick these changes. I've reconfigured and booted with that patch. It works.

Thanks a lot for the fast reply! That patch was really helpful.

I'll try to extract the parts that are relevant for grub-efi support
while keeping other bootloaders in mind. Hoping to largely piggy-back on
the regular grub stuff for grub-efi, as it only needs two extra lines in
grub.cfg and adding "--bootloader-id=GNU" "--efi-directory=<boot-dir>"
to the grub-install command.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]

Re: How to use custom grub?

[Ludovic Courtès]

Hello!

Marius Bakke <mbakke@fastmail.com> skribis:

> I have a working UEFI GuixSD system, by adding FAT32 support to
> base-initrd and installing the initial UEFI bootloader from a non-Guix
> live CD.

Cool!  We should integrate the necessary changes in ‘master’.

> The "grub-configuration" object takes a 'grub' argument which defaults
> to (@ (gnu packages grub) grub). However setting that to 'grub-efi' in
> the operating-system declaration or even directly in gnu/system/grub.scm
> has no effect.

Ideally it should have the desired effect.  :-)

> Looking at guix/scripts/system.scm, grub is defined as
> (package->derivation grub). I tried exporting grub-configuration-grub
> and using that in the (package->derivation) procedure, but that broke
> the world :)
>
> What is the correct approach here?

What you describe above looks like the right approach.  Maybe we could
see exactly what broke and work from there, unless Danny’s patch already
solves the issue?

Thanks,
Ludo’.

Re: How to use custom grub?

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 2534 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

>> Looking at guix/scripts/system.scm, grub is defined as
>> (package->derivation grub). I tried exporting grub-configuration-grub
>> and using that in the (package->derivation) procedure, but that broke
>> the world :)
>>
>> What is the correct approach here?
>
> What you describe above looks like the right approach.  Maybe we could
> see exactly what broke and work from there, unless Danny’s patch already
> solves the issue?

I haven't gotten around to really try Dannys patch, but here is the
backtrace when exporting grub-configuration-grub in gnu/system/grub.scm
and changing to (package->derivation grub-configuration-grub) in
guix/scripts/system.scm:

Backtrace:
In unknown file:
   ?: 19 [apply-smob/1 #<catch-closure 19426a0>]
In ice-9/boot-9.scm:
  63: 18 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 17 [eval # #]
In ice-9/boot-9.scm:
2401: 16 [save-module-excursion #<procedure 1960940 at ice-9/boot-9.scm:4045:3 ()>]
4050: 15 [#<procedure 1960940 at ice-9/boot-9.scm:4045:3 ()>]
1724: 14 [%start-stack load-stack ...]
1729: 13 [#<procedure 1977ea0 ()>]
In unknown file:
   ?: 12 [primitive-load "/home/guix/scripts/guix"]
In guix/ui.scm:
1220: 11 [run-guix-command system "reconfigure" "/etc/config.scm" "--fallback"]
In ice-9/boot-9.scm:
 157: 10 [catch srfi-34 #<procedure 5963f80 at guix/ui.scm:426:2 ()> ...]
 157: 9 [catch system-error #<procedure 5aeca20 at ice-9/eval.scm:416:20 ()> ...]
In ice-9/eval.scm:
 481: 8 [lp (#<fluid 32>) (#t)]
In ice-9/r4rs.scm:
  90: 7 [dynamic-wind #<procedure 47daa40 at ice-9/boot-9.scm:1133:2 _> ...]
In guix/store.scm:
1182: 6 [run-with-store # #<procedure 4624720 at ice-9/eval.scm:416:20 (a)> # ...]
In ice-9/r4rs.scm:
  39: 5 [call-with-values #<procedure 3be11b0 at ice-9/eval.scm:416:20 ()> ...]
In guix/store.scm:
1105: 4 [#<procedure 3be1240 at guix/store.scm:1104:28 (store)> #]
In guix/packages.scm:
 793: 3 [cache! #<weak-key-hash-table 2104540 846/1759> ...]
1099: 2 [thunk]
 793: 1 [cache! #<weak-key-hash-table 2104240 940/1759> ...]
 878: 0 [thunk]

guix/packages.scm:878:27: In procedure thunk:
guix/packages.scm:878:27: In procedure struct_vtable: Wrong type argument in position 1 (expectingstruct): #<procedure %grub-configuration-grub-procedure (a)>

I don't really understand what's going on in that file, so hopefully the
problem is obvious :)

I will post the other patches for UEFI support shortly, as they are
mostly unrelated to this.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]

Re: How to use custom grub?

[Danny Milosavljevic]

Hi Marius,

On Sat, 05 Nov 2016 12:37:07 +0000
Marius Bakke <mbakke@fastmail.com> wrote:
> I haven't gotten around to really try Dannys patch, but here is the
> backtrace when exporting grub-configuration-grub in gnu/system/grub.scm
> and changing to (package->derivation grub-configuration-grub) in
> guix/scripts/system.scm:

Hmm, did you mean (package->drivation (grub-configuration-grub (operating-system-bootloader os))) ? Otherwise it can't work.

Re: How to use custom grub?

[Danny Milosavljevic]

> Hmm, did you mean (package->drivation (grub-configuration-grub (operating-system-bootloader os))) ? Otherwise it can't work.

Fixing typo:

Hmm, did you mean (package->derivation (grub-configuration-grub (operating-system-bootloader os))) ? Otherwise it can't work.

It gives you a derivation which you can use derivation->output-path (which returns a file path) on and then invoke the executable or whatever.

Re: How to use custom grub?

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1097 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

>> Hmm, did you mean (package->drivation (grub-configuration-grub (operating-system-bootloader os))) ? Otherwise it can't work.
>
> Fixing typo:
>
> Hmm, did you mean (package->derivation (grub-configuration-grub (operating-system-bootloader os))) ? Otherwise it can't work.
>
> It gives you a derivation which you can use derivation->output-path
> (which returns a file path) on and then invoke the executable or
> whatever.

That's it! This makes it pick up (bootloader (grub-configuration (grub
grub-efi))) from my config.scm:

making '/gnu/store/kgk9rrawq9fxh1g2j6121gl3lcz47395-system' the current system...
Installing for x86_64-efi platform.
Installation finished. No error reported.

Even though I'm now working on multi-platform grub, I think we should
have this anyway so that the "grub" argument works as expected. WDYT?

Also, since this works, maybe it's better to have grub-efi as a separate
package, as it will add ~8MiB to the grub closure (the x86_64-efi
platform directory is 6.1MiB; efibootmgr adds another ~2MiB).

Thanks!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]

Re: How to use custom grub?

[Danny Milosavljevic]

Hi Marius,

On Mon, 07 Nov 2016 11:36:51 +0000
Marius Bakke <mbakke@fastmail.com> wrote:

> That's it! This makes it pick up (bootloader (grub-configuration (grub
> grub-efi))) from my config.scm:
> 
> making '/gnu/store/kgk9rrawq9fxh1g2j6121gl3lcz47395-system' the current system...
> Installing for x86_64-efi platform.
> Installation finished. No error reported.
> 
> Even though I'm now working on multi-platform grub, I think we should
> have this anyway so that the "grub" argument works as expected. WDYT?

Yes, I agree that grub should not be magically picked up but rather be read from the configuration - as you do here. This makes alternative bootloaders possible and is also less surprising in any case.

Also in guix/scripts/system.scm in perform-action there's a (setenv "PATH" ...) form. I think that one should be replaced, too - for much the same reasons. It would be better to just pass grub to install-grub* (which would need its parameter list adapted) instead of mucking with PATH :P

In this way the package variable would flow from the os configuration all the way to the actual "grub-install" invocation call without magical environment variables, packages that just happen to be pulled in from some imported module etc.

If you want, you can also fix this one up, too. (If not, I'll wait until your stuff is merged and fix it myself - no worries)

A first test whether it was enough is to remove the #:use-module (gnu packages grub) from guix/scripts/system.scm and see whether it still works (it should). For clarity I would make the final patch remove it, too.

Re: How to use custom grub?

[Marius Bakke]

[-- Attachment #1.1: Type: text/plain, Size: 2248 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> Hi Marius,
>
> On Mon, 07 Nov 2016 11:36:51 +0000
> Marius Bakke <mbakke@fastmail.com> wrote:
>
>> That's it! This makes it pick up (bootloader (grub-configuration (grub
>> grub-efi))) from my config.scm:
>> 
>> making '/gnu/store/kgk9rrawq9fxh1g2j6121gl3lcz47395-system' the current system...
>> Installing for x86_64-efi platform.
>> Installation finished. No error reported.
>> 
>> Even though I'm now working on multi-platform grub, I think we should
>> have this anyway so that the "grub" argument works as expected. WDYT?
>
> Yes, I agree that grub should not be magically picked up but rather be read from the configuration - as you do here. This makes alternative bootloaders possible and is also less surprising in any case.
>
> Also in guix/scripts/system.scm in perform-action there's a (setenv "PATH" ...) form. I think that one should be replaced, too - for much the same reasons. It would be better to just pass grub to install-grub* (which would need its parameter list adapted) instead of mucking with PATH :P
>
> In this way the package variable would flow from the os configuration all the way to the actual "grub-install" invocation call without magical environment variables, packages that just happen to be pulled in from some imported module etc.
>
> If you want, you can also fix this one up, too. (If not, I'll wait until your stuff is merged and fix it myself - no worries)
>
> A first test whether it was enough is to remove the #:use-module (gnu packages grub) from guix/scripts/system.scm and see whether it still works (it should). For clarity I would make the final patch remove it, too.

Hi Danny!

You raise some very good points. The patch I just sent indeed works
without #:use-module (gnu packages grub), so that should be included.

Passing the grub object to grub-install seems like it is better suited
for a separate patch. I have a couple of other things on my list before
ready to hack on grub-install (need to pass "--efi-directory" somehow),
but happy to review any work on it.

Attached is the same patch without loading the grub module. Would be
great to clear this out of the patch queue. Thanks a lot! :)


[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-system-Use-grub-from-bootloader-configuration.patch --]
[-- Type: text/x-patch, Size: 1907 bytes --]

From 5e31312aeae87d63ab2c64e92835231b59c804db Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke@fastmail.com>
Date: Mon, 7 Nov 2016 11:56:52 +0000
Subject: [PATCH] system: Use grub from bootloader configuration.

* gnu/system/grub.scm (gnu): Export grub-configuration-grub.
* guix/scripts/system.scm (perform-action): Use it.
(define-module): Don't import (gnu packages grub).

Co-authored-by: Danny Milosavljevic <dannym@scratchpost.org>
---
 gnu/system/grub.scm     | 1 +
 guix/scripts/system.scm | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/system/grub.scm b/gnu/system/grub.scm
index 5c9d0f1..4657b06 100644
--- a/gnu/system/grub.scm
+++ b/gnu/system/grub.scm
@@ -51,6 +51,7 @@
             grub-configuration
             grub-configuration?
             grub-configuration-device
+            grub-configuration-grub
 
             menu-entry
             menu-entry?
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index df9b37d..71ddccf 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -44,7 +44,6 @@
   #:use-module (gnu services)
   #:use-module (gnu services shepherd)
   #:use-module (gnu services herd)
-  #:use-module (gnu packages grub)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-19)
@@ -617,7 +616,8 @@ building anything."
                                                 #:image-size image-size
                                                 #:full-boot? full-boot?
                                                 #:mappings mappings))
-       (grub      (package->derivation grub))
+       (grub      (package->derivation (grub-configuration-grub
+                                        (operating-system-bootloader os))))
        (grub.cfg  (if (eq? 'container action)
                       (return #f)
                       (operating-system-grub.cfg os
-- 
2.10.2

Re: How to use custom grub?

[Danny Milosavljevic]

Looks good to me!

I also invoked make in the guix directory - which didn't print any new warnings -, reconfigured the system (for non-uefi grub) and successfully booted with it. So all good!

Re: How to use custom grub?

[Ludovic Courtès]

Marius Bakke <mbakke@fastmail.com> skribis:

> From 5e31312aeae87d63ab2c64e92835231b59c804db Mon Sep 17 00:00:00 2001
> From: Marius Bakke <mbakke@fastmail.com>
> Date: Mon, 7 Nov 2016 11:56:52 +0000
> Subject: [PATCH] system: Use grub from bootloader configuration.
>
> * gnu/system/grub.scm (gnu): Export grub-configuration-grub.
> * guix/scripts/system.scm (perform-action): Use it.
> (define-module): Don't import (gnu packages grub).
>
> Co-authored-by: Danny Milosavljevic <dannym@scratchpost.org>

Also an “LGTM” from me.  Thanks to both of you!

Ludo’.

Re: How to use custom grub?

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 701 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Marius Bakke <mbakke@fastmail.com> skribis:
>
>> From 5e31312aeae87d63ab2c64e92835231b59c804db Mon Sep 17 00:00:00 2001
>> From: Marius Bakke <mbakke@fastmail.com>
>> Date: Mon, 7 Nov 2016 11:56:52 +0000
>> Subject: [PATCH] system: Use grub from bootloader configuration.
>>
>> * gnu/system/grub.scm (gnu): Export grub-configuration-grub.
>> * guix/scripts/system.scm (perform-action): Use it.
>> (define-module): Don't import (gnu packages grub).
>>
>> Co-authored-by: Danny Milosavljevic <dannym@scratchpost.org>
>
> Also an “LGTM” from me.  Thanks to both of you!

Cool! I've pushed this as 81bf2ccbc408fc2e959d3f5ab019938dad2ce616.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 454 bytes --]