[bug#45225] [PATCH] gnu: grub: Fix unicode font loading when booting over TFTP.

[bug#45225] [PATCH] gnu: grub: Fix unicode font loading when booting over TFTP.

[Stefan]

* gnu/bootloader/grub.scm (font-file): Using (canonicalize-path), as symlinks
 from a bootloader profile do not work on a tftp server when network booting.
---
gnu/bootloader/grub.scm | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm
index af7b7561ff..3177452dfb 100644
--- a/gnu/bootloader/grub.scm
+++ b/gnu/bootloader/grub.scm
@@ -191,9 +191,18 @@ fi~%"
  (define font-file
    (let* ((bootloader (bootloader-configuration-bootloader config))
           (grub (bootloader-package bootloader)))
-      (normalize-file (file-append grub "/share/grub/unicode.pf2")
-                      store-mount-point
-                      store-directory-prefix)))
+      ;; The bootloader-package may be a profile with only symlinks.
+      ;; If network booting, then a symlink to the font may not work on the
+      ;; server side.  Therefore we canonicalize the file name of the font.
+      ;; TODO:  The font gets installed by (install-grub-efi-netboot) and
+      ;; (install-grub-efi).  The installed font could be referred to as
+      ;; "unicode".  But it is currently unclear if (install-grub-disk-image)
+      ;; and (install-grub) both install the font as well.
+      ;; Actually this should be preferred.
+      #~(canonicalize-path
+         #+(normalize-file (file-append grub "/share/grub/unicode.pf2")
+                           store-mount-point
+                           store-directory-prefix))))

  (define image
    (normalize-file (grub-background-image config)
-- 
2.29.2

bug#45225: [PATCH] gnu: grub: Fix unicode font loading when booting over TFTP.

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 884 bytes --]

Hi Stefan,

your patch file arrived mangled in my mailbox.

On Sun, 13 Dec 2020 18:25:24 +0100
Stefan <stefan-guix@vodafonemail.de> wrote:

>diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm
>index af7b7561ff..3177452dfb 100644
>--- a/gnu/bootloader/grub.scm
>+++ b/gnu/bootloader/grub.scm
>@@ -191,9 +191,18 @@ fi~%"
>  (define font-file

For example, the original source file had here two spaces before the "define".
But the Patch format needs one control character in front.

In this case (for the Context mark), the control character should also be a
space.
That means there should be three spaces there.  But there are two spaces there.

It goes on similarly in the entire patch file.

I've fixed it up manually and pushed to guix master--but you might want
to look into why this happens in general before something important gets
mangled.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[bug#45225] [PATCH] gnu: grub: Fix unicode font loading when booting over TFTP.

[Stefan]

Hi Danny and Ludo!

This patch got pushed and later reverted, as it is causing issues, because /gnu/store…-grub…/share/grub/unicode.pf2 is sometimes not visible when building the grub.cfg file.

To understand what is happening here, I modified the creation of the grub.cfg file to write the directory entries of /gnu/store visible at build time into a file.

While the /gnu/store/ on disk shows thousands of entries, when building grub.cfg there are only about 400 entries visible in my case doing a guix system reconfigure. Although the grub-efi package is referred to with a file-append gexp to build the path to the unicode.pf2 in gnu/bootloader/grub.scm, it happens that the grub package is not available in all cases in /gnu/store. The grub-efi package is gotten via (bootloader-package bootloader) and the bootloader possibly from a /run/*-system/parameters file.

I also noticed that even all the Linux kernels and initrds – except for the recent pair –, which end up being referred to from the grub.cfg, are not visible in /gnu/store while building the grub.cfg.

What makes the list of visible entries in /gnu/store when building the grub.cfg? And why might the gexp-referred grub-efi package not always be visible? This feels like a bug to me.


Bye

Stefan

[bug#45225] [PATCH] gnu: grub: Fix unicode font loading when booting over TFTP.

[Ludovic Courtès]

Hi Stefan,

Stefan <stefan-guix@vodafonemail.de> skribis:

> This patch got pushed and later reverted, as it is causing issues, because /gnu/store…-grub…/share/grub/unicode.pf2 is sometimes not visible when building the grub.cfg file.

Yeah, I wanted to send you a heads-up and then I forgot, apologies!

In general, my suggestion when fiddling with these things is to at least
run:

  make check-system TESTS=basic

This one builds and runs quickly and catches the most obvious issues, as
in this case.

Then, when fiddling with the bootloader, file systems, and that kind of
stuff, it’s a good idea to run an installation test, as in:

  make check-system TESTS=installed-os

This is more expensive to build (it first has to build ‘guix’, then an
ISO image, and then run the installation), but it’s a good idea when
low-level things are modified.  See:

  https://guix.gnu.org/manual/en/html_node/Running-the-Test-Suite.html

I’m not sure I understand what this change meant to fix though.  What
was the initial problem that led to the addition of this
‘canonicalize-path’ call?

Thanks,
Ludo’.

[bug#45225] [PATCH] gnu: grub: Fix unicode font loading when booting over TFTP.

[Stefan]

Hi Ludo’!

> I’m not sure I understand what this change meant to fix though.  What
> was the initial problem that led to the addition of this
> ‘canonicalize-path’ call?

When I am booting over TFTP, the Guix related files are located somewhere on that server and therefore symlinks do not work. Let's make an example:

There is some path like /volume/systems/guix-system/. There you can find gnu/store/ and boot/.

That server serves /volume/systems/guix-system/ via NFS as the root file system for a raspberry. That server also serves /volume/systems/guix-system/boot/ as the TFTP root for booting.

Now due to the fact that I need some firmware, configuration files, U-Boot and GRUB, there is now the possibility to create a bootloader-profile, which is basically a collection of symlinks to all this.

Due to this, there is somewhere a /volume/systems/guix-system/gnu/store/…-bootloader-profile/share/grub/unicode.pf2, which is an *absolute* symlink to /gnu/store/…-grub-efi-2.04/share/grub/unicode.pf2. 

The generated grub.cfg now normally refers to /gnu/store/…-bootloader-profile/share/grub/unicode.pf2.

When GRUB tries to load gnu/store/…-bootloader-profile/share/grub/unicode.pf2 via TFTP, due to a profile prepared *relative* symlink at /volume/systems/guix-system/boot/gnu/store to the real store location at ../..gnu/store/guix-system/gnu/store/ the TFTP server is able to access gnu/store/…-bootloader-profile/share/grub/unicode.pf2.

And there is the problem: That /gnu/store/…-bootloader-profile/share/grub/unicode.pf2 is an *absolute* symlink with no meaning to the TFTP server, so loading the font fails.

Simply speaking: Files for booting via TFTP must not be *absolute* symlinks.

Meanwhile I know that the font can be referred to as simply "unicode" without any path and extension, as it is installed with install-grub-efi, install-grub-efi-netboot and install-grub. I’m just still unsure about install-grub-disk-image. So hopefully the font reference can be simplified.

> Yeah, I wanted to send you a heads-up and then I forgot, apologies!

Nevermind. I’m sorry to have caused this trouble.

> In general, my suggestion when fiddling with these things is to at least
> run …

My trouble is that I only have that raspberry with only 1 GB RAM yet, and running tests on it is kind of impossible.

I tried to install Guix in a virtual machine on a NAS system during the weekend. The installation image is working, but the installed system hangs during boot. At least this was enough to find out that a simple "unicode" in grub.cfg is also sufficient for install-grub. :-)

I’m sorry that I broke Guix. I didn’t imagine that a canonicalize-path would not work, when the store path is known during build.

But this brings me back to my original question: Could you please explain why the grub package, which is referred to with a file-append gexp, is finally not visible when the grub.cfg is created?


Bye

Stefan 

[bug#45225] [PATCH] gnu: grub: Fix unicode font loading when booting over TFTP.

[Ludovic Courtès]

Hi Stefan,

Stefan <stefan-guix@vodafonemail.de> skribis:

> Due to this, there is somewhere a /volume/systems/guix-system/gnu/store/…-bootloader-profile/share/grub/unicode.pf2, which is an *absolute* symlink to /gnu/store/…-grub-efi-2.04/share/grub/unicode.pf2. 
>
> The generated grub.cfg now normally refers to /gnu/store/…-bootloader-profile/share/grub/unicode.pf2.
>
> When GRUB tries to load gnu/store/…-bootloader-profile/share/grub/unicode.pf2 via TFTP, due to a profile prepared *relative* symlink at /volume/systems/guix-system/boot/gnu/store to the real store location at ../..gnu/store/guix-system/gnu/store/ the TFTP server is able to access gnu/store/…-bootloader-profile/share/grub/unicode.pf2.
>
> And there is the problem: That /gnu/store/…-bootloader-profile/share/grub/unicode.pf2 is an *absolute* symlink with no meaning to the TFTP server, so loading the font fails.
>
> Simply speaking: Files for booting via TFTP must not be *absolute* symlinks.

It’s possible to create profiles that use relative symlinks:

  (profile-derivation … #:relative-symlinks? #t)

or, for the declarative counterpart:

  (profile (content …)
           (relative-symlinks? #t))

Would that solve your problem?

BTW, I don’t see “bootloader-profile” in ‘master’; is it on a branch?

Thanks,
Ludo’.

[bug#45225] [PATCH] gnu: grub: Fix unicode font loading when booting over TFTP.

[Stefan]

Hi Ludo’!

> It’s possible to create profiles that use relative symlinks:

> Would that solve your problem?

Oh, that can be helpful. Thanks!

However, the unicode font is the only file referring from grub.cfg to the bootloader-profile inside the store and I’m meanwhile confident that this is not necessary at all, as the font is installed with all the other grub files in /boot/.

> BTW, I don’t see “bootloader-profile” in ‘master’; is it on a branch?

It’s in master in gnu/bootloader.scm, the function to use is efi-bootloader-profile. However, there’s still a patch pending.

https://issues.guix.gnu.org/41066#33

I’m sorry for my nagging, but I’d really like to understand, why the grub package is not always visible in /gnu/store/ when building the grub.cfg file, although it is referred to by a file-append gexp.


Bye

Stefan

[bug#45225] [PATCH] gnu: grub: Use installed unicode font, fix loading when booting over TFTP.

[Stefan]

* gnu/bootloader/grub.scm (setup-gfxterm): Removed parameter font-file.
Refer to the installed font, not into the store.
(font-file): Removed obsolete function.
(install-grub): When populating a disk-image, install the unicode font, too.
---
 gnu/bootloader/grub.scm | 26 +++++++++++---------------
 1 file changed, 11 insertions(+), 15 deletions(-)

diff --git a/gnu/bootloader/grub.scm b/gnu/bootloader/grub.scm
index 29c81ae641..13ee695909 100644
--- a/gnu/bootloader/grub.scm
+++ b/gnu/bootloader/grub.scm
@@ -168,15 +168,14 @@ STORE-DEVICE designates the device holding the store, and STORE-MOUNT-POINT is
  its mount point; these are used to determine where the background image and
  fonts must be searched for.  STORE-DIRECTORY-PREFIX is a directory prefix to
  prepend to any store file name."
-  (define (setup-gfxterm config font-file)
+  (define (setup-gfxterm config)
      (if (memq 'gfxterm (bootloader-configuration-terminal-outputs config))
          #~(format #f "
-if loadfont ~a; then
+if loadfont unicode; then
    set gfxmode=~a
    insmod all_video
    insmod gfxterm
  fi~%"
-                  #+font-file
                    #$(string-join
                       (grub-theme-gfxmode (bootloader-theme config))
                       ";"))
@@ -188,13 +187,6 @@ fi~%"
        (string-append (symbol->string (assoc-ref colors 'fg)) "/"
                       (symbol->string (assoc-ref colors 'bg)))))
  
-  (define font-file
-    (let* ((bootloader (bootloader-configuration-bootloader config))
-           (grub (bootloader-package bootloader)))
-      (normalize-file (file-append grub "/share/grub/unicode.pf2")
-                      store-mount-point
-                      store-directory-prefix)))
-
    (define image
      (normalize-file (grub-background-image config)
                      store-mount-point
@@ -216,8 +208,8 @@ else
    set menu_color_normal=cyan/blue
    set menu_color_highlight=white/blue
  fi~%"
-                 #$(grub-root-search store-device font-file)
-                 #$(setup-gfxterm config font-file)
+                 #$(grub-root-search store-device image)
+                 #$(setup-gfxterm config)
                   #$(grub-setup-io config)
  
                   #$image
@@ -545,9 +537,13 @@ fi~%"))))
                (invoke/quiet grub "--no-floppy" "--target=i386-pc"
                              "--boot-directory" install-dir
                              device))
-            ;; When creating a disk-image, only install GRUB modules.
-            (copy-recursively (string-append bootloader "/lib/")
-                              install-dir)))))
+            ;; When creating a disk-image, only install a font and GRUB modules.
+            (let* ((fonts (string-append install-dir "/grub/fonts")))
+              (mkdir-p fonts)
+              (copy-file (string-append bootloader "/share/grub/unicode.pf2")
+                         (string-append fonts "/unicode.pf2"))
+              (copy-recursively (string-append bootloader "/lib/")
+                                install-dir))))))
  
  (define install-grub-disk-image
    #~(lambda (bootloader root-index image)
-- 
2.29.2

[bug#45225] [PATCH] gnu: grub: Use installed unicode font, fix loading when booting over TFTP.

[Stefan]

Hi!

A friendly ping. :-)

<https://issues.guix.gnu.org/issue/45225#7>


Bye

Stefan

[bug#45225] [PATCH] gnu: grub: Use installed unicode font, fix loading when booting over TFTP.

[Stefan]

Hi!

Another friendly ping. :-)

<https://issues.guix.gnu.org/issue/45225#7>


Bye

Stefan

[bug#45225] [PATCH] gnu: grub: Fix unicode font loading when booting over TFTP.

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 1644 bytes --]

Hi Stefan,

On Tue, 22 Dec 2020 16:21:40 +0100
Stefan <stefan-guix@vodafonemail.de> wrote:

> I’m sorry for my nagging, but I’d really like to understand, why the grub package is not always visible in /gnu/store/ when building the grub.cfg file, although it is referred to by a file-append gexp.

I just want to say that I'm not ignoring the question--I honesty don't know.

In any case, until we do know, I've applied your patch
https://issues.guix.gnu.org/issue/45225#7  to guix master as commit
b57056a9fd83c61efb5343853948a798da6e975e--after all, the font was in /boot
for regular Guix Systems anyway, so I don't see a downside loading it from
/boot--especially since it's part of the bootloader anyway, which does not
have different generations for itself.  Also, the code is simpler now.

I also want to note that all 4 hunks in that patch had failed to apply with
git am.  I implore you to please download the patch files from
https://issues.guix.gnu.org/issue/45225#7 using the little blue download arrow
are the top right corner and try to "git am" them yourself.  It won't work
because the spaces in your patch files are mangled.

If it's that way because git send-email doesn't work, I have a block like the
following in ~/.gitconfig--and my patches arrive unmangled that way:

[sendemail]
        from = Example User <user@example.com>
        smtpEncryption = tls
        smtpServer = xxx
        smtpUser = xxx
        smtpPass = xxx
        smtpServerPort = 587

Then you can use

  git format-patch -1 -o ../outgoing-topic1
  git send-email ../outgoing-topic1/0001*

and it will work.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[bug#45225] [PATCH] gnu: grub: Fix unicode font loading when booting over TFTP.

[Stefan]

Hi Danny!

> I just want to say that I'm not ignoring the question--I honesty don't know.

OK.

> In any case, until we do know, I've applied your patch

Many thanks! :-)

> I also want to note that all 4 hunks in that patch had failed to apply with
> git am.

I’m sorry about this. I think that patch was older than your hint to me about troubles with spaces.

I’ll change my setup. Thanks for the hints.


Bye

Stefan