bug#44717: ISO grub config points to nonexistent drive UUID.

bug#44717: ISO grub config points to nonexistent drive UUID.

[Jesse Gibbons]

On multiple computers, when I try to copy a custom operating system iso 
to an sd card and boot, the boot operation fails. Here's the output on 
one of the computers:

GC Warning: pthread_getattr_up or pthread_getattr_getstack failed for 
main thread
GC Warning: couldn't read /proc/stat
Welcome, this is GNU's early boot Guile.
Use --repl for an initrd REPL.

loading kernel modules...
waiting for partition '31393730-3031-3031-3139-333534353239' to appear...
waiting for partition '31393730-3031-3031-3139-333534353239' to appear...
waiting for partition '31393730-3031-3031-3139-333534353239' to appear...
[    5.044226] sd 0:0:0:0: [sda] No Caching mode page found
[    5.044336] sd 0:0:0:0: [sda] Assuming drive cache: write through
No file system check procedure for /dev/sda; skipping
ice-9/boot-9.scm:1669:16: In procedure raise-exception:
In procedure mount: Permission denied

Entering a new prompt. Type `,bt' for a backtrace or `,q` to continue.
GNU Guile 3.0.2
Copyright (C) 1995-2020 Free Software foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help\ for help.
scheme@(guile-user)>

When I type ,q or C-d I trigger a kernel panic.

This happens on at least three different computers:
- a lenovo ideapad s100 (intel atom, only successfully boots 32-bit 
image, output above)
- a laptop (unidentified brand) with an amd athlon-x2, only tried 64-bit iso
- an acer aspire one (cnet says intel atom), only tried 64-bit iso

I can confirm the lenovo successfully boots the i686 install iso 
(version 1.1.0) on the guix website and lets me see the info page on 
installing the system by hand, though the UI installer fails (most 
likely because the network hardware is disgustingly blobby).

I built the install image for i686-linux and flashed it to the sd card. 
It looks for the exact same partition and falls back on a boot guile repl.

$ guix describe
Generation 334    Nov 17 2020 17:52:52    (current)
   guix 129b9b1
     repository URL: https://git.savannah.gnu.org/git/guix.git
     branch: master
     commit: 129b9b16d9b588316cc997cf8f4fefe30961a417

I generated the iso with the command
`guix system disk-image -t iso9660 --root=installer.BaNl/install-x86.iso 
--system=i686-linux gnu/system/install.scm`
and flash the sd card with the command
`sudo bash -c "echo success" && time sudo dd if=install-x86.iso of=/dev/sdc`

When I inspect the GRUB menu, I see the option
--root=31393730-3031-3031-3139-333534353239
but in the gnome disk utility on my main laptop I do not see the above 
UUID in any of the partitions on the SD card I'm using, still with the 
freshly built install iso flashed onto it. Instead I see the UUIDs 
1970-01-01-19-49-46-83 for partition 1 and 3495-32E0 for partition 2.

Thanks,
-Jesse

bug#44717: ISO grub config points to nonexistent drive UUID.

[Ludovic Courtès]

Hi,

Jesse Gibbons <jgibbons2357@gmail.com> skribis:

> I generated the iso with the command
> `guix system disk-image -t iso9660
> --root=installer.BaNl/install-x86.iso --system=i686-linux
> gnu/system/install.scm`
> and flash the sd card with the command
> `sudo bash -c "echo success" && time sudo dd if=install-x86.iso of=/dev/sdc`
>
> When I inspect the GRUB menu, I see the option
> --root=31393730-3031-3031-3139-333534353239
> but in the gnome disk utility on my main laptop I do not see the above
> UUID in any of the partitions on the SD card I'm using, still with the 
> freshly built install iso flashed onto it. Instead I see the UUIDs
> 1970-01-01-19-49-46-83 for partition 1 and 3495-32E0 for partition 2.

The option in the GRUB menu uses the “DCE” format for the UUID, but if
you convert it to an ISO-9660 UUID, it looks almost the same:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(gnu system uuid)
scheme@(guile-user)> (string->uuid "31393730-3031-3031-3139-333534353239")
$60 = #vu8(49 57 55 48 48 49 48 49 49 57 51 53 52 53 50 57)
scheme@(guile-user)> (bytevector->uuid $60 'iso9660)
$61 = #<<uuid> type: iso9660 bv: #vu8(49 57 55 48 48 49 48 49 49 57 51 53 52 53 50 57)>
scheme@(guile-user)> (uuid->string $61)
$62 = "1970-01-01-19-35-45-29"
--8<---------------cut here---------------end--------------->8---

The ISO UUID is computed in a deterministic fashion.  Are you sure
you’re looking at the same ISO?

For example, if you pick
<https://alpha.gnu.org/gnu/guix/guix-system-install-1.2.0rc1.x86_64-linux.iso.xz>,
it boots just fine.  In the GRUB menu entry (type ‘e’ in the menu), you
can see both the DCE UUID for ‘--root’ and the ISO UUID for ‘search.fs’,
which are actually the same.

HTH!

Ludo’.

bug#44717: ISO grub config points to nonexistent drive UUID.

[Jesse Gibbons]

[-- Attachment #1: Type: text/plain, Size: 2818 bytes --]


On 11/18/20 3:31 AM, Ludovic Courtès wrote:
> Hi,
>
> Jesse Gibbons <jgibbons2357@gmail.com> skribis:
>
>> I generated the iso with the command
>> `guix system disk-image -t iso9660
>> --root=installer.BaNl/install-x86.iso --system=i686-linux
>> gnu/system/install.scm`
>> and flash the sd card with the command
>> `sudo bash -c "echo success" && time sudo dd if=install-x86.iso of=/dev/sdc`
>>
>> When I inspect the GRUB menu, I see the option
>> --root=31393730-3031-3031-3139-333534353239
>> but in the gnome disk utility on my main laptop I do not see the above
>> UUID in any of the partitions on the SD card I'm using, still with the
>> freshly built install iso flashed onto it. Instead I see the UUIDs
>> 1970-01-01-19-49-46-83 for partition 1 and 3495-32E0 for partition 2.
> The option in the GRUB menu uses the “DCE” format for the UUID, but if
> you convert it to an ISO-9660 UUID, it looks almost the same:
>
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> ,use(gnu system uuid)
> scheme@(guile-user)> (string->uuid "31393730-3031-3031-3139-333534353239")
> $60 = #vu8(49 57 55 48 48 49 48 49 49 57 51 53 52 53 50 57)
> scheme@(guile-user)> (bytevector->uuid $60 'iso9660)
> $61 = #<<uuid> type: iso9660 bv: #vu8(49 57 55 48 48 49 48 49 49 57 51 53 52 53 50 57)>
> scheme@(guile-user)> (uuid->string $61)
> $62 = "1970-01-01-19-35-45-29"
> --8<---------------cut here---------------end--------------->8---
>
> The ISO UUID is computed in a deterministic fashion.  Are you sure
> you’re looking at the same ISO?
>
> For example, if you pick
> <https://alpha.gnu.org/gnu/guix/guix-system-install-1.2.0rc1.x86_64-linux.iso.xz>,
> it boots just fine.  In the GRUB menu entry (type ‘e’ in the menu), you
> can see both the DCE UUID for ‘--root’ and the ISO UUID for ‘search.fs’,
> which are actually the same.
>
> HTH!
>
> Ludo’.

When I posted this initial bug report, I reported what happened when I 
flashed the built iso to an SD card and tried it on another laptop. Just 
to be sure, I remade the image (in the same directory as my guix checkout)

guix system disk-image -t iso9660 --root=$(mktemp -p /tmp -d 
install.XXX)/install-x86.iso --system=i686-linux gnu/system/install.scm

and I mounted the ISO itself and took a look at it. The grub.conf 
specifies both UUIDs as you described.

When I try it on a VM, it opens a repl with a completely different error 
which I'm too lazy to type out by hand. See attached screenshot.

When I download the iso you linked and run it on a vm, it works just 
fine. Just to see if it has anything to do with the architecture, I also 
decided to try a vm with the i686-linux counterpart. It also works fine.

I do not have access to any of those old laptops right now, so I can't 
experiment further.

[-- Attachment #2: Screenshot_guix-1.1_2020-11-18_07:57:16.png --]
[-- Type: image/png, Size: 10338 bytes --]

bug#44717: ISO grub config points to nonexistent drive UUID.

[Ludovic Courtès]

Hi Jesse,

Jesse Gibbons <jgibbons2357@gmail.com> skribis:

>> For example, if you pick
>> <https://alpha.gnu.org/gnu/guix/guix-system-install-1.2.0rc1.x86_64-linux.iso.xz>,
>> it boots just fine.  In the GRUB menu entry (type ‘e’ in the menu), you
>> can see both the DCE UUID for ‘--root’ and the ISO UUID for ‘search.fs’,
>> which are actually the same.

[...]

> guix system disk-image -t iso9660 --root=$(mktemp -p /tmp -d
> install.XXX)/install-x86.iso --system=i686-linux
> gnu/system/install.scm
>
> and I mounted the ISO itself and took a look at it. The grub.conf
> specifies both UUIDs as you described.
>
> When I try it on a VM, it opens a repl with a completely different
> error which I'm too lazy to type out by hand. See attached screenshot.

Do you observe the same problem with the image I linked to above?  It’s
built with ‘guix system disk-image -t iso9660 --label=GUIX…
gnu/system/install.scm’.

The error you sent looks as if it’s trying to mount the root file system
read/write.

Thinking about it: does it work if you pass ‘--volatile’ on the
‘disk-image’ command line?  This flag was added recently on ‘master’
(commit 41f27bf8702838f19b1dc5ffee8eec1d4315d4e6), so perhaps what
you’re seeing is a regression here.

Maxim, could it be that we need (volatile-root? #t) for the ISO9660
image type and/or passing ‘--volatile’ in Makefile.am (‘release’ target)
and updating the “Building the Installation Image” node of the manual?

Thanks,
Ludo’.

bug#44717: ISO grub config points to nonexistent drive UUID.

[Jesse Gibbons]

On 11/18/20 10:17 AM, Ludovic Courtès wrote:
> Hi Jesse,
>
> Jesse Gibbons <jgibbons2357@gmail.com> skribis:
>
>>> For example, if you pick
>>> <https://alpha.gnu.org/gnu/guix/guix-system-install-1.2.0rc1.x86_64-linux.iso.xz>,
>>> it boots just fine.  In the GRUB menu entry (type ‘e’ in the menu), you
>>> can see both the DCE UUID for ‘--root’ and the ISO UUID for ‘search.fs’,
>>> which are actually the same.
> [...]
>
>> guix system disk-image -t iso9660 --root=$(mktemp -p /tmp -d
>> install.XXX)/install-x86.iso --system=i686-linux
>> gnu/system/install.scm
>>
>> and I mounted the ISO itself and took a look at it. The grub.conf
>> specifies both UUIDs as you described.
>>
>> When I try it on a VM, it opens a repl with a completely different
>> error which I'm too lazy to type out by hand. See attached screenshot.
> Do you observe the same problem with the image I linked to above?  It’s
> built with ‘guix system disk-image -t iso9660 --label=GUIX…
> gnu/system/install.scm’.
No, I did not have any trouble booting either of the "official" 
installer images in a VM.
>
> The error you sent looks as if it’s trying to mount the root file system
> read/write.
>
> Thinking about it: does it work if you pass ‘--volatile’ on the
> ‘disk-image’ command line?  This flag was added recently on ‘master’
> (commit 41f27bf8702838f19b1dc5ffee8eec1d4315d4e6), so perhaps what
> you’re seeing is a regression here.
>
> Maxim, could it be that we need (volatile-root? #t) for the ISO9660
> image type and/or passing ‘--volatile’ in Makefile.am (‘release’ target)
> and updating the “Building the Installation Image” node of the manual?
>
> Thanks,
> Ludo’.
I looked up the commit for the v1.2.0rc1 tag and ran
`guix time-machine --commit=1e272d42f6217b70c9801b93e46b144e9ab27664 -- 
system disk-image -t iso9660 --system=i686-linux --root=$(mktemp -p /tmp 
-d install.x86.XXX)/install.x86.iso gnu/system/install.scm`
I tried the output in a vm. It booted fine. I'll try including the 
--volatile flag without using the time machine.

bug#44717: ISO grub config points to nonexistent drive UUID.

[Bengt Richter]

Hi Jesse,

On +2020-11-17 21:56:32 -0700, Jesse Gibbons wrote:
[...]
> 
> I generated the iso with the command
> `guix system disk-image -t iso9660 --root=installer.BaNl/install-x86.iso
> --system=i686-linux gnu/system/install.scm`
> and flash the sd card with the command
> `sudo bash -c "echo success" && time sudo dd if=install-x86.iso of=/dev/sdc`
>

&& sync ??

I think I have gotten corrupted results before, by assuming that DD does its own sync
of its output all the way to completed writing to destination disk before returning.
(not sure if of= was an ordinary file when I got hit, though ;)

> When I inspect the GRUB menu, I see the option
> --root=31393730-3031-3031-3139-333534353239
> but in the gnome disk utility on my main laptop I do not see the above UUID
> in any of the partitions on the SD card I'm using, still with the freshly
> built install iso flashed onto it. Instead I see the UUIDs
> 1970-01-01-19-49-46-83 for partition 1 and 3495-32E0 for partition 2.
> 
> Thanks,
> -Jesse
> 
> 
> 

-- 
Regards,
Bengt Richter

bug#44717: ISO grub config points to nonexistent drive UUID.

[Jesse Gibbons]

On 11/18/20 11:09 AM, Bengt Richter wrote:
> Hi Jesse,
>
> On +2020-11-17 21:56:32 -0700, Jesse Gibbons wrote:
> [...]
>> I generated the iso with the command
>> `guix system disk-image -t iso9660 --root=installer.BaNl/install-x86.iso
>> --system=i686-linux gnu/system/install.scm`
>> and flash the sd card with the command
>> `sudo bash -c "echo success" && time sudo dd if=install-x86.iso of=/dev/sdc`
>>
> && sync ??
>
> I think I have gotten corrupted results before, by assuming that DD does its own sync
> of its output all the way to completed writing to destination disk before returning.
> (not sure if of= was an ordinary file when I got hit, though ;)
I'm not sure how I can get it to sync. I would normally use the gnome 
disk utiltity to flash an image to an external drive, but for a while it 
didn't respond when I tried to "restore disk image". Now it's working 
fine though. I'll try using gnome disk utility to flash and see if that 
makes a difference.
>> When I inspect the GRUB menu, I see the option
>> --root=31393730-3031-3031-3139-333534353239
>> but in the gnome disk utility on my main laptop I do not see the above UUID
>> in any of the partitions on the SD card I'm using, still with the freshly
>> built install iso flashed onto it. Instead I see the UUIDs
>> 1970-01-01-19-49-46-83 for partition 1 and 3495-32E0 for partition 2.
>>
>> Thanks,
>> -Jesse
>>
>>
>>

bug#44717: ISO grub config points to nonexistent drive UUID.

[Bengt Richter]

Hi again,

On +2020-11-18 12:17:14 -0700, Jesse Gibbons wrote:
> 
> 
> On 11/18/20 11:09 AM, Bengt Richter wrote:
> > Hi Jesse,
> > 
> > On +2020-11-17 21:56:32 -0700, Jesse Gibbons wrote:
> > [...]
> > > I generated the iso with the command
> > > `guix system disk-image -t iso9660 --root=installer.BaNl/install-x86.iso
> > > --system=i686-linux gnu/system/install.scm`
> > > and flash the sd card with the command
> > > `sudo bash -c "echo success" && time sudo dd if=install-x86.iso of=/dev/sdc`
> > > 
> > && sync ??
> > 
> > I think I have gotten corrupted results before, by assuming that DD does its own sync
> > of its output all the way to completed writing to destination disk before returning.
> > (not sure if of= was an ordinary file when I got hit, though ;)
> I'm not sure how I can get it to sync. I would normally use the gnome disk

I actually meant try adding " && sync" to your command line.
Or, when it comes back after printing timing info, do the
sync command separately, right then.

Either way, it may seem to "hang" after seeing the sync command.

Writing to SDHC cards is pretty (glacially :) slow, so if the OS has cached a
lot of write buffers from DD output, it may well seem to "hang"
after you type sync. If it does, it will confirm this theory :)
Don't kill it, make some coffee and come back ;-)
Or spawn another terminal and look with top to see what's going on.

> utiltity to flash an image to an external drive, but for a while it didn't
> respond when I tried to "restore disk image". Now it's working fine though.
> I'll try using gnome disk utility to flash and see if that makes a
> difference.
> > > When I inspect the GRUB menu, I see the option
> > > --root=31393730-3031-3031-3139-333534353239
> > > but in the gnome disk utility on my main laptop I do not see the above UUID
> > > in any of the partitions on the SD card I'm using, still with the freshly
> > > built install iso flashed onto it. Instead I see the UUIDs
> > > 1970-01-01-19-49-46-83 for partition 1 and 3495-32E0 for partition 2.
> > > 
> > > Thanks,
> > > -Jesse
> > > 
> > > 
> > > 
> 

-- 
Regards,
Bengt Richter

bug#44717: ISO grub config points to nonexistent drive UUID.

[Jesse Gibbons]

On 11/18/20 10:17 AM, Ludovic Courtès wrote:
> Hi Jesse,
>
> Jesse Gibbons <jgibbons2357@gmail.com> skribis:
>
>>> For example, if you pick
>>> <https://alpha.gnu.org/gnu/guix/guix-system-install-1.2.0rc1.x86_64-linux.iso.xz>,
>>> it boots just fine.  In the GRUB menu entry (type ‘e’ in the menu), you
>>> can see both the DCE UUID for ‘--root’ and the ISO UUID for ‘search.fs’,
>>> which are actually the same.
> [...]
>
>> guix system disk-image -t iso9660 --root=$(mktemp -p /tmp -d
>> install.XXX)/install-x86.iso --system=i686-linux
>> gnu/system/install.scm
>>
>> and I mounted the ISO itself and took a look at it. The grub.conf
>> specifies both UUIDs as you described.
>>
>> When I try it on a VM, it opens a repl with a completely different
>> error which I'm too lazy to type out by hand. See attached screenshot.
> Do you observe the same problem with the image I linked to above?  It’s
> built with ‘guix system disk-image -t iso9660 --label=GUIX…
> gnu/system/install.scm’.
>
> The error you sent looks as if it’s trying to mount the root file system
> read/write.
>
> Thinking about it: does it work if you pass ‘--volatile’ on the
> ‘disk-image’ command line?  This flag was added recently on ‘master’
> (commit 41f27bf8702838f19b1dc5ffee8eec1d4315d4e6), so perhaps what
> you’re seeing is a regression here.
>
> Maxim, could it be that we need (volatile-root? #t) for the ISO9660
> image type and/or passing ‘--volatile’ in Makefile.am (‘release’ target)
> and updating the “Building the Installation Image” node of the manual?
>
> Thanks,
> Ludo’.
Passing --volatile to the iso build command fixes the issue with the VM. 
I'm flashing an the resulting to my external drive right now. In a few 
hours, I should be able to test if the problem was that the drive didn't 
fully sync, and hopefully we can close this issue tonight.

bug#44717: ISO grub config points to nonexistent drive UUID.

[Maxim Cournoyer]

Hi,

I've unearth this forgotten issue (now fixed, thank you Mathieu!) from
my INBOX.  Sorry for failing to address myself at the time!

Jesse Gibbons <jgibbons2357@gmail.com> writes:

> On 11/18/20 10:17 AM, Ludovic Courtès wrote:
>> Hi Jesse,
>>
>> Jesse Gibbons <jgibbons2357@gmail.com> skribis:
>>
>>>> For example, if you pick
>>>> <https://alpha.gnu.org/gnu/guix/guix-system-install-1.2.0rc1.x86_64-linux.iso.xz>,
>>>> it boots just fine.  In the GRUB menu entry (type ‘e’ in the menu), you
>>>> can see both the DCE UUID for ‘--root’ and the ISO UUID for ‘search.fs’,
>>>> which are actually the same.
>> [...]
>>
>>> guix system disk-image -t iso9660 --root=$(mktemp -p /tmp -d
>>> install.XXX)/install-x86.iso --system=i686-linux
>>> gnu/system/install.scm
>>>
>>> and I mounted the ISO itself and took a look at it. The grub.conf
>>> specifies both UUIDs as you described.
>>>
>>> When I try it on a VM, it opens a repl with a completely different
>>> error which I'm too lazy to type out by hand. See attached screenshot.
>> Do you observe the same problem with the image I linked to above?  It’s
>> built with ‘guix system disk-image -t iso9660 --label=GUIX…
>> gnu/system/install.scm’.
>>
>> The error you sent looks as if it’s trying to mount the root file system
>> read/write.
>>
>> Thinking about it: does it work if you pass ‘--volatile’ on the
>> ‘disk-image’ command line?  This flag was added recently on ‘master’
>> (commit 41f27bf8702838f19b1dc5ffee8eec1d4315d4e6), so perhaps what
>> you’re seeing is a regression here.
>>
>> Maxim, could it be that we need (volatile-root? #t) for the ISO9660
>> image type and/or passing ‘--volatile’ in Makefile.am (‘release’ target)
>> and updating the “Building the Installation Image” node of the manual?
>>
>> Thanks,
>> Ludo’.
> Passing --volatile to the iso build command fixes the issue with the
> VM. I'm flashing an the resulting to my external drive right now. In a
> few hours, I should be able to test if the problem was that the drive
> didn't fully sync, and hopefully we can close this issue tonight.

For the record, this was fixed by Mathieu by special-casing the ISO9660
so that it always has the volatile-root? property set to #t in commit
83de7ee662fd42cce410dd8c395922647f3aeb13.

Thanks,

Maxim