bug#44027: 1.2-29a2eb3 Installer fails at final stage installing GRUB.

bug#44027: 1.2-29a2eb3 Installer fails at final stage installing GRUB.

[Brendan Tildesley]

[-- Attachment #1: Type: text/plain, Size: 873 bytes --]

|Going through the GUI installer selecting all the defaults, and install 
with LVM and ecryption, AND for the normal option without encryption I 
get these errors on TWO different laptops, both ~10 years old with BIOS. 
Encrypted: Installing for i386-pc platform. grub-install: warning: this 
GPT partition label contains no BIOS Boot Partition; embedding won't be 
possible. grub-install: error: embedding is not possible, but this is 
required for RAID and LVM install. Unencrypted: ||Installing for i386-pc platform. grub-install: warning: this GPT 
partition label contains no BIOS Boot Partition; embedding won't be 
possible. grub-install: warning: Embedding is not possible. GRUB can 
only be installed in this setup by using blocklists. However, blocklists 
are UNRELIABLE and their use is discouraged.. grub-install: error: will 
not proceed with blocklists.|
||


[-- Attachment #2: Type: text/html, Size: 1041 bytes --]

bug#44027: 1.2-29a2eb3 Installer fails at final stage installing GRUB.

[Ludovic Courtès]

Hi Brendan,

Brendan Tildesley <mail@brendan.scot> skribis:

> Going through the GUI installer selecting all the defaults, and install with LVM and ecryption, AND for the normal option without encryption I get these errors on TWO different laptops, both ~10 years old with BIOS.
>
> Encrypted:
>
> Installing for i386-pc  platform.
> grub-install: warning: this GPT partition label contains no BIOS Boot Partition; embedding won't be possible.
> grub-install: error: embedding is not possible, but this is required for RAID and LVM install.
>
> Unencrypted:
>
> Installing for i386-pc  platform.
> grub-install: warning: this GPT partition label contains no BIOS Boot Partition; embedding won't be possible.
> grub-install: warning: Embedding is not possible. GRUB can only be installed in this setup by using blocklists. However, blocklists are UNRELIABLE and their use is discouraged..
> grub-install: error: will not proceed with blocklists.

Shouldn’t it create a “legacy” partition table rather than GPT since
we’re on an old, non-UEFI platform?

Ludo’.

bug#44027: 1.2-29a2eb3 Installer fails at final stage installing GRUB.

[Brett Gilio]

Ludovic Courtès <ludo@gnu.org> writes:
>
> Shouldn’t it create a “legacy” partition table rather than GPT since
> we’re on an old, non-UEFI platform?
>
> Ludo’.
>

That is my thinking as well, it should create a legacy MBR table.

-- 
Brett M. Gilio
brettg@gnu.org
https://brettgilio.com/
E82A C026 95D6 FF02 43CA 1E5C F6C5 2DD1 BA27 CB87

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

[Miguel Ángel Arruga Vivas]

[-- Attachment #1.1: Type: text/plain, Size: 792 bytes --]

Hi,

Brett Gilio <brettg@gnu.org> writes:
> Ludovic Courtès <ludo@gnu.org> writes:
>> Shouldn’t it create a “legacy” partition table rather than GPT since
>> we’re on an old, non-UEFI platform?
>
> That is my thinking as well, it should create a legacy MBR table.

IMHO the old format should be avoided completely when possible.  Why
should we enforce it?

I think this problem involves having a previous ESP partition on the
disk (at least identified as such by parted), because auto-partition!
currently checks that before checking if the booted system has EFI
support.  When that's the case, it doesn't create the needed bios_grub
partition that might have been removed previously.

The attached patch solves that.  What do you think?

Happy hacking,
Miguel

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-installer-Create-bios_grub-partition-when-it-is-needed.patch --]
[-- Type: text/x-patch, Size: 1467 bytes --]

From a2f13b21a631398689cc5471c1910af294454e80 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
 <rosen644835@gmail.com>
Date: Sat, 17 Oct 2020 14:20:41 +0200
Subject: [PATCH] installer: Create bios_grub partition when it is needed.

* gnu/installer/parted.scm (auto-partition!): Only check for ESP on EFI
installations.
---
 gnu/installer/parted.scm | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/gnu/installer/parted.scm b/gnu/installer/parted.scm
index b0c73b837e..fffd5abf3b 100644
--- a/gnu/installer/parted.scm
+++ b/gnu/installer/parted.scm
@@ -911,13 +911,13 @@ exists."
 
     (let* ((start-partition
             (and (not has-extended?)
-                 (not esp-partition)
                  (if (efi-installation?)
-                     (user-partition
-                      (fs-type 'fat32)
-                      (esp? #t)
-                      (size new-esp-size)
-                      (mount-point (default-esp-mount-point)))
+                     (and (not esp-partition)
+                          (user-partition
+                           (fs-type 'fat32)
+                           (esp? #t)
+                           (size new-esp-size)
+                           (mount-point (default-esp-mount-point))))
                      (user-partition
                       (fs-type 'ext4)
                       (bootable? #t)
-- 
2.28.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

[Brendan Tildesley]

On 17/10/20 11:09 pm, Miguel Ángel Arruga Vivas wrote:
> Hi,
>
> Brett Gilio <brettg@gnu.org> writes:
>> Ludovic Courtès <ludo@gnu.org> writes:
>>> Shouldn’t it create a “legacy” partition table rather than GPT since
>>> we’re on an old, non-UEFI platform?
>> That is my thinking as well, it should create a legacy MBR table.
> IMHO the old format should be avoided completely when possible.  Why
> should we enforce it?
>
> I think this problem involves having a previous ESP partition on the
> disk (at least identified as such by parted), because auto-partition!
> currently checks that before checking if the booted system has EFI
> support.  When that's the case, it doesn't create the needed bios_grub
> partition that might have been removed previously.
>
> The attached patch solves that.  What do you think?
>
> Happy hacking,
> Miguel

I'm about to give it a test run. Unrelated to the patch, but, I'm 
building the installer (gnu/system/install.scm) on latest master and I 
get the following warnings/errors, although it doesn't stop the build 
from completing. is it a bug?

https://paste.debian.net/plain/1167398

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

[Brendan Tildesley]

On 17/10/20 11:09 pm, Miguel Ángel Arruga Vivas wrote:
> [...]
> The attached patch solves that.  What do you think?
>
> Happy hacking,
> Miguel

I have tested the patch and the Installer mostly worked. One thing I 
noticed is that the partition scheme I was given automatically looks 
like this:

1 537MB fat32 boot,esp /boot/efi EFI System Partition
2 3146Kb ext4 bios_grub
3 2001MB linux-swap
4 37.5GB ext4 /

It is creating the bios_grub partition, but the EFI is still being made 
also. Is that necessary?

---

After completing the install, I was able to make it to the next bug!: 
https://issues.guix.gnu.org/44054

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

[Miguel Ángel Arruga Vivas]

Hi!

Brendan Tildesley <mail@brendan.scot> writes:
> On 17/10/20 11:09 pm, Miguel Ángel Arruga Vivas wrote:
>> [...]
>> The attached patch solves that.  What do you think?
>>
>> Happy hacking,
>> Miguel
>
> I have tested the patch and the Installer mostly worked. One thing I
> noticed is that the partition scheme I was given automatically looks 
> like this:
>
> 1 537MB fat32 boot,esp /boot/efi EFI System Partition
> 2 3146Kb ext4 bios_grub
> 3 2001MB linux-swap
> 4 37.5GB ext4 /
>
> It is creating the bios_grub partition, but the EFI is still being
> made also. Is that necessary?

AFAIU from the code, the first partition should not be created by the
installer but it won't be removed if it was found on the disk
beforehand.  Tomorrow I'll give a try at the full installation process,
I must have overlooked something if it still being created in that
case...

Happy hacking!
Miguel

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

[Mathieu Othacehe]

Hello Miguel & Brendan,

> AFAIU from the code, the first partition should not be created by the
> installer but it won't be removed if it was found on the disk
> beforehand.  Tomorrow I'll give a try at the full installation process,
> I must have overlooked something if it still being created in that
> case...

Thanks for your help on this topic! Brendan is probably using a GPT
partitionned disk on a non-UEFI compatible machine. Hence, as you
noticed, the installer does not create a bios_grub partition.

This is a problem as this partition is necessary for GRUB and I think
that your patch is fixing the problem. The "auto-partition!" procedure
is also leaving a probably pre-existing ESP partition, but I don't
really understand how it appeared in the first place.

Brendan, did you use "manual partitioning" to create an ESP partition?

Thanks,

Mathieu

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

[Brendan Tildesley]

On 19/10/20 3:03 am, Mathieu Othacehe wrote:
> Hello Miguel & Brendan,
>
>> AFAIU from the code, the first partition should not be created by the
>> installer but it won't be removed if it was found on the disk
>> beforehand.  Tomorrow I'll give a try at the full installation process,
>> I must have overlooked something if it still being created in that
>> case...
> Thanks for your help on this topic! Brendan is probably using a GPT
> partitionned disk on a non-UEFI compatible machine. Hence, as you
> noticed, the installer does not create a bios_grub partition.
>
> This is a problem as this partition is necessary for GRUB and I think
> that your patch is fixing the problem. The "auto-partition!" procedure
> is also leaving a probably pre-existing ESP partition, but I don't
> really understand how it appeared in the first place.
>
> Brendan, did you use "manual partitioning" to create an ESP partition?
Hmm I forget what was on this drive. I used to have Windows 10 on it, 
then I can't
remember. I may have installed Arch Linux on it. What ever was on it, I 
just ran the
installer letting it do it's thing selecting Encrypted root with LVM 
because I wanted to
see if that worked. it didn't, then I tried the default, before posting 
the bug report.
Finally I went though with Miguel's patch selecting all the defaults and 
that was
the result.
I feel the installer should not care what was on the drive to begin 
with, it should just
format the drive how it sees fit. Why should the existing contents of 
the drive
affect the outcome when we are reformatting everything anyway?
>
> Thanks,
>
> Mathieu

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

[Mathieu Othacehe]

Hello Brendan,

> I feel the installer should not care what was on the drive to begin with, it
> should just
> format the drive how it sees fit. Why should the existing contents of the
> drive
> affect the outcome when we are reformatting everything anyway?

That's the case except for the ESP partition that is preserved by the
installer because it could contain vendor / operating system specific
files that should not be wiped.

Mathieu

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

[Miguel Ángel Arruga Vivas]

[-- Attachment #1: Type: text/plain, Size: 644 bytes --]

Hi Mathieu,

Mathieu Othacehe <othacehe@gnu.org> writes:
> Hello Brendan,
>
>> I feel the installer should not care what was on the drive to begin with, it
>> should just
>> format the drive how it sees fit. Why should the existing contents of the
>> drive
>> affect the outcome when we are reformatting everything anyway?
>
> That's the case except for the ESP partition that is preserved by the
> installer because it could contain vendor / operating system specific
> files that should not be wiped.

Is there any other issue remaining?  It doesn't modify this logic at
all.  Should I apply this patch to master then?

Happy hacking!
Miguel

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

[Mathieu Othacehe]

Hello Miguel,

> Is there any other issue remaining?  It doesn't modify this logic at
> all.  Should I apply this patch to master then?

Yes, your patch LGTM. Please make sure that system tests are still
working by running:

--8<---------------cut here---------------start------------->8---
make check-system TESTS="gui-installed-os gui-installed-os-encrypted
gui-installed-desktop-os-encrypted"
--8<---------------cut here---------------end--------------->8---

Thanks,

Mathieu

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

[Miguel Ángel Arruga Vivas]

[-- Attachment #1: Type: text/plain, Size: 526 bytes --]

Hi,

Mathieu Othacehe <othacehe@gnu.org> writes:
> Hello Miguel,
>
>> Is there any other issue remaining?  It doesn't modify this logic at
>> all.  Should I apply this patch to master then?
>
> Yes, your patch LGTM. Please make sure that system tests are still
> working by running:
>
> make check-system TESTS="gui-installed-os gui-installed-os-encrypted
> gui-installed-desktop-os-encrypted"

I pushed it to master as 19c14d95b3 after running successfully these
system tests.

Thank you very much, and happy hacking!
Miguel

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]

bug#44027: [PATCH] installer: Create bios_grub partition when it is needed.

[Bengt Richter]

Hi,

On +2020-10-19 16:57:49 +1100, Brendan Tildesley wrote:
> On 19/10/20 3:03 am, Mathieu Othacehe wrote:
> > Hello Miguel & Brendan,
> > 
> > > AFAIU from the code, the first partition should not be created by the
> > > installer but it won't be removed if it was found on the disk
> > > beforehand.  Tomorrow I'll give a try at the full installation process,
> > > I must have overlooked something if it still being created in that
> > > case...
> > Thanks for your help on this topic! Brendan is probably using a GPT
> > partitionned disk on a non-UEFI compatible machine. Hence, as you
> > noticed, the installer does not create a bios_grub partition.
> > 
> > This is a problem as this partition is necessary for GRUB and I think
> > that your patch is fixing the problem. The "auto-partition!" procedure
> > is also leaving a probably pre-existing ESP partition, but I don't
> > really understand how it appeared in the first place.
> > 
> > Brendan, did you use "manual partitioning" to create an ESP partition?
> Hmm I forget what was on this drive. I used to have Windows 10 on it, then I
> can't
> remember. I may have installed Arch Linux on it. What ever was on it, I just
> ran the
> installer letting it do it's thing selecting Encrypted root with LVM because
> I wanted to
> see if that worked. it didn't, then I tried the default, before posting the
> bug report.
> Finally I went though with Miguel's patch selecting all the defaults and
> that was
> the result.
> I feel the installer should not care what was on the drive to begin with, it
> should just
> format the drive how it sees fit. Why should the existing contents of the
> drive
> affect the outcome when we are reformatting everything anyway?
> > 
> > Thanks,
> > 
> > Mathieu
>

I agree that existing contents should not matter, UNLESS:
You are about to shoot yourself in the foot by overwriting the wrong disk.

I think it is risky to target disks just by /dev/<kname>, so if doing that,
with e.g. /dev/sdb, I would like a safe confirmation dialog displaying the output of

$ lsblk -o mountpoint,type,kname,model,serial /dev/sdb
MOUNTPOINT      TYPE  KNAME     MODEL              SERIAL
                disk  sdb       SanDisk_3.2Gen1    xxxxxx...

with automatic verification that it is not mounted, is type disk, and asking me
if the model and serial number look right. The nice thing about the serial number
is (AFAIK) that it persists for selection even after zapping GPT/MBR UUIDs.

Similar checks before overwriting any pre-existing disk or partition, please. 

Maybe also have a target-disk-serial parameter that can be part of my-system.scm
to lock in the right target disk? Similary for partitions for /boot and / and any others.

(Hm, BTW, how are stable disk type, model and serial number lsblk outputs above
provided for virtual disks? )

[later]
Hm, I decided to test the serial number persistence, and 
the brand new Sandisk_3.2Gen1 made lsblk show a whopping 120-character string
for that xxxxxx... serial, but after zapping it fully with gdisk, the output
was the first 39 characters, Presumably gdisk enforced a 40-byte
buffer size with a null in the last byte.

BUT: Waitaminit -- how could it do that if that buffer was manufactured read-only??
I'll try dd-ing zeroes over the front end, and seeing if there's a serial after that...

So it that a bug in lsblk, or in SanDisk manufacturing?
Well, the confirmation dialog would work either way, but the target-disk-serial parameter
would have to do a prefix-match or something, until all parties get their acts together ;-)
[later]
zapped the first 1GiB of /dev/sdb with dd, and lsblk returned 39 chars like after gdisk zap,
BUT: now it's back to the same long number, which also shows up at /dev/disk/by-id/*

Meanwhile, did: apt install sdparm (mentioned at [1]), and then sdparm got me the first 20 characters
of lsblk's version:

--8<---------------cut here---------------start------------->8---
sudo sdparm --page=sn /dev/sdb
[sudo] password for bokr: 
    /dev/sdb:  USB   SanDisk 3.2Gen1  1.00
Unit serial number VPD page:
  xxxxxxxxxxxxxxxxxxxx
--8<---------------cut here---------------end--------------->8---

and lsblk and /dev/disk/by-id both show the original long serial, 
which suggests they share some code, since they agree and both
disagree with the 20-char version from sdparm.

Per [1], the serial number isn't stored in the data writeable part,
it's in the scsi controller innards, retrieved by saying the right
thing to the controller, unlike other ways of serializing storage.

But is sdparm right? 20 characters seems stingy.

┌──────────────────────────────────┐
│ ISTM there's a bug somewhere ;-) │
└──────────────────────────────────┘

I'm mCurrently running (per uname -a):
Linux LionPure 4.19.0-9-amd64 #1 SMP Debian 4.19.118-2+deb10u1 (2020-06-07) x86_64 GNU/Linux

...without guix though, sorry, I like guix :/
(though there's lots of scraps around, until I can install one with absolute
minimal use of sudo :) 

[1] https://stackoverflow.com/questions/2432759/usb-drive-serial-number-under-linux-c

HTH make things better (at least with the confirmation dialog part :)
And maybe a clue to a bug.
-- 
Regards,
Bengt Richter