Supporting *multiple* bootloaders for arm64 on a single install?

Supporting *multiple* bootloaders for arm64 on a single install?

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 3679 bytes --]

So, I've managed to get a single image that supports booting both the
Pinebook and Pinebook Pro reasonably well! I can pop the microSD card
out of one and put it into the other, and it boots!

Maybe this is not a big deal to those used to x86, but in the ARM world,
it is unfortunately more complicated than one would hope:

  https://archive.fosdem.org/2019/schedule/event/one_image_to_rule_them_all/

The only problem with my single image for multiple pinebooks variants is
it requires manually installing u-boot to different offsets for Pinebook
Pro (e.g. idbloader.img at sector 2112 rather than sector 64), as parts
of the Pinebook bootloader are installed at overlapping offsets.

I don't think it would be terribly hard to create an image that also
supports booting EFI! And as u-boot can provide an EFI implementation,
it would not be totally unreasonable to want to install both u-boot and
EFI in the same system configuration...

But as I understand it, guix only supports a single bootloader entry. To
support all of the above, I would need three separate bootloader
instances... one for Pinebook, one for Pinebook Pro, and lastly a
grub-efi bootloader.

Installing u-boot-pinebook uses:

(define install-allwinner64-u-boot
  #~(lambda (bootloader root-index image)
      (let ((spl (string-append bootloader "/libexec/u-boot-sunxi-with-spl.bin"))
            (u-boot (string-append bootloader "/libexec/u-boot-sunxi-with-spl.fit.itb")))
        (write-file-on-device spl (stat:size (stat spl))
                              image (* 8 1024))
        (write-file-on-device u-boot (stat:size (stat u-boot))
                              image (* 40 1024)))))

Installing u-boot-pinebook-pro-rk3399 uses:

(define install-rockpro64-rk3399-u-boot
  #~(lambda (bootloader root-index image)
      (let ((idb (string-append bootloader "/libexec/idbloader.img"))
            (u-boot (string-append bootloader "/libexec/u-boot.itb")))
        (write-file-on-device idb (stat:size (stat idb))
                              image (* 64 512))
        (write-file-on-device u-boot (stat:size (stat u-boot))
                              image (* 16384 512)))))

(define install-pinebook-pro-rk3399-u-boot install-rockpro64-rk3399-u-boot)

;; yes, these could be refactored into a single function!

But now I need to figure out how to pass a non-default offset for the
"idb" part for rockchip platforms.

In a system config.scm, you'd define it like so:

  (bootloader (bootloader-configuration
               (bootloader u-boot-pinebook-pro-rk3399-bootloader)
               (target "/dev/mmcblk0")))

u-boot-pinebook-pro-rk3399-bootloader is defined in
gnu/bootloader/u-boot.scm, which inherits from u-boot-bootloader, which
inherits from extlinux-bootloader in gnu/bootloader/extlinux.scm...

And somewhere along the way I've lost track of how we get to
install-pinebook-pro-rk3399-u-boot...


Is it possible to definte multiple bootloaders currently, or if not,
what would need to change to be able to support that? Where would one
pass non-default offsets for a given platform?


Strictly speaking, the extlinux.conf generation would be optional for an
EFI generated image(as u-boot can load grub-efi), although at the moment
I'd prefer using extlinux.conf if available as it makes for more
consistent matching of device-tree/.dtb file with the running kernel...


A related idea would be to generate an EFI bootable image with
sufficient emtpy space before the first partition (16MB) and then one
could manually install the appropriate u-boot, maybe with some helper
scripts to avoid having to do it completely manually...


Thoughts, advice, help? :)


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Supporting *multiple* bootloaders for arm64 on a single install?

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 6660 bytes --]

Hi Vagrant,

On Sun, 06 Jun 2021 14:38:27 -0700
Vagrant Cascadian <vagrant@debian.org> wrote:

> So, I've managed to get a single image that supports booting both the
> Pinebook and Pinebook Pro reasonably well! I can pop the microSD card
> out of one and put it into the other, and it boots!

Awesome!

> The only problem with my single image for multiple pinebooks variants is
> it requires manually installing u-boot to different offsets for Pinebook
> Pro (e.g. idbloader.img at sector 2112 rather than sector 64), 

>as parts of the Pinebook bootloader are installed at overlapping offsets.

Uhhh?  Doesn't sound possible to me then to use the same image for both?
I'm probably missing something.

> But as I understand it, guix only supports a single bootloader entry. 

That is correct.

> To support all of the above, I would need three separate bootloader
> instances... one for Pinebook, one for Pinebook Pro, and lastly a
> grub-efi bootloader.

Stefan wrote a Guix chain bootloader that is in master--but it's meant
to be only used for bootloaders where there is a primary "bare-metal-loaded"
bootloader and the others are chain-loaded one-after-another from ONE filesystem
(for example Raspberry Pi and/or EFI bootloaders).

(It's currently used in order to use an EFI bootloader hosted on NFS--which
is an awesome way to manage embedded boards)

The chain bootloader itself is one Guix bootloader.

I advise you to search for mails by Stefan on the guix-devel mailing list--those
are very illuminating.

> Installing u-boot-pinebook uses:

> (define install-allwinner64-u-boot
>   #~(lambda (bootloader root-index image)
>       (let ((spl (string-append bootloader "/libexec/u-boot-sunxi-with-spl.bin"))
>             (u-boot (string-append bootloader "/libexec/u-boot-sunxi-with-spl.fit.itb")))
>         (write-file-on-device spl (stat:size (stat spl))
>                               image (* 8 1024))
>         (write-file-on-device u-boot (stat:size (stat u-boot))
>                               image (* 40 1024)))))
> 
> Installing u-boot-pinebook-pro-rk3399 uses:
> 
> (define install-rockpro64-rk3399-u-boot
>   #~(lambda (bootloader root-index image)

    #~(lambda* (bootloader root-index image #:key idb-destination-offset)

>       (let ((idb (string-append bootloader "/libexec/idbloader.img"))
>             (u-boot (string-append bootloader "/libexec/u-boot.itb")))
>         (write-file-on-device idb (stat:size (stat idb))
>                               image (* 64 512))


                                image (or idb-destination-offset (* 64 512)))

>         (write-file-on-device u-boot (stat:size (stat u-boot))
>                               image (* 16384 512)))))
> 
> (define install-pinebook-pro-rk3399-u-boot install-rockpro64-rk3399-u-boot)
> 
> But now I need to figure out how to pass a non-default offset for the
> "idb" part for rockchip platforms.
> 
> In a system config.scm, you'd define it like so:
> 
>   (bootloader (bootloader-configuration
>                (bootloader u-boot-pinebook-pro-rk3399-bootloader)
>                (target "/dev/mmcblk0")))
> 
> u-boot-pinebook-pro-rk3399-bootloader is defined in
> gnu/bootloader/u-boot.scm, which inherits from u-boot-bootloader, which
> inherits from extlinux-bootloader in gnu/bootloader/extlinux.scm...
> 
> And somewhere along the way I've lost track of how we get to
> install-pinebook-pro-rk3399-u-boot...

It's in the "bootloader" record, field "disk-image-installer".

If you search for "bootloader-disk-image-installer" in the Guix source code,
you find the (two) callers of it.

> Is it possible to definte multiple bootloaders [in the same operating-system] currently, 

No--and I don't think that would have any advantages.

If the thing is not a chain then Guix doesn't need to logically split the
bootloaders in the first place.

Ok: stage1 -> stage2 -> stage3 represented as (chain-bootloaders stage1 stage2 stage3).

Not ok: stage1-arch1 stage1-arch2 represented as two different bootloaders
that both are specified in the same operating-system inside the same list.

Instead, I think it's fine to just model that as one "weird" bootloader
(arm64-pinebook-generic-bootloader)--especially if both use u-boot anyway and also
especially if they reuse some of their parts anyway (it's gonna have weird
interdependencies in any case--so any apparent modularity would just be a ruse).

> or if not,
> what would need to change to be able to support that? Where would one
> pass non-default offsets for a given platform?

Could you elaborate on what you mean?  Pass where?  On the Guix command line?

If you mean the disk-image-installer, that's just a procedure (which is run
build-side).  You can make it do whatever you want--including hardcoding weird
offsets.  I'm pretty sure it gets the entire image (one could potentially mangle)
as a parameter anyway.

> Strictly speaking, the extlinux.conf generation would be optional for an
> EFI generated image(as u-boot can load grub-efi), although at the moment
> I'd prefer using extlinux.conf if available as it makes for more
> consistent matching of device-tree/.dtb file with the running kernel...

If you do want to chainload grub-efi eventually, that's supported in Guix master,
see "efi-bootloader-chain".  It's meant for the end user to be able to use.

> A related idea would be to generate an EFI bootable image with
> sufficient emtpy space before the first partition (16MB) and then one
> could manually install the appropriate u-boot, maybe with some helper
> scripts to avoid having to do it completely manually...

Yeah, that's possible right now.  After all, you don't have to load the generated
guix system disk image on bare metal.  Just boot it in qemu and install u-boot
there for example.  Or even edit the image manually by using dd.  (unfortunately,
on almost every ARM system I set up Guix on so far, one or both of those were
necessary)

That said, it's nice to have officially supported ARM systems where you can just
generate a disk image and it will boot without having to do weird extra stuff
outside Guix.

(In general, I want to reiterate that the buildroot project already has genimage
scripts for almost all ARM boards--and Guix uses genimage anyway!  It makes a lot
of sense to write an importer for those instead of reinventing the wheel.  You
really do not want to develop & maintain u-boot installation recipes for all
34432 ARM variants out there yourself--just use the recipes already developed
instead)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Supporting *multiple* bootloaders for arm64 on a single install?

[Stefan]

Hi!

>> But as I understand it, guix only supports a single bootloader entry. 
> 
> That is correct.
> 
>> To support all of the above, I would need three separate bootloader
>> instances... one for Pinebook, one for Pinebook Pro, and lastly a
>> grub-efi bootloader.
> 
> Stefan wrote a Guix chain bootloader that is in master--but it's meant
> to be only used for bootloaders where there is a primary "bare-metal-loaded"
> bootloader and the others are chain-loaded one-after-another from ONE filesystem
> (for example Raspberry Pi and/or EFI bootloaders).
> 
> (It's currently used in order to use an EFI bootloader hosted on NFS--which
> is an awesome way to manage embedded boards)
> 
> The chain bootloader itself is one Guix bootloader.
> 
> I advise you to search for mails by Stefan on the guix-devel mailing list--those
> are very illuminating.

By the way, there is <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=48314> since a month meanwhile, which now makes the support for the Raspberry Pi complete. The same bootloader can be used for an NFS root installation and an installation on micro SD card or an USB drive by just changing the mount point of the root file-system.

Danny, I’d appreciate your review very much. This time you can apply the patches and just use the gnu/system/examples/raspberry-pi-64.tmpl with guix system init. :-) 

>> And somewhere along the way I've lost track of how we get to
>> install-pinebook-pro-rk3399-u-boot…

The same happened to me. I’ll first wait for the patch #48314 to be accepted, before I’ll look again into creating a disk image for the Raspberry Pi.

> If you do want to chainload grub-efi eventually, that's supported in Guix master,
> see "efi-bootloader-chain".  It's meant for the end user to be able to use.

Referring to the patch #48314, the efi-bootloader-chain got simplified. It basically prepares a profile with all files to be copied as is, no special “collection” folder any more. And the installer of the grub-efi-netboot-(removable-)bootloader is now merely a simple “(copy-recursively bootloader-profile)“ procedure.

>> A related idea would be to generate an EFI bootable image with
>> sufficient emtpy space before the first partition (16MB) and then one
>> could manually install the appropriate u-boot, maybe with some helper
>> scripts to avoid having to do it completely manually...
> 
> Yeah, that's possible right now.  After all, you don't have to load the generated
> guix system disk image on bare metal.  Just boot it in qemu and install u-boot
> there for example.  Or even edit the image manually by using dd.  (unfortunately,
> on almost every ARM system I set up Guix on so far, one or both of those were
> necessary)

I believe that the ARM future is UEFI, like on PCs. A lot of distributions already chainload U-Boot and GRUB an arm systems. There is a specification for ARM servers which demands UEFI¹. And there is even the choice between U-Boot and TianoCore/EDK II.

I think meanwhile that it would be beneficial, if, beside a bootloader field, Guix would accept an optional firmware field. Then the U-Boot or TianoCore/EDK II (or coreboot) could become some board specific firmware, and the actual bootloader would be grub-efi installed on an EFI System Partition. For systems like the Raspberry, which require their firmware on a FAT partition, there is the already working efi-bootloader-chain solution.

The firmware could only be installed on explicit request. It's not expected to see frequent changes – every re-installation is a risk to brick the system. (For my taste the bootloader is re-installed too often already.)


Bye

Stefan


¹ <https://documentation-service.arm.com/static/5fb7e415d77dd807b9a80c80>

Re: Supporting *multiple* bootloaders for arm64 on a single install?

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 3268 bytes --]

On 2021-06-19, Stefan wrote:
>>> To support all of the above, I would need three separate bootloader
>>> instances... one for Pinebook, one for Pinebook Pro, and lastly a
>>> grub-efi bootloader.
...
>> The chain bootloader itself is one Guix bootloader.
>> 
>> I advise you to search for mails by Stefan on the guix-devel mailing list--those
>> are very illuminating.
...
>> If you do want to chainload grub-efi eventually, that's supported in Guix master,
>> see "efi-bootloader-chain".  It's meant for the end user to be able to use.
>
> Referring to the patch #48314, the efi-bootloader-chain got
> simplified. It basically prepares a profile with all files to be
> copied as is, no special “collection” folder any more. And the
> installer of the grub-efi-netboot-(removable-)bootloader is now merely
> a simple “(copy-recursively bootloader-profile)“ procedure.

>>> A related idea would be to generate an EFI bootable image with
>>> sufficient emtpy space before the first partition (16MB) and then one
>>> could manually install the appropriate u-boot, maybe with some helper
>>> scripts to avoid having to do it completely manually...
...
> I believe that the ARM future is UEFI, like on PCs. A lot of
> distributions already chainload U-Boot and GRUB an arm systems. There
> is a specification for ARM servers which demands UEFI¹. And there is
> even the choice between U-Boot and TianoCore/EDK II.

Yes, standardizing on UEFI in general would be a good thing, especially
because u-boot can increasingly fill that role for boards that don't
have a tianocore/edkII implementation.

My heart is more with u-boot, but i also value standardization. :)


> I think meanwhile that it would be beneficial, if, beside a bootloader
> field, Guix would accept an optional firmware field. Then the U-Boot
> or TianoCore/EDK II (or coreboot) could become some board specific
> firmware, and the actual bootloader would be grub-efi installed on an
> EFI System Partition. For systems like the Raspberry, which require
> their firmware on a FAT partition, there is the already working
> efi-bootloader-chain solution.

Yeah, I like that idea! Das U-boot functionally is both kind of boot
firmware as well as a bootloader rolled into one; I could see treating
it more as a firmware which may not actually even need a separate
bootloader (other than maybe generating a boot script or extlinux.conf),
or a firmware implments EFI (and thus can load grub-efi).


> The firmware could only be installed on explicit request. It's not
> expected to see frequent changes – every re-installation is a risk to
> brick the system. (For my taste the bootloader is re-installed too
> often already.)

Mixed feelings on that; it puts you in the situation of not knowing when
the current guix bootloader/firmware is broken if you don't reinstall it
(although maybe it could be made smarter and only reinstall
bootloader/firmware when it actually changes).

It's already possible to pass an empty bootloader that just generates
the configuration files for grub or extlinux which doesn't re-install(to
my knowledge). Extending that to the proposed firmware idea would make
some sense...


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Supporting *multiple* bootloaders for arm64 on a single install?

[Bengt Richter]

Hi Stefan, et al,

On +2021-06-19 17:11:45 +0200, Stefan wrote:
> Hi!
> 
> >> But as I understand it, guix only supports a single bootloader entry. 
> > 
> > That is correct.
> > 
> >> To support all of the above, I would need three separate bootloader
> >> instances... one for Pinebook, one for Pinebook Pro, and lastly a
> >> grub-efi bootloader.
> > 
> > Stefan wrote a Guix chain bootloader that is in master--but it's meant
> > to be only used for bootloaders where there is a primary "bare-metal-loaded"
> > bootloader and the others are chain-loaded one-after-another from ONE filesystem
> > (for example Raspberry Pi and/or EFI bootloaders).
> > 
> > (It's currently used in order to use an EFI bootloader hosted on NFS--which
> > is an awesome way to manage embedded boards)
> > 
> > The chain bootloader itself is one Guix bootloader.
> > 
> > I advise you to search for mails by Stefan on the guix-devel mailing list--those
> > are very illuminating.
> 
> By the way, there is <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=48314> since a month meanwhile, which now makes the support for the Raspberry Pi complete. The same bootloader can be used for an NFS root installation and an installation on micro SD card or an USB drive by just changing the mount point of the root file-system.
> 
> Danny, I’d appreciate your review very much. This time you can apply the patches and just use the gnu/system/examples/raspberry-pi-64.tmpl with guix system init. :-) 
> 
> >> And somewhere along the way I've lost track of how we get to
> >> install-pinebook-pro-rk3399-u-boot…
> 
> The same happened to me. I’ll first wait for the patch #48314 to be accepted, before I’ll look again into creating a disk image for the Raspberry Pi.
> 
> > If you do want to chainload grub-efi eventually, that's supported in Guix master,
> > see "efi-bootloader-chain".  It's meant for the end user to be able to use.
> 
> Referring to the patch #48314, the efi-bootloader-chain got simplified. It basically prepares a profile with all files to be copied as is, no special “collection” folder any more. And the installer of the grub-efi-netboot-(removable-)bootloader is now merely a simple “(copy-recursively bootloader-profile)“ procedure.
> 
> >> A related idea would be to generate an EFI bootable image with
> >> sufficient emtpy space before the first partition (16MB) and then one
> >> could manually install the appropriate u-boot, maybe with some helper
> >> scripts to avoid having to do it completely manually...
> > 
> > Yeah, that's possible right now.  After all, you don't have to load the generated
> > guix system disk image on bare metal.  Just boot it in qemu and install u-boot
> > there for example.  Or even edit the image manually by using dd.  (unfortunately,
> > on almost every ARM system I set up Guix on so far, one or both of those were
> > necessary)
> 
> I believe that the ARM future is UEFI, like on PCs. A lot of distributions already chainload U-Boot and GRUB an arm systems. There is a specification for ARM servers which demands UEFI¹. And there is even the choice between U-Boot and TianoCore/EDK II.
> 
> I think meanwhile that it would be beneficial, if, beside a bootloader field, Guix would accept an optional firmware field. Then the U-Boot or TianoCore/EDK II (or coreboot) could become some board specific firmware, and the actual bootloader would be grub-efi installed on an EFI System Partition. For systems like the Raspberry, which require their firmware on a FAT partition, there is the already working efi-bootloader-chain solution.
> 
> The firmware could only be installed on explicit request. It's not expected to see frequent changes – every re-installation is a risk to brick the system. (For my taste the bootloader is re-installed too often already.)
>

What form would a "firmware field" take?

On principle, I am against boundless _incorporation_ of new dangerous capabilities into a piece of software, as opposed
to incoporating the ability to chain-load or otherwise encapsulate execution of a single-purpose,
minimal-source-for-better-inspection-and-exclusion-of-attacks-piece-of-software that does something dangerous.

ISTM UEFI is way more complicated than booting needs to be. What does the boot process need to do besides
(after post) identify a series of untrusted(!) block stream sources to try, load the first image whose secure hash either matches
a white list -- or securely display the hash of the unrecognized image and ask authorized operator for ok + hash nickname
for inclusion in the whitelist or reject? If ok, jump into boot image as normal.

If a developer I trust says (in a securely communicated way) that I can safely load something with a hash of whatever,
I think that is more trustworthy than pretty much anything else I can think of. And on a forum, someone else can say,
"Don't trust that thing with hash xxx...zzz, it blew up for me," and I can hold off until there's a consensus.

WDYT?

BTW, why not build multiple installer ISOs targeted for different architectures, and specialized needs?
(for smaller ISOs and other benefits). I assume one could already do this with guix, but why not leave the
whole ball-of-wax to git clone, and let people with common architectures have less to download and less
irrelevant-to-them choices?

> 
> Bye
> 
> Stefan
> 
> 
> ¹ <https://documentation-service.arm.com/static/5fb7e415d77dd807b9a80c80>

-- 
Regards,
Bengt Richter

Re: Supporting *multiple* bootloaders for arm64 on a single install?

[Stefan]

Hi Bengt!

> What form would a "firmware field" take?

In principle a gexp, which installs a firmware package, or #f. On x86 only #f is needed – unless you want Guix to install e.g. coreboot as your firmware.

> On principle, I am against boundless _incorporation_ of new dangerous capabilities into a piece of software, as opposed
> to incoporating the ability to chain-load or otherwise encapsulate execution of a single-purpose,
> minimal-source-for-better-inspection-and-exclusion-of-attacks-piece-of-software that does something dangerous.

Well, GRUB has some nice features, which e.g. U-Boot hasn’t. It would be comfortable to use these features on other architectures than x86, too, e.g. graphics, modifying the kernel arguments, LUKS. Treating U-Boot as firmware makes GRUB the de-facto bootloader and brings ARM and other architectures on par with x86.

> ISTM UEFI is way more complicated than booting needs to be. What does the boot process need to do besides
> (after post) identify a series of untrusted(!) block stream sources to try, load the first image whose secure hash either matches
> a white list -- or securely display the hash of the unrecognized image and ask authorized operator for ok + hash nickname
> for inclusion in the whitelist or reject? If ok, jump into boot image as normal.
> 
> If a developer I trust says (in a securely communicated way) that I can safely load something with a hash of whatever,
> I think that is more trustworthy than pretty much anything else I can think of. And on a forum, someone else can say,
> "Don't trust that thing with hash xxx...zzz, it blew up for me," and I can hold off until there's a consensus.
> 
> WDYT?

Chain loading U-Boot, iPXE and GRUB offers iSCSI capabilities, using the network driver from U-Boot, the TCP and iSCSI stack from iPXE, GRUB as the front-end¹ and iSCSI as block device for the OS. This enables possibilities you otherwise would not have. But of course this is not against your argument, that it gets more complex. But it offers new possibilities, which were not possible otherwise.

Anyway, I do not suggest to degrade U-Boot to be a firmware only, it can be a bootloader. But offering U-Boot as a firmware makes other architectures similar to x86, offering the same possibilities.

Actually I think any firmware to make GRUB usable is a good fit for a firmware. Using coreboot as firmware and GRUB as its payload is a good fit. If your system has a firmware capable to start GRUB, fine as well, nothing to do. If Guix needs to install a firmware first, before GRUB becomes usable, then I think there is a need for a firmware field. If you want to treat U-Boot as a firmware-and-bootloader, then use it as bootloader and omit the firmware field.

> BTW, why not build multiple installer ISOs targeted for different architectures, and specialized needs?
> (for smaller ISOs and other benefits). I assume one could already do this with guix, but why not leave the
> whole ball-of-wax to git clone, and let people with common architectures have less to download and less
> irrelevant-to-them choices?

I think I don't understand what you are talking about here.


Bye

Stefan


¹ <https://fosdem.org/2020/schedule/event/firmware_duwu/attachments/slides/3867/export/events/attachments/firmware_duwu/slides/3867/Discover_UEFI_with_U_Boot_16_9.pdf> page 22

Re: Supporting *multiple* bootloaders for arm64 on a single install?

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 10441 bytes --]

On 2021-06-19, Danny Milosavljevic wrote:
> On Sun, 06 Jun 2021 14:38:27 -0700
> Vagrant Cascadian <vagrant@debian.org> wrote:
>
>> So, I've managed to get a single image that supports booting both the
>> Pinebook and Pinebook Pro reasonably well! I can pop the microSD card
>> out of one and put it into the other, and it boots!
>
> Awesome!

Thanks! and thanks for stirring up this quiet topic again :)

>> The only problem with my single image for multiple pinebooks variants is
>> it requires manually installing u-boot to different offsets for Pinebook
>> Pro (e.g. idbloader.img at sector 2112 rather than sector 64), 
>
>>as parts of the Pinebook bootloader are installed at overlapping offsets.
>
> Uhhh?  Doesn't sound possible to me then to use the same image for both?
> I'm probably missing something.

Different parts of the image can be written to different offsets on the
boot media in a way that doesn't conflict. The boot ROM on allwinner and
rockchip have a series of offsets they check for compatible boot code,
and the offsets are different enough that you can have a pairing of
allwinner and rockchip boards installed on the same boot media. The pair
that makes "obvious" sense is the pinebook and pinebook-pro, though
nearly any pairing of allwinner and rockchip boards would likely work.


>> To support all of the above, I would need three separate bootloader
>> instances... one for Pinebook, one for Pinebook Pro, and lastly a
>> grub-efi bootloader.
>
> Stefan wrote a Guix chain bootloader that is in master--but it's meant
> to be only used for bootloaders where there is a primary "bare-metal-loaded"
> bootloader and the others are chain-loaded one-after-another from ONE filesystem
> (for example Raspberry Pi and/or EFI bootloaders).
>
> (It's currently used in order to use an EFI bootloader hosted on NFS--which
> is an awesome way to manage embedded boards)
>
> The chain bootloader itself is one Guix bootloader.
>
> I advise you to search for mails by Stefan on the guix-devel mailing list--those
> are very illuminating.

Sounds possibly useful for this, yes....


>> Installing u-boot-pinebook uses:
>
>> (define install-allwinner64-u-boot
>>   #~(lambda (bootloader root-index image)
>>       (let ((spl (string-append bootloader "/libexec/u-boot-sunxi-with-spl.bin"))
>>             (u-boot (string-append bootloader "/libexec/u-boot-sunxi-with-spl.fit.itb")))
>>         (write-file-on-device spl (stat:size (stat spl))
>>                               image (* 8 1024))
>>         (write-file-on-device u-boot (stat:size (stat u-boot))
>>                               image (* 40 1024)))))
>> 
>> Installing u-boot-pinebook-pro-rk3399 uses:
>> 
>> (define install-rockpro64-rk3399-u-boot
>>   #~(lambda (bootloader root-index image)
>
>     #~(lambda* (bootloader root-index image #:key idb-destination-offset)
>
>>       (let ((idb (string-append bootloader "/libexec/idbloader.img"))
>>             (u-boot (string-append bootloader "/libexec/u-boot.itb")))
>>         (write-file-on-device idb (stat:size (stat idb))
>>                               image (* 64 512))
>
>
>                                 image (or idb-destination-offset (* 64 512)))
>
>>         (write-file-on-device u-boot (stat:size (stat u-boot))
>>                               image (* 16384 512)))))
>> 
>> (define install-pinebook-pro-rk3399-u-boot install-rockpro64-rk3399-u-boot)

Thanks! So I would need to add this in the bootloader section of
config.scm and add the optional idb-destination-offset argument to all
the intermediary functions...


>> But now I need to figure out how to pass a non-default offset for the
>> "idb" part for rockchip platforms.
>> 
>> In a system config.scm, you'd define it like so:
>> 
>>   (bootloader (bootloader-configuration
>>                (bootloader u-boot-pinebook-pro-rk3399-bootloader)
>>                (target "/dev/mmcblk0")))  
>> 
>> u-boot-pinebook-pro-rk3399-bootloader is defined in
>> gnu/bootloader/u-boot.scm, which inherits from u-boot-bootloader, which
>> inherits from extlinux-bootloader in gnu/bootloader/extlinux.scm...
>> 
>> And somewhere along the way I've lost track of how we get to
>> install-pinebook-pro-rk3399-u-boot...
>
> It's in the "bootloader" record, field "disk-image-installer".
>
> If you search for "bootloader-disk-image-installer" in the Guix source code,
> you find the (two) callers of it.
>

>> Is it possible to definte multiple bootloaders [in the same operating-system] currently, 
>
> No--and I don't think that would have any advantages.
>
> If the thing is not a chain then Guix doesn't need to logically split the
> bootloaders in the first place.
>
> Ok: stage1 -> stage2 -> stage3 represented as (chain-bootloaders stage1 stage2 stage3).
>
> Not ok: stage1-arch1 stage1-arch2 represented as two different bootloaders
> that both are specified in the same operating-system inside the same list.
>
> Instead, I think it's fine to just model that as one "weird" bootloader
> (arm64-pinebook-generic-bootloader)--especially if both use u-boot anyway and also
> especially if they reuse some of their parts anyway (it's gonna have weird
> interdependencies in any case--so any apparent modularity would just be a ruse).

Yeah, the two u-boot (pinebook, pinebook-pro-rk3399) used are entirely
independent, neither of those are part of a single chain. But either
u-boot variant *could* load grub-efi (and does look for potentialy EFI
images if there's no extlinux.conf or boot.scr) as part of a bootloader
chain...


>> or if not,
>> what would need to change to be able to support that? Where would one
>> pass non-default offsets for a given platform?
>
> Could you elaborate on what you mean?  Pass where?  On the Guix command line?

In config.scm. Sounds like making a specific
arm64-pinebook-generic-bootloader like you suggested above might be an
option, although it would be nice to be able to generalize it enough so
that you could pick semi-arbitrary u-boot combinations (as long as
they're compatible).


> If you mean the disk-image-installer, that's just a procedure (which is run
> build-side).  You can make it do whatever you want--including hardcoding weird
> offsets.  I'm pretty sure it gets the entire image (one could potentially mangle)
> as a parameter anyway.

I've never successfully or knowingly used the disk-image-installer, as
it doesn't support native builds. :/

Mostly, I've been doing "guix system init" and "guix system
reconfigure", using the u-boot-pinebook-bootloader in config.scm since
the default offsets work, and then manually installing the appropriate
u-boot-pinebook-rk3399 bits, due to needing to use non-default offsets.


>> Strictly speaking, the extlinux.conf generation would be optional for an
>> EFI generated image(as u-boot can load grub-efi), although at the moment
>> I'd prefer using extlinux.conf if available as it makes for more
>> consistent matching of device-tree/.dtb file with the running kernel...
>
> If you do want to chainload grub-efi eventually, that's supported in Guix master,
> see "efi-bootloader-chain".  It's meant for the end user to be able to use.

I'll look into it, thanks!


>> A related idea would be to generate an EFI bootable image with
>> sufficient emtpy space before the first partition (16MB) and then one
>> could manually install the appropriate u-boot, maybe with some helper
>> scripts to avoid having to do it completely manually...
>
> Yeah, that's possible right now.  After all, you don't have to load the generated
> guix system disk image on bare metal.  Just boot it in qemu and install u-boot
> there for example.

No need to boot into qemu to install u-boot, it's just typically write a
couple binaries to a specific offset in the resulting image...


> Or even edit the image manually by using dd.  (unfortunately,
> on almost every ARM system I set up Guix on so far, one or both of those were
> necessary)

Curious. I've had good luck with:

  (bootloader (bootloader-configuration
                (bootloader u-boot-pinebook-pro-rk3399-bootloader)
                (target "/dev/mmcblk0")))  

Occasionally having to do weird things like passing /dev/sda instead if
installing to a USB adapter for microSD or something.


> That said, it's nice to have officially supported ARM systems where you can just
> generate a disk image and it will boot without having to do weird extra stuff
> outside Guix.

That is the goal!


> (In general, I want to reiterate that the buildroot project already has genimage
> scripts for almost all ARM boards--and Guix uses genimage anyway!  It makes a lot
> of sense to write an importer for those instead of reinventing the wheel.  You
> really do not want to develop & maintain u-boot installation recipes for all
> 34432 ARM variants out there yourself--just use the recipes already developed
> instead)

Admittedly, I haven't looked into buildroot's genimage scripts beyond
just a cursory glance... though have been meaning to even since you
mentioned it what now seems ages ago. Maybe an importer would make a lot
of sense, though it would likely loose some flexibility along the way
(like my mad scientist schemes to install two u-boots on the same
system).

I've written some simple shell scripts for allwinner and rockchip
systems for Debian that basically just dd various bits to offsets.

And of course, I have looked at the code to do essentially the same
thing in guix but in guile... I do wonder if the u-boot-FOO to
install-u-boot-FOO chain could be refactored to remove a few
intermediate steps, which is what I was hinting at in my earlier mail.

When using upstream u-boot, it is a relatively small set of
instructions; e.g. allwinner uses one set of instructions, rockchip uses
another set of instructions, ti-family processors another, imx
processors, and maybe a 32-bit variant for some of them... so ~6
standard installation sets covers hundreds or even thousands of boards,
with a few outliers that differ. Most other platforms I'm aware of
require non-free bits so are out of scope for guix.


And thanks for reading thus far my rambling post about niche corner
cases that are actually useful! :)


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]