How does system-disk-image work with embedded-style boot loaders?

How does system-disk-image work with embedded-style boot loaders?

[Richard Sent]

Hi Guix!

I'm a bit confused as to how a system image is generated when certain
bootloader types are used. For example, rockpro64-rk3399-u-boot.

In (gnu bootloader u-boot) install-rockpro64-rk3399-u-boot has this
definition:

--8<---------------cut here---------------start------------->8---
(define install-rockpro64-rk3399-u-boot
  #~(lambda (bootloader root-index image)
      (let ((idb (string-append bootloader "/libexec/idbloader.img"))
            (u-boot (string-append bootloader "/libexec/u-boot.itb")))
        (write-file-on-device idb (stat:size (stat idb))
                              image (* 64 512))
        (write-file-on-device u-boot (stat:size (stat u-boot))
                              image (* 16384 512)))))
--8<---------------cut here---------------end--------------->8---

Which makes sense. The onboard ROM bootlaoder looks for u-boot at a
specific offset in flash and loads that.

However, I don't understand how we "orchestrate" that with the rest of
the system image to avoid a collision.

At the end of system-disk-image in (gnu system image), we have this:

--8<---------------cut here---------------start------------->8---
(let* ((image-name (image-name image))
         (name (if image-name
                   (symbol->string image-name)
                   name))
         (format (image-format image))
         (substitutable? (image-substitutable? image))
         (builder
          (with-imported-modules*
           (let ((inputs '#+(list genimage coreutils findutils qemu-minimal))
                 (bootloader-installer
                  #+(bootloader-disk-image-installer bootloader))
                 (out-image (string-append "images/" #$genimage-name)))
             (set-path-environment-variable "PATH" '("bin" "sbin") inputs)
             (genimage #$(image->genimage-cfg image))
             ;; Install the bootloader directly on the disk-image.
             (when bootloader-installer
               (bootloader-installer
                #+(bootloader-package bootloader)
                #$(root-partition-index image)
                out-image))
             (convert-disk-image out-image '#$format #$output)))))
    (computed-file name builder
                   #:local-build? #f              ;too I/O-intensive
                   #:options `(#:substitutable? ,substitutable?)))
--8<---------------cut here---------------end--------------->8---

From what I can tell this is a 4 step process.

1. Generate a genimage.cfg file via image->genimage-cfg. This does not
appear to take the bootloader into account.

2. Call genimage in (gnu build image) to create the image defined in the
genimage.cfg file.

3. use bootloader-installer to write directly into the image file that
was just created. In this case we write two files at specific offsets.

4. Convert the raw disk image into the desired output format.

How do we avoid "screwing up" the image by writing directly to the
image? What stops genimage from putting important data there that gets
overwritten in step 3? Is it just coincidence? A convention in the image
file to leave a large amount of space at the beginning? Some other
factor? What if another embedded board uses a different offset that
genimage does happen to use?

I'm *guessing* that the answer lies in the image type and
raw-with-offset-disk-image function, but I'd appreciate any
clarification on this! I see that pinebook-pro-image-type, which also
uses Pine64's rk3399 SoC has an offset of 2^20, or 1,048,576. However

1) There isn't a rockpro64-image-type, only a pinebook-pro-image-type.
Why only have one image type but two bootloaders?

2) install-rockpro64-rk3399-u-boot writes u-boot.itb at (* 16384 512),
or 8,388,608, past the 2^20 offset in the image type. (Likely not
coincidentally 8,388,608 / 8 = 1,048,576. I don't know what to make of this
because it feels weird that bytes are used in one situation while
another uses bits.)

Appreciate any clarification on this!

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: How does system-disk-image work with embedded-style boot loaders?

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 2724 bytes --]

On 2024-05-11, Richard Sent wrote:
> I'm a bit confused as to how a system image is generated when certain
> bootloader types are used. For example, rockpro64-rk3399-u-boot.
...
> From what I can tell this is a 4 step process.
>
> 1. Generate a genimage.cfg file via image->genimage-cfg. This does not
> appear to take the bootloader into account.
>
> 2. Call genimage in (gnu build image) to create the image defined in the
> genimage.cfg file.
>
> 3. use bootloader-installer to write directly into the image file that
> was just created. In this case we write two files at specific offsets.
>
> 4. Convert the raw disk image into the desired output format.
>
> How do we avoid "screwing up" the image by writing directly to the
> image? What stops genimage from putting important data there that gets
> overwritten in step 3? Is it just coincidence? A convention in the image
> file to leave a large amount of space at the beginning? Some other
> factor? What if another embedded board uses a different offset that
> genimage does happen to use?

I'd hope guix is at least using "A convention in the image file to leave
a large amount of space at the beginning" if it is not doing anything
fancy.

Leaving the first 16MB of the partition table empty, or partitioning it
with dedicated partitions at appropriate offsets, or with one big blank
partition ... was at least the approach I used when manually
installing.

16MB covers most rockchip based systems, and all others systems I am
aware of tend to be much smaller than that, thankfully, and... dare I
tempt fate, hopefully stays that way.

I was too involved in the image generation switch to genimage much, so
cannot answer your question of what the code is actually is doing.


> I'm *guessing* that the answer lies in the image type and
> raw-with-offset-disk-image function, but I'd appreciate any
> clarification on this! I see that pinebook-pro-image-type, which also
> uses Pine64's rk3399 SoC has an offset of 2^20, or 1,048,576. However
>
> 1) There isn't a rockpro64-image-type, only a pinebook-pro-image-type.
> Why only have one image type but two bootloaders?
>
> 2) install-rockpro64-rk3399-u-boot writes u-boot.itb at (* 16384 512),
> or 8,388,608, past the 2^20 offset in the image type. (Likely not
> coincidentally 8,388,608 / 8 = 1,048,576. I don't know what to make of this
> because it feels weird that bytes are used in one situation while
> another uses bits.)

Hopefully this is just a bit of confusion, or some of the images are
broken and nobody noticed!

I would suggest to inspect the partition table of a generated image to
confirm if there is an empty space or blank/reserved partition(s)
falling in that range...


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: How does system-disk-image work with embedded-style boot loaders?

[Richard Sent]

>> 2) install-rockpro64-rk3399-u-boot writes u-boot.itb at (* 16384 512),
>> or 8,388,608, past the 2^20 offset in the image type. (Likely not
>> coincidentally 8,388,608 / 8 = 1,048,576. I don't know what to make of this
>> because it feels weird that bytes are used in one situation while
>> another uses bits.)
>
> Hopefully this is just a bit of confusion, or some of the images are
> broken and nobody noticed!
>
> I would suggest to inspect the partition table of a generated image to
> confirm if there is an empty space or blank/reserved partition(s)
> falling in that range...

Hi Vagrant!

Thanks for the response! I looked at the code again and I did realize I
missed one thing regarding point 2. pinebook-pro-image-type isn't using
a 1,048,576 offset, but a 9,437,184 offset. There's a *9 I missed,
oopsie.

Alas, there's still some confusion. We place the u-boot.itb image at an
offset of 16384*512, or 8,388,608. When building the u-boot image—part
of the output from the u-boot-pinebook-pro-rk3399-2024.01 derivation
built as part of $ guix system image gnu/system/images/pinebook-pro.scm
--system=aarch64-linux—the u-boot.itb file is 1,089,024 bytes long. Our
raw image offset is system offset is 9 * 2^20, or 9,437,184 (AKA 9 MiB).

So that means writing a 1,089,024 byte image to 8,388,608 spills over to
9,477,632. This is beyond the 9 MiB offset, so our u-boot image should
be overwriting the root FS.

Running fdisk on the image produces (emulated build due to
cross-compilation failures):

--8<---------------cut here---------------start------------->8---
gibraltar :( guix$ fdisk -l $(guix system image gnu/system/images/pinebook-pro.scm --system=aarch64-linux)
Disk /gnu/store/832w3d7dh2vdfdm2qdv5025w8sfm8zrl-pinebook-pro-barebones-raw-image: 1.62 GiB, 1741840384 bytes, 3402032 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device                                                                        Boot Start     End Sectors  Size Id Type
/gnu/store/832w3d7dh2vdfdm2qdv5025w8sfm8zrl-pinebook-pro-barebones-raw-image1 *    18432 3402031 3383600  1.6G 83 Linux
--8<---------------cut here---------------end--------------->8---

I can't figure out where that 18432 offset is coming from. Also, based
on the genimage configuration used to create this image, I'd expect at
least two partitions:

--8<---------------cut here---------------start------------->8---
;; /gnu/store/2ah9p62crpar2iq2jcni85i4fz4yx9x4-genimage.cfg
image image {
	hdimage {
		partition-table-type = "mbr"
	}
	partition GNU-ESP {
  		partition-type = 0xEF
  		image = "/gnu/store/d8irqwcay0w1wm54l1ydrs3kp6vcdd1a-partition.img"
  		offset = "1048576"
  		bootable = "false"
  	}
	partition Guix_image {
  		partition-type = 0x83
  		image = "/gnu/store/823mxbjirpcqgjkybkq60vc34p6nc218-partition.img"
  		offset = "0"
  		bootable = "true"
  	}
}
--8<---------------cut here---------------end--------------->8---

I do find it surprisingly we split "generate base image with genimage"
and "copy bootloader into base image" into different steps. From what I
see at https://github.com/pengutronix/genimage genimage is entirely
capable of placing the bootloader itself. Perhaps there is some value in
reworking the direct bootloader installers (mostly u-boot I think?) to
use genimage.

--8<---------------cut here---------------start------------->8---
;; example from README
partition bootloader {
    in-partition-table = false
    offset = 0
    image = "/path/to/bootloader.img"
}
--8<---------------cut here---------------end--------------->8---

If we are indeed deleting part of our Linux partition due to an overly
large u-boot image (perhaps the image grew over time?), having genimage
be in charge of copying both the partition images AND the bootloader
file would presumably catch that error. At least, I assume genimage
performs those kinds of sanity checks.

I may very well be missing something here, particularly with
interpreting the fdisk output. Sorry if I didn't make sense. This stuff
is hard! :)

I haven't even started looking into what, if any, logic is done on the
Guix side for u-boot to find the Linux kernel. Spooky. 👻

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: How does system-disk-image work with embedded-style boot loaders?

[Richard Sent]

Richard Sent <richard@freakingpenguin.com> writes:

>
> Running fdisk on the image produces (emulated build due to
> cross-compilation failures):
>
> gibraltar :( guix$ fdisk -l $(guix system image gnu/system/images/pinebook-pro.scm --system=aarch64-linux)
> Disk /gnu/store/832w3d7dh2vdfdm2qdv5025w8sfm8zrl-pinebook-pro-barebones-raw-image: 1.62 GiB, 1741840384 bytes, 3402032 sectors
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disklabel type: dos
> Disk identifier: 0x00000000
>
> Device                                                                        Boot Start     End Sectors  Size Id Type
> /gnu/store/832w3d7dh2vdfdm2qdv5025w8sfm8zrl-pinebook-pro-barebones-raw-image1 *    18432 3402031 3383600  1.6G 83 Linux
>
>
> I can't figure out where that 18432 offset is coming from.
> ...
> I may very well be missing something here, particularly with
> interpreting the fdisk output.

Well, I did figure out one thing. fdisk is giving the offset in sector
size, NOT bytes. With a sector size of 512 bytes, the offset in bytes is
18432*512, or 9437184 bytes. This is what we expect. Yay.

I still suspect u-boot is overwriting part of the root FS in this
particular image.

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.