cannot boot from microSD card on aarch64 machine

cannot boot from microSD card on aarch64 machine

[Ricardo Wurmus]

Hi,

I got myself a rockpro64 board.  I put armbian
(Armbian_21.08.1_Rockpro64_bullseye_current_5.10.60.img) on a microSD
card to check that it’s all working correctly.  Then I downloaded the
latest Guix System image for a pinebook pro and dd’d it to a USB drive.
With the armbian microSD card inserted (which contains an old version of
u-boot) I can boot from USB into Guix System.

So I then decided to build my own system and install a new u-boot onto
the microSD card.  Here’s the system I built:

--8<---------------cut here---------------start------------->8---
(operating-system
  (host-name "foo")
  (timezone "Europe/Berlin")
  (locale "en_US.utf8")
  (bootloader (bootloader-configuration
               (bootloader u-boot-rockpro64-rk3399-bootloader)
               ;; SD card/eMMC (SD priority) storage
               (target "/dev/mmcblk1")))
  (kernel linux-libre)
  (kernel-arguments
   (list "console=ttyS2")) ; UART2 connected on the Pi2 bus
  (file-systems (cons (file-system
                        (device "my-root")
                        (title 'label)
                        (mount-point "/")
                        (type "ext4"))
                      %base-file-systems))
  ...)
--8<---------------cut here---------------end--------------->8---

The actual system also has one more initrd module, which I can’t
remember now, but that shouldn’t matter.

I mounted the target disk (an SSD) at /mnt, made sure that the microSD
card was inserted, unmounted, and available at /dev/mmcblk1, and then
ran

   guix system init config.scm /mnt

This was successful.

Unfortunately, rebooting failed.  There is seemingly no activity at all
when I power on the board, so something’s not right with how u-boot was
installed on the microSD card.

Does it perhaps install the u-boot files at the wrong offsets?  How
would I begin to debug this?

I’d appreciate any hints.

-- 
Ricardo

Re: cannot boot from microSD card on aarch64 machine

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2904 bytes --]

On Wed, Mar 02, 2022 at 12:37:53PM +0100, Ricardo Wurmus wrote:
> Hi,
> 
> I got myself a rockpro64 board.  I put armbian
> (Armbian_21.08.1_Rockpro64_bullseye_current_5.10.60.img) on a microSD
> card to check that it’s all working correctly.  Then I downloaded the
> latest Guix System image for a pinebook pro and dd’d it to a USB drive.
> With the armbian microSD card inserted (which contains an old version of
> u-boot) I can boot from USB into Guix System.
> 
> So I then decided to build my own system and install a new u-boot onto
> the microSD card.  Here’s the system I built:
> 
> --8<---------------cut here---------------start------------->8---
> (operating-system
>   (host-name "foo")
>   (timezone "Europe/Berlin")
>   (locale "en_US.utf8")
>   (bootloader (bootloader-configuration
>                (bootloader u-boot-rockpro64-rk3399-bootloader)
>                ;; SD card/eMMC (SD priority) storage
>                (target "/dev/mmcblk1")))
>   (kernel linux-libre)
>   (kernel-arguments
>    (list "console=ttyS2")) ; UART2 connected on the Pi2 bus
>   (file-systems (cons (file-system
>                         (device "my-root")
>                         (title 'label)
>                         (mount-point "/")
>                         (type "ext4"))
>                       %base-file-systems))
>   ...)
> --8<---------------cut here---------------end--------------->8---
> 
> The actual system also has one more initrd module, which I can’t
> remember now, but that shouldn’t matter.
> 
> I mounted the target disk (an SSD) at /mnt, made sure that the microSD
> card was inserted, unmounted, and available at /dev/mmcblk1, and then
> ran
> 
>    guix system init config.scm /mnt
> 
> This was successful.
> 
> Unfortunately, rebooting failed.  There is seemingly no activity at all
> when I power on the board, so something’s not right with how u-boot was
> installed on the microSD card.
> 
> Does it perhaps install the u-boot files at the wrong offsets?  How
> would I begin to debug this?
> 
> I’d appreciate any hints.

Are you sure the primary device is /dev/mmcblk1? On my pine64 it's
mmcblk0.

I'd start by putting u-boot and the image on the sd-card and see if Guix
boots up that way. If it works then there's a larger chance there's a
mistake somewhere in a config.

Perhaps you need to mark the USB drive as a dependency for your root
partition?

Actually, better idea. Try it with linux-libre-arm64-generic for the
kernel. I haven't tried recently but in the past I haven't been able to
use the linux-libre kernel with my pine64 and needed the arm64-generic
version.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: cannot boot from microSD card on aarch64 machine

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1692 bytes --]

On 2022-03-02, Ricardo Wurmus wrote:
> I got myself a rockpro64 board.
...
> So I then decided to build my own system and install a new u-boot onto
> the microSD card.  Here’s the system I built:
...
>   (bootloader (bootloader-configuration
>                (bootloader u-boot-rockpro64-rk3399-bootloader)
>                ;; SD card/eMMC (SD priority) storage
>                (target "/dev/mmcblk1")))

You need to leave a pretty generous gap at the partition table of your
microSD, either a GPT partition table with various partitions for the
various u-boot bits:

  http://opensource.rock-chips.com/wiki_Partitions

or an MBR partition table with the first partition starting at sector
32768, and the previous sectors empty.

Otherwise, the u-boot installation may clobber parts of your partition
table or vice-versa.


Also, if you have a different u-boot installed to SPI or eMMC, those
will take priority over microSD on most rockchip platforms.


>   (kernel linux-libre)
>   (kernel-arguments
>    (list "console=ttyS2")) ; UART2 connected on the Pi2 bus

I assume you're using a serial console. The rockchip platforms usually
default 1500000 for baud ... usually I use:

  screen /dev/ttyUSB0 1500000

Oh, now that I think about it, you might also want to set the speed for
your console in kernel-arguments:

   (kernel-arguments
    (list "console=ttyS2,1500000"))



> Does it perhaps install the u-boot files at the wrong offsets?  How
> would I begin to debug this?

It is also entirely possible something is wrong with rockpro64 u-boot
offsets or u-boot itself ... been a while since I've tested.


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: cannot boot from microSD card on aarch64 machine

[Ricardo Wurmus]

Efraim Flashner <efraim@flashner.co.il> writes:

> Are you sure the primary device is /dev/mmcblk1? On my pine64 it's
> mmcblk0.

There is no /dev/mmcblk0 when I boot either Guix System (from USB) or armbian
(from microsd).  /dev/mmcblk1 is the microSD card, at least in a booted
system.  The device name only matters for the installation of u-boot,
doesn’t it?  It has no impact on any runtime behavior, or does it?

> I'd start by putting u-boot and the image on the sd-card and see if Guix
> boots up that way. If it works then there's a larger chance there's a
> mistake somewhere in a config.

Okay.

> Perhaps you need to mark the USB drive as a dependency for your root
> partition?

I don’t want to boot from USB going forward.  I only do this so the
microSD card is not used for the system.

I want to have u-boot on the microsd card and then boot the system from
SSD.

> Actually, better idea. Try it with linux-libre-arm64-generic for the
> kernel. I haven't tried recently but in the past I haven't been able to
> use the linux-libre kernel with my pine64 and needed the arm64-generic
> version.

Okay, I’ll do that.  At least I know that this one works, because that’s
what the pinebook pro image uses — and that one works fine (from USB).

But at this point I’m not even complaining about the kernel, because
nothing at all happens when I power on the machine.  U-boot does not
seem to start.

-- 
Ricardo

Re: cannot boot from microSD card on aarch64 machine

[Ricardo Wurmus]

Vagrant Cascadian <vagrant@debian.org> writes:

> [[PGP Signed Part:Undecided]]
> On 2022-03-02, Ricardo Wurmus wrote:
>> I got myself a rockpro64 board.
> ...
>> So I then decided to build my own system and install a new u-boot onto
>> the microSD card.  Here’s the system I built:
> ...
>>   (bootloader (bootloader-configuration
>>                (bootloader u-boot-rockpro64-rk3399-bootloader)
>>                ;; SD card/eMMC (SD priority) storage
>>                (target "/dev/mmcblk1")))
>
> You need to leave a pretty generous gap at the partition table of your
> microSD, either a GPT partition table with various partitions for the
> various u-boot bits:
>
>   http://opensource.rock-chips.com/wiki_Partitions
>
> or an MBR partition table with the first partition starting at sector
> 32768, and the previous sectors empty.
>
> Otherwise, the u-boot installation may clobber parts of your partition
> table or vice-versa.

Do I need to prepare the microSD card at all?  There’s no documentation
in the manual, so I have no idea what to do with it prior to “guix
system init”.  Looking at the code I assumed that the
disk-image-installer procedures take care of placing whatever blobs are
needed at the expected locations.

> Also, if you have a different u-boot installed to SPI or eMMC, those
> will take priority over microSD on most rockchip platforms.

There is no other u-boot installed.  The machine does not show any
activity when powered on without microSD card.

>>   (kernel linux-libre)
>>   (kernel-arguments
>>    (list "console=ttyS2")) ; UART2 connected on the Pi2 bus
>
> I assume you're using a serial console. The rockchip platforms usually
> default 1500000 for baud ... usually I use:
>
>   screen /dev/ttyUSB0 1500000
>
> Oh, now that I think about it, you might also want to set the speed for
> your console in kernel-arguments:
>
>    (kernel-arguments
>     (list "console=ttyS2,1500000"))

I’m not yet using a serial console, but I’ll come back to this later if
necessary.

>> Does it perhaps install the u-boot files at the wrong offsets?  How
>> would I begin to debug this?
>
> It is also entirely possible something is wrong with rockpro64 u-boot
> offsets or u-boot itself ... been a while since I've tested.

Hmm, okay.

Thanks for the hints!

-- 
Ricardo

Re: cannot boot from microSD card on aarch64 machine

[Maxim Cournoyer]

Hi Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi,
>
> I got myself a rockpro64 board.  I put armbian
> (Armbian_21.08.1_Rockpro64_bullseye_current_5.10.60.img) on a microSD
> card to check that it’s all working correctly.  Then I downloaded the
> latest Guix System image for a pinebook pro and dd’d it to a USB drive.
> With the armbian microSD card inserted (which contains an old version of
> u-boot) I can boot from USB into Guix System.
>
> So I then decided to build my own system and install a new u-boot onto
> the microSD card.  Here’s the system I built:
>
> (operating-system
>   (host-name "foo")
>   (timezone "Europe/Berlin")
>   (locale "en_US.utf8")
>   (bootloader (bootloader-configuration
>                (bootloader u-boot-rockpro64-rk3399-bootloader)
>                ;; SD card/eMMC (SD priority) storage
>                (target "/dev/mmcblk1")))
>   (kernel linux-libre)
>   (kernel-arguments
>    (list "console=ttyS2")) ; UART2 connected on the Pi2 bus
>   (file-systems (cons (file-system
>                         (device "my-root")
>                         (title 'label)
>                         (mount-point "/")
>                         (type "ext4"))
>                       %base-file-systems))
>   ...)
>
> The actual system also has one more initrd module, which I can’t
> remember now, but that shouldn’t matter.
>
> I mounted the target disk (an SSD) at /mnt, made sure that the microSD
> card was inserted, unmounted, and available at /dev/mmcblk1, and then
> ran
>
>    guix system init config.scm /mnt
>
> This was successful.
>
> Unfortunately, rebooting failed.  There is seemingly no activity at all
> when I power on the board, so something’s not right with how u-boot was
> installed on the microSD card.
>
> Does it perhaps install the u-boot files at the wrong offsets?  How
> would I begin to debug this?
>
> I’d appreciate any hints.

Sorry, I don't have any clue to offer, but were you able to solve or
workaround your problem?

Thanks,

Maxim

Re: cannot boot from microSD card on aarch64 machine

[Ricardo Wurmus]

Hi,

>> I mounted the target disk (an SSD) at /mnt, made sure that the microSD
>> card was inserted, unmounted, and available at /dev/mmcblk1, and then
>> ran
>>
>>    guix system init config.scm /mnt
>>
>> This was successful.
>>
>> Unfortunately, rebooting failed.  There is seemingly no activity at all
>> when I power on the board, so something’s not right with how u-boot was
>> installed on the microSD card.
>>
>> Does it perhaps install the u-boot files at the wrong offsets?  How
>> would I begin to debug this?
>>
>> I’d appreciate any hints.
>
> Sorry, I don't have any clue to offer, but were you able to solve or
> workaround your problem?

Not quite, but I’ve learned more and understand the problem better.

I manually copied the two u-boot files to the expected offsets on the
microsd card, and that let me boot Guix System from USB.  But I have so
far been unsuccessful in my attempts to do this automatically as part of
“guix system init” (I haven’t tried this much) or to successfully patch
u-boot to boot from the SSD on the PCIe card (I have tried this a lot).

My next step is to attempt to install the whole system onto the microsd
card, just to end up with a usable system.  If that works then nothing
is wrong with Guix (except for a lack of documentation), but it would be
a disappointing workaround.

I think it should be possible to patch u-boot to initialize the PCIe
card and boot from the attached disk (with the patched u-boot residing
on the microsd card).  I tried the relevant subset of the netbsd
patches, but the CPU keeps resetting after (successfully) enumerating
the PCI devices.

I hope I can make some more time for this soon and overcome that last
hurdle — and then write a cookbook entry for all this :)

-- 
Ricardo

Re: cannot boot from microSD card on aarch64 machine

[Ricardo Wurmus]

Hi Vagrant,

> On 2022-03-02, Ricardo Wurmus wrote:
>> I got myself a rockpro64 board.
> ...
>> So I then decided to build my own system and install a new u-boot onto
>> the microSD card.  Here’s the system I built:
> ...
>>   (bootloader (bootloader-configuration
>>                (bootloader u-boot-rockpro64-rk3399-bootloader)
>>                ;; SD card/eMMC (SD priority) storage
>>                (target "/dev/mmcblk1")))
>
> You need to leave a pretty generous gap at the partition table of your
> microSD, either a GPT partition table with various partitions for the
> various u-boot bits:
>
>   http://opensource.rock-chips.com/wiki_Partitions
>
> or an MBR partition table with the first partition starting at sector
> 32768, and the previous sectors empty.
>
> Otherwise, the u-boot installation may clobber parts of your partition
> table or vice-versa.

Thank you, this was interesting.

I still don’t quite understand how the partition layout relates to what
Guix does when it writes the files to the offsets.  I tried building
this exact partition table (the default settings of partitioning tools
force some other sector alignment, which makes this impossible without
overriding alignment settings in expert mode) and then used dd to write
the u-boot files to the partitions, but I wasn’t successful as the
microsd card seemed not to work at all.

Using dd to copy the two files to the expected offsets did let me into
u-boot and allowed me to boot from USB.  I haven’t yet tried putting the
system on a partition on the microsd card, because I’m still confused
about how I should build the partition table.

> Also, if you have a different u-boot installed to SPI or eMMC, those
> will take priority over microSD on most rockchip platforms.

Good to know.  No u-boot has been installed to SPI yet.

>> Does it perhaps install the u-boot files at the wrong offsets?  How
>> would I begin to debug this?
>
> It is also entirely possible something is wrong with rockpro64 u-boot
> offsets or u-boot itself ... been a while since I've tested.

When I used the same offsets with dd it works fine.  So I’m clearly
missing some information about how to prepare the microsd card.

-- 
Ricardo

Re: cannot boot from microSD card on aarch64 machine

[Maxim Cournoyer]

Hi Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi,
>
>>> I mounted the target disk (an SSD) at /mnt, made sure that the microSD
>>> card was inserted, unmounted, and available at /dev/mmcblk1, and then
>>> ran
>>>
>>>    guix system init config.scm /mnt
>>>
>>> This was successful.
>>>
>>> Unfortunately, rebooting failed.  There is seemingly no activity at all
>>> when I power on the board, so something’s not right with how u-boot was
>>> installed on the microSD card.
>>>
>>> Does it perhaps install the u-boot files at the wrong offsets?  How
>>> would I begin to debug this?
>>>
>>> I’d appreciate any hints.
>>
>> Sorry, I don't have any clue to offer, but were you able to solve or
>> workaround your problem?
>
> Not quite, but I’ve learned more and understand the problem better.
>
> I manually copied the two u-boot files to the expected offsets on the
> microsd card, and that let me boot Guix System from USB.  But I have so
> far been unsuccessful in my attempts to do this automatically as part of
> “guix system init” (I haven’t tried this much) or to successfully patch
> u-boot to boot from the SSD on the PCIe card (I have tried this a
> lot).

Heh, sounds tricky.

> My next step is to attempt to install the whole system onto the microsd
> card, just to end up with a usable system.  If that works then nothing
> is wrong with Guix (except for a lack of documentation), but it would be
> a disappointing workaround.
>
> I think it should be possible to patch u-boot to initialize the PCIe
> card and boot from the attached disk (with the patched u-boot residing
> on the microsd card).  I tried the relevant subset of the netbsd
> patches, but the CPU keeps resetting after (successfully) enumerating
> the PCI devices.

If I recall from my limited experience with an embedded ARM, you'd have
to register the newly supported hardware in the device tree of
U-Boot/the kernel; do the patches applied take care of that?

> I hope I can make some more time for this soon and overcome that last
> hurdle — and then write a cookbook entry for all this :)

I wish you luck!

Cheers,

Maxim

Re: cannot boot from microSD card on aarch64 machine

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1943 bytes --]

On 2022-03-29, Ricardo Wurmus wrote:
>>> I mounted the target disk (an SSD) at /mnt, made sure that the microSD
>>> card was inserted, unmounted, and available at /dev/mmcblk1, and then
>>> ran
>>>
>>>    guix system init config.scm /mnt
>>>
>>> This was successful.
>>>
>>> Unfortunately, rebooting failed.  There is seemingly no activity at all
>>> when I power on the board, so something’s not right with how u-boot was
>>> installed on the microSD card.
...
> I manually copied the two u-boot files to the expected offsets on the
> microsd card, and that let me boot Guix System from USB.  But I have so
> far been unsuccessful in my attempts to do this automatically as part of
> “guix system init” (I haven’t tried this much) or to successfully patch
> u-boot to boot from the SSD on the PCIe card (I have tried this a lot).

I don't understand this problem, works for me with both reconfigure and
init... does your operating system config reference the same device that
is present when you're initializing the card? ... sometimes i use a
usb-to-microSD adapter, which shows up as /dev/sdX rather than
/dev/mmcblkX if i'm initializing from another machine.


> I think it should be possible to patch u-boot to initialize the PCIe
> card and boot from the attached disk (with the patched u-boot residing
> on the microsd card).  I tried the relevant subset of the netbsd
> patches, but the CPU keeps resetting after (successfully) enumerating
> the PCI devices.
>
> I hope I can make some more time for this soon and overcome that last
> hurdle — and then write a cookbook entry for all this :)

I was able to get it to work! With your updated patch to disable openssl
(and rebasing against newer u-boot), and the phase modifying the config
and the the rockchip include file to add support for SCSI devices.

The netbsd patches weren't needed with 2022.04-rc5.


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: cannot boot from microSD card on aarch64 machine

[Ricardo Wurmus]

Vagrant Cascadian <vagrant@debian.org> writes:

>> I think it should be possible to patch u-boot to initialize the PCIe
>> card and boot from the attached disk (with the patched u-boot residing
>> on the microsd card).  I tried the relevant subset of the netbsd
>> patches, but the CPU keeps resetting after (successfully) enumerating
>> the PCI devices.
>>
>> I hope I can make some more time for this soon and overcome that last
>> hurdle — and then write a cookbook entry for all this :)
>
> I was able to get it to work! With your updated patch to disable openssl
> (and rebasing against newer u-boot), and the phase modifying the config
> and the the rockchip include file to add support for SCSI devices.
>
> The netbsd patches weren't needed with 2022.04-rc5.

Oh, that’s great!  Do you have patches for the u-boot upgrade that I
could test?

Are you using a PCIe card with your rockpro64?

-- 
Ricardo

Re: cannot boot from microSD card on aarch64 machine

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 3250 bytes --]

On 2022-04-02, Ricardo Wurmus wrote:
> Vagrant Cascadian <vagrant@debian.org> writes:
>> I was able to get it to work! With your updated patch to disable openssl
>> (and rebasing against newer u-boot), and the phase modifying the config
>> and the the rockchip include file to add support for SCSI devices.
>>
>> The netbsd patches weren't needed with 2022.04-rc5.
>
> Oh, that’s great!

It really is, thanks for getting it as far as you did!

Although very mildly disappointed that this might deprioritize making
split /boot in guix a reality. :)


> Do you have patches for the u-boot upgrade that I could test?

https://www.aikidev.net/~vagrant/guix/u-boot-2022.04-rc5-with-rockpro64-scsi.patch

Pretty similar to what you had already, with some added native-inputs
for the newer u-boot version, minor adjustments to the patch to disable
openssl, and I only tweaked the upstream files in the
u-boot-rockpro64-rk3399 package, not for all u-boot packages. There is
also an unrelated extraneous patch, but I left it out of the above patch
for brevity.

I think it *might* be possible to get the patches a little
smaller... and of course should get submitted upstream!

I'd consider pushing the updates now, but u-boot 2022.04 is planned to
be released monday, so may as well wait till then.


> Are you using a PCIe card with your rockpro64?

Yup. Pretty sure it's whatever PCIe dual-port sata card I bought right
along with the rockpro64.

U-boot is still on microSD, but it loads the kernel+initrd+dtb from
SCSI. The scsi target ends up pretty low in the boot order, so you have
to either manually set the boot_targets or make sure your mmc/nvme
devices aren't bootable (e.g. change partition type, rename boot
extlinux.conf, etc.):

  Hit any key to stop autoboot:  0
  => printenv boot_targets
  boot_targets=mmc1 mmc0 nvme0 scsi0 usb0 pxe dhcp sf0
  => setenv boot_targets scsi0
  => boot
  scanning bus for devices...
  SATA link 0 timeout.
  Target spinup took 0 ms.
  AHCI 0001.0200 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
  flags: 64bit ncq stag led clo pmp pio slum part ccc sxs
    Device 0: (1:0) Vendor: ATA Prod.: Faspeed F510-120 Rev: O080
              Type: Hard Disk
              Capacity: 114473.4 MB = 111.7 GB (234441648 x 512)
  
  Device 0: (1:0) Vendor: ATA Prod.: Faspeed F510-120 Rev: O080
              Type: Hard Disk
              Capacity: 114473.4 MB = 111.7 GB (234441648 x 512)
  ... is now current device
  Scanning scsi 0:2...
  Found /boot/extlinux/extlinux.conf
  Retrieving file: /boot/extlinux/extlinux.conf
  Ignoring unknown command: UI
  1:      GNU with Linux-Libre 5.17
  Retrieving file: /gnu/store/fxrc92smbx16k29fw8yz1s0mi7nslnan-raw-initrd/initrd.cpio.gz
  Retrieving file: /gnu/store/z5kqlcqf4r0hpyj6cdkcylh3gmcp4x9x-linux-libre-5.17/Image
  append: root=/dev/sda2 gnu.system=/gnu/store/fz4g427hcdjp70wh7rjh88vrvwi8r5yp-system gnu.load=/gnu/store/fz4g427hcdjp70wh7rjh88vrvwi8r5yp-system/boot modprobe.blacklist=usbmouse,usbkbd quiet
  Retrieving file: /gnu/store/z5kqlcqf4r0hpyj6cdkcylh3gmcp4x9x-linux-libre-5.17/lib/dtbs/rockchip/rk3399-rockpro64.dtb


Happy guixing on your shiny rockpro64!


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: cannot boot from microSD card on aarch64 machine

[Ricardo Wurmus]

Hi Vagrant,

> On 2022-04-02, Ricardo Wurmus wrote:
>> Vagrant Cascadian <vagrant@debian.org> writes:
>>> I was able to get it to work! With your updated patch to disable openssl
>>> (and rebasing against newer u-boot), and the phase modifying the config
>>> and the the rockchip include file to add support for SCSI devices.
>>>
>>> The netbsd patches weren't needed with 2022.04-rc5.
>>
>> Oh, that’s great!
>
> It really is, thanks for getting it as far as you did!
>
> Although very mildly disappointed that this might deprioritize making
> split /boot in guix a reality. :)
>
>
>> Do you have patches for the u-boot upgrade that I could test?
>
> https://www.aikidev.net/~vagrant/guix/u-boot-2022.04-rc5-with-rockpro64-scsi.patch

Thank you!

>> Are you using a PCIe card with your rockpro64?
>
> Yup. Pretty sure it's whatever PCIe dual-port sata card I bought right
> along with the rockpro64.

I’ve got that one, too.

> U-boot is still on microSD, but it loads the kernel+initrd+dtb from
> SCSI. The scsi target ends up pretty low in the boot order, so you have
> to either manually set the boot_targets or make sure your mmc/nvme
> devices aren't bootable (e.g. change partition type, rename boot
> extlinux.conf, etc.):
>
>   Hit any key to stop autoboot:  0
>   => printenv boot_targets
>   boot_targets=mmc1 mmc0 nvme0 scsi0 usb0 pxe dhcp sf0
>   => setenv boot_targets scsi0
>   => boot
>   scanning bus for devices...
>   SATA link 0 timeout.
>   Target spinup took 0 ms.
>   AHCI 0001.0200 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
>   flags: 64bit ncq stag led clo pmp pio slum part ccc sxs
>     Device 0: (1:0) Vendor: ATA Prod.: Faspeed F510-120 Rev: O080
>               Type: Hard Disk
>               Capacity: 114473.4 MB = 111.7 GB (234441648 x 512)
>   
>   Device 0: (1:0) Vendor: ATA Prod.: Faspeed F510-120 Rev: O080
>               Type: Hard Disk
>               Capacity: 114473.4 MB = 111.7 GB (234441648 x 512)
>   ... is now current device
>   Scanning scsi 0:2...
>   Found /boot/extlinux/extlinux.conf
>   Retrieving file: /boot/extlinux/extlinux.conf
>   Ignoring unknown command: UI
>   1:      GNU with Linux-Libre 5.17
>   Retrieving file: /gnu/store/fxrc92smbx16k29fw8yz1s0mi7nslnan-raw-initrd/initrd.cpio.gz
>   Retrieving file: /gnu/store/z5kqlcqf4r0hpyj6cdkcylh3gmcp4x9x-linux-libre-5.17/Image
>   append: root=/dev/sda2 gnu.system=/gnu/store/fz4g427hcdjp70wh7rjh88vrvwi8r5yp-system gnu.load=/gnu/store/fz4g427hcdjp70wh7rjh88vrvwi8r5yp-system/boot modprobe.blacklist=usbmouse,usbkbd quiet
>   Retrieving file: /gnu/store/z5kqlcqf4r0hpyj6cdkcylh3gmcp4x9x-linux-libre-5.17/lib/dtbs/rockchip/rk3399-rockpro64.dtb

Hmm, unfortunately, this is not working for me :(

I applied the patch, built u-boot-rockpro64-rk3399 with

    ./pre-inst-env guix build u-boot-rockpro64-rk3399

and got

    /gnu/store/xw7nyd0fvg9rlsyhxbbk0hg5k18q5pd9-u-boot-rockpro64-rk3399-2022.04-rc5

I then wiped the old partitions on the microsd card with

    sudo wipefs -f -a /dev/sdb

and copied the two files with dd:

    export where=/gnu/store/xw7nyd0fvg9rlsyhxbbk0hg5k18q5pd9-u-boot-rockpro64-rk3399-2022.04-rc5
    sudo dd seek=64 obs=512 conv=notrunc if=$where/libexec/idbloader.img of=/dev/sdb
    sudo dd seek=16384 obs=512 conv=notrunc if=$where/libexec/u-boot.itb of=/dev/sdb
    sudo sync

Moved the microsd to the rockpro64 slot and powered it on.  u-boot comes
up in the expected version, and since it successfully fails to boot from
the microsd card it falls through to scanning for SCSI devices.
Immediately after listing connected devices (even when none are
connected to the PCIe card) a long error message appears, the display
goes blank, and the CPU seemingly gets stuck.

I can reliably reproduce this by typing on the u-boot console:

    pci enum
    scsi scan

I don’t think the PCIe card is broken, because I can use it just fine
after booting from USB.

Maybe I made a mistake that would seem really obvious to others?

-- 
Ricardo

Re: cannot boot from microSD card on aarch64 machine

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1381 bytes --]

On 2022-04-03, Ricardo Wurmus wrote:
>> On 2022-04-02, Ricardo Wurmus wrote:
>>> Vagrant Cascadian <vagrant@debian.org> writes:

>>   Device 0: (1:0) Vendor: ATA Prod.: Faspeed F510-120 Rev: O080
>>               Type: Hard Disk
>>               Capacity: 114473.4 MB = 111.7 GB (234441648 x 512)
>>   ... is now current device
>>   Scanning scsi 0:2...
>>   Found /boot/extlinux/extlinux.conf
>>   Retrieving file: /boot/extlinux/extlinux.conf
>>   Ignoring unknown command: UI
>>   1:      GNU with Linux-Libre 5.17
...

> Hmm, unfortunately, this is not working for me :(
>
> I applied the patch, built u-boot-rockpro64-rk3399 with
>
>     ./pre-inst-env guix build u-boot-rockpro64-rk3399

Most perplexing. Since it at least worked for me, I went ahead and
pushed to master with the u-boot 2022.04 update.


> Maybe I made a mistake that would seem really obvious to others?

Nothing obvious so far, unfortunately...

My best guess is some difference in actual board model, card model, etc.

  $ lspci
  00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI Express Root Port
  01:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)

Is your board a v2.1 ? The version is clearly printed on the board next
to the CPU. I'm not sure what the differences are with the v2 model, or
if there are newer variants.


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: cannot boot from microSD card on aarch64 machine

[Ricardo Wurmus]

Vagrant Cascadian <vagrant@debian.org> writes:

> My best guess is some difference in actual board model, card model, etc.
>
>   $ lspci
>   00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI Express Root Port
>   01:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)

My SATA controller differs:

  01:00.0 SATA controller: JMicron Technology Corp. Device 0585

It’s good to know that it works for a different PCIe card.  Maybe I can
find some more information about the differences.

> Is your board a v2.1 ? The version is clearly printed on the board next
> to the CPU. I'm not sure what the differences are with the v2 model, or
> if there are newer variants.

Yes, my board is V2.1 from 2018.

-- 
Ricardo

Re: cannot boot from microSD card on aarch64 machine

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 1516 bytes --]

On Wed, 02 Mar 2022 19:52:18 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:

> Do I need to prepare the microSD card at all?  There’s no
> documentation in the manual, so I have no idea what to do with it
> prior to “guix system init”.  Looking at the code I assumed that the
> disk-image-installer procedures take care of placing whatever blobs
> are needed at the expected locations.

Long time ago I managed to boot Guix on a Lime 1 A20 and it worked. 

I used a command similar to this one (probably the equivalent with guix
image that is now deprecated or removed):
> guix system image --target=arm-linux-gnueabihf -t raw-with-offset \
> system.scm

This takes care of writing the bootloader at the right location on the
image. However I'm not sure what happens if the bootloader gets too big
and get overwritten by the GPT or MBR partition table.

With the command above, you'll have to adjust the target for ARM 64bit.

The issue for me is that at some point the boot broke: I saw u-boot and
the kernel and the init on the serial port, but the init never
completed and I didn't find enough time to correctly investigate the
issue. But at least the bootloader is probably still working fine.

In my TODO list I also wanted to add u-boot for qemu_arm
(qemu_arm_defconfig) and qemu_arm64 (qemu_arm64_defconfig) to be able
to enable people to more easily test ARM support without any special
hardware, but that doesn't advance fast at all due to the lack of time.

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]