wip blog post: running Guix System on ARM

wip blog post: running Guix System on ARM

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 140 bytes --]

Hi, attached is a draft for a blog post (or a section in the cookbook)
for explaining how to install the Guix System on an ARM board. WDYT?

[-- Attachment #2: guix-arm.md --]
[-- Type: text/markdown, Size: 11129 bytes --]

Guix on an ARM Board
====================

Increasingly people discovering Guix want to try it on an ARM board, instead of
their x86 computer.  There might be various reasons for that, from power consumption
to security.  In my case, I found these ARM boards practical for self-hosting,
and I think the unique properties of GNU Guix are making it very suitable for that
purpose.  I have installed GNU Guix on a Cubietruck, so my examples below will be
about that board.  However, you should be able to change the examples for your
own use case.

Installing the Guix System on an ARM board is not as easy as installing it on an
x86 desktop computer: there is no installation image.  However, Guix supports
ARM and can be installed on a foreign distribution running on that architecture.
The trick is to use the Guix installed on that foreign distribution to initialize
the Guix System.

Most boards can be booted from an existing GNU+Linux distribution.  You will
need to install a distribution (any of them) and install GNU Guix on it, using
e.g. the installer script.  Then, my plan was to install the Guix System on an
external SSD drive, instead of the SD card, but we will see that both are perfectly
possible.

The first part of the article will focus on creating a proper u-boot configuration
and an operating system declaration that suits your board. The second part of this
article will focus on the installation procedure, when there is no installer working
for your system.

Writing a configuration file for an ARM board
---------------------------------------------

A configuration file for an ARM board is not very different from a configuration file
for a desktop or a server running on another architecture.  However, most boards use
the u-boot bootloader and require some less common modules to be available at boot time.

### The root file system

First of all, you should decide where your root file system is going to be installed. In
my case, I wanted to install is on the external SSD, so I chose it:

```scheme
(file-systems
  (cons* (file-system
           (mount-point "/")
           (device "/dev/sda1")
           (type "ext4"))
         %base-file-systems))
```

If you instead want to install the root file system on an SD card, you'll need to find its
device name, usually `/dev/mmcblk0` and the partition number.  The device corresponding to the
first partition should be `/dev/mmcblk0p1`.  In that case, you would have:

```scheme
(file-systems
  (cons* (file-system
           (mount-point "/")
           (device "/dev/mmcblk0p1")
           (type "ext4"))
         %base-file-systems))
```

### The bootloader

Because of the way the Guix System is designed, you cannot use an already existing bootloader
to boot your system: it wouldn't know where to look for the kernel, because it doesn't know
its store path.  It wouldn't be able to let you boot older generations either.  Most boards
use the u-boot bootloader, so we will focus on that bootloader here.

Contrary to grub, there are multiple variants of u-boot, one per board type.  The installation
procedure for u-boot is also somewhat specific to the board, so there are two things that you
need to take care of: the u-boot package and the bootloader declaration.

Guix already define a few u-boot based bootloaders, such as `u-boot-a20-olinuxino-lime-bootloader`
or `u-boot-pine64-plus-bootloader` among others.  If your board already has a `u-boot-*-bootloader`
defined in `(gnu bootloader u-boot)`, you're lucky and you can skip this part of the article!

Otherwise, maybe the bootloader package is defined in `(gnu packages bootloaders)`, such as
the `u-boot-cubietruck` package.  If so, you're a bit lucky and you can skip creating your
own package definition.

If your board doesn't have a `u-boot-*` package defined, you can create one.  It could be
as simple as `(make-u-boot-package "Cubietruck" "arm-linux-gnueabihf")`.  The first argument
is the board name, as expected by the u-boot build sysetem.  The second argument is the
target triplet that corresponds to the architecture of the board.  You should refer to the
documentation of your board for selecting the correct values.  If you're really unlucky,
you'll need to do some extra work to make the u-boot package you just created work, as is
the case for the `u-boot-puma-rk3399` for instance: it needs additional phases to install
firmware.

You can add the package definition to your operating system configuration file like so,
before the operating-system declaration:

```scheme
(use-modules (gnu packages bootloaders))

(define u-boot-my-board
  (make-u-boot-package "Myboard" "arm-linux-gnueabihf"))

(operating-system
  [...])
```

Then, you need to define the bootloader.  A bootloader is a structure that has a name,
a package, an installer, a configuration file and a configuration file generator.  Fortunately,
Guix already defines a base u-boot bootloader, so we can inherit from it and only redefine a few
things.

The Cubietruck happens to be based on an allwinner core, for which there is already a
u-boot bootloader definition `u-boot-allwinner-bootloader`.  This bootloader is not
usable as is for the Cubietruck, but it defines most of what we need.  In order to get
a proper bootloader for the Cubietruck, we define a new bootloader based on the
Allwinner bootloader definition:

```scheme
(define u-boot-cubietruck-bootloader
  (bootloader
    (inherit u-boot-allwinner-bootloader)
    (package u-boot-cubietruck)))
```

Now that we have our definitions, we can choose where to install the bootloader.  In the
case of the Cubietruck, I decided to install it on the SD card, because it cannot boot from
the SSD directly.  Refer to your board documentation to make sure you install u-boot on
a bootable device.  As we said earlier, the SD card is `/dev/mmcblk0` on my device.

We can now put everything together like so:

```scheme
(use-modules (gnu packages bootloaders))

(define u-boot-cubietruck
  (make-u-boot-package "Cubietruck" "arm-linux-gnueabihf"))

;; u-boot-allwinner-bootloader is not exported by (gnu bootloader u-boot) so
;; we use @@ to get it.  (@ (module) variable) means: get the value of "variable"
;; as defined (and exported) in (module).  (@@ (module) variable) is the same, but
;; it doesn't care whether it is exported or not.
(define u-boot-allwinner-bootloader
  (@@ (gnu bootloader u-boot) u-boot-allwinner-bootloader))

(define u-boot-cubietruck-bootloader
  (bootloader
    (inherit u-boot-allwinner-bootloader)
    (package u-boot-cubietruck)))

(operating-system
  [...]
  (bootloader
    (bootloader-configuration
      (target "/dev/mmcblk0")
      (bootloader u-boot-cubietruck-bootloader)))
  [...])
```

### The kernel modules

In order for Guix to be able to load the system from the initramfs, it will probably need
to load some modules, especially to access the root file system.  In my case, the SSD is
on an ahci device, so I need a driver for it.  The kernel defines `ahci_sunxi` for that
device on any sunxi board.  The SD card itself also requires two drivers: `sunxi-mmc` and
`sd_mod`.

Your own board may need other kernel modules to boot properly, however it is hard to discover
them.  Guix can tell you when a module is missing in your configuration file if it is loaded
as a module.  Most distros however build these modules in the kernel directly, so Guix cannot
detect them reliably.  Another way to find what drivers might be needed is to look at the output
of `dmesg`.  You'll find messages such as:

```
[    5.193684] sunxi-mmc 1c0f000.mmc: Got CD GPIO
[    5.219697] sunxi-mmc 1c0f000.mmc: initialized, max. request size: 16384 KB
[    5.221819] sunxi-mmc 1c12000.mmc: allocated mmc-pwrseq
[    5.245620] sunxi-mmc 1c12000.mmc: initialized, max. request size: 16384 KB
[    5.255341] mmc0: host does not support reading read-only switch, assuming write-enable
[    5.265310] mmc0: new high speed SDHC card at address 0007
[    5.268723] mmcblk0: mmc0:0007 SD32G 29.9 GiB
```

or

```
[    5.614961] ahci-sunxi 1c18000.sata: controller can't do PMP, turning off CAP_PMP
[    5.614981] ahci-sunxi 1c18000.sata: forcing PORTS_IMPL to 0x1
[    5.615067] ahci-sunxi 1c18000.sata: AHCI 0001.0100 32 slots 1 ports 3 Gbps 0x1 impl platform mode
[    5.615083] ahci-sunxi 1c18000.sata: flags: ncq sntf pm led clo only pio slum part ccc 
[    5.616840] scsi host0: ahci-sunxi
[    5.617458] ata1: SATA max UDMA/133 mmio [mem 0x01c18000-0x01c18fff] port 0x100 irq 37
[    5.933494] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
```

Also note that module names are not consistent between what Guix expects and what is printed by
dmesg, especially when the contain a "-" or a "_".  You will find the correct file name by building
(or using a substitute for) linux-libre beforehand:

```
find `guix build linux-libre`/lib/modules -name '*mmc*'
```

Here, I could find a file named "kernel/drivers/mmc/host/sunxi-mmc.ko", hence the module
name `sunxi-mmc`.  For the other driver, I found a "kernel/drivers/ata/ahci_sunxi.ko",
hence the name `ahci_sunxi`, even if dmesg suggested `ahci-sunxi`.

Once you have found the modules you want to load before mounting the root partition, you can
add them to your operating-system declaration file:

```scheme
(initrd-modules (cons* "sunxi-mmc" "sd_mod" "ahci_sunxi" %base-initrd-modules))
```

Installing the Guix System
--------------------------

### Installing on another drive

In my case, I wanted to install the system on an external SSD, while the currently running
foreign distribution was running from the SD card.  What is nice with this setup is that,
in case of real trouble (you SSD caught fire or broke), you can still boot from the old
foreign system with an installed Guix and all your tools by re-flashing only the bootloader.

In this scenario, we use the foreign system as we would the installer iso, using the manual
installation procedures described in the manual.  Essentially, you have to partition your SSD
to your liking, format your new partations and make sure to reference the correct partition
for the root file system in your configuration file.  Then, initialize the system with:

```bash
mount /dev/sda1 /mnt
mkdir /mnt/etc
$EDITOR /mnt/etc/config.scm # create the configuration file
guix system init /mnt/etc/config.scm /mnt
```

You can now reboot and enjoy your new Guix System!

### Installing on the same drive

Another option is to install the Guix System over the existing foreign distribution, replacing
it entirely.  Note that the root filesystem for the new Guix System is the current root filesystem,
so no need to mount it.  The following will initialize your system:

```bash
$EDITOR /etc/config.scm # create the configuration file
guix system init /etc/config.scm /
```

Make sure to remove the files from the old system.  You should at least get rid of the
old `/etc` directory, like so:

```bash
mv /etc{,.bak}
mkdir /etc
```

Make sure there is an empty /etc, or the new system won't boot properly.  You can
copy your config.scm to the new `/etc` directory.  You can now reboot and enjoy your
new Guix System!

Re: wip blog post: running Guix System on ARM

[Giovanni Biscuolo]

[-- Attachment #1: Type: text/plain, Size: 4585 bytes --]

Hi Julien,

Julien Lepiller <julien@lepiller.eu> writes:

> Hi, attached is a draft for a blog post (or a section in the cookbook)
> for explaining how to install the Guix System on an ARM board. WDYT?

LOL! \O/

This should be a cookbook section, and released soon unless other people
find some error or important things missing (I still do not use some ARM
SoC, but hope to test soon)

What about to start a wip-cookbook-install-on-arm branch with a 1/2
weeks time frame to allow patch proposals from interested
parties?... then publish? :-)

I have just a couple of comments and one typo... BTW discussion based on
my comments should be postponed after your section is published in the
cookbook, since I don't have patch proposals :-D

[...]

> Most boards can be booted from an existing GNU+Linux distribution.  You will
> need to install a distribution (any of them) and install GNU Guix on it, using
> e.g. the installer script.

Is it possible to install to SDD (eventually with external HDD) using
another host (e.g. my laptop) with `guix system disk-image...` and then
``dd ... of=/dev/sdX`` on SDD?

[...]

> ### The kernel modules

[...]

AFAIU this is the only "tricky" part for the user: can we find a way for
Guix to help them automagically populate the list of needed initrd
kernel modules for their SoC?

> Your own board may need other kernel modules to boot properly, however it is hard to discover
> them.  Guix can tell you when a module is missing in your configuration file if it is loaded
> as a module.  Most distros however build these modules in the kernel directly, so Guix cannot
> detect them reliably.  Another way to find what drivers might be needed is to look at the output
> of `dmesg`.  You'll find messages such as:

This implies that we are booting a distro on the SoC, so (if possible)
we cannot install using `guix system disk-image`.  This means that the
only way to make Guix automagically know the list of base modules needed
for each SoC should be to... define them in Guix.

Should be possible to extend (gnu system linux-initrd) whith code
defining default-initrd-modules if string-match some specific SoC
architectures (are they detected?) ?

--8<---------------cut here---------------start------------->8---
(define* (default-initrd-modules
           #:optional
           (system (or (%current-target-system)
                       (%current-system))))
  "Return the list of modules included in the initrd by default."
  (define virtio-modules
    ;; Modules for Linux para-virtualized devices, for use in QEMU guests.
    '("virtio_pci" "virtio_balloon" "virtio_blk" "virtio_net"
      "virtio_console" "virtio-rng"))

  `("ahci"                                  ;for SATA controllers
    "usb-storage" "uas"                     ;for the installation image etc.
    "usbhid" "hid-generic" "hid-apple"      ;keyboards during early boot
    "dm-crypt" "xts" "serpent_generic" "wp512" ;for encrypted root partitions
    "nls_iso8859-1"                            ;for `mkfs.fat`, et.al
    ,@(if (string-match "^(x86_64|i[3-6]86)-" system)
          '("pata_acpi" "pata_atiixp"    ;for ATA controllers
            "isci")                      ;for SAS controllers like Intel C602
          '())

    ,@virtio-modules))
--8<---------------cut here---------------end--------------->8---

I mean something like adding a proper form of this pseudocode:

--8<---------------cut here---------------start------------->8---
    ,@(if (string-match "^(<architecture+SoC>)-" system)
          '("sd_mod" "ahci_sunxi"    ;for sunxi ATA controller
            "sunxi-mmc")             ;for sunxi MMC
          '())
--8<---------------cut here---------------end--------------->8---

Obviously there should be quite some work to do to extend our
default-initrd-modules this way for each SoC (or class of SoCs) since we
should find them reading some appropriate documentation or testing
things like you described in this draft, but once done it's forever and
for all :-)

> Installing the Guix System
> --------------------------

[...]

> In this scenario, we use the foreign system as we would the installer iso, using the manual
> installation procedures described in the manual.  Essentially, you have to partition your SSD
> to your liking, format your new partations and make sure to reference the correct partition
                                      ^ i (typo)
[...]

Thanks! Gio'

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: wip blog post: running Guix System on ARM

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 2147 bytes --]

Neat, thanks for this article!

Question: how do you intend to convert the markdown to Texinfo?


Some typos / nits below:


> ### The kernel modules
>
> In order for Guix to be able to load the system from the initramfs, it will probably need

Maybe define initramfs?  Or just link to https://en.wikipedia.org/wiki/Initramfs.

> to load some modules, especially to access the root file system.  In my case, the SSD is
> on an ahci device, so I need a driver for it.  The kernel defines `ahci_sunxi` for that

AHCI

> Your own board may need other kernel modules to boot properly, however it is hard to discover
> them.  Guix can tell you when a module is missing in your configuration file if it is loaded
> as a module.  Most distros however build these modules in the kernel directly, so Guix cannot

Maybe "distributions" unabridged?

> Also note that module names are not consistent between what Guix expects and what is printed by
> dmesg, especially when the contain a "-" or a "_".  You will find the correct file name by building

Word missing here: "when the contain".

> Here, I could find a file named "kernel/drivers/mmc/host/sunxi-mmc.ko", hence the module
> name `sunxi-mmc`.  For the other driver, I found a "kernel/drivers/ata/ahci_sunxi.ko",
> hence the name `ahci_sunxi`, even if dmesg suggested `ahci-sunxi`.

Maybe add "(notice the '_' and the '-' difference)."

> In my case, I wanted to install the system on an external SSD, while the currently running
> foreign distribution was running from the SD card.  What is nice with this setup is that,
> in case of real trouble (you SSD caught fire or broke), you can still boot from the old

"your SSD"

> foreign system with an installed Guix and all your tools by re-flashing only the bootloader.
>
> In this scenario, we use the foreign system as we would the installer iso, using the manual

Word missing: "we would the installer iso".

Also: iso -> ISO.

> Make sure there is an empty /etc, or the new system won't boot
> properly.

Isn't this a bug in Guix?


Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: wip blog post: running Guix System on ARM

[zimoun]

Hi,

Thank you for the article!


I would find interesting to point somewhere in your post to these
previous blog posts [1] [2] [3] about ARM and Guix. :-)

[1] https://guix.gnu.org/blog/2017/porting-guixsd-to-armv7/
[2] http://guix.gnu.org/blog/2018/guix-on-android/
[3] http://guix.gnu.org/blog/2019/guix-days-bootstrapping-arm/



à tantôt,
simon

Re: wip blog post: running Guix System on ARM

[pelzflorian (Florian Pelz)]

On Thu, Nov 14, 2019 at 10:29:22AM +0100, Pierre Neidhardt wrote:
> Neat, thanks for this article!
> 

Yes, thank you!  I have not tried yet though.

Maybe add the top add instructions what to do if installation fails
(i.e. flash the SD with another operating system and start anew).


> > Make sure there is an empty /etc, or the new system won't boot
> > properly.
> 
> Isn't this a bug in Guix?
> 
> 

Is installing on the same drive an “official” installation method?  I
mean, it probably works and people can ask for help after using it,
but remaining files from the old system could be problematic.  If this
is dangerous (is it?) then please add a warning.

Also, I wonder:

On Wed, Nov 13, 2019 at 10:21:54PM +0100, Julien Lepiller wrote:
> Then, initialize the system with:
> 
> ```bash
> mount /dev/sda1 /mnt
> mkdir /mnt/etc
> $EDITOR /mnt/etc/config.scm # create the configuration file
> guix system init /mnt/etc/config.scm /mnt
> ```

So the mmcblk you install on is different from the running system’s
mmcblk, otherwise how could you keep your old system’s SD card?

Then the mmcblk device number will change on the running Guix System
and the config.scm will have to be adapted to use the mmcblk before
reconfiguring from the installed Guix System.

Otherwise if you install on the same mmcblk as the running system,
then maybe this could fail if the u-boot partition is too small?  That
would leave both the existing operating system and the new Guix System
unusable.  Maybe there should be more of a warning.

> ### The bootloader
> 
> Because of the way the Guix System is designed, you cannot use an already existing bootloader
> to boot your system: it wouldn't know where to look for the kernel, because it doesn't know
> its store path.  It wouldn't be able to let you boot older generations either.  Most boards
> use the u-boot bootloader, so we will focus on that bootloader here.


More generally, since no old Guix generation can be selected in pure
u-boot when booting (I think) this warrants more of a warning that one
important feature of Guix is missing.

For later: What would a rescue of a broken Guix System look like?  I
do not know if all this works better with grub-efi on supported ARM
systems; I have never tried.

For later, maybe in the manual: Maybe it would be interesting how to
create a reusable SD install image for Guix.  I also remember there
were discussions about making ci.guix.gnu.org build a two-part
bootable installation image in the past, one part with a bootloader
for a specific board and another part general for all boards.

Regards,
Florian

Re: wip blog post: running Guix System on ARM

[Julien Lepiller]

Le Thu, 14 Nov 2019 12:05:34 +0100,
"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> a écrit :

> On Thu, Nov 14, 2019 at 10:29:22AM +0100, Pierre Neidhardt wrote:
> > Neat, thanks for this article!
> >   
> 
> Yes, thank you!  I have not tried yet though.
> 
> Maybe add the top add instructions what to do if installation fails
> (i.e. flash the SD with another operating system and start anew).
> 
> 
> > > Make sure there is an empty /etc, or the new system won't boot
> > > properly.  
> > 
> > Isn't this a bug in Guix?
> > 
> >   
> 
> Is installing on the same drive an “official” installation method?  I
> mean, it probably works and people can ask for help after using it,
> but remaining files from the old system could be problematic.  If this
> is dangerous (is it?) then please add a warning.

It's not officially recommended, but we have used and proposed it on
IRC in the past.  The overdrive at my place, as well as my own server
both were installed using that method.

> 
> Also, I wonder:
> 
> On Wed, Nov 13, 2019 at 10:21:54PM +0100, Julien Lepiller wrote:
> > Then, initialize the system with:
> > 
> > ```bash
> > mount /dev/sda1 /mnt
> > mkdir /mnt/etc
> > $EDITOR /mnt/etc/config.scm # create the configuration file
> > guix system init /mnt/etc/config.scm /mnt
> > ```  
> 
> So the mmcblk you install on is different from the running system’s
> mmcblk, otherwise how could you keep your old system’s SD card?
> 
> Then the mmcblk device number will change on the running Guix System
> and the config.scm will have to be adapted to use the mmcblk before
> reconfiguring from the installed Guix System.
> 
> Otherwise if you install on the same mmcblk as the running system,
> then maybe this could fail if the u-boot partition is too small?  That
> would leave both the existing operating system and the new Guix System
> unusable.  Maybe there should be more of a warning.
> 
> > ### The bootloader
> > 
> > Because of the way the Guix System is designed, you cannot use an
> > already existing bootloader to boot your system: it wouldn't know
> > where to look for the kernel, because it doesn't know its store
> > path.  It wouldn't be able to let you boot older generations
> > either.  Most boards use the u-boot bootloader, so we will focus on
> > that bootloader here.  
> 
> 
> More generally, since no old Guix generation can be selected in pure
> u-boot when booting (I think) this warrants more of a warning that one
> important feature of Guix is missing.

I must have expressed myself badly, sorry. It's perfectly possible to
select older generations from u-boot.  I was talking about the u-boot
from the foreign distro, but I was wrong on that too ^^'. So you can
always select an older generation.

> 
> For later: What would a rescue of a broken Guix System look like?  I
> do not know if all this works better with grub-efi on supported ARM
> systems; I have never tried.
> 
> For later, maybe in the manual: Maybe it would be interesting how to
> create a reusable SD install image for Guix.  I also remember there
> were discussions about making ci.guix.gnu.org build a two-part
> bootable installation image in the past, one part with a bootloader
> for a specific board and another part general for all boards.
> 
> Regards,
> Florian

Re: wip blog post: running Guix System on ARM

[Ludovic Courtès]

Hi,

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

>> ### The kernel modules
>>
>> In order for Guix to be able to load the system from the initramfs, it will probably need
>
> Maybe define initramfs?  Or just link to https://en.wikipedia.org/wiki/Initramfs.

I’d suggest writing “initial RAM disk (initrd)” to (1) be consistent
with the terminology used in Guix, and (2) to avoid obscure abbreviations.
:-)

Ludo’.

Re: wip blog post: running Guix System on ARM

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 2094 bytes --]

Hi Julien,

On Wed, 13 Nov 2019 22:21:54 +0100
Julien Lepiller <julien@lepiller.eu> wrote:

> Hi, attached is a draft for a blog post (or a section in the cookbook)
> for explaining how to install the Guix System on an ARM board. WDYT?

I guess it can't hurt to describe this stuff as it works now, but the goal is not
to need to do any of this complicated stuff--and there's no fundamental reason
why one would have to.  There are projects like buildroot which have all the
u-boot configuration (and more--like the partitioning, which also can sometimes
have funny requirements, for example on Allwinner) that Guix could import to
generically install Guix on any ARM board.
Eventually there will be an importer for those.

The kernel module situation is awful.  It would be good if we could automate
it more in the future (if the information is statically available in the first
place).  If the foreign distribution doesn't have it, not much can be done.

I'm not sure what buildroot does--whether we have kernels with different
built-in modules for different modules or what?

Some comments on your draft:

(1) You absolutely can use an existing u-boot to boot Guix (bugs nonwithstanding).
Guix only generates "extlinux.conf" and doesn't touch the other u-boot config.
The default u-boot-bootloader installer (as opposed to config installer) is
"do nothing".  So your existing u-boot (for example in NAND flash) would display
a boot menu and then you could boot from SSD or SD, for example.
If this doesn't work, please file a bug report.  Also, the old generations
should be able to be selected, too.  Doesn't it work?

FWIW, I'm also using an existing Grub inside Libreboot to boot Guix on X86,
with exactly the same mechanism (grub.conf generated, grub-install not invoked).

(2) If possible, can you mention that non-exported, non-documented variables
are subject to change?  I mean I like that "@@" is possible, but let's make sure
the user knows that it would be better to contact us for including his stuff
if it's supposed to continue to work.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: wip blog post (v2): running Guix System on ARM

[Julien Lepiller]

Le Thu, 14 Nov 2019 13:12:11 +0100,
Danny Milosavljevic <dannym@scratchpost.org> a écrit :

> Hi Julien,
> 
> On Wed, 13 Nov 2019 22:21:54 +0100
> Julien Lepiller <julien@lepiller.eu> wrote:
> 
> > Hi, attached is a draft for a blog post (or a section in the
> > cookbook) for explaining how to install the Guix System on an ARM
> > board. WDYT?  
> 
> I guess it can't hurt to describe this stuff as it works now, but the
> goal is not to need to do any of this complicated stuff--and there's
> no fundamental reason why one would have to.  There are projects like
> buildroot which have all the u-boot configuration (and more--like the
> partitioning, which also can sometimes have funny requirements, for
> example on Allwinner) that Guix could import to generically install
> Guix on any ARM board. Eventually there will be an importer for those.
> 
> The kernel module situation is awful.  It would be good if we could
> automate it more in the future (if the information is statically
> available in the first place).  If the foreign distribution doesn't
> have it, not much can be done.
> 
> I'm not sure what buildroot does--whether we have kernels with
> different built-in modules for different modules or what?
> 
> Some comments on your draft:
> 
> (1) You absolutely can use an existing u-boot to boot Guix (bugs
> nonwithstanding). Guix only generates "extlinux.conf" and doesn't
> touch the other u-boot config. The default u-boot-bootloader
> installer (as opposed to config installer) is "do nothing".  So your
> existing u-boot (for example in NAND flash) would display a boot menu
> and then you could boot from SSD or SD, for example. If this doesn't
> work, please file a bug report.  Also, the old generations should be
> able to be selected, too.  Doesn't it work?
> 
> FWIW, I'm also using an existing Grub inside Libreboot to boot Guix
> on X86, with exactly the same mechanism (grub.conf generated,
> grub-install not invoked).
> 
> (2) If possible, can you mention that non-exported, non-documented
> variables are subject to change?  I mean I like that "@@" is
> possible, but let's make sure the user knows that it would be better
> to contact us for including his stuff if it's supposed to continue to
> work.

Thank you! attached is a new version of that blog post. I think I'll
convert it to texinfo and add it to the cookbook, instead of making it
a blog post, as we then could change or remove it more easily if we
make progress on this :)

I've changed the first paragraph of the u-boot section and the comment
before using @@. I also fixed a few typos others found. If it looks
good, I'll push the change to master in a few days.

Re: wip blog post (v2): running Guix System on ARM

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 459 bytes --]

Hi Julien,

On Sun, 17 Nov 2019 17:57:28 +0100
Julien Lepiller <julien@lepiller.eu> wrote:

> Thank you! attached is a new version of that blog post. I think I'll
> convert it to texinfo and add it to the cookbook, instead of making it
> a blog post, as we then could change or remove it more easily if we
> make progress on this :)

Hmm, did you forget to attach it?  I can't see it so far.

I think it's a good idea to have it in the cookbook.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: wip blog post (v2): running Guix System on ARM

[Ludovic Courtès]

Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Sun, 17 Nov 2019 17:57:28 +0100
> Julien Lepiller <julien@lepiller.eu> wrote:
>
>> Thank you! attached is a new version of that blog post. I think I'll
>> convert it to texinfo and add it to the cookbook, instead of making it
>> a blog post, as we then could change or remove it more easily if we
>> make progress on this :)
>
> Hmm, did you forget to attach it?  I can't see it so far.

I can’t see it either.

What are we missing, Julien?  :-)

Ludo’.

Re: wip blog post (v2): running Guix System on ARM

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

Le Sat, 23 Nov 2019 18:19:22 +0100,
Ludovic Courtès <ludo@gnu.org> a écrit :

> Hello,
> 
> Danny Milosavljevic <dannym@scratchpost.org> skribis:
> 
> > On Sun, 17 Nov 2019 17:57:28 +0100
> > Julien Lepiller <julien@lepiller.eu> wrote:
> >  
> >> Thank you! attached is a new version of that blog post. I think
> >> I'll convert it to texinfo and add it to the cookbook, instead of
> >> making it a blog post, as we then could change or remove it more
> >> easily if we make progress on this :)  
> >
> > Hmm, did you forget to attach it?  I can't see it so far.  
> 
> I can’t see it either.
> 
> What are we missing, Julien?  :-)
> 
> Ludo’.

Right, sorry for that!

[-- Attachment #2: guix-arm.md --]
[-- Type: text/markdown, Size: 12139 bytes --]

Guix on an ARM Board
====================

Increasingly people discovering Guix want to try it on an ARM board, instead of
their x86 computer.  There might be various reasons for that, from power consumption
to security.  In my case, I found these ARM boards practical for self-hosting,
and I think the unique properties of GNU Guix are making it very suitable for that
purpose.  I have installed GNU Guix on a Cubietruck, so my examples below will be
about that board.  However, you should be able to change the examples for your
own use case.

Installing the Guix System on an ARM board is not as easy as installing it on an
x86 desktop computer: there is no installation image.  However, Guix supports
ARM (you can even run it on your [Android phone!](http://guix.gnu.org/blog/2018/guix-on-android/)
and can be installed on a foreign distribution running on that architecture.
The trick is to use the Guix installed on that foreign distribution to initialize
the Guix System.  As we have previously
[mentionned](https://guix.gnu.org/blog/2017/porting-guixsd-to-armv7/) it is
possible to generate an installation image yourself, if your board is supported.
There is also some work on integrating
[buildroot](https://git.savannah.gnu.org/cgit/guix.git/log/?h=wip-buildroot) with Guix,
so that all of these manual steps will no longer be necessary.

Most boards can be booted from an existing GNU+Linux distribution.  You will
need to install a distribution (any of them) and
[install](http://guix.gnu.org/manual/en/html_node/Binary-Installation.html) GNU Guix
on it, using e.g. the installer script.  Then, my plan was to install the Guix System
on an
external SSD drive, instead of the SD card, but we will see that both are perfectly
possible.

The first part of the article will focus on creating a proper U-Boot configuration
and an operating system declaration that suits your board. The second part of this
article will focus on the installation procedure, when there is no installer working
for your system.

Writing a configuration file for an ARM board
---------------------------------------------

A configuration file for an ARM board is not very different from a configuration file
for a desktop or a server running on another architecture.  However, most boards use
the U-Boot bootloader and require some less common modules to be available at boot time.

### The root file system

First of all, you should decide where your root file system is going to be installed. In
my case, I wanted to install is on the external SSD, so I chose it:

```scheme
(file-systems
  (cons* (file-system
           (mount-point "/")
           (device "/dev/sda1")
           (type "ext4"))
         %base-file-systems))
```

If you instead want to install the root file system on an SD card, you'll need to find its
device name, usually `/dev/mmcblk0` and the partition number.  The device corresponding to the
first partition should be `/dev/mmcblk0p1`.  In that case, you would have:

```scheme
(file-systems
  (cons* (file-system
           (mount-point "/")
           (device "/dev/mmcblk0p1")
           (type "ext4"))
         %base-file-systems))
```

### The bootloader

Contrary to grub, there are multiple variants of U-Boot, one per board type.  The installation
procedure for U-Boot is also somewhat specific to the board, so there are two things that you
need to take care of: the U-Boot package and the bootloader declaration.  You can also use
the default U-Boot-bootloader to install a configuration, and let any existing U-Boot do its
job.

Guix already defines a few U-Boot based bootloaders, such as `u-boot-a20-olinuxino-lime-bootloader`
or `u-boot-pine64-plus-bootloader` among others.  If your board already has a `u-boot-*-bootloader`
defined in `(gnu bootloader u-boot)`, you're lucky and you can skip this part of the article!

Otherwise, maybe the bootloader package is defined in `(gnu packages bootloaders)`, such as
the `u-boot-cubietruck` package.  If so, you're a bit lucky and you can skip creating your
own package definition.

If your board doesn't have a `u-boot-*` package defined, you can create one.  It could be
as simple as `(make-u-boot-package "Cubietruck" "arm-linux-gnueabihf")`.  The first argument
is the board name, as expected by the U-Boot build sysetem.  The second argument is the
target triplet that corresponds to the architecture of the board.  You should refer to the
documentation of your board for selecting the correct values.  If you're really unlucky,
you'll need to do some extra work to make the U-Boot package you just created work, as is
the case for the `u-boot-puma-rk3399` for instance: it needs additional phases to install
firmware.

You can add the package definition to your operating system configuration file like so,
before the operating-system declaration:

```scheme
(use-modules (gnu packages bootloaders))

(define u-boot-my-board
  (make-u-boot-package "Myboard" "arm-linux-gnueabihf"))

(operating-system
  [...])
```

Then, you need to define the bootloader.  A bootloader is a structure that has a name,
a package, an installer, a configuration file and a configuration file generator.  Fortunately,
Guix already defines a base U-Boot bootloader, so we can inherit from it and only redefine a few
things.

The Cubietruck happens to be based on an allwinner core, for which there is already a
U-Boot bootloader definition `u-boot-allwinner-bootloader`.  This bootloader is not
usable as is for the Cubietruck, but it defines most of what we need.  In order to get
a proper bootloader for the Cubietruck, we define a new bootloader based on the
Allwinner bootloader definition:

```scheme
(define u-boot-cubietruck-bootloader
  (bootloader
    (inherit u-boot-allwinner-bootloader)
    (package u-boot-cubietruck)))
```

Now that we have our definitions, we can choose where to install the bootloader.  In the
case of the Cubietruck, I decided to install it on the SD card, because it cannot boot from
the SSD directly.  Refer to your board documentation to make sure you install U-Boot on
a bootable device.  As we said earlier, the SD card is `/dev/mmcblk0` on my device.

We can now put everything together like so:

```scheme
(use-modules (gnu packages bootloaders))

(define u-boot-cubietruck
  (make-u-boot-package "Cubietruck" "arm-linux-gnueabihf"))

;; u-boot-allwinner-bootloader is not exported by (gnu bootloader u-boot) so
;; we use @@ to get it.  (@ (module) variable) means: get the value of "variable"
;; as defined (and exported) in (module).  (@@ (module) variable) is the same, but
;; it doesn't care whether it is exported or not.
;; A note of caution though: this is not part of the public API and is subject to
;; change at any time, without warning.  If you need to use it for your board,
;; consider contributing your bootloader instead.
(define u-boot-allwinner-bootloader
  (@@ (gnu bootloader u-boot) u-boot-allwinner-bootloader))

(define u-boot-cubietruck-bootloader
  (bootloader
    (inherit u-boot-allwinner-bootloader)
    (package u-boot-cubietruck)))

(operating-system
  [...]
  (bootloader
    (bootloader-configuration
      (target "/dev/mmcblk0")
      (bootloader u-boot-cubietruck-bootloader)))
  [...])
```

### The kernel modules

In order for Guix to be able to load the system from the initial RAM disk
(initrd), it will probably need to load some modules, especially to access the
root file system.  In my case, the SSD is on an AHCI device, so I need a driver
for it.  The kernel defines `ahci_sunxi` for that device on any sunxi board.
The SD card itself also requires two drivers: `sunxi-mmc` and `sd_mod`.

Your own board may need other kernel modules to boot properly, however it is hard to discover
them.  Guix can tell you when a module is missing in your configuration file if it is loaded
as a module.  Most distributions however build these modules in the kernel directly, so Guix
cannot detect them reliably.  Another way to find what drivers might be needed is to look at
the output of `dmesg`.  The most important modules are modules used for the hardware you'll
be installing the root file system on.  You'll find messages such as:

```
[    5.193684] sunxi-mmc 1c0f000.mmc: Got CD GPIO
[    5.219697] sunxi-mmc 1c0f000.mmc: initialized, max. request size: 16384 KB
[    5.221819] sunxi-mmc 1c12000.mmc: allocated mmc-pwrseq
[    5.245620] sunxi-mmc 1c12000.mmc: initialized, max. request size: 16384 KB
[    5.255341] mmc0: host does not support reading read-only switch, assuming write-enable
[    5.265310] mmc0: new high speed SDHC card at address 0007
[    5.268723] mmcblk0: mmc0:0007 SD32G 29.9 GiB
```

or

```
[    5.614961] ahci-sunxi 1c18000.sata: controller can't do PMP, turning off CAP_PMP
[    5.614981] ahci-sunxi 1c18000.sata: forcing PORTS_IMPL to 0x1
[    5.615067] ahci-sunxi 1c18000.sata: AHCI 0001.0100 32 slots 1 ports 3 Gbps 0x1 impl platform mode
[    5.615083] ahci-sunxi 1c18000.sata: flags: ncq sntf pm led clo only pio slum part ccc 
[    5.616840] scsi host0: ahci-sunxi
[    5.617458] ata1: SATA max UDMA/133 mmio [mem 0x01c18000-0x01c18fff] port 0x100 irq 37
[    5.933494] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
```

Also note that module names are not consistent between what Guix expects and what is printed by
dmesg, especially when the name contains a "-" or a "_".  You will find the correct file name by
building (or using a substitute for) linux-libre beforehand:

```
find `guix build linux-libre`/lib/modules -name '*mmc*'
```

Here, I could find a file named "kernel/drivers/mmc/host/sunxi-mmc.ko", hence the module
name `sunxi-mmc`.  For the other driver, I found a "kernel/drivers/ata/ahci_sunxi.ko",
hence the name `ahci_sunxi`, even if dmesg suggested `ahci-sunxi` (notice the "_" and the
"-" difference).

Once you have found the modules you want to load before mounting the root partition, you can
add them to your operating-system declaration file:

```scheme
(initrd-modules (cons* "sunxi-mmc" "sd_mod" "ahci_sunxi" %base-initrd-modules))
```

Installing the Guix System
--------------------------

### Installing on another drive

In my case, I wanted to install the system on an external SSD, while the currently running
foreign distribution was running from the SD card.  What is nice with this setup is that,
in case of real trouble (your SSD caught fire or broke), you can still boot from the old
foreign system with an installed Guix and all your tools by re-flashing only the bootloader.

In this scenario, we use the foreign system as we would use the installer ISO, using the
[manual installation procedure](http://guix.gnu.org/manual/en/html_node/Proceeding-with-the-Installation.html)
described in the manual.  Essentially, you have to partition your SSD to your liking,
format your new partitions and make sure to reference the correct partition for the
root file system in your configuration file.  Then, initialize the system with:

```bash
mount /dev/sda1 /mnt
mkdir /mnt/etc
$EDITOR /mnt/etc/config.scm # create the configuration file
guix system init /mnt/etc/config.scm /mnt
```

You can now reboot and enjoy your new Guix System!  Note that we didn't need to use
the cow-store service that is mentionned in the manual.  That is because builds and
downloads happen in the store of the foreign distribution, not in memory.  Guix will
copy necessary files to the target.

### Installing on the same drive

Another option is to install the Guix System over the existing foreign distribution, replacing
it entirely.  Note that the root filesystem for the new Guix System is the current root filesystem,
so no need to mount it.  The following will initialize your system:

```bash
$EDITOR /etc/config.scm # create the configuration file
guix system init /etc/config.scm /
```

Make sure to remove the files from the old system.  You should at least get rid of the
old `/etc` directory, like so:

```bash
mv /etc{,.bak}
mkdir /etc
```

Make sure there is an empty /etc, or the new system won't boot properly.  You can
copy your config.scm to the new `/etc` directory.  You can now reboot and enjoy your
new Guix System!

Re: wip blog post: running Guix System on ARM

[Ludovic Courtès]

Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Wed, 13 Nov 2019 22:21:54 +0100
> Julien Lepiller <julien@lepiller.eu> wrote:
>
>> Hi, attached is a draft for a blog post (or a section in the cookbook)
>> for explaining how to install the Guix System on an ARM board. WDYT?
>
> I guess it can't hurt to describe this stuff as it works now, but the goal is not
> to need to do any of this complicated stuff--and there's no fundamental reason
> why one would have to.  There are projects like buildroot which have all the
> u-boot configuration (and more--like the partitioning, which also can sometimes
> have funny requirements, for example on Allwinner) that Guix could import to
> generically install Guix on any ARM board.
> Eventually there will be an importer for those.

Perhaps the closing could link to the work you’ve started in this area
and to explain the goals?

  https://git.savannah.gnu.org/cgit/guix.git/log/?h=wip-buildroot
  (Was there also a mailing list thread on this topic?)

Thanks,
Ludo’.

Re: wip blog post: running Guix System on ARM

[Ludovic Courtès]

Hello!

Julien Lepiller <julien@lepiller.eu> skribis:

> Hi, attached is a draft for a blog post (or a section in the cookbook)
> for explaining how to install the Guix System on an ARM board. WDYT?

I really like it!

Nitpick: s/grub/GRUB/ and s/u-?boot/U-Boot/ (when referring to the
software).

Perhaps a reference at the end to Danny’s work to follow Buildroot more
closely (and to Buildroot itself) would be interesting, but anyhow, it
looks very nice!

I guess you could post it on Monday noon CET (tomorrow) if you feel
ready.  Note that you’ll need to adjust the header just like the other
.md files in guix-artwork.git/website/posts.

Thanks!

Ludo’.

Re: wip blog post: running Guix System on ARM

[Pjotr Prins]

On Sun, Nov 17, 2019 at 08:43:33PM +0100, Ludovic Courtès wrote:
> Hello!
> 
> Julien Lepiller <julien@lepiller.eu> skribis:
> 
> > Hi, attached is a draft for a blog post (or a section in the cookbook)
> > for explaining how to install the Guix System on an ARM board. WDYT?
> 
> I really like it!

Yes, it makes we want to try! I think it should be a BLOG AND a
section in the cookbook.

Pj.