title: powerpc64le-linux support in GNU Guix
date: 2021-03-26 00:00
author: Chris Marusich and Léo Le Bouter
tags: porting, powerpc64le
---

It is a pleasure to announce that support for powerpc64le-linux
(PowerISA v.2.07 and later) has now been
[merged](https://issues.guix.gnu.org/47182) to the master branch of
GNU Guix!

This means that GNU Guix can be used immediately on this platform from
a [from a Git
checkout](https://guix.gnu.org/manual/en/html_node/Building-from-Git.html).
Starting with the next release (Guix v1.2.1), you will also be able to
[download a copy of Guix pre-built for
powerpc64le-linux](https://guix.gnu.org/manual/en/guix.html#Binary-Installation).
Regardless of how you get it, you can run the new powerpc64le-linux
port of GNU Guix on top of any existing powerpc64le GNU/Linux
distribution.

This new platform is available as a "technology preview".  This means
that although it is supported, substitutes are not yet available from
the build farm, and some packages may fail to build.  Although
powerpc64le-linux support is nascent, the Guix community is actively
working on improving it, and this is a great time to [get
involved](https://guix.gnu.org/manual/en/html_node/Contributing.html)!

### Why Is This Important?

This is important because it means that GNU Guix now works on the [RYF
Talos II and Talos II Lite
mainboards](https://www.fsf.org/news/talos-ii-mainboard-and-talos-ii-lite-mainboard-now-fsf-certified-to-respect-your-freedom)
and it's IBM POWER9 processor.  This is a modern, performant hardware
platform that respects your freedom.  It can run without any non-free
code, all the way down to its bootloader and firmware.  It's a
freedom-friendly platform that aligns well with GNU Guix's commitment
to software freedom.

How is this any different from existing RYF hardware, you might ask?
The existing RYF
[laptops](https://ryf.fsf.org/products?category=1&vendor=All&sort_by=created&sort_order=DESC),
[mainboards](https://ryf.fsf.org/products?category=5&vendor=All&sort_by=created&sort_order=DESC),
and
[workstations](https://ryf.fsf.org/products?category=30&vendor=All&sort_by=created&sort_order=DESC)
can only really be used with Intel Core Duo or AMD Opteron processors.
Those processors were released over 15 years ago.  Since then,
processor performance has increased drastically.  People should not
have to choose between performance and freedom, but the fact is that
for many years, that is exactly what we were forced to do.  However,
the Talos II and Talos II Lite have changed this: the free software
community now has an RYF-certified option that can compete with the
performance of modern Intel and AMD systems.

Although the performance of POWER9 processors is competitive with
modern Intel and AMD processors, its real advantage is that it<
respects your freedom.  Modern processors from [both Intel and AMD
include back
doors](https://www.fsf.org/blogs/sysadmin/the-management-engine-an-attack-on-computer-users-freedom)
over which you are given no control.  Additionally, hardware design
defects in the processors of both vendors have been discovered, giving
rise to critical security vulnerabilities like
[Spectre](https://en.wikipedia.org/wiki/Spectre_(security_vulnerability))
and
[Meltdown](https://en.wikipedia.org/wiki/Meltdown_(security_vulnerability)).
In many cases, these vulnerabilities can only be fixed by installing
[non-free CPU microcode](https://wiki.debian.org/Microcode) - unless,
of course, [the vendor decides not to provide any fix at
all](https://arstechnica.com/gadgets/2018/04/intel-drops-plans-to-develop-spectre-microcode-for-ancient-chips/)!

Compared to that, the RYF Talos II and Talos II Lite are a breath of
fresh air that the free software community really deserves.  Raptor
Computing Systems' commitment to software freedom and owner control is
an inspiring reminder that it **is** possible to ship a great product
that respects the freedom of your customers.  And going forward, the
future looks bright for the open, royalty-free Power ISA, [which is
now a Linux Foundation
project](https://www.linuxfoundation.org/press-release/2019/08/the-linux-foundation-announces-new-open-hardware-technologies-and-collaboration/)
(see also: [the same announcement from The OpenPOWER
Foundation](https://openpowerfoundation.org/the-next-step-in-the-openpower-foundation-journey/).

### About other freedom-friendly platforms

The authors believe that the Power ISA, in particular the RYF Talos II
from Raptor Computing Systems, is a good, freedom-friendly choice for
a server or a desktop workstation.  If we didn't, we wouldn't be
writing this blog post!  However, it is worth noting that other
freedom-friendly platforms exist, too.  For example, RISC-V is another
platform that is quite interesting from a freedom standpoint.

In the opinion of these authors, any platform that furthers the cause
of software freedom is a good platform.  For this reason, we want to
make it clear that we do not consider freedom-friendly platforms like
Power ISA or RISC-V (or others) to be in competition with one another.
All people deserve freedom, thus all people deserve a computing
platform that respects their freedom.  It doesn't matter if it's Power
ISA, RISC-V, or something else; if it suits your needs and respects
your freedom, it's good, and we want more of it!

On that topic, Guix still needs a RISC-V port.  Perhaps you (yes, you,
the RISC-V fan reading this right now!) could be the one to start the
porting work for that platform?  The Guix community is very friendly,
so [come and join us](https://guix.gnu.org/en/contribute/), and let's
build it together!

### Bootstrapping powerpc64le-linux: A Journey

To build software, you need software.  How can one port Guix to a
platform before support for that platform exists?  This is a
[bootstrapping
problem](https://guix.gnu.org/manual/en/html_node/Bootstrapping.html).

In Guix, all software for a given system (e.g., powerpc64le-linux) is
built starting from its bootstrap binaries.  It is intended that the
bootstrap binaries are the only pieces of software in the entire
package collection that Guix cannot build from source.  In practice,
[additional bootstrap roots are
possible](https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00814.html),
but introducing them in Guix is highly discouraged, and our community
[actively](https://guix.gnu.org/en/blog/2019/guix-reduces-bootstrap-seed-by-50/)
[works](https://guix.gnu.org/en/blog/2020/guix-further-reduces-bootstrap-seed-to-25/)
to [reduce](https://guix.gnu.org/en/blog/2018/bootstrapping-rust/) our
overall bootstrap footprint.

So first you need to build the the bootstrap binaries for your
platform.  In theory, you can do this in many ways.  For example, you
might try to manually compile them on an existing system.  However,
Guix has [package
definitions](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/make-bootstrap.scm?id=5d8c2c00d60196c46a32b68c618ccbe2b3aa48f4)
that you can use to build them - using Guix, of course!

Commonly, the first step in [porting Guix to a new
platform](https://guix.gnu.org/manual/en/html_node/Porting.html) is to
use Guix to cross-compile the bootstrap binaries for that new platform
from a platform on which Guix is already supported. This can be done
by running a command like the following on a system where Guix is
already installed:

```scheme
guix build --target=powerpc64le-linux-gnu bootstrap-tarballs
```

This is the route that we took when building the powerpc64le-linux
bootstrap binaries, as described in commit
[8a1118a](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a1118a96c9ae128302c3d435ae77cb3dd693aea).

Note that before you can even do this, you must first update the
glibc-dynamic-linker and system->linux-architecture procedures in
Guix's code, as described in
[Porting](https://guix.gnu.org/manual/en/html_node/Porting.html).  In
addition, the versions of packages in Guix that make up the GNU
toolchain (gcc, glibc, etc.) must already support the target platform.
This pre-existing toolchain support needs to be good enough so that
Guix can (1) build, on some already-supported platform, a
cross-compilation toolchain for that target, (2) use, on that same
platform, the cross-compilation toolchain to cross-compile the
bootstrap binaries for the target platform, and (3) use, on the target
platform, the bootstrap binaries to natively build the rest of the
Guix package collection.  The above "guix build" command takes care of
steps (1) and (2) automatically.

Step (3) is a little more involved.  Once the bootstrap binaries for
the target platform have been built, they must be made available for
anyone to download.  In addition, Guix's code must be updated so that
(a) it recognizes the system (e.g., "powerpc64le-linux") that will be
used to identify the new platform and (b) it fetches the new system's
bootstrap binaries from the right location.  Commit
[8a1118a](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a1118a96c9ae128302c3d435ae77cb3dd693aea)
accomplishes both of those things.  After that, you just have to try
building things and see what breaks, for example by running
`./pre-inst-env guix build hello` from your Git checkout to build GNU
Hello.

The actual bootstrap binaries for powerpc64le-linux are stored on the
[alpha.gnu.org FTP
server](https://alpha.gnu.org/gnu/guix/bootstrap/powerpc64le-linux/20210106/).
Chris Marusich built these bootstrap binaries in an x86_64-linux Guix
System VM which was running on hardware owned by Léo Le Bouter.  Chris
then signed the binaries and provided them to Ludovic Courtès, who in
turn verified their authenticity, signed them, and [uploaded them to
alpha.gnu.org](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669#125).

With the bootstrap binaries finally in place, we could get started on
step (3) above: building the rest of the package collection.  Or
trying to, at least.  There were many stumbling blocks.  For example,
to resolve some test failures, we had to update the code in Guix that
allows it to make
[certain](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=b57de27d0331198c9cafb09a1cf8a5fa4f691e36)
[syscalls](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c29bfbfc78ccd9e5c10d38faf3d269eafed12854)
from scheme.  As another example, we had to [patch GCC so that it
looks for the 64-bit libraries in
/lib](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46253), rather
than /lib64, since that is where Guix puts its 64-bit libraries by
convention.  Some packages required for Guix failed to build, so we
had to debug those build failures, too.

For a list of all the changes, see [the patch
series](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47182) or the
actual commits, which are:

```
$ git log --oneline --no-decorate 8a1118a96c9ae128302c3d435ae77cb3dd693aea..65c46e79e0495fe4d32f6f2725d7233fff10fd70
65c46e79e04 gnu: sed: Make it build on SELinux-enabled kernels.
93f21e1a35e utils: Fix target-64bit? on powerpc64le-linux.
8d9aece8c49 ci: %cross-targets: Add powerpc64le-linux-gnu.
c29bfbfc78c syscalls: Fix RNDADDTOENTCNT on powerpc64le-linux.
b57de27d033 syscalls: Fix clone on powerpc64le-linux.
a16eb6c5f97 Add powerpc64le-linux as a supported Guix architecture.
b50f4268035 gnu: libelf: Fix compilation for powerpc64le-linux.
1a0f4013d33 gnu: texlive-latex-base: Fix compilation on powerpc64le*.
e9938dc8f0e gnu: texlive-bin: Fix compilation on powerpc64le*.
69b3907adf6 gnu: guile-avahi: Fix compilation on powerpc64le-linux.
4cc2d2aa599 gnu: bdb-4.8: Fix configure on powerpc64le-linux.
be4b1cf53bd gnu: binutils-final: Support more Power architectures.
060478c32c9 gnu: binutils-final: Provide bash for binary on powerpc-linux.
b2135b5d576 gnu: gcc-boot0: Enable 128-bit long double for POWER9.
6e98e9ca923 gnu: glibc: Fix ldd path on powerpc*.
cac88b28b83 gnu: gcc-4.7: On powerpc64le, fix /lib64 references.
fc7cf0c1ecc utils: Add target-powerpc? procedure.
```

In the end, through the combined efforts of multiple people, we slowly
worked through the issues until we reached a point where we could do
all of the following things successfully:

- Build Guix manually on a [Debian GNU/Linux
  ppc64el](https://wiki.debian.org/ppc64el) machine (yet another name
  for the powerpc64le-linux-gnu triplet), and all of its "make check"
  tests passed.
- Build GNU Hello using Guix and run it.
- Run "guix pull" to build and install the most recent version of
  Guix, with powerpc64le-linux support.
- Build a release binary tarball for powerpc64le-linux using "make
  guix-binary.powerpc64le-linux.tar.xz"
- Use that binary to install a version of Guix that could build/run
  GNU Hello and run "guix pull" successfully.

This was an exciting moment!  But there was still more work to be
done.

Originally, we did this work on the wip-ppc64le branch, with the
intent of merging it into core-updates.  By convention, the
"[core-updates](https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html)"
branch in Guix is where changes are made if they cause too many
rebuilds.  Since we were updating package definitions so deep in the
dependency graph of the package collection, we assumed it wouldn't be
possible to avoid rebuilding the world.  For this reason, we had based
the wip-ppc64le branch on core-updates.

However, Efraim Flashner proved us wrong!  He created a separate
branch, wip-ppc64le-for-master, where he adjusted some of the
wip-ppc64le commits to avoid rebuilding the world on other platforms.
Thanks to his work, we were able to merge the changes directly to
master!  This meant that we would be able to include it in the next
release (Guix v.1.2.1).

In short, the initial porting work is done, and it is now possible for
anyone to easily try out Guix on this new platform.  Because "guix
pull" works, too, it is also easy to iterate on what we have and work
towards improving support for the platform.  It took a lot of
cooperation and effort to get this far, but there are multiple people
actively contributing to this port in the Guix community who want to
see it succeed, and we hope you will join us in exploring the limits
of this exciting new freedom-friendly platform!


### Hiccups Along the Way

Along the way, there were a few problems that stymied our porting
attempts.

First, we thought we would try to port to powerpc64-linux
(big-endian).  This did not prove to be any easier than the
little-endian port.  In addition, other distributions (e.g.,
[Debian](https://www.debian.org/ports/) and
[Fedora](https://fedoraproject.org/wiki/Architectures)) have recently
dropped their big-endian powerpc64 ports, so the little-endian variant
is more likely to be tested and supported in the community.  So we
decided to focus our efforts on the little-endian variant, and so far
we haven't looked back.

In both the big-endian and little-endian case, we were saddened to
discover that the bootstrap binaries are not entirely reproducible.
This fact is documented in [bug
41669](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669), along
with our extensive investigations.  In short, if you build the
bootstrap binaries on two separate machines without using any
substitutes, you will find that the derivation which cross-compiles
%gcc-static (the bootstrap GCC, version 5.5.0) produces different
output on the two systems.  However, if you build %gcc-static twice on
the same system, it builds reproducibly.  This suggests that something
in the transitive closure of inputs of %gcc-static is perhaps
contributing to its non-reproducibility.  There is an interesting graph toward the end of the bug report that suggests that the non-reproducibliity is introduced by one or more of 21 derivations in the transitive closure of %gcc-static's inputs that do not build reproducibly across systems.

At some point, you have to cut your losses and move on.  After months
of investigation without resolving the reproducibility issue, we
finally decided to move forward with the bootstrap binaries produced
earlier.  If necessary, we can always go back and try to fix this
issue, but it seemed more important to get started with the
bootstrapping work.  Anyone who is interested in solving this problem
can comment on the bug report and help us figure out the mystery.  We
are very interested in solving it, but at the moment we are more
interested in bootstrapping the rest of the system with the existing
bootstrap binaries.

### Next Steps

It is now possible to install Guix on a powerpc64le-linux system and
use it to build some useful software - in particular, Guix itself.  So
Guix is now "self-hosted" on this platform, which gives us a
comfortable place to begin further work.

The following tasks still need to be done.  Anyone can help, so please
get in touch if you want to contribute!

- Solve the GCC bootstrap binary reproducibility issue.
- Get Guix System to work on powerpc64le-linux.
- Get CI infrastructure to work (Cuirass, guix-build-coordinator,
  substitutes, etc.)
- Fix all package build failures.
- Try building rust, and if it works, judiciously re-introduce the
  librsvg dependency for powerpc64le-linux in gtk+ and gtk+-2, since
  [it is currently
  missing](https://git.savannah.gnu.org/cgit/guix.git).
- Upgrade GCC to 8 on core-updates, fix issues that occur.
- Merge core-updates to master after that.