Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich <cmmarusich@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 1234 bytes --]

Hi,

Here's a new version of the blog post.  It incorporates all the feedback
so far.  I've also removed the "About other freedom-friendly platforms"
because it didn't seem to add much substance.

I significantly rewrote the "Why Is This Important?" section, mainly
because I realized that I was incorrectly and unfairly implying that
POWER9 CPUs are not vulnerable to Spectre/Meltdown-style
vulnerabilities.  In fact, some POWER9 CPUs were found to be vulnerable,
but the most recent models have been fixed.  I've rewritten this section
so that it focuses more on explaining why the RYF Talos II and Talos II
Lite are "more free" than the popular Intel and AMD mainstays (even the
older, RYF-certified models, where you still have to jump over the
hurdle of removing the Intel ME or equivalent.)

For details on Spectre/Meltdown on POWER9, see:

https://wiki.raptorcs.com/wiki/Speculative_Execution_Vulnerabilities_of_2018

I added a footer describing GNU Guix, as is customary on most of our
blog posts.

I changed the title.

I also fixed various links and rephrased a few things.

Anyway, if you can cast your eye over it once more, I would appreciate
it.  I think it's just about done!

-- 
Chris

[-- Attachment #1.2: 0001-website-drafts-Add-powerpc64le-linux-announcement.patch --]
[-- Type: text/x-patch, Size: 22627 bytes --]

From 4d9133e51fc666f14074c1da18bb16af0d76066f Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Tue, 6 Apr 2021 00:10:35 -0700
Subject: [PATCH] website: drafts: Add powerpc64le-linux announcement.

* website/drafts/new-system-powerpc64le-linux.md: New file.
---
 .../drafts/new-system-powerpc64le-linux.md    | 389 ++++++++++++++++++
 1 file changed, 389 insertions(+)
 create mode 100644 website/drafts/new-system-powerpc64le-linux.md

diff --git a/website/drafts/new-system-powerpc64le-linux.md b/website/drafts/new-system-powerpc64le-linux.md
new file mode 100644
index 0000000..d2104aa
--- /dev/null
+++ b/website/drafts/new-system-powerpc64le-linux.md
@@ -0,0 +1,389 @@
+title: New Supported Platform: powerpc64le-linux
+date: 2021-04-08 00:00
+author: Chris Marusich and Léo Le Bouter
+tags: porting, powerpc64le, bootstrapping, cross-compilation, reproducibility
+---
+
+It is a pleasure to announce that support for powerpc64le-linux
+(PowerISA v.2.07 and later) has now been
+[merged](https://issues.guix.gnu.org/47182) to the master branch of
+GNU Guix!
+
+This means that GNU Guix can be used immediately on this platform from
+a [from a Git
+checkout](https://guix.gnu.org/manual/en/html_node/Building-from-Git.html).
+Starting with the next release (Guix v1.2.1), you will also be able to
+[download a copy of Guix pre-built for
+powerpc64le-linux](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html#Binary-Installation).
+Regardless of how you get it, you can run the new powerpc64le-linux
+port of GNU Guix on top of any existing powerpc64le GNU/Linux
+distribution.
+
+This new platform is available as a "technology preview".  This means
+that although it is supported,
+[substitutes](https://guix.gnu.org/manual/en/html_node/Substitutes.html)
+are not yet available from the build farm, and some packages may fail
+to build.  Although powerpc64le-linux support is nascent, the Guix
+community is actively working on improving it, and this is a great
+time to [get
+involved](https://guix.gnu.org/manual/en/html_node/Contributing.html)!
+
+### Why Is This Important?
+
+This is important because it means that GNU Guix now works on the
+[Talos II and Talos II Lite
+mainboards](https://www.fsf.org/news/talos-ii-mainboard-and-talos-ii-lite-mainboard-now-fsf-certified-to-respect-your-freedom),
+which use [IBM POWER9
+processors](https://wiki.raptorcs.com/wiki/POWER9).  This is a modern,
+performant hardware platform that has recently received [Respects Your
+Freedom (RYF) certification](https://ryf.fsf.org/) from the FSF.  It
+can run without any non-free code, all the way down to its bootloader
+and firmware.  In other words, it's a freedom-friendly platform that
+aligns well with GNU Guix's commitment to software freedom.
+
+How is this any different from existing RYF hardware, you might ask?
+One reason is performance.  The existing RYF
+[laptops](https://ryf.fsf.org/products?category=1&vendor=All&sort_by=created&sort_order=DESC),
+[mainboards](https://ryf.fsf.org/products?category=5&vendor=All&sort_by=created&sort_order=DESC),
+and
+[workstations](https://ryf.fsf.org/products?category=30&vendor=All&sort_by=created&sort_order=DESC)
+can only really be used with Intel Core Duo or AMD Opteron processors.
+Those processors were released over 15 years ago.  Since then,
+processor performance has increased drastically.  People should not
+have to choose between performance and freedom, but for many years
+that is exactly what we were forced to do.  However, the Talos II and
+Talos II Lite have changed this: the free software community now has
+an RYF-certified option that [can compete with the performance of
+modern Intel and AMD
+systems](https://www.phoronix.com/scan.php?page=article&item=power9-threadripper-core9&num=1).
+
+Although the performance of POWER9 processors is competitive with
+modern Intel and AMD processors, the real advantage of the Talos II
+and Talos II Lite is that they were designed from the start to respect
+your freedom.  Modern processors from [both Intel and AMD include back
+doors](https://www.fsf.org/blogs/sysadmin/the-management-engine-an-attack-on-computer-users-freedom)
+over which you are given no control.  Even though the back doors can
+be removed [in some cases with significant
+effort](https://www.fsf.org/news/libreboot-x200-laptop-now-fsf-certified-to-respect-your-freedom),
+this is an obstacle that nobody should have to overcome just to
+control their own computer.  No such obstacles exist when using the
+Talos II or Talos II Lite.  In fact, although
+[Intel](https://arstechnica.com/gadgets/2020/10/in-a-first-researchers-extract-secret-key-used-to-encrypt-intel-cpu-code/)
+and
+[AMD](https://www.extremetech.com/computing/292722-amds-secure-processor-firmware-is-now-explorable-thanks-to-new-tool)
+both go out of their way to keep you from understanding what is going
+on in your own computer, Raptor Computing Systems releases [all of the
+software and firmware used on the
+board](https://git.raptorcs.com/git/) as free software.  They even
+include circuit diagrams when they ship you the machine.
+
+Compared to the existing options, the RYF Talos II and Talos II Lite
+are a breath of fresh air that the free software community really
+deserves.  Raptor Computing Systems' commitment to software freedom
+and owner control is an inspiring reminder that it **is** possible to
+ship a great product that respects the freedom of your customers.  And
+going forward, the future looks bright for the open, royalty-free
+Power ISA, [which is now a Linux Foundation
+project](https://www.linuxfoundation.org/press-release/2019/08/the-linux-foundation-announces-new-open-hardware-technologies-and-collaboration/)
+(see also: [the same announcement from The OpenPOWER
+Foundation](https://openpowerfoundation.org/the-next-step-in-the-openpower-foundation-journey/).
+
+### Bootstrapping powerpc64le-linux: A Journey
+
+To build software, you need software.  How can one port Guix to a
+platform before support for that platform exists?  This is a
+[bootstrapping
+problem](https://guix.gnu.org/manual/en/html_node/Bootstrapping.html).
+
+In Guix, all software for a given platform (e.g., powerpc64le-linux)
+is built starting from a small set of "bootstrap binaries".  These are
+binaries of Guile, GCC, Binutils, libc, and a few other packages,
+pre-built for the relevant platform.  It is intended that the
+bootstrap binaries are the only pieces of software in the entire
+package collection that Guix cannot build from source.  In practice,
+[additional bootstrap roots are
+possible](https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00814.html),
+but introducing them in Guix is highly discouraged, and our community
+[actively](https://guix.gnu.org/en/blog/2019/guix-reduces-bootstrap-seed-by-50/)
+[works](https://guix.gnu.org/en/blog/2020/guix-further-reduces-bootstrap-seed-to-25/)
+to [reduce](https://guix.gnu.org/en/blog/2018/bootstrapping-rust/) our
+overall bootstrap footprint.  There is one set of bootstrap binaries
+for each platform that Guix supports.
+
+This means that to port Guix to a new platform, you must first build
+the bootstrap binaries for that platform.  In theory, you can do this
+in many ways.  For example, you might try to manually compile them on
+an existing system.  However, Guix has [package
+definitions](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/make-bootstrap.scm?id=5d8c2c00d60196c46a32b68c618ccbe2b3aa48f4)
+that you can use to build them - using Guix, of course!
+
+Commonly, the first step in [porting Guix to a new
+platform](https://guix.gnu.org/manual/en/html_node/Porting.html) is to
+use Guix to cross-compile the bootstrap binaries for that new platform
+from a platform on which Guix is already supported. This can be done
+by running a command like the following on a system where Guix is
+already installed:
+
+```scheme
+guix build --target=powerpc64le-linux-gnu bootstrap-tarballs
+```
+
+This is the route that we took when building the powerpc64le-linux
+bootstrap binaries, as described in commit
+[8a1118a](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a1118a96c9ae128302c3d435ae77cb3dd693aea).
+You might wonder why the target above is "powerpc64le-linux-gnu" even
+though the new Guix platform is called "powerpc64le-linux".  This is
+because "powerpc64le-linux-gnu" is a GNU
+[triplet](https://wiki.osdev.org/Target_Triplet) identifying the new
+platform, but "powerpc64le-linux" is the name of a "system" (i.e., a
+platform) in Guix.  Guix contains code that converts between the two
+as needed (see `nix-system->gnu-triplet` and `gnu-triplet->nix-system`
+in
+[`guix/utils.scm`](https://git.savannah.gnu.org/cgit/guix.git/tree/guix/utils.scm?id=83991a34d5c1d4985e54dd029a81412277ad062a).
+When cross-compiling, you only need to specify the GNU triplet.
+
+Note that before you can even do this, you must first update the
+`glibc-dynamic-linker` and `system->linux-architecture` procedures in
+Guix's code, as described in
+[Porting](https://guix.gnu.org/manual/en/html_node/Porting.html).  In
+addition, the versions of packages in Guix that make up the GNU
+toolchain (gcc, glibc, etc.) must already support the target platform.
+This pre-existing toolchain support needs to be good enough so that
+Guix can (1) build, on some already-supported platform, a
+cross-compilation toolchain for the target platform, (2) use, on the
+already-supported platform, the cross-compilation toolchain to
+cross-compile the bootstrap binaries for the target platform, and (3)
+use, on the target platform, the bootstrap binaries to natively build
+the rest of the Guix package collection.  The above [`guix
+build`](https://guix.gnu.org/manual/en/html_node/Invoking-guix-build.html#Invoking-guix-build)
+command takes care of steps (1) and (2) automatically.
+
+Step (3) is a little more involved.  Once the bootstrap binaries for
+the target platform have been built, they must be published online for
+anyone to download.  After that, Guix's code must be updated so that
+(a) it recognizes the "system" name (e.g., "powerpc64le-linux") that
+will be used to identify the new platform and (b) it fetches the new
+platform's bootstrap binaries from the right location.  After all that
+is done, you just have to try building things and see what breaks.
+For example, you can run `./pre-inst-env guix build hello` from your
+Git checkout to try building GNU Hello.
+
+The actual bootstrap binaries for powerpc64le-linux are stored on the
+[alpha.gnu.org FTP
+server](https://alpha.gnu.org/gnu/guix/bootstrap/powerpc64le-linux/20210106/).
+Chris Marusich built these bootstrap binaries in an x86_64-linux Guix
+System VM which was running on hardware owned by Léo Le Bouter.  Chris
+then signed the binaries and provided them to Ludovic Courtès, who in
+turn verified their authenticity, signed them, and [uploaded them to
+alpha.gnu.org](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669#125).
+After that, we updated the code to use the newly published bootstrap
+binaries in commit
+[8a1118a](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a1118a96c9ae128302c3d435ae77cb3dd693aea).
+Once all that was done, we could begin bootstrapping the rest of the
+system - or trying to, at least.
+
+There were many stumbling blocks.  For example, to resolve some test
+failures, we had to update the code in Guix that enables it to make
+[certain](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=b57de27d0331198c9cafb09a1cf8a5fa4f691e36)
+[syscalls](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c29bfbfc78ccd9e5c10d38faf3d269eafed12854)
+from scheme.  In another example, we had to [patch GCC so that it
+looks for the 64-bit libraries in
+/lib](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46253), rather
+than /lib64, since that is where Guix puts its 64-bit libraries by
+convention.  In addition, some packages required in order to build
+Guix failed to build, so we had to debug those build failures, too.
+
+For a list of all the changes, see [the patch
+series](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47182) or the
+actual commits, which are:
+
+```
+$ git log --oneline --no-decorate 8a1118a96c9ae128302c3d435ae77cb3dd693aea..65c46e79e0495fe4d32f6f2725d7233fff10fd70
+65c46e79e04 gnu: sed: Make it build on SELinux-enabled kernels.
+93f21e1a35e utils: Fix target-64bit? on powerpc64le-linux.
+8d9aece8c49 ci: %cross-targets: Add powerpc64le-linux-gnu.
+c29bfbfc78c syscalls: Fix RNDADDTOENTCNT on powerpc64le-linux.
+b57de27d033 syscalls: Fix clone on powerpc64le-linux.
+a16eb6c5f97 Add powerpc64le-linux as a supported Guix architecture.
+b50f4268035 gnu: libelf: Fix compilation for powerpc64le-linux.
+1a0f4013d33 gnu: texlive-latex-base: Fix compilation on powerpc64le*.
+e9938dc8f0e gnu: texlive-bin: Fix compilation on powerpc64le*.
+69b3907adf6 gnu: guile-avahi: Fix compilation on powerpc64le-linux.
+4cc2d2aa599 gnu: bdb-4.8: Fix configure on powerpc64le-linux.
+be4b1cf53bd gnu: binutils-final: Support more Power architectures.
+060478c32c9 gnu: binutils-final: Provide bash for binary on powerpc-linux.
+b2135b5d576 gnu: gcc-boot0: Enable 128-bit long double for POWER9.
+6e98e9ca923 gnu: glibc: Fix ldd path on powerpc*.
+cac88b28b83 gnu: gcc-4.7: On powerpc64le, fix /lib64 references.
+fc7cf0c1ecc utils: Add target-powerpc? procedure.
+```
+
+In the end, through the combined efforts of multiple people, we slowly
+worked through the issues until we reached a point where we could do
+all of the following things successfully:
+
+- Build Guix manually on a [Debian GNU/Linux
+  ppc64el](https://wiki.debian.org/ppc64el) machine (this is Debian's
+  name for a system using the powerpc64le-linux-gnu triplet), and
+  verify that its `make check` tests passed.
+- Build GNU Hello using Guix and run it.
+- Run [`guix
+  pull`](https://guix.gnu.org/manual/en/html_node/Invoking-guix-pull.html#Invoking-guix-pull)
+  to build and install the most recent version of Guix, with
+  powerpc64le-linux support.
+- Build a release binary tarball for powerpc64le-linux via: `make
+  guix-binary.powerpc64le-linux.tar.xz`
+- Use that binary to install a version of Guix that could build/run
+  GNU Hello and run `guix pull` successfully.
+
+This was an exciting moment!  But there was still more work to be
+done.
+
+Originally, we did this work on the
+[wip-ppc64le](https://git.savannah.gnu.org/cgit/guix.git/log/?h=wip-ppc64le)
+branch, with the intent of merging it into core-updates.  By
+convention, the "core-updates" branch in Guix is [where changes are
+made if they cause too many
+rebuilds](https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html).
+Since we were updating package definitions so deep in the dependency
+graph of the package collection, we assumed it wouldn't be possible to
+avoid rebuilding the world.  For this reason, we had based the
+wip-ppc64le branch on core-updates.
+
+However, Efraim Flashner proved us wrong!  He created a separate
+branch, wip-ppc64le-for-master, where he adjusted some of the
+wip-ppc64le commits to avoid rebuilding the world on other platforms.
+Thanks to his work, we were able to merge the changes directly to
+master!  This meant that we would be able to include it in the next
+release (Guix v.1.2.1).
+
+In short, the initial porting work is done, and it is now possible for
+anyone to easily try out Guix on this new platform.  Because `guix
+pull` works, too, it is also easy to iterate on what we have and work
+towards improving support for the platform.  It took a lot of
+cooperation and effort to get this far, but there are multiple people
+actively contributing to this port in the Guix community who want to
+see it succeed.  We hope you will join us in exploring the limits of
+this exciting new freedom-friendly platform!
+
+### Other Porting Challenges
+
+Very early in the porting process, there were some other problems that
+stymied our work.
+
+First, we actually thought we would try to port to powerpc64-linux
+(big-endian).  However, this did not prove to be any easier than the
+little-endian port.  In addition, other distributions (e.g.,
+[Debian](https://www.debian.org/ports/) and
+[Fedora](https://fedoraproject.org/wiki/Architectures)) have recently
+dropped their big-endian powerpc64 ports, so the little-endian variant
+is more likely to be tested and supported in the community.  For these
+reasons, we decided to focus our efforts on the little-endian variant,
+and so far we haven't looked back.
+
+In both the big-endian and little-endian case, we were saddened to
+discover that the bootstrap binaries are not entirely reproducible.
+This fact is documented in [bug
+41669](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669), along
+with our extensive investigations.
+
+In short, if you build the bootstrap binaries on two separate machines
+without using any substitutes, you will find that the derivation which
+cross-compiles %gcc-static (the bootstrap GCC, version 5.5.0) produces
+different output on the two systems.  However, if you build
+%gcc-static twice on the same system, it builds reproducibly.  This
+suggests that something in the transitive closure of inputs of
+%gcc-static is perhaps contributing to its non-reproducibility.  There
+is an interesting graph [toward the end of the bug
+report](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669#137),
+shown below:
+
+![Differing
+Derivations](https://debbugs.gnu.org/cgi/bugreport.cgi?att=1;msg=137;filename=nonreproducible-drvs-small.png;bug=41669)
+
+This graph shows the derivations that produce differing outputs across
+two Guix System machines, when everything is built without
+substitutes.  It starts from the derivation that cross-compiles
+%gcc-static for powerpc64-linux-gnu (from x86_64-linux) using Guix at
+commit 1ced8379c7641788fa607b19b7a66d18f045362b.  Then, it walks the
+graph of derivation inputs, recording only those derivations which
+produce differing output on the two different machines.  If the
+non-reproducibility (across systems) of %gcc-static is caused by a
+non-reproducible input, then it is probably caused by one or more of
+the derivations shown in this graph.
+
+At some point, you have to cut your losses and move on.  After months
+of investigation without resolving the reproducibility issue, we
+finally decided to move forward with the bootstrap binaries produced
+earlier.  If necessary, we can always go back and try to fix this
+issue.  However, it seemed more important to get started with the
+bootstrapping work.
+
+Anyone who is interested in solving this problem is welcome to comment
+on the bug report and help us to figure out the mystery.  We are very
+interested in solving it, but at the moment we are more focused on
+building the rest of the Guix package collection on the
+powerpc64le-linux platform using the existing bootstrap binaries.
+
+### Next Steps
+
+It is now possible to install Guix on a powerpc64le-linux system and
+use it to build some useful software - in particular, Guix itself.  So
+Guix is now "self-hosted" on this platform, which gives us a
+comfortable place to begin further work.
+
+The following tasks still need to be done.  Anyone can help, so please
+get in touch if you want to contribute!
+
+- Solve [the GCC bootstrap binary reproducibility
+  issue](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669)
+  described above.
+- Get [Guix
+  System](https://guix.gnu.org/manual/en/html_node/System-Installation.html)
+  to work on powerpc64le-linux.
+- Get CI infrastructure to work
+  ([Cuirass](https://guix.gnu.org/en/cuirass/) (see also: [Cuirass in
+  the Guix
+  manual](https://guix.gnu.org/manual/en/html_node/Continuous-Integration.html),
+  [guix-build-coordinator](https://git.cbaines.net/guix/build-coordinator/)
+  (see also: [Guix Build Coordinator in the Guix
+  manual](https://guix.gnu.org/manual/en/html_node/Guix-Services.html),
+  [substitutes](https://guix.gnu.org/manual/en/html_node/Substitutes.html),
+  etc.)
+- Try to build your favorite packages using Guix, [report
+  problems](https://guix.gnu.org/manual/en/html_node/Tracking-Bugs-and-Patches.html),
+  [try to fix
+  them](https://guix.gnu.org/manual/en/html_node/Contributing.html),
+  and [ask for help](https://guix.gnu.org/en/help/) if you're feeling
+  stuck or not sure how to start.
+- Try building rust, and if it works, judiciously re-introduce the
+  librsvg dependency for powerpc64le-linux in gtk+ and gtk+-2, since
+  [it is currently
+  missing](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=5d2863dfe4613d5091e61800fcd5a48922c8ce4e).
+- Upgrade the default GCC to 8 on core-updates, try to build guix
+  (e.g., `./pre-inst-env guix build guix`), and report/fix whatever
+  issues occur.  We want to upgrade GCC to 8 because, on the
+  core-updates branch, glibc has been upgraded from 2.31 to 2.32.
+  Unfortunately, on powerpc64le-linux, upgrading glibc from 2.31 to
+  2.32 without also upgrading the default GCC (it's currently 7.5.0)
+  causes a lot of problems.  Right now, we believe the best path
+  forward is probably just to upgrade to GCC 8 on core-updates.
+- Merge core-updates to master after that.
+
+#### About GNU Guix
+
+[GNU Guix](https://guix.gnu.org) is a transactional package manager and
+an advanced distribution of the GNU system that [respects user
+freedom](https://www.gnu.org/distros/free-system-distribution-guidelines.html).
+Guix can be used on top of any system running the Hurd or the Linux
+kernel, or it can be used as a standalone operating system distribution
+for i686, x86_64, ARMv7, and AArch64 machines.
+
+In addition to standard package management features, Guix supports
+transactional upgrades and roll-backs, unprivileged package management,
+per-user profiles, and garbage collection.  When used as a standalone
+GNU/Linux distribution, Guix offers a declarative, stateless approach to
+operating system configuration management.  Guix is highly customizable
+and hackable through [Guile](https://www.gnu.org/software/guile)
+programming interfaces and extensions to the
+[Scheme](http://schemers.org) language.
-- 
2.30.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]