Please review blog post draft: powerpc64le-linux support

Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

[-- Attachment #1.1: Type: text/plain, Size: 473 bytes --]

Hi,

Léo and I have drafted the following blog post.  Could you take a few
minutes to read it and give us your thoughts?

It's a work in progress.  The primary goal is to announce the new
powerpc64le-linux support and explain why it matters (POWER9 is an
exciting, freedom-friendly platform!).  The secondary goal is to explain
some of the details about what we did, and invite people to get
involved.

Your feedback would be highly appreciated!

-- 
Chris

[-- Attachment #1.2: 0001-website-drafts-Add-powerpc64le-linux-announcement.patch --]
[-- Type: text/x-patch, Size: 18755 bytes --]

From 8900d918106f6a70b20df461c5f086b5275773cc Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Tue, 6 Apr 2021 00:10:35 -0700
Subject: [PATCH] website: drafts: Add powerpc64le-linux announcement.

* website/drafts/new-system-powerpc64le-linux.md: New file.
---
 .../drafts/new-system-powerpc64le-linux.md    | 326 ++++++++++++++++++
 1 file changed, 326 insertions(+)
 create mode 100644 website/drafts/new-system-powerpc64le-linux.md

diff --git a/website/drafts/new-system-powerpc64le-linux.md b/website/drafts/new-system-powerpc64le-linux.md
new file mode 100644
index 0000000..e3de5ba
--- /dev/null
+++ b/website/drafts/new-system-powerpc64le-linux.md
@@ -0,0 +1,326 @@
+title: powerpc64le-linux support in GNU Guix
+date: 2021-03-26 00:00
+author: Chris Marusich and Léo Le Bouter
+tags: porting, powerpc64le
+---
+
+It is a pleasure to announce that support for powerpc64le-linux
+(PowerISA v.2.07 and later) has now been
+[merged](https://issues.guix.gnu.org/47182) to the master branch of
+GNU Guix!
+
+This means that GNU Guix can be used immediately on this platform from
+a [from a Git
+checkout](https://guix.gnu.org/manual/en/html_node/Building-from-Git.html).
+Starting with the next release (Guix v1.2.1), you will also be able to
+[download a copy of Guix pre-built for
+powerpc64le-linux](https://guix.gnu.org/manual/en/guix.html#Binary-Installation).
+Regardless of how you get it, you can run the new powerpc64le-linux
+port of GNU Guix on top of any existing powerpc64le GNU/Linux
+distribution.
+
+This new platform is available as a "technology preview".  This means
+that although it is supported, substitutes are not yet available from
+the build farm, and some packages may fail to build.  Although
+powerpc64le-linux support is nascent, the Guix community is actively
+working on improving it, and this is a great time to [get
+involved](https://guix.gnu.org/manual/en/html_node/Contributing.html)!
+
+### Why Is This Important?
+
+This is important because it means that GNU Guix now works on the [RYF
+Talos II and Talos II Lite
+mainboards](https://www.fsf.org/news/talos-ii-mainboard-and-talos-ii-lite-mainboard-now-fsf-certified-to-respect-your-freedom)
+and it's IBM POWER9 processor.  This is a modern, performant hardware
+platform that respects your freedom.  It can run without any non-free
+code, all the way down to its bootloader and firmware.  It's a
+freedom-friendly platform that aligns well with GNU Guix's commitment
+to software freedom.
+
+How is this any different from existing RYF hardware, you might ask?
+The existing RYF
+[laptops](https://ryf.fsf.org/products?category=1&vendor=All&sort_by=created&sort_order=DESC),
+[mainboards](https://ryf.fsf.org/products?category=5&vendor=All&sort_by=created&sort_order=DESC),
+and
+[workstations](https://ryf.fsf.org/products?category=30&vendor=All&sort_by=created&sort_order=DESC)
+can only really be used with Intel Core Duo or AMD Opteron processors.
+Those processors were released over 15 years ago.  Since then,
+processor performance has increased drastically.  People should not
+have to choose between performance and freedom, but the fact is that
+for many years, that is exactly what we were forced to do.  However,
+the Talos II and Talos II Lite have changed this: the free software
+community now has an RYF-certified option that can compete with the
+performance of modern Intel and AMD systems.
+
+Although the performance of POWER9 processors is competitive with
+modern Intel and AMD processors, its real advantage is that it<
+respects your freedom.  Modern processors from [both Intel and AMD
+include back
+doors](https://www.fsf.org/blogs/sysadmin/the-management-engine-an-attack-on-computer-users-freedom)
+over which you are given no control.  Additionally, hardware design
+defects in the processors of both vendors have been discovered, giving
+rise to critical security vulnerabilities like
+[Spectre](https://en.wikipedia.org/wiki/Spectre_(security_vulnerability))
+and
+[Meltdown](https://en.wikipedia.org/wiki/Meltdown_(security_vulnerability)).
+In many cases, these vulnerabilities can only be fixed by installing
+[non-free CPU microcode](https://wiki.debian.org/Microcode) - unless,
+of course, [the vendor decides not to provide any fix at
+all](https://arstechnica.com/gadgets/2018/04/intel-drops-plans-to-develop-spectre-microcode-for-ancient-chips/)!
+
+Compared to that, the RYF Talos II and Talos II Lite are a breath of
+fresh air that the free software community really deserves.  Raptor
+Computing Systems' commitment to software freedom and owner control is
+an inspiring reminder that it **is** possible to ship a great product
+that respects the freedom of your customers.  And going forward, the
+future looks bright for the open, royalty-free Power ISA, [which is
+now a Linux Foundation
+project](https://www.linuxfoundation.org/press-release/2019/08/the-linux-foundation-announces-new-open-hardware-technologies-and-collaboration/)
+(see also: [the same announcement from The OpenPOWER
+Foundation](https://openpowerfoundation.org/the-next-step-in-the-openpower-foundation-journey/).
+
+### About other freedom-friendly platforms
+
+The authors believe that the Power ISA, in particular the RYF Talos II
+from Raptor Computing Systems, is a good, freedom-friendly choice for
+a server or a desktop workstation.  If we didn't, we wouldn't be
+writing this blog post!  However, it is worth noting that other
+freedom-friendly platforms exist, too.  For example, RISC-V is another
+platform that is quite interesting from a freedom standpoint.
+
+In the opinion of these authors, any platform that furthers the cause
+of software freedom is a good platform.  For this reason, we want to
+make it clear that we do not consider freedom-friendly platforms like
+Power ISA or RISC-V (or others) to be in competition with one another.
+All people deserve freedom, thus all people deserve a computing
+platform that respects their freedom.  It doesn't matter if it's Power
+ISA, RISC-V, or something else; if it suits your needs and respects
+your freedom, it's good, and we want more of it!
+
+On that topic, Guix still needs a RISC-V port.  Perhaps you (yes, you,
+the RISC-V fan reading this right now!) could be the one to start the
+porting work for that platform?  The Guix community is very friendly,
+so [come and join us](https://guix.gnu.org/en/contribute/), and let's
+build it together!
+
+### Bootstrapping powerpc64le-linux: A Journey
+
+To build software, you need software.  How can one port Guix to a
+platform before support for that platform exists?  This is a
+[bootstrapping
+problem](https://guix.gnu.org/manual/en/html_node/Bootstrapping.html).
+
+In Guix, all software for a given system (e.g., powerpc64le-linux) is
+built starting from its bootstrap binaries.  It is intended that the
+bootstrap binaries are the only pieces of software in the entire
+package collection that Guix cannot build from source.  In practice,
+[additional bootstrap roots are
+possible](https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00814.html),
+but introducing them in Guix is highly discouraged, and our community
+[actively](https://guix.gnu.org/en/blog/2019/guix-reduces-bootstrap-seed-by-50/)
+[works](https://guix.gnu.org/en/blog/2020/guix-further-reduces-bootstrap-seed-to-25/)
+to [reduce](https://guix.gnu.org/en/blog/2018/bootstrapping-rust/) our
+overall bootstrap footprint.
+
+So first you need to build the the bootstrap binaries for your
+platform.  In theory, you can do this in many ways.  For example, you
+might try to manually compile them on an existing system.  However,
+Guix has [package
+definitions](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/make-bootstrap.scm?id=5d8c2c00d60196c46a32b68c618ccbe2b3aa48f4)
+that you can use to build them - using Guix, of course!
+
+Commonly, the first step in [porting Guix to a new
+platform](https://guix.gnu.org/manual/en/html_node/Porting.html) is to
+use Guix to cross-compile the bootstrap binaries for that new platform
+from a platform on which Guix is already supported. This can be done
+by running a command like the following on a system where Guix is
+already installed:
+
+```scheme
+guix build --target=powerpc64le-linux-gnu bootstrap-tarballs
+```
+
+This is the route that we took when building the powerpc64le-linux
+bootstrap binaries, as described in commit
+[8a1118a](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a1118a96c9ae128302c3d435ae77cb3dd693aea).
+
+Note that before you can even do this, you must first update the
+glibc-dynamic-linker and system->linux-architecture procedures in
+Guix's code, as described in
+[Porting](https://guix.gnu.org/manual/en/html_node/Porting.html).  In
+addition, the versions of packages in Guix that make up the GNU
+toolchain (gcc, glibc, etc.) must already support the target platform.
+This pre-existing toolchain support needs to be good enough so that
+Guix can (1) build, on some already-supported platform, a
+cross-compilation toolchain for that target, (2) use, on that same
+platform, the cross-compilation toolchain to cross-compile the
+bootstrap binaries for the target platform, and (3) use, on the target
+platform, the bootstrap binaries to natively build the rest of the
+Guix package collection.  The above "guix build" command takes care of
+steps (1) and (2) automatically.
+
+Step (3) is a little more involved.  Once the bootstrap binaries for
+the target platform have been built, they must be made available for
+anyone to download.  In addition, Guix's code must be updated so that
+(a) it recognizes the system (e.g., "powerpc64le-linux") that will be
+used to identify the new platform and (b) it fetches the new system's
+bootstrap binaries from the right location.  Commit
+[8a1118a](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a1118a96c9ae128302c3d435ae77cb3dd693aea)
+accomplishes both of those things.  After that, you just have to try
+building things and see what breaks, for example by running
+`./pre-inst-env guix build hello` from your Git checkout to build GNU
+Hello.
+
+The actual bootstrap binaries for powerpc64le-linux are stored on the
+[alpha.gnu.org FTP
+server](https://alpha.gnu.org/gnu/guix/bootstrap/powerpc64le-linux/20210106/).
+Chris Marusich built these bootstrap binaries in an x86_64-linux Guix
+System VM which was running on hardware owned by Léo Le Bouter.  Chris
+then signed the binaries and provided them to Ludovic Courtès, who in
+turn verified their authenticity, signed them, and [uploaded them to
+alpha.gnu.org](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669#125).
+
+With the bootstrap binaries finally in place, we could get started on
+step (3) above: building the rest of the package collection.  Or
+trying to, at least.  There were many stumbling blocks.  For example,
+to resolve some test failures, we had to update the code in Guix that
+allows it to make
+[certain](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=b57de27d0331198c9cafb09a1cf8a5fa4f691e36)
+[syscalls](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c29bfbfc78ccd9e5c10d38faf3d269eafed12854)
+from scheme.  As another example, we had to [patch GCC so that it
+looks for the 64-bit libraries in
+/lib](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46253), rather
+than /lib64, since that is where Guix puts its 64-bit libraries by
+convention.  Some packages required for Guix failed to build, so we
+had to debug those build failures, too.
+
+For a list of all the changes, see [the patch
+series](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47182) or the
+actual commits, which are:
+
+```
+$ git log --oneline --no-decorate 8a1118a96c9ae128302c3d435ae77cb3dd693aea..65c46e79e0495fe4d32f6f2725d7233fff10fd70
+65c46e79e04 gnu: sed: Make it build on SELinux-enabled kernels.
+93f21e1a35e utils: Fix target-64bit? on powerpc64le-linux.
+8d9aece8c49 ci: %cross-targets: Add powerpc64le-linux-gnu.
+c29bfbfc78c syscalls: Fix RNDADDTOENTCNT on powerpc64le-linux.
+b57de27d033 syscalls: Fix clone on powerpc64le-linux.
+a16eb6c5f97 Add powerpc64le-linux as a supported Guix architecture.
+b50f4268035 gnu: libelf: Fix compilation for powerpc64le-linux.
+1a0f4013d33 gnu: texlive-latex-base: Fix compilation on powerpc64le*.
+e9938dc8f0e gnu: texlive-bin: Fix compilation on powerpc64le*.
+69b3907adf6 gnu: guile-avahi: Fix compilation on powerpc64le-linux.
+4cc2d2aa599 gnu: bdb-4.8: Fix configure on powerpc64le-linux.
+be4b1cf53bd gnu: binutils-final: Support more Power architectures.
+060478c32c9 gnu: binutils-final: Provide bash for binary on powerpc-linux.
+b2135b5d576 gnu: gcc-boot0: Enable 128-bit long double for POWER9.
+6e98e9ca923 gnu: glibc: Fix ldd path on powerpc*.
+cac88b28b83 gnu: gcc-4.7: On powerpc64le, fix /lib64 references.
+fc7cf0c1ecc utils: Add target-powerpc? procedure.
+```
+
+In the end, through the combined efforts of multiple people, we slowly
+worked through the issues until we reached a point where we could do
+all of the following things successfully:
+
+- Build Guix manually on a [Debian GNU/Linux
+  ppc64el](https://wiki.debian.org/ppc64el) machine (yet another name
+  for the powerpc64le-linux-gnu triplet), and all of its "make check"
+  tests passed.
+- Build GNU Hello using Guix and run it.
+- Run "guix pull" to build and install the most recent version of
+  Guix, with powerpc64le-linux support.
+- Build a release binary tarball for powerpc64le-linux using "make
+  guix-binary.powerpc64le-linux.tar.xz"
+- Use that binary to install a version of Guix that could build/run
+  GNU Hello and run "guix pull" successfully.
+
+This was an exciting moment!  But there was still more work to be
+done.
+
+Originally, we did this work on the wip-ppc64le branch, with the
+intent of merging it into core-updates.  By convention, the
+"[core-updates](https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html)"
+branch in Guix is where changes are made if they cause too many
+rebuilds.  Since we were updating package definitions so deep in the
+dependency graph of the package collection, we assumed it wouldn't be
+possible to avoid rebuilding the world.  For this reason, we had based
+the wip-ppc64le branch on core-updates.
+
+However, Efraim Flashner proved us wrong!  He created a separate
+branch, wip-ppc64le-for-master, where he adjusted some of the
+wip-ppc64le commits to avoid rebuilding the world on other platforms.
+Thanks to his work, we were able to merge the changes directly to
+master!  This meant that we would be able to include it in the next
+release (Guix v.1.2.1).
+
+In short, the initial porting work is done, and it is now possible for
+anyone to easily try out Guix on this new platform.  Because "guix
+pull" works, too, it is also easy to iterate on what we have and work
+towards improving support for the platform.  It took a lot of
+cooperation and effort to get this far, but there are multiple people
+actively contributing to this port in the Guix community who want to
+see it succeed, and we hope you will join us in exploring the limits
+of this exciting new freedom-friendly platform!
+
+
+### Hiccups Along the Way
+
+Along the way, there were a few problems that stymied our porting
+attempts.
+
+First, we thought we would try to port to powerpc64-linux
+(big-endian).  This did not prove to be any easier than the
+little-endian port.  In addition, other distributions (e.g.,
+[Debian](https://www.debian.org/ports/) and
+[Fedora](https://fedoraproject.org/wiki/Architectures)) have recently
+dropped their big-endian powerpc64 ports, so the little-endian variant
+is more likely to be tested and supported in the community.  So we
+decided to focus our efforts on the little-endian variant, and so far
+we haven't looked back.
+
+In both the big-endian and little-endian case, we were saddened to
+discover that the bootstrap binaries are not entirely reproducible.
+This fact is documented in [bug
+41669](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669), along
+with our extensive investigations.  In short, if you build the
+bootstrap binaries on two separate machines without using any
+substitutes, you will find that the derivation which cross-compiles
+%gcc-static (the bootstrap GCC, version 5.5.0) produces different
+output on the two systems.  However, if you build %gcc-static twice on
+the same system, it builds reproducibly.  This suggests that something
+in the transitive closure of inputs of %gcc-static is perhaps
+contributing to its non-reproducibility.  There is an interesting graph toward the end of the bug report that suggests that the non-reproducibliity is introduced by one or more of 21 derivations in the transitive closure of %gcc-static's inputs that do not build reproducibly across systems.
+
+At some point, you have to cut your losses and move on.  After months
+of investigation without resolving the reproducibility issue, we
+finally decided to move forward with the bootstrap binaries produced
+earlier.  If necessary, we can always go back and try to fix this
+issue, but it seemed more important to get started with the
+bootstrapping work.  Anyone who is interested in solving this problem
+can comment on the bug report and help us figure out the mystery.  We
+are very interested in solving it, but at the moment we are more
+interested in bootstrapping the rest of the system with the existing
+bootstrap binaries.
+
+### Next Steps
+
+It is now possible to install Guix on a powerpc64le-linux system and
+use it to build some useful software - in particular, Guix itself.  So
+Guix is now "self-hosted" on this platform, which gives us a
+comfortable place to begin further work.
+
+The following tasks still need to be done.  Anyone can help, so please
+get in touch if you want to contribute!
+
+- Solve the GCC bootstrap binary reproducibility issue.
+- Get Guix System to work on powerpc64le-linux.
+- Get CI infrastructure to work (Cuirass, guix-build-coordinator,
+  substitutes, etc.)
+- Fix all package build failures.
+- Try building rust, and if it works, judiciously re-introduce the
+  librsvg dependency for powerpc64le-linux in gtk+ and gtk+-2, since
+  [it is currently
+  missing](https://git.savannah.gnu.org/cgit/guix.git).
+- Upgrade GCC to 8 on core-updates, fix issues that occur.
+- Merge core-updates to master after that.
-- 
2.30.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Please review blog post draft: powerpc64le-linux support

[Joshua Branson]

Awesome!  Great work!  I read the below draft blog post like a Harry
Potter novel!  It is superbly written.  And it makes a lot of sense!

Chris Marusich <cmmarusich@gmail.com> writes:

> Hi,
>
> Léo and I have drafted the following blog post.  Could you take a few
> minutes to read it and give us your thoughts?
>
> It's a work in progress.  The primary goal is to announce the new
> powerpc64le-linux support and explain why it matters (POWER9 is an
> exciting, freedom-friendly platform!).  The secondary goal is to explain
> some of the details about what we did, and invite people to get
> involved.
>
> Your feedback would be highly appreciated!
>
> --
> Chris

Seriously good job on this blog post, and all involved in the powerPC
porting work!  Fabulous job chaps!  Cheerio!  Also below are my tiny
edits:

> +
> +### Why Is This Important?
> +
> +This is important because it means that GNU Guix now works on the [RYF
> +Talos II and Talos II Lite
> +mainboards](https://www.fsf.org/news/talos-ii-mainboard-and-talos-ii-lite-mainboard-now-fsf-certified-to-respect-your-freedom)
> +and it's IBM POWER9 processor.  This is a modern, performant hardware

I believe you should use "its".  it's is short for "it is".

> +platform that respects your freedom.  It can run without any non-free
> +code, all the way down to its bootloader and firmware.  It's a
> +freedom-friendly platform that aligns well with GNU Guix's commitment
> +to software freedom.
> +
> +How is this any different from existing RYF hardware, you might ask?
> +The existing RYF
> +[laptops](https://ryf.fsf.org/products?category=1&vendor=All&sort_by=created&sort_order=DESC),
> +[mainboards](https://ryf.fsf.org/products?category=5&vendor=All&sort_by=created&sort_order=DESC),
> +and
> +[workstations](https://ryf.fsf.org/products?category=30&vendor=All&sort_by=created&sort_order=DESC)
> +can only really be used with Intel Core Duo or AMD Opteron processors.
> +Those processors were released over 15 years ago.  Since then,
> +processor performance has increased drastically.  People should not
> +have to choose between performance and freedom, but the fact is that
> +for many years, that is exactly what we were forced to do.  However,
> +the Talos II and Talos II Lite have changed this: the free software
> +community now has an RYF-certified option that can compete with the
> +performance of modern Intel and AMD systems.
> +
> +Although the performance of POWER9 processors is competitive with
> +modern Intel and AMD processors, its real advantage is that it<

Why is there "it<"  ?  Is that some markup I'm not familiar with?

> +respects your freedom.  Modern processors from [both Intel and AMD
> +include back
> +doors](https://www.fsf.org/blogs/sysadmin/the-management-engine-an-attack-on-computer-users-freedom)
> +over which you are given no control.  Additionally, hardware design
> +defects in the processors of both vendors have been discovered, giving
> +rise to critical security vulnerabilities like
> +[Spectre](https://en.wikipedia.org/wiki/Spectre_(security_vulnerability))
> +and
> +[Meltdown](https://en.wikipedia.org/wiki/Meltdown_(security_vulnerability)).
> +In many cases, these vulnerabilities can only be fixed by installing
> +[non-free CPU microcode](https://wiki.debian.org/Microcode) - unless,
> +of course, [the vendor decides not to provide any fix at
> +all](https://arstechnica.com/gadgets/2018/04/intel-drops-plans-to-develop-spectre-microcode-for-ancient-chips/)!
> +
> +Compared to that, the RYF Talos II and Talos II Lite are a breath of
> +fresh air that the free software community really deserves.  Raptor
> +Computing Systems' commitment to software freedom and owner control is
> +an inspiring reminder that it **is** possible to ship a great product
> +that respects the freedom of your customers.  And going forward, the
> +future looks bright for the open, royalty-free Power ISA, [which is
> +now a Linux Foundation
> +project](https://www.linuxfoundation.org/press-release/2019/08/the-linux-foundation-announces-new-open-hardware-technologies-and-collaboration/)
> +(see also: [the same announcement from The OpenPOWER
> +Foundation](https://openpowerfoundation.org/the-next-step-in-the-openpower-foundation-journey/).
> +
> +
> +In Guix, all software for a given system (e.g., powerpc64le-linux) is
> +built starting from its bootstrap binaries.  It is intended that the
> +bootstrap binaries are the only pieces of software in the entire
> +package collection that Guix cannot build from source.  In practice,
> +[additional bootstrap roots are
> +possible](https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00814.html),
> +but introducing them in Guix is highly discouraged, and our community
> +[actively](https://guix.gnu.org/en/blog/2019/guix-reduces-bootstrap-seed-by-50/)
> +[works](https://guix.gnu.org/en/blog/2020/guix-further-reduces-bootstrap-seed-to-25/)
> +to [reduce](https://guix.gnu.org/en/blog/2018/bootstrapping-rust/) our
> +overall bootstrap footprint.
> +
> +So first you need to build the the bootstrap binaries for your

"the the" --> "the"

> +platform.  In theory, you can do this in many ways.  For example, you
> +might try to manually compile them on an existing system.  However,
> +Guix has [package
> +definitions](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/make-bootstrap.scm?id=5d8c2c00d60196c46a32b68c618ccbe2b3aa48f4)
> +that you can use to build them - using Guix, of course!
>
> +In both the big-endian and little-endian case, we were saddened to
> +discover that the bootstrap binaries are not entirely reproducible.
> +This fact is documented in [bug
> +41669](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669), along
> +with our extensive investigations.  In short, if you build the
> +bootstrap binaries on two separate machines without using any
> +substitutes, you will find that the derivation which cross-compiles
> +%gcc-static (the bootstrap GCC, version 5.5.0) produces different
> +output on the two systems.  However, if you build %gcc-static twice on
> +the same system, it builds reproducibly.  This suggests that something
> +in the transitive closure of inputs of %gcc-static is perhaps
> +contributing to its non-reproducibility.  There is an interesting graph toward the end of the bug report that suggests that the non-reproducibliity is introduced by one or more of 21 derivations in the transitive closure of %gcc-static's inputs that do not build reproducibly across systems.

On my computer, the above line is SUPER LONG.  Should you do a M-x
fill-paragraph on the above line?  Or does that not really matter?

> +### Next Steps
> +
> +It is now possible to install Guix on a powerpc64le-linux system and
> +use it to build some useful software - in particular, Guix itself.  So
> +Guix is now "self-hosted" on this platform, which gives us a
> +comfortable place to begin further work.
> +
> +The following tasks still need to be done.  Anyone can help, so please
> +get in touch if you want to contribute!
> +
> +- Solve the GCC bootstrap binary reproducibility issue.
> +- Get Guix System to work on powerpc64le-linux.
> +- Get CI infrastructure to work (Cuirass, guix-build-coordinator,
> +  substitutes, etc.)
> +- Fix all package build failures.
> +- Try building rust, and if it works, judiciously re-introduce the
> +  librsvg dependency for powerpc64le-linux in gtk+ and gtk+-2, since
> +  [it is currently
> +  missing](https://git.savannah.gnu.org/cgit/guix.git).

Just out of curiosity, does rust work on power?  I'm under the
impression that rust, while memory safe, is not so easy to bootstrap.
Though I am eager to see someone bootstrap rust to power!  Best of luck
you brave soul!

> +- Upgrade GCC to 8 on core-updates, fix issues that occur.
> +- Merge core-updates to master after that.

--
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar

Re: Please review blog post draft: powerpc64le-linux support

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 1251 bytes --]

On Tue, 2021-04-06 at 00:15 -0700, Chris Marusich wrote:
> Hi,
> 
> Léo and I have drafted the following blog post.  Could you take a few
> minutes to read it and give us your thoughts?
> 
> It's a work in progress.  The primary goal is to announce the new
> powerpc64le-linux support and explain why it matters (POWER9 is an
> exciting, freedom-friendly platform!).  The secondary goal is to
> explain
> some of the details about what we did, and invite people to get
> involved.
> 
> Your feedback would be highly appreciated!
> 

It's been mostly you here Chris, thank you so much for writing it, as
others said, it is really beautifully written! Unfortunately I havent
felt enough peace of mind to write like you did :-(

I would've liked to write about the early days where I met some
problems with the core-updates branch having to rebase several times
and learning GNU Guix at the same time since my first ever project
related to GNU Guix was porting before even trying to use it elsewhere.
Having to learn the GNU commit message guidelines, then learning git-
send-email and GNU Emacs (since that's where all dev tools are), all
that to contribute to GNU Guix and get this port in. Aaaahh very long
story!

Léo

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 8352 bytes --]

Hi Joshua,

Joshua Branson <jbranso@dismail.de> writes:

> Awesome!  Great work!  I read the below draft blog post like a Harry
> Potter novel!  It is superbly written.  And it makes a lot of sense!

Thank you for the kind words, and your feedback!

>> +### Why Is This Important?
>> +
>> +This is important because it means that GNU Guix now works on the [RYF
>> +Talos II and Talos II Lite
>> +mainboards](https://www.fsf.org/news/talos-ii-mainboard-and-talos-ii-lite-mainboard-now-fsf-certified-to-respect-your-freedom)
>> +and it's IBM POWER9 processor.  This is a modern, performant hardware
>
> I believe you should use "its".  it's is short for "it is".

Good catch!

>> +platform that respects your freedom.  It can run without any non-free
>> +code, all the way down to its bootloader and firmware.  It's a
>> +freedom-friendly platform that aligns well with GNU Guix's commitment
>> +to software freedom.
>> +
>> +How is this any different from existing RYF hardware, you might ask?
>> +The existing RYF
>> +[laptops](https://ryf.fsf.org/products?category=1&vendor=All&sort_by=created&sort_order=DESC),
>> +[mainboards](https://ryf.fsf.org/products?category=5&vendor=All&sort_by=created&sort_order=DESC),
>> +and
>> +[workstations](https://ryf.fsf.org/products?category=30&vendor=All&sort_by=created&sort_order=DESC)
>> +can only really be used with Intel Core Duo or AMD Opteron processors.
>> +Those processors were released over 15 years ago.  Since then,
>> +processor performance has increased drastically.  People should not
>> +have to choose between performance and freedom, but the fact is that
>> +for many years, that is exactly what we were forced to do.  However,
>> +the Talos II and Talos II Lite have changed this: the free software
>> +community now has an RYF-certified option that can compete with the
>> +performance of modern Intel and AMD systems.
>> +
>> +Although the performance of POWER9 processors is competitive with
>> +modern Intel and AMD processors, its real advantage is that it<
>
> Why is there "it<"  ?  Is that some markup I'm not familiar with?

Nope, it's a typo.  Good eyes!

>> +respects your freedom.  Modern processors from [both Intel and AMD
>> +include back
>> +doors](https://www.fsf.org/blogs/sysadmin/the-management-engine-an-attack-on-computer-users-freedom)
>> +over which you are given no control.  Additionally, hardware design
>> +defects in the processors of both vendors have been discovered, giving
>> +rise to critical security vulnerabilities like
>> +[Spectre](https://en.wikipedia.org/wiki/Spectre_(security_vulnerability))
>> +and
>> +[Meltdown](https://en.wikipedia.org/wiki/Meltdown_(security_vulnerability)).
>> +In many cases, these vulnerabilities can only be fixed by installing
>> +[non-free CPU microcode](https://wiki.debian.org/Microcode) - unless,
>> +of course, [the vendor decides not to provide any fix at
>> +all](https://arstechnica.com/gadgets/2018/04/intel-drops-plans-to-develop-spectre-microcode-for-ancient-chips/)!
>> +
>> +Compared to that, the RYF Talos II and Talos II Lite are a breath of
>> +fresh air that the free software community really deserves.  Raptor
>> +Computing Systems' commitment to software freedom and owner control is
>> +an inspiring reminder that it **is** possible to ship a great product
>> +that respects the freedom of your customers.  And going forward, the
>> +future looks bright for the open, royalty-free Power ISA, [which is
>> +now a Linux Foundation
>> +project](https://www.linuxfoundation.org/press-release/2019/08/the-linux-foundation-announces-new-open-hardware-technologies-and-collaboration/)
>> +(see also: [the same announcement from The OpenPOWER
>> +Foundation](https://openpowerfoundation.org/the-next-step-in-the-openpower-foundation-journey/).
>> +
>> +
>> +In Guix, all software for a given system (e.g., powerpc64le-linux) is
>> +built starting from its bootstrap binaries.  It is intended that the
>> +bootstrap binaries are the only pieces of software in the entire
>> +package collection that Guix cannot build from source.  In practice,
>> +[additional bootstrap roots are
>> +possible](https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00814.html),
>> +but introducing them in Guix is highly discouraged, and our community
>> +[actively](https://guix.gnu.org/en/blog/2019/guix-reduces-bootstrap-seed-by-50/)
>> +[works](https://guix.gnu.org/en/blog/2020/guix-further-reduces-bootstrap-seed-to-25/)
>> +to [reduce](https://guix.gnu.org/en/blog/2018/bootstrapping-rust/) our
>> +overall bootstrap footprint.
>> +
>> +So first you need to build the the bootstrap binaries for your
>
> "the the" --> "the"

Fixed!

>> +platform.  In theory, you can do this in many ways.  For example, you
>> +might try to manually compile them on an existing system.  However,
>> +Guix has [package
>> +definitions](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/make-bootstrap.scm?id=5d8c2c00d60196c46a32b68c618ccbe2b3aa48f4)
>> +that you can use to build them - using Guix, of course!
>>
>> +In both the big-endian and little-endian case, we were saddened to
>> +discover that the bootstrap binaries are not entirely reproducible.
>> +This fact is documented in [bug
>> +41669](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669), along
>> +with our extensive investigations.  In short, if you build the
>> +bootstrap binaries on two separate machines without using any
>> +substitutes, you will find that the derivation which cross-compiles
>> +%gcc-static (the bootstrap GCC, version 5.5.0) produces different
>> +output on the two systems.  However, if you build %gcc-static twice on
>> +the same system, it builds reproducibly.  This suggests that something
>> +in the transitive closure of inputs of %gcc-static is perhaps
>> +contributing to its non-reproducibility.  There is an interesting
>> graph toward the end of the bug report that suggests that the
>> non-reproducibliity is introduced by one or more of 21 derivations
>> in the transitive closure of %gcc-static's inputs that do not build
>> reproducibly across systems.
>
> On my computer, the above line is SUPER LONG.  Should you do a M-x
> fill-paragraph on the above line?  Or does that not really matter?

Yeah, I forgot to do that on this line.  It's wrapped now.

>> +### Next Steps
>> +
>> +It is now possible to install Guix on a powerpc64le-linux system and
>> +use it to build some useful software - in particular, Guix itself.  So
>> +Guix is now "self-hosted" on this platform, which gives us a
>> +comfortable place to begin further work.
>> +
>> +The following tasks still need to be done.  Anyone can help, so please
>> +get in touch if you want to contribute!
>> +
>> +- Solve the GCC bootstrap binary reproducibility issue.
>> +- Get Guix System to work on powerpc64le-linux.
>> +- Get CI infrastructure to work (Cuirass, guix-build-coordinator,
>> +  substitutes, etc.)
>> +- Fix all package build failures.
>> +- Try building rust, and if it works, judiciously re-introduce the
>> +  librsvg dependency for powerpc64le-linux in gtk+ and gtk+-2, since
>> +  [it is currently
>> +  missing](https://git.savannah.gnu.org/cgit/guix.git).
>
> Just out of curiosity, does rust work on power?  I'm under the
> impression that rust, while memory safe, is not so easy to bootstrap.
> Though I am eager to see someone bootstrap rust to power!  Best of luck
> you brave soul!

Yes, Rust should work.  There's a Debian package on their ppc64el port
(the package is rustc), and the GNU triplet
"powerpc64le-unknown-linux-gnu" (which is the triplet corresponding to
the Guix system "powerpc64le-linux") is listed as a Tier 2 supported
platform in the Rust docs:

https://doc.rust-lang.org/nightly/rustc/platform-support.html

Whether or not it does work is another question.  I'll be surprised if
it builds in Guix on powerpc64le-linux the first time we try, but we
just have to try it and fix any problems that occur.

I'll build and try using it eventually, but others are more than welcome
to beat me to the punch!  I've been focusing on finishing this blog post
in my spare time first.

Again, thank you for the feedback!  I really appreciate it.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2478 bytes --]

Hi Léo,

Léo Le Bouter <lle-bout@zaclys.net> writes:

> It's been mostly you here Chris, thank you so much for writing it, as
> others said, it is really beautifully written! Unfortunately I havent
> felt enough peace of mind to write like you did :-(

You've been busy!  It's totally understandable.  The encouragement from
you and others has been very useful and motivating for me, so thank you.

> I would've liked to write about the early days where I met some
> problems with the core-updates branch having to rebase several times
> and learning GNU Guix at the same time since my first ever project
> related to GNU Guix was porting before even trying to use it elsewhere.
> Having to learn the GNU commit message guidelines, then learning git-
> send-email and GNU Emacs (since that's where all dev tools are), all
> that to contribute to GNU Guix and get this port in. Aaaahh very long
> story!

I agree.  Those are interesting topics.  I tried to include some
discussion about it, but the post became too lengthy.  I just want it to
be about the new support, mainly, and why it's exciting.

I think that the following related topics would make good candidates for
future blog posts:

- An analysis of trust in Guix, with an eye towards bootstrapping.  If
  you use substitutes, what are you implicitly trusting?  If you build
  without substitutes, what are you implicitly trusting?  If you build
  Guix from source without using Guix, like you have to do when you
  first port Guix to a new platform, what are you trusting?  A
  comparison of similar paths of trust when using other software.  Make
  a script to find out if there are any forgotten "bootstrap roots"
  beyond the bootstrap binaries, like there apparently used to be for
  some self-hosted compilers (see:
  https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00814.html).
  Stuff like that.  I think it is not obvious.

- An analysis of the hurdles / friction involved in contributing.
  Preferably with suggestions for ways to remove the hurdles and reduce
  friction.  It is easy to complain or bikeshed, of course, but the
  point is not to do that.  The point is to discuss issues to try and
  make things better.

Thank you again for your help!  This is just the beginning - let's keep
hacking away at it and improving POWER9 support together!  Hopefully
others will see the benefits of the platform and join us along the way.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

[-- Attachment #1.1: Type: text/plain, Size: 1234 bytes --]

Hi,

Here's a new version of the blog post.  It incorporates all the feedback
so far.  I've also removed the "About other freedom-friendly platforms"
because it didn't seem to add much substance.

I significantly rewrote the "Why Is This Important?" section, mainly
because I realized that I was incorrectly and unfairly implying that
POWER9 CPUs are not vulnerable to Spectre/Meltdown-style
vulnerabilities.  In fact, some POWER9 CPUs were found to be vulnerable,
but the most recent models have been fixed.  I've rewritten this section
so that it focuses more on explaining why the RYF Talos II and Talos II
Lite are "more free" than the popular Intel and AMD mainstays (even the
older, RYF-certified models, where you still have to jump over the
hurdle of removing the Intel ME or equivalent.)

For details on Spectre/Meltdown on POWER9, see:

https://wiki.raptorcs.com/wiki/Speculative_Execution_Vulnerabilities_of_2018

I added a footer describing GNU Guix, as is customary on most of our
blog posts.

I changed the title.

I also fixed various links and rephrased a few things.

Anyway, if you can cast your eye over it once more, I would appreciate
it.  I think it's just about done!

-- 
Chris

[-- Attachment #1.2: 0001-website-drafts-Add-powerpc64le-linux-announcement.patch --]
[-- Type: text/x-patch, Size: 22627 bytes --]

From 4d9133e51fc666f14074c1da18bb16af0d76066f Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Tue, 6 Apr 2021 00:10:35 -0700
Subject: [PATCH] website: drafts: Add powerpc64le-linux announcement.

* website/drafts/new-system-powerpc64le-linux.md: New file.
---
 .../drafts/new-system-powerpc64le-linux.md    | 389 ++++++++++++++++++
 1 file changed, 389 insertions(+)
 create mode 100644 website/drafts/new-system-powerpc64le-linux.md

diff --git a/website/drafts/new-system-powerpc64le-linux.md b/website/drafts/new-system-powerpc64le-linux.md
new file mode 100644
index 0000000..d2104aa
--- /dev/null
+++ b/website/drafts/new-system-powerpc64le-linux.md
@@ -0,0 +1,389 @@
+title: New Supported Platform: powerpc64le-linux
+date: 2021-04-08 00:00
+author: Chris Marusich and Léo Le Bouter
+tags: porting, powerpc64le, bootstrapping, cross-compilation, reproducibility
+---
+
+It is a pleasure to announce that support for powerpc64le-linux
+(PowerISA v.2.07 and later) has now been
+[merged](https://issues.guix.gnu.org/47182) to the master branch of
+GNU Guix!
+
+This means that GNU Guix can be used immediately on this platform from
+a [from a Git
+checkout](https://guix.gnu.org/manual/en/html_node/Building-from-Git.html).
+Starting with the next release (Guix v1.2.1), you will also be able to
+[download a copy of Guix pre-built for
+powerpc64le-linux](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html#Binary-Installation).
+Regardless of how you get it, you can run the new powerpc64le-linux
+port of GNU Guix on top of any existing powerpc64le GNU/Linux
+distribution.
+
+This new platform is available as a "technology preview".  This means
+that although it is supported,
+[substitutes](https://guix.gnu.org/manual/en/html_node/Substitutes.html)
+are not yet available from the build farm, and some packages may fail
+to build.  Although powerpc64le-linux support is nascent, the Guix
+community is actively working on improving it, and this is a great
+time to [get
+involved](https://guix.gnu.org/manual/en/html_node/Contributing.html)!
+
+### Why Is This Important?
+
+This is important because it means that GNU Guix now works on the
+[Talos II and Talos II Lite
+mainboards](https://www.fsf.org/news/talos-ii-mainboard-and-talos-ii-lite-mainboard-now-fsf-certified-to-respect-your-freedom),
+which use [IBM POWER9
+processors](https://wiki.raptorcs.com/wiki/POWER9).  This is a modern,
+performant hardware platform that has recently received [Respects Your
+Freedom (RYF) certification](https://ryf.fsf.org/) from the FSF.  It
+can run without any non-free code, all the way down to its bootloader
+and firmware.  In other words, it's a freedom-friendly platform that
+aligns well with GNU Guix's commitment to software freedom.
+
+How is this any different from existing RYF hardware, you might ask?
+One reason is performance.  The existing RYF
+[laptops](https://ryf.fsf.org/products?category=1&vendor=All&sort_by=created&sort_order=DESC),
+[mainboards](https://ryf.fsf.org/products?category=5&vendor=All&sort_by=created&sort_order=DESC),
+and
+[workstations](https://ryf.fsf.org/products?category=30&vendor=All&sort_by=created&sort_order=DESC)
+can only really be used with Intel Core Duo or AMD Opteron processors.
+Those processors were released over 15 years ago.  Since then,
+processor performance has increased drastically.  People should not
+have to choose between performance and freedom, but for many years
+that is exactly what we were forced to do.  However, the Talos II and
+Talos II Lite have changed this: the free software community now has
+an RYF-certified option that [can compete with the performance of
+modern Intel and AMD
+systems](https://www.phoronix.com/scan.php?page=article&item=power9-threadripper-core9&num=1).
+
+Although the performance of POWER9 processors is competitive with
+modern Intel and AMD processors, the real advantage of the Talos II
+and Talos II Lite is that they were designed from the start to respect
+your freedom.  Modern processors from [both Intel and AMD include back
+doors](https://www.fsf.org/blogs/sysadmin/the-management-engine-an-attack-on-computer-users-freedom)
+over which you are given no control.  Even though the back doors can
+be removed [in some cases with significant
+effort](https://www.fsf.org/news/libreboot-x200-laptop-now-fsf-certified-to-respect-your-freedom),
+this is an obstacle that nobody should have to overcome just to
+control their own computer.  No such obstacles exist when using the
+Talos II or Talos II Lite.  In fact, although
+[Intel](https://arstechnica.com/gadgets/2020/10/in-a-first-researchers-extract-secret-key-used-to-encrypt-intel-cpu-code/)
+and
+[AMD](https://www.extremetech.com/computing/292722-amds-secure-processor-firmware-is-now-explorable-thanks-to-new-tool)
+both go out of their way to keep you from understanding what is going
+on in your own computer, Raptor Computing Systems releases [all of the
+software and firmware used on the
+board](https://git.raptorcs.com/git/) as free software.  They even
+include circuit diagrams when they ship you the machine.
+
+Compared to the existing options, the RYF Talos II and Talos II Lite
+are a breath of fresh air that the free software community really
+deserves.  Raptor Computing Systems' commitment to software freedom
+and owner control is an inspiring reminder that it **is** possible to
+ship a great product that respects the freedom of your customers.  And
+going forward, the future looks bright for the open, royalty-free
+Power ISA, [which is now a Linux Foundation
+project](https://www.linuxfoundation.org/press-release/2019/08/the-linux-foundation-announces-new-open-hardware-technologies-and-collaboration/)
+(see also: [the same announcement from The OpenPOWER
+Foundation](https://openpowerfoundation.org/the-next-step-in-the-openpower-foundation-journey/).
+
+### Bootstrapping powerpc64le-linux: A Journey
+
+To build software, you need software.  How can one port Guix to a
+platform before support for that platform exists?  This is a
+[bootstrapping
+problem](https://guix.gnu.org/manual/en/html_node/Bootstrapping.html).
+
+In Guix, all software for a given platform (e.g., powerpc64le-linux)
+is built starting from a small set of "bootstrap binaries".  These are
+binaries of Guile, GCC, Binutils, libc, and a few other packages,
+pre-built for the relevant platform.  It is intended that the
+bootstrap binaries are the only pieces of software in the entire
+package collection that Guix cannot build from source.  In practice,
+[additional bootstrap roots are
+possible](https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00814.html),
+but introducing them in Guix is highly discouraged, and our community
+[actively](https://guix.gnu.org/en/blog/2019/guix-reduces-bootstrap-seed-by-50/)
+[works](https://guix.gnu.org/en/blog/2020/guix-further-reduces-bootstrap-seed-to-25/)
+to [reduce](https://guix.gnu.org/en/blog/2018/bootstrapping-rust/) our
+overall bootstrap footprint.  There is one set of bootstrap binaries
+for each platform that Guix supports.
+
+This means that to port Guix to a new platform, you must first build
+the bootstrap binaries for that platform.  In theory, you can do this
+in many ways.  For example, you might try to manually compile them on
+an existing system.  However, Guix has [package
+definitions](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/make-bootstrap.scm?id=5d8c2c00d60196c46a32b68c618ccbe2b3aa48f4)
+that you can use to build them - using Guix, of course!
+
+Commonly, the first step in [porting Guix to a new
+platform](https://guix.gnu.org/manual/en/html_node/Porting.html) is to
+use Guix to cross-compile the bootstrap binaries for that new platform
+from a platform on which Guix is already supported. This can be done
+by running a command like the following on a system where Guix is
+already installed:
+
+```scheme
+guix build --target=powerpc64le-linux-gnu bootstrap-tarballs
+```
+
+This is the route that we took when building the powerpc64le-linux
+bootstrap binaries, as described in commit
+[8a1118a](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a1118a96c9ae128302c3d435ae77cb3dd693aea).
+You might wonder why the target above is "powerpc64le-linux-gnu" even
+though the new Guix platform is called "powerpc64le-linux".  This is
+because "powerpc64le-linux-gnu" is a GNU
+[triplet](https://wiki.osdev.org/Target_Triplet) identifying the new
+platform, but "powerpc64le-linux" is the name of a "system" (i.e., a
+platform) in Guix.  Guix contains code that converts between the two
+as needed (see `nix-system->gnu-triplet` and `gnu-triplet->nix-system`
+in
+[`guix/utils.scm`](https://git.savannah.gnu.org/cgit/guix.git/tree/guix/utils.scm?id=83991a34d5c1d4985e54dd029a81412277ad062a).
+When cross-compiling, you only need to specify the GNU triplet.
+
+Note that before you can even do this, you must first update the
+`glibc-dynamic-linker` and `system->linux-architecture` procedures in
+Guix's code, as described in
+[Porting](https://guix.gnu.org/manual/en/html_node/Porting.html).  In
+addition, the versions of packages in Guix that make up the GNU
+toolchain (gcc, glibc, etc.) must already support the target platform.
+This pre-existing toolchain support needs to be good enough so that
+Guix can (1) build, on some already-supported platform, a
+cross-compilation toolchain for the target platform, (2) use, on the
+already-supported platform, the cross-compilation toolchain to
+cross-compile the bootstrap binaries for the target platform, and (3)
+use, on the target platform, the bootstrap binaries to natively build
+the rest of the Guix package collection.  The above [`guix
+build`](https://guix.gnu.org/manual/en/html_node/Invoking-guix-build.html#Invoking-guix-build)
+command takes care of steps (1) and (2) automatically.
+
+Step (3) is a little more involved.  Once the bootstrap binaries for
+the target platform have been built, they must be published online for
+anyone to download.  After that, Guix's code must be updated so that
+(a) it recognizes the "system" name (e.g., "powerpc64le-linux") that
+will be used to identify the new platform and (b) it fetches the new
+platform's bootstrap binaries from the right location.  After all that
+is done, you just have to try building things and see what breaks.
+For example, you can run `./pre-inst-env guix build hello` from your
+Git checkout to try building GNU Hello.
+
+The actual bootstrap binaries for powerpc64le-linux are stored on the
+[alpha.gnu.org FTP
+server](https://alpha.gnu.org/gnu/guix/bootstrap/powerpc64le-linux/20210106/).
+Chris Marusich built these bootstrap binaries in an x86_64-linux Guix
+System VM which was running on hardware owned by Léo Le Bouter.  Chris
+then signed the binaries and provided them to Ludovic Courtès, who in
+turn verified their authenticity, signed them, and [uploaded them to
+alpha.gnu.org](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669#125).
+After that, we updated the code to use the newly published bootstrap
+binaries in commit
+[8a1118a](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a1118a96c9ae128302c3d435ae77cb3dd693aea).
+Once all that was done, we could begin bootstrapping the rest of the
+system - or trying to, at least.
+
+There were many stumbling blocks.  For example, to resolve some test
+failures, we had to update the code in Guix that enables it to make
+[certain](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=b57de27d0331198c9cafb09a1cf8a5fa4f691e36)
+[syscalls](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c29bfbfc78ccd9e5c10d38faf3d269eafed12854)
+from scheme.  In another example, we had to [patch GCC so that it
+looks for the 64-bit libraries in
+/lib](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46253), rather
+than /lib64, since that is where Guix puts its 64-bit libraries by
+convention.  In addition, some packages required in order to build
+Guix failed to build, so we had to debug those build failures, too.
+
+For a list of all the changes, see [the patch
+series](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47182) or the
+actual commits, which are:
+
+```
+$ git log --oneline --no-decorate 8a1118a96c9ae128302c3d435ae77cb3dd693aea..65c46e79e0495fe4d32f6f2725d7233fff10fd70
+65c46e79e04 gnu: sed: Make it build on SELinux-enabled kernels.
+93f21e1a35e utils: Fix target-64bit? on powerpc64le-linux.
+8d9aece8c49 ci: %cross-targets: Add powerpc64le-linux-gnu.
+c29bfbfc78c syscalls: Fix RNDADDTOENTCNT on powerpc64le-linux.
+b57de27d033 syscalls: Fix clone on powerpc64le-linux.
+a16eb6c5f97 Add powerpc64le-linux as a supported Guix architecture.
+b50f4268035 gnu: libelf: Fix compilation for powerpc64le-linux.
+1a0f4013d33 gnu: texlive-latex-base: Fix compilation on powerpc64le*.
+e9938dc8f0e gnu: texlive-bin: Fix compilation on powerpc64le*.
+69b3907adf6 gnu: guile-avahi: Fix compilation on powerpc64le-linux.
+4cc2d2aa599 gnu: bdb-4.8: Fix configure on powerpc64le-linux.
+be4b1cf53bd gnu: binutils-final: Support more Power architectures.
+060478c32c9 gnu: binutils-final: Provide bash for binary on powerpc-linux.
+b2135b5d576 gnu: gcc-boot0: Enable 128-bit long double for POWER9.
+6e98e9ca923 gnu: glibc: Fix ldd path on powerpc*.
+cac88b28b83 gnu: gcc-4.7: On powerpc64le, fix /lib64 references.
+fc7cf0c1ecc utils: Add target-powerpc? procedure.
+```
+
+In the end, through the combined efforts of multiple people, we slowly
+worked through the issues until we reached a point where we could do
+all of the following things successfully:
+
+- Build Guix manually on a [Debian GNU/Linux
+  ppc64el](https://wiki.debian.org/ppc64el) machine (this is Debian's
+  name for a system using the powerpc64le-linux-gnu triplet), and
+  verify that its `make check` tests passed.
+- Build GNU Hello using Guix and run it.
+- Run [`guix
+  pull`](https://guix.gnu.org/manual/en/html_node/Invoking-guix-pull.html#Invoking-guix-pull)
+  to build and install the most recent version of Guix, with
+  powerpc64le-linux support.
+- Build a release binary tarball for powerpc64le-linux via: `make
+  guix-binary.powerpc64le-linux.tar.xz`
+- Use that binary to install a version of Guix that could build/run
+  GNU Hello and run `guix pull` successfully.
+
+This was an exciting moment!  But there was still more work to be
+done.
+
+Originally, we did this work on the
+[wip-ppc64le](https://git.savannah.gnu.org/cgit/guix.git/log/?h=wip-ppc64le)
+branch, with the intent of merging it into core-updates.  By
+convention, the "core-updates" branch in Guix is [where changes are
+made if they cause too many
+rebuilds](https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html).
+Since we were updating package definitions so deep in the dependency
+graph of the package collection, we assumed it wouldn't be possible to
+avoid rebuilding the world.  For this reason, we had based the
+wip-ppc64le branch on core-updates.
+
+However, Efraim Flashner proved us wrong!  He created a separate
+branch, wip-ppc64le-for-master, where he adjusted some of the
+wip-ppc64le commits to avoid rebuilding the world on other platforms.
+Thanks to his work, we were able to merge the changes directly to
+master!  This meant that we would be able to include it in the next
+release (Guix v.1.2.1).
+
+In short, the initial porting work is done, and it is now possible for
+anyone to easily try out Guix on this new platform.  Because `guix
+pull` works, too, it is also easy to iterate on what we have and work
+towards improving support for the platform.  It took a lot of
+cooperation and effort to get this far, but there are multiple people
+actively contributing to this port in the Guix community who want to
+see it succeed.  We hope you will join us in exploring the limits of
+this exciting new freedom-friendly platform!
+
+### Other Porting Challenges
+
+Very early in the porting process, there were some other problems that
+stymied our work.
+
+First, we actually thought we would try to port to powerpc64-linux
+(big-endian).  However, this did not prove to be any easier than the
+little-endian port.  In addition, other distributions (e.g.,
+[Debian](https://www.debian.org/ports/) and
+[Fedora](https://fedoraproject.org/wiki/Architectures)) have recently
+dropped their big-endian powerpc64 ports, so the little-endian variant
+is more likely to be tested and supported in the community.  For these
+reasons, we decided to focus our efforts on the little-endian variant,
+and so far we haven't looked back.
+
+In both the big-endian and little-endian case, we were saddened to
+discover that the bootstrap binaries are not entirely reproducible.
+This fact is documented in [bug
+41669](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669), along
+with our extensive investigations.
+
+In short, if you build the bootstrap binaries on two separate machines
+without using any substitutes, you will find that the derivation which
+cross-compiles %gcc-static (the bootstrap GCC, version 5.5.0) produces
+different output on the two systems.  However, if you build
+%gcc-static twice on the same system, it builds reproducibly.  This
+suggests that something in the transitive closure of inputs of
+%gcc-static is perhaps contributing to its non-reproducibility.  There
+is an interesting graph [toward the end of the bug
+report](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669#137),
+shown below:
+
+![Differing
+Derivations](https://debbugs.gnu.org/cgi/bugreport.cgi?att=1;msg=137;filename=nonreproducible-drvs-small.png;bug=41669)
+
+This graph shows the derivations that produce differing outputs across
+two Guix System machines, when everything is built without
+substitutes.  It starts from the derivation that cross-compiles
+%gcc-static for powerpc64-linux-gnu (from x86_64-linux) using Guix at
+commit 1ced8379c7641788fa607b19b7a66d18f045362b.  Then, it walks the
+graph of derivation inputs, recording only those derivations which
+produce differing output on the two different machines.  If the
+non-reproducibility (across systems) of %gcc-static is caused by a
+non-reproducible input, then it is probably caused by one or more of
+the derivations shown in this graph.
+
+At some point, you have to cut your losses and move on.  After months
+of investigation without resolving the reproducibility issue, we
+finally decided to move forward with the bootstrap binaries produced
+earlier.  If necessary, we can always go back and try to fix this
+issue.  However, it seemed more important to get started with the
+bootstrapping work.
+
+Anyone who is interested in solving this problem is welcome to comment
+on the bug report and help us to figure out the mystery.  We are very
+interested in solving it, but at the moment we are more focused on
+building the rest of the Guix package collection on the
+powerpc64le-linux platform using the existing bootstrap binaries.
+
+### Next Steps
+
+It is now possible to install Guix on a powerpc64le-linux system and
+use it to build some useful software - in particular, Guix itself.  So
+Guix is now "self-hosted" on this platform, which gives us a
+comfortable place to begin further work.
+
+The following tasks still need to be done.  Anyone can help, so please
+get in touch if you want to contribute!
+
+- Solve [the GCC bootstrap binary reproducibility
+  issue](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669)
+  described above.
+- Get [Guix
+  System](https://guix.gnu.org/manual/en/html_node/System-Installation.html)
+  to work on powerpc64le-linux.
+- Get CI infrastructure to work
+  ([Cuirass](https://guix.gnu.org/en/cuirass/) (see also: [Cuirass in
+  the Guix
+  manual](https://guix.gnu.org/manual/en/html_node/Continuous-Integration.html),
+  [guix-build-coordinator](https://git.cbaines.net/guix/build-coordinator/)
+  (see also: [Guix Build Coordinator in the Guix
+  manual](https://guix.gnu.org/manual/en/html_node/Guix-Services.html),
+  [substitutes](https://guix.gnu.org/manual/en/html_node/Substitutes.html),
+  etc.)
+- Try to build your favorite packages using Guix, [report
+  problems](https://guix.gnu.org/manual/en/html_node/Tracking-Bugs-and-Patches.html),
+  [try to fix
+  them](https://guix.gnu.org/manual/en/html_node/Contributing.html),
+  and [ask for help](https://guix.gnu.org/en/help/) if you're feeling
+  stuck or not sure how to start.
+- Try building rust, and if it works, judiciously re-introduce the
+  librsvg dependency for powerpc64le-linux in gtk+ and gtk+-2, since
+  [it is currently
+  missing](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=5d2863dfe4613d5091e61800fcd5a48922c8ce4e).
+- Upgrade the default GCC to 8 on core-updates, try to build guix
+  (e.g., `./pre-inst-env guix build guix`), and report/fix whatever
+  issues occur.  We want to upgrade GCC to 8 because, on the
+  core-updates branch, glibc has been upgraded from 2.31 to 2.32.
+  Unfortunately, on powerpc64le-linux, upgrading glibc from 2.31 to
+  2.32 without also upgrading the default GCC (it's currently 7.5.0)
+  causes a lot of problems.  Right now, we believe the best path
+  forward is probably just to upgrade to GCC 8 on core-updates.
+- Merge core-updates to master after that.
+
+#### About GNU Guix
+
+[GNU Guix](https://guix.gnu.org) is a transactional package manager and
+an advanced distribution of the GNU system that [respects user
+freedom](https://www.gnu.org/distros/free-system-distribution-guidelines.html).
+Guix can be used on top of any system running the Hurd or the Linux
+kernel, or it can be used as a standalone operating system distribution
+for i686, x86_64, ARMv7, and AArch64 machines.
+
+In addition to standard package management features, Guix supports
+transactional upgrades and roll-backs, unprivileged package management,
+per-user profiles, and garbage collection.  When used as a standalone
+GNU/Linux distribution, Guix offers a declarative, stateless approach to
+operating system configuration management.  Guix is highly customizable
+and hackable through [Guile](https://www.gnu.org/software/guile)
+programming interfaces and extensions to the
+[Scheme](http://schemers.org) language.
-- 
2.30.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Please review blog post draft: powerpc64le-linux support

[Vincent Legoll]

Hello Chris,

Great blog post !
I've not seen anything more than the already reported issues.

On Thu, Apr 8, 2021 at 10:55 AM Chris Marusich <cmmarusich@gmail.com> wrote:
> Talos II and Talos II Lite

Why only speaking of Talos and not about the 3rd option: the blackbird ?
Maybe just concentrate on the vendor, more than on particular models...

Thanks

-- 
Vincent Legoll

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2379 bytes --]

Vincent Legoll <vincent.legoll@gmail.com> writes:

> Why only speaking of Talos and not about the 3rd option: the blackbird ?
> Maybe just concentrate on the vendor, more than on particular models...

I specifically avoided speaking about the Blackbird, only because it's
not yet RYF-certified.  However, perhaps I'm being too strict about it.

I actually own a Blackbird, myself.  I chose to buy it instead of the
Talos II or Talos II Lite because of its physically smaller form factor
and its lower cost.  I don't know why it isn't RYF-certified yet, but
according to this Phoronix article, they are "pursuing RYF
certification" for Blackbird, too:

https://www.phoronix.com/scan.php?page=news_item&px=FSF-RYF-Talos-II

Raptor Computing Systems claims that the Blackbird is "completely blob
free":

https://twitter.com/RaptorCompSys/status/1048373354695208960

They also say in that Twitter thread: "We have been putting together our
systems from blob-free components only (sans NIC as is known and being
actively worked), and this is an area where no low-cost blob-free
silicon is available right now."

However, the Talos II and Blackbird both use the same NIC, so I guess
that wouldn't stop it from meeting the RYF requirements:

https://wiki.raptorcs.com/wiki/Talos_II
Networking: 2x GbE (Broadcom BCM5719)

https://wiki.raptorcs.com/wiki/Blackbird
Networking: 3x GbE (Broadcom BCM5719)

See also:

https://wiki.raptorcs.com/wiki/BCM5719

"As the BCM5719 is the only on-board device on the non-SAS Talos™ II
variants to use proprietary firmware, Raptor Computing Systems has
started a contest to see who can create a truly libre replacement
firmware[1]. Anyone with the appropriate skill set is encouraged to take
up the challenge, and contributions to this page as the device is
analyzed in detail are welcomed.

While the BCM5719 does, at least for now, execute proprietary firmware
it is prevented from corrupting the operating system and/or other
protected memory regions via the system IOMMU[2]."

Thinking about this more, I think we should mention Blackbird in our
blog post as a more affordable option.  Let's explain that it doesn't
yet have RYF certification, but the platform is very similar to the
Talos II, and Raptor Computing Systems is currently pursuing RYF
certification for it, too.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Please review blog post draft: powerpc64le-linux support

[Vincent Legoll]

Hello,

On Thu, Apr 8, 2021 at 6:37 PM Chris Marusich <cmmarusich@gmail.com> wrote:
> I specifically avoided speaking about the Blackbird, only because it's
> not yet RYF-certified.  However, perhaps I'm being too strict about it.

Ah, yes, I forgot about this detail. I'd have chosen the blackbird myself,
for the same reasons. But it's still a bit pricey for me though.

I'd say you can talk about it, the way you proposed, as there's a high
probability that it will get the certification.

-- 
Vincent Legoll

Re: Please review blog post draft: powerpc64le-linux support

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 477 bytes --]

On Thu, 2021-04-08 at 09:37 -0700, Chris Marusich wrote:
> They also say in that Twitter thread: "We have been putting together
> our
> systems from blob-free components only (sans NIC as is known and
> being
> actively worked), and this is an area where no low-cost blob-free
> silicon is available right now."
> 

I've been using the Free Software firmware replacement for the NIC
since a while now and it's working great: 
https://github.com/meklort/bcm5719-fw/

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Please review blog post draft: powerpc64le-linux support

[Tobias Platen]

On Fri, 09 Apr 2021 00:59:44 +0200
Léo Le Bouter <lle-bout@zaclys.net> wrote:

> On Thu, 2021-04-08 at 09:37 -0700, Chris Marusich wrote:
> > They also say in that Twitter thread: "We have been putting together
> > our
> > systems from blob-free components only (sans NIC as is known and
> > being
> > actively worked), and this is an area where no low-cost blob-free
> > silicon is available right now."
> > 
> 
> I've been using the Free Software firmware replacement for the NIC
> since a while now and it's working great: 
> https://github.com/meklort/bcm5719-fw/

I've install that firmware on my Talos II and I can confirm that it works.
I have reviewed 0001-website-drafts-Add-powerpc64le-linux-announcement.patch and it looks good.
It would be good to mention the Libre-SoC project(https://libre-soc.org/), which might be a good target for the future.

Tobias

--
Sent from my IBM 7094

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1509 bytes --]

Hi Tobias,

Thank you very much for taking the time to review the blog post!

Tobias Platen <guix@platen-software.de> writes:

> On Fri, 09 Apr 2021 00:59:44 +0200
> Léo Le Bouter <lle-bout@zaclys.net> wrote:
>
>> On Thu, 2021-04-08 at 09:37 -0700, Chris Marusich wrote:
>> > They also say in that Twitter thread: "We have been putting together
>> > our
>> > systems from blob-free components only (sans NIC as is known and
>> > being
>> > actively worked), and this is an area where no low-cost blob-free
>> > silicon is available right now."
>> > 
>> 
>> I've been using the Free Software firmware replacement for the NIC
>> since a while now and it's working great: 
>> https://github.com/meklort/bcm5719-fw/
>
> I've install that firmware on my Talos II and I can confirm that it works.
> I have reviewed 0001-website-drafts-Add-powerpc64le-linux-announcement.patch and it looks good.
> It would be good to mention the Libre-SoC project(https://libre-soc.org/), which might be a good target for the future.

I think that project is interesting, but I don't think I'll add a
section expliclty mentioning it in the post this time.  I originally did
discuss RISC-V in passing, but it felt out of place, since the focus of
the blog post is really on the POWER9 support.  The blog post is already
quite long, so I tried to cut out what I could.

I appreciate the suggestion, though.  I hope that somebody will try
porting to Libre-SoC as well, and RISC-V, too!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

[-- Attachment #1.1: Type: text/plain, Size: 322 bytes --]

Hi,

This is the final draft, I think.  I intend to commit it to the "posts"
directory in guix-artwork on Monday morning, USA time, at which point I
believe it will automatically show up on the blog.

Thank you again for your help, everyone!  If you see any last-minute
typos, please do let me know.

-- 
Chris

[-- Attachment #1.2: 0001-website-drafts-Add-powerpc64le-linux-announcement.patch --]
[-- Type: text/x-patch, Size: 23549 bytes --]

From e4300631958b75d996b9b57c595e74539da5f938 Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Tue, 6 Apr 2021 00:10:35 -0700
Subject: [PATCH] website: drafts: Add powerpc64le-linux announcement.

* website/drafts/new-system-powerpc64le-linux.md: New file.
---
 .../drafts/new-system-powerpc64le-linux.md    | 405 ++++++++++++++++++
 1 file changed, 405 insertions(+)
 create mode 100644 website/drafts/new-system-powerpc64le-linux.md

diff --git a/website/drafts/new-system-powerpc64le-linux.md b/website/drafts/new-system-powerpc64le-linux.md
new file mode 100644
index 0000000..18f3fc4
--- /dev/null
+++ b/website/drafts/new-system-powerpc64le-linux.md
@@ -0,0 +1,405 @@
+title: New Supported Platform: powerpc64le-linux
+date: 2021-04-12 00:00
+author: Chris Marusich and Léo Le Bouter
+tags: porting, powerpc64le, bootstrapping, cross-compilation, reproducibility
+---
+
+It is a pleasure to announce that support for powerpc64le-linux
+(PowerISA v.2.07 and later) has now been
+[merged](https://issues.guix.gnu.org/47182) to the master branch of
+GNU Guix!
+
+This means that GNU Guix can be used immediately on this platform
+[from a Git
+checkout](https://guix.gnu.org/manual/en/html_node/Building-from-Git.html).
+Starting with the next release (Guix v1.2.1), you will also be able to
+[download a copy of Guix pre-built for
+powerpc64le-linux](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html#Binary-Installation).
+Regardless of how you get it, you can run the new powerpc64le-linux
+port of GNU Guix on top of any existing powerpc64le GNU/Linux
+distribution.
+
+This new platform is available as a "technology preview".  This means
+that although it is supported,
+[substitutes](https://guix.gnu.org/manual/en/html_node/Substitutes.html)
+are not yet available from the build farm, and some packages may fail
+to build.  Although powerpc64le-linux support is nascent, the Guix
+community is actively working on improving it, and this is a great
+time to [get
+involved](https://guix.gnu.org/manual/en/html_node/Contributing.html)!
+
+### Why Is This Important?
+
+This is important because it means that GNU Guix now works on the
+[Talos II, Talos II Lite, and Blackbird
+mainboards](https://www.raptorcs.com/content/base/products.html) sold
+by [Raptor Computing Systems](https://www.raptorcs.com/).  This
+modern, performant hardware uses [IBM
+POWER9](https://wiki.raptorcs.com/wiki/POWER9) processors, and it is
+designed to respect your freedom.  The Talos II and Talos II Lite have
+[recently received Respects Your Freedom (RYF)
+certification](https://www.fsf.org/news/talos-ii-mainboard-and-talos-ii-lite-mainboard-now-fsf-certified-to-respect-your-freedom)
+from the FSF, and Raptor Computing Systems is currently pursuing RYF
+certification for the more affordable Blackbird, too.  All of this
+hardware [can run without any non-free
+code](https://wiki.raptorcs.com/wiki/Platform_Comparison), even the
+bootloader and firmware.  In other words, this is a freedom-friendly
+hardware platform that aligns well with GNU Guix's commitment to
+software freedom.
+
+How is this any different from existing RYF hardware, you might ask?
+One reason is performance.  The existing RYF
+[laptops](https://ryf.fsf.org/products?category=1&vendor=All&sort_by=created&sort_order=DESC),
+[mainboards](https://ryf.fsf.org/products?category=5&vendor=All&sort_by=created&sort_order=DESC),
+and
+[workstations](https://ryf.fsf.org/products?category=30&vendor=All&sort_by=created&sort_order=DESC)
+can only really be used with Intel Core Duo or AMD Opteron processors.
+Those processors were released over 15 years ago.  Since then,
+processor performance has increased drastically.  People should not
+have to choose between performance and freedom, but for many years
+that is exactly what we were forced to do.  However, the POWER9
+machines sold by Raptor Computing Systems have changed this: the free
+software community now has an RYF-certified option that [can compete
+with the performance of modern Intel and AMD
+systems](https://www.phoronix.com/scan.php?page=article&item=power9-threadripper-core9&num=1).
+
+Although the performance of POWER9 processors is competitive with
+modern Intel and AMD processors, the real advantage of the Talos II,
+Talos II Lite, and Blackbird is that they were designed from the start
+to respect your freedom.  Modern processors from [both Intel and AMD
+include back
+doors](https://www.fsf.org/blogs/sysadmin/the-management-engine-an-attack-on-computer-users-freedom)
+over which you are given no control.  Even though the back doors can
+be removed [with significant effort on older hardware in some
+cases](https://www.fsf.org/news/libreboot-x200-laptop-now-fsf-certified-to-respect-your-freedom),
+this is an obstacle that nobody should have to overcome just to
+control their own computer.  Many of the existing RYF-certified
+options (e.g., the venerable Lenovo x200) use hardware that can only
+be considered RYF-certified after someone has gone through the extra
+effort of removing those back doors.  No such obstacles exist when
+using the Talos II, Talos II Lite, or Blackbird.  In fact, although
+[Intel](https://arstechnica.com/gadgets/2020/10/in-a-first-researchers-extract-secret-key-used-to-encrypt-intel-cpu-code/)
+and
+[AMD](https://www.extremetech.com/computing/292722-amds-secure-processor-firmware-is-now-explorable-thanks-to-new-tool)
+both go out of their way to keep you from understanding what is going
+on in your own computer, Raptor Computing Systems releases [all of the
+software and firmware used in their
+boards](https://git.raptorcs.com/git/) as free software.  They even
+include circuit diagrams when they ship you the machine!
+
+Compared to the existing options, the Talos II, Talos II Lite, and
+Blackbird are a breath of fresh air that the free software community
+really deserves.  Raptor Computing Systems' commitment to software
+freedom and owner control is an inspiring reminder that it **is**
+possible to ship a great product while still respecting the freedom of
+your customers.  And going forward, the future looks bright for the
+open, royalty-free Power ISA stewarded by the OpenPOWER Foundation,
+[which is now a Linux Foundation
+project](https://www.linuxfoundation.org/press-release/2019/08/the-linux-foundation-announces-new-open-hardware-technologies-and-collaboration/)
+(see also: [the same announcement from the OpenPOWER
+Foundation](https://openpowerfoundation.org/the-next-step-in-the-openpower-foundation-journey/).
+
+In the rest of this blog post, we will discuss the steps we took to
+port Guix to powerpc64le-linux, the issues we encountered, and the
+steps we can take going forward to further solidify support for this
+exciting new platform.
+
+### Bootstrapping powerpc64le-linux: A Journey
+
+To build software, you need software.  How can one port Guix to a
+platform before support for that platform exists?  This is a
+[bootstrapping
+problem](https://guix.gnu.org/manual/en/html_node/Bootstrapping.html).
+
+In Guix, all software for a given platform (e.g., powerpc64le-linux)
+is built starting from a small set of "bootstrap binaries".  These are
+binaries of Guile, GCC, Binutils, libc, and a few other packages,
+pre-built for the relevant platform.  It is intended that the
+bootstrap binaries are the only pieces of software in the entire
+package collection that Guix cannot build from source.  In practice,
+[additional bootstrap roots are
+possible](https://lists.gnu.org/archive/html/guix-devel/2015-02/msg00814.html),
+but introducing them in Guix is highly discouraged, and our community
+[actively](https://guix.gnu.org/en/blog/2019/guix-reduces-bootstrap-seed-by-50/)
+[works](https://guix.gnu.org/en/blog/2020/guix-further-reduces-bootstrap-seed-to-25/)
+to [reduce](https://guix.gnu.org/en/blog/2018/bootstrapping-rust/) our
+overall bootstrap footprint.  There is one set of bootstrap binaries
+for each platform that Guix supports.
+
+This means that to port Guix to a new platform, you must first build
+the bootstrap binaries for that platform.  In theory, you can do this
+in many ways.  For example, you might try to manually compile them on
+an existing system.  However, Guix has [package
+definitions](https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/make-bootstrap.scm?id=5d8c2c00d60196c46a32b68c618ccbe2b3aa48f4)
+that you can use to build them - using Guix, of course!
+
+Commonly, the first step in [porting Guix to a new
+platform](https://guix.gnu.org/manual/en/html_node/Porting.html) is to
+use Guix to cross-compile the bootstrap binaries for that new platform
+from a platform on which Guix is already supported. This can be done
+by running a command like the following on a system where Guix is
+already installed:
+
+```scheme
+guix build --target=powerpc64le-linux-gnu bootstrap-tarballs
+```
+
+This is the route that we took when building the powerpc64le-linux
+bootstrap binaries, as described in commit
+[8a1118a](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a1118a96c9ae128302c3d435ae77cb3dd693aea).
+You might wonder why the target above is "powerpc64le-linux-gnu" even
+though the new Guix platform is called "powerpc64le-linux".  This is
+because "powerpc64le-linux-gnu" is a GNU
+[triplet](https://wiki.osdev.org/Target_Triplet) identifying the new
+platform, but "powerpc64le-linux" is the name of a "system" (i.e., a
+platform) in Guix.  Guix contains code that converts between the two
+as needed (see `nix-system->gnu-triplet` and `gnu-triplet->nix-system`
+in
+[`guix/utils.scm`](https://git.savannah.gnu.org/cgit/guix.git/tree/guix/utils.scm?id=83991a34d5c1d4985e54dd029a81412277ad062a).
+When cross-compiling, you only need to specify the GNU triplet.
+
+Note that before you can even do this, you must first update the
+`glibc-dynamic-linker` and `system->linux-architecture` procedures in
+Guix's code, as described in
+[Porting](https://guix.gnu.org/manual/en/html_node/Porting.html).  In
+addition, the versions of packages in Guix that make up the GNU
+toolchain (gcc, glibc, etc.) must already support the target platform.
+This pre-existing toolchain support needs to be good enough so that
+Guix can (1) build, on some already-supported platform, a
+cross-compilation toolchain for the target platform, (2) use, on the
+already-supported platform, the cross-compilation toolchain to
+cross-compile the bootstrap binaries for the target platform, and (3)
+use, on the target platform, the bootstrap binaries to natively build
+the rest of the Guix package collection.  The above [`guix
+build`](https://guix.gnu.org/manual/en/html_node/Invoking-guix-build.html#Invoking-guix-build)
+command takes care of steps (1) and (2) automatically.
+
+Step (3) is a little more involved.  Once the bootstrap binaries for
+the target platform have been built, they must be published online for
+anyone to download.  After that, Guix's code must be updated so that
+(a) it recognizes the "system" name (e.g., "powerpc64le-linux") that
+will be used to identify the new platform and (b) it fetches the new
+platform's bootstrap binaries from the right location.  After all that
+is done, you just have to try building things and see what breaks.
+For example, you can run `./pre-inst-env guix build hello` from your
+Git checkout to try building GNU Hello.
+
+The actual bootstrap binaries for powerpc64le-linux are stored on the
+[alpha.gnu.org FTP
+server](https://alpha.gnu.org/gnu/guix/bootstrap/powerpc64le-linux/20210106/).
+Chris Marusich built these bootstrap binaries in an x86_64-linux Guix
+System VM which was running on hardware owned by Léo Le Bouter.  Chris
+then signed the binaries and provided them to Ludovic Courtès, who in
+turn verified their authenticity, signed them, and [uploaded them to
+alpha.gnu.org](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669#125).
+After that, we updated the code to use the newly published bootstrap
+binaries in commit
+[8a1118a](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=8a1118a96c9ae128302c3d435ae77cb3dd693aea).
+Once all that was done, we could begin bootstrapping the rest of the
+system - or trying to, at least.
+
+There were many stumbling blocks.  For example, to resolve some test
+failures, we had to update the code in Guix that enables it to make
+[certain](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=b57de27d0331198c9cafb09a1cf8a5fa4f691e36)
+[syscalls](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c29bfbfc78ccd9e5c10d38faf3d269eafed12854)
+from scheme.  In another example, we had to [patch GCC so that it
+looks for the 64-bit libraries in
+/lib](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46253), rather
+than /lib64, since that is where Guix puts its 64-bit libraries by
+convention.  In addition, some packages required in order to build
+Guix failed to build, so we had to debug those build failures, too.
+
+For a list of all the changes, see [the patch
+series](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47182) or the
+actual commits, which are:
+
+```
+$ git log --oneline --no-decorate 8a1118a96c9ae128302c3d435ae77cb3dd693aea..65c46e79e0495fe4d32f6f2725d7233fff10fd70
+65c46e79e04 gnu: sed: Make it build on SELinux-enabled kernels.
+93f21e1a35e utils: Fix target-64bit? on powerpc64le-linux.
+8d9aece8c49 ci: %cross-targets: Add powerpc64le-linux-gnu.
+c29bfbfc78c syscalls: Fix RNDADDTOENTCNT on powerpc64le-linux.
+b57de27d033 syscalls: Fix clone on powerpc64le-linux.
+a16eb6c5f97 Add powerpc64le-linux as a supported Guix architecture.
+b50f4268035 gnu: libelf: Fix compilation for powerpc64le-linux.
+1a0f4013d33 gnu: texlive-latex-base: Fix compilation on powerpc64le*.
+e9938dc8f0e gnu: texlive-bin: Fix compilation on powerpc64le*.
+69b3907adf6 gnu: guile-avahi: Fix compilation on powerpc64le-linux.
+4cc2d2aa599 gnu: bdb-4.8: Fix configure on powerpc64le-linux.
+be4b1cf53bd gnu: binutils-final: Support more Power architectures.
+060478c32c9 gnu: binutils-final: Provide bash for binary on powerpc-linux.
+b2135b5d576 gnu: gcc-boot0: Enable 128-bit long double for POWER9.
+6e98e9ca923 gnu: glibc: Fix ldd path on powerpc*.
+cac88b28b83 gnu: gcc-4.7: On powerpc64le, fix /lib64 references.
+fc7cf0c1ecc utils: Add target-powerpc? procedure.
+```
+
+In the end, through the combined efforts of multiple people, we slowly
+worked through the issues until we reached a point where we could do
+all of the following things successfully:
+
+- Build Guix manually on a [Debian GNU/Linux
+  ppc64el](https://wiki.debian.org/ppc64el) machine (this is Debian's
+  name for a system using the powerpc64le-linux-gnu triplet), and
+  verify that its `make check` tests passed.
+- Build GNU Hello using Guix and run it.
+- Run [`guix
+  pull`](https://guix.gnu.org/manual/en/html_node/Invoking-guix-pull.html#Invoking-guix-pull)
+  to build and install the most recent version of Guix, with
+  powerpc64le-linux support.
+- Build a release binary tarball for powerpc64le-linux via: `make
+  guix-binary.powerpc64le-linux.tar.xz`
+- Use that binary to install a version of Guix that could build/run
+  GNU Hello and run `guix pull` successfully.
+
+This was an exciting moment!  But there was still more work to be
+done.
+
+Originally, we did this work on the
+[wip-ppc64le](https://git.savannah.gnu.org/cgit/guix.git/log/?h=wip-ppc64le)
+branch, with the intent of merging it into core-updates.  By
+convention, the "core-updates" branch in Guix is [where changes are
+made if they cause too many
+rebuilds](https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html).
+Since we were updating package definitions so deep in the dependency
+graph of the package collection, we assumed it wouldn't be possible to
+avoid rebuilding the world.  For this reason, we had based the
+wip-ppc64le branch on core-updates.
+
+However, Efraim Flashner proved us wrong!  He created a separate
+branch, wip-ppc64le-for-master, where he adjusted some of the
+wip-ppc64le commits to avoid rebuilding the world on other platforms.
+Thanks to his work, we were able to merge the changes directly to
+master!  This meant that we would be able to include it in the next
+release (Guix v.1.2.1).
+
+In short, the initial porting work is done, and it is now possible for
+anyone to easily try out Guix on this new platform.  Because `guix
+pull` works, too, it is also easy to iterate on what we have and work
+towards improving support for the platform.  It took a lot of
+cooperation and effort to get this far, but there are multiple people
+actively contributing to this port in the Guix community who want to
+see it succeed.  We hope you will join us in exploring the limits of
+this exciting new freedom-friendly platform!
+
+### Other Porting Challenges
+
+Very early in the porting process, there were some other problems that
+stymied our work.
+
+First, we actually thought we would try to port to powerpc64-linux
+(big-endian).  However, this did not prove to be any easier than the
+little-endian port.  In addition, other distributions (e.g.,
+[Debian](https://www.debian.org/ports/) and
+[Fedora](https://fedoraproject.org/wiki/Architectures)) have recently
+dropped their big-endian powerpc64 ports, so the little-endian variant
+is more likely to be tested and supported in the community.  For these
+reasons, we decided to focus our efforts on the little-endian variant,
+and so far we haven't looked back.
+
+In both the big-endian and little-endian case, we were saddened to
+discover that the bootstrap binaries are not entirely reproducible.
+This fact is documented in [bug
+41669](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669), along
+with our extensive investigations.
+
+In short, if you build the bootstrap binaries on two separate machines
+without using any substitutes, you will find that the derivation which
+cross-compiles %gcc-static (the bootstrap GCC, version 5.5.0) produces
+different output on the two systems.  However, if you build
+%gcc-static twice on the same system, it builds reproducibly.  This
+suggests that something in the transitive closure of inputs of
+%gcc-static is perhaps contributing to its non-reproducibility.  There
+is an interesting graph [toward the end of the bug
+report](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669#137),
+shown below:
+
+![Differing
+Derivations](https://debbugs.gnu.org/cgi/bugreport.cgi?att=1;msg=137;filename=nonreproducible-drvs-small.png;bug=41669)
+
+This graph shows the derivations that produce differing outputs across
+two Guix System machines, when everything is built without
+substitutes.  It starts from the derivation that cross-compiles
+%gcc-static for powerpc64-linux-gnu (from x86_64-linux) using Guix at
+commit 1ced8379c7641788fa607b19b7a66d18f045362b.  Then, it walks the
+graph of derivation inputs, recording only those derivations which
+produce differing output on the two different machines.  If the
+non-reproducibility (across systems) of %gcc-static is caused by a
+non-reproducible input, then it is probably caused by one or more of
+the derivations shown in this graph.
+
+At some point, you have to cut your losses and move on.  After months
+of investigation without resolving the reproducibility issue, we
+finally decided to move forward with the bootstrap binaries produced
+earlier.  If necessary, we can always go back and try to fix this
+issue.  However, it seemed more important to get started with the
+bootstrapping work.
+
+Anyone who is interested in solving this problem is welcome to comment
+on the bug report and help us to figure out the mystery.  We are very
+interested in solving it, but at the moment we are more focused on
+building the rest of the Guix package collection on the
+powerpc64le-linux platform using the existing bootstrap binaries.
+
+### Next Steps
+
+It is now possible to install Guix on a powerpc64le-linux system and
+use it to build some useful software - in particular, Guix itself.  So
+Guix is now "self-hosted" on this platform, which gives us a
+comfortable place to begin further work.
+
+The following tasks still need to be done.  Anyone can help, so please
+get in touch if you want to contribute!
+
+- Solve [the GCC bootstrap binary reproducibility
+  issue](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669)
+  described above.
+- Get [Guix
+  System](https://guix.gnu.org/manual/en/html_node/System-Installation.html)
+  to work on powerpc64le-linux.
+- Get CI infrastructure to work
+  ([Cuirass](https://guix.gnu.org/en/cuirass/) (see also: [Cuirass in
+  the Guix
+  manual](https://guix.gnu.org/manual/en/html_node/Continuous-Integration.html),
+  [guix-build-coordinator](https://git.cbaines.net/guix/build-coordinator/)
+  (see also: [Guix Build Coordinator in the Guix
+  manual](https://guix.gnu.org/manual/en/html_node/Guix-Services.html),
+  [substitutes](https://guix.gnu.org/manual/en/html_node/Substitutes.html),
+  etc.)
+- Try to build your favorite packages using Guix, [report
+  problems](https://guix.gnu.org/manual/en/html_node/Tracking-Bugs-and-Patches.html),
+  [try to fix
+  them](https://guix.gnu.org/manual/en/html_node/Contributing.html),
+  and [ask for help](https://guix.gnu.org/en/help/) if you're feeling
+  stuck or not sure how to start.
+- Try building rust, and if it works, judiciously re-introduce the
+  librsvg dependency for powerpc64le-linux in gtk+ and gtk+-2, since
+  [it is currently
+  missing](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=5d2863dfe4613d5091e61800fcd5a48922c8ce4e).
+- Upgrade the default GCC to 8 on core-updates, try to build guix
+  (e.g., `./pre-inst-env guix build guix`), and report/fix whatever
+  issues occur.  We want to upgrade GCC to 8 because, on the
+  core-updates branch, glibc has been upgraded from 2.31 to 2.32.
+  Unfortunately, on powerpc64le-linux, upgrading glibc from 2.31 to
+  2.32 without also upgrading the default GCC (it's currently 7.5.0)
+  causes a lot of problems.  Right now, we believe the best path
+  forward is probably just to upgrade to GCC 8 on core-updates.
+- Merge core-updates to master after that.
+
+#### About GNU Guix
+
+[GNU Guix](https://guix.gnu.org) is a transactional package manager and
+an advanced distribution of the GNU system that [respects user
+freedom](https://www.gnu.org/distros/free-system-distribution-guidelines.html).
+Guix can be used on top of any system running the Hurd or the Linux
+kernel, or it can be used as a standalone operating system distribution
+for i686, x86_64, ARMv7, and AArch64 machines.
+
+In addition to standard package management features, Guix supports
+transactional upgrades and roll-backs, unprivileged package management,
+per-user profiles, and garbage collection.  When used as a standalone
+GNU/Linux distribution, Guix offers a declarative, stateless approach to
+operating system configuration management.  Guix is highly customizable
+and hackable through [Guile](https://www.gnu.org/software/guile)
+programming interfaces and extensions to the
+[Scheme](http://schemers.org) language.
-- 
2.30.2


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Please review blog post draft: powerpc64le-linux support

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 393 bytes --]

Hi,

Chris Marusich <cmmarusich@gmail.com> writes:

> This is the final draft, I think.  I intend to commit it to the "posts"
> directory in guix-artwork on Monday morning, USA time, at which point I
> believe it will automatically show up on the blog.

I have published it in commit 0129dd529347bfefee96644ac9fbabc29adbe772.
Thank you again to everyone for your help!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Please review blog post draft: powerpc64le-linux support

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 627 bytes --]

On Mon, 2021-04-12 at 12:46 -0700, Chris Marusich wrote:
> Hi,
> 
> Chris Marusich <cmmarusich@gmail.com> writes:
> 
> > This is the final draft, I think.  I intend to commit it to the
> > "posts"
> > directory in guix-artwork on Monday morning, USA time, at which
> > point I
> > believe it will automatically show up on the blog.
> 
> I have published it in commit
> 0129dd529347bfefee96644ac9fbabc29adbe772.
> Thank you again to everyone for your help!
> 

Awesome! I also sent an email to Micheal from Phoronix earlier, but it
seems they either didnt take it into account or didnt see it for now.

Léo

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]