# Table of Contents

1.  [Overview](#org2633a51)
2.  [Anatomy of a toolchain](#orgc440e9e)
3.  [Bootstrapping a Toolchain](#orgd42b6c3)
4.  [Defining the Packages](#org55042c5)
	1.  [Binutils](#org67da1ec)
	2.  [GCC sans libc](#org82d6f83)
	3.  [Newlib(-nano)](#orgf6bafbc)
	4.  [Complete toolchain](#org052f2a2)
5.  [Integrating with Zephyr Build System](#orgc3f87f4)
	1.  [Testing](#org9f3c314)

All code is available at [guix-zephyr](https://github.com/paperclip4465/guix-zephyr) channel.


<a id="org2633a51"></a>

# Overview

In order to deploy embedded software using Guix we first need to teach Guix
how to build it. Since Guix bootstraps everything this means we must teach Guix
how to build our toolchain.

The [Zephyr Project](https://zephyrproject.org) uses its own fork of GCC with custom configs for
the architectures supported by the project.


<a id="orgc440e9e"></a>

# Anatomy of a toolchain

Toolchains are responsible for taking high level descriptions of programs
and lowering them down to a series of equivalent machine instructions.
This process involves more than just a compiler. The compiler uses the `binutils`
to manipulate it's internal representation down to a given architecture.
It also needs the use of the C standard library as well as a few other libraries
needed for some compiler optimizations.

The C library provides the interface to the underlying kernel. System calls like `write`
and `read` are provided by `Glibc` on most Linux distributions.

In embedded systems smaller implementations like `newlib` and `newlib-nano` are used.


<a id="orgd42b6c3"></a>

# Bootstrapping a Toolchain

In order to compile GCC we need a C library that's been compiled for
our target architecture. How can we cross compile our C library if we
need our C library to build a cross compiler? The solution is to build
a simpler compiler that doesn't require the C library to function.
It will not be capable of as many optimizations and it will be very slow,
however it will be able to build the C libraries as well as the complete version
of GCC.

In order to build the simpler compiler we need to compile the `binutils` to
work with our target architecture.
The `binutils` can be bootstrapped with our host GCC and have no target dependencies.

[For more information read this.](https://crosstool-ng.github.io/docs/toolchain-construction/)

Doesn't sound so bad right? It isn't&#x2026; in theory.
However internet forums since time immemorial have been
littered with the laments of those who came before.
From incorrect versions of `ISL` to the wrong C library being linked
or the host linker being used, etc.
The one commonality between all of these issues is the environment.
Building a cross toolchain is difficult because isolating build
environments is difficult.

In fact as of `v0.14.2` the zephyr SDK repository took down the build
instructions and posted a sign that read
"Building this is too complicated, don't worry about it."<sup><a id="fnr.1" class="footref" href="#fn.1" role="doc-backlink">1</a></sup>

We will neatly side step all of these problems and not
risk destroying or polluting our host system with garbage
by using Guix to manage our environments for us.

Our toolchain only requires the first pass compiler because
newlib(-nano) is statically linked and introduced to the toolchain
by normal package composition.


<a id="org55042c5"></a>

# Defining the Packages

All of the base packages are defined in `zephyr/packages/zephyr.scm`.
Zephyr modules are defined in `zephyr/packages/zephyr-xyz.scm`, following
the pattern of other module systems implemented by Guix.


<a id="org67da1ec"></a>

## Binutils

First thing we need to build is the `arm-zephyr-eabi` binutils.
This is very easy in Guix.

	(define-module (zephyr packages zephyr)
	  #:use-module (guix packages))

	(define-public arm-zephyr-eabi-binutils
	  (let ((xbinutils (cross-binutils "arm-zephyr-eabi")))
		(package
		  (inherit xbinutils)
		  (name "arm-zephyr-eabi-binutils")
		  (version "2.38")
		  (source
		   (origin (method git-fetch)
				   (uri (git-reference
						 (url "https://github.com/zephyrproject-rtos/binutils-gdb")
						 (commit "6a1be1a6a571957fea8b130e4ca2dcc65e753469")))
				   (file-name (git-file-name name version))
				   (sha256 (base32 "0ylnl48jj5jk3jrmvfx5zf8byvwg7g7my7jwwyqw3a95qcyh0isr"))))
		  (arguments
		   `(#:tests? #f
			 ,@(substitute-keyword-arguments (package-arguments xbinutils)
				 ((#:configure-flags flags)
				  `(cons "--program-prefix=arm-zephyr-eabi-" ,flags)))))
		  (native-inputs
		   (append
			(list texinfo
				  bison
				  flex
				  gmp
				  dejagnu)
			(package-native-inputs xbinutils)))
		  (home-page "https://zephyrproject.org")
		  (synopsis "binutils for zephyr RTOS"))))

The function `cross-binutils` returns a package which has been
configured for the given gnu triplet.  We simply inherit that package
and replace the source.
The zephyr build system expects the binutils to be prefixed with
`arm-zephyr-eabi-` which is accomplished by adding another flag to the
`#:configure-flags` argument.

We can test our package definition using the `-L` flag with `guix build`
to add our packages.

	guix build -L guix-zephyr zephyr-binutils

	/gnu/store/...-zephyr-binutils-2.38

This directory contains the results of `make install`.


<a id="org82d6f83"></a>

## GCC sans libc

This one is a bit more involved. Don't be afraid!
This version of GCC wants ISL version 0.15. It's easy enough
to make that happen. Inherit the current version of ISL and swap
out the source and update the version. For most packages the build process doesn't
change that much between versions.

	(define-public isl-0.15
	  (package
		(inherit isl)
		(version "0.15")
		(source (origin
				  (method url-fetch)
				  (uri (list (string-append "mirror://sourceforge/libisl/isl-"
											version ".tar.gz")))
				  (sha256
				   (base32
					"11vrpznpdh7w8jp4wm4i8zqhzq2h7nix71xfdddp8xnzhz26gyq2"))))))

Like the binutils, there is a function for creating cross-gcc packages.
This one accepts keywords specifying which binutils and libc to use.
If libc isn't given (like here), gcc is configured with many options disabled
to facilitate being built without libc. Therefore we need to add the extra options
we want <sup><a id="fnr.2" class="footref" href="#fn.2" role="doc-backlink">2</a></sup>

	(define-public gcc-arm-zephyr-eabi-12
	  (let ((xgcc (cross-gcc "arm-zephyr-eabi"
							 #:xbinutils zephyr-binutils)))
		(package
		  (inherit xgcc)
		  (version "12.1.0")
		  (source (origin (method git-fetch)
						  (uri (git-reference
								(url "https://github.com/zephyrproject-rtos/gcc")
								(commit "0218469df050c33479a1d5be3e5239ac0eb351bf")))
						  (file-name (git-file-name (package-name xgcc) version))
						  (sha256
						   (base32 "1s409qmidlvzaw1ns6jaanigh3azcxisjplzwn7j2n3s33b76zjk"))
						  (patches
						   (search-patches "gcc-12-cross-environment-variables.patch"
										   "gcc-cross-gxx-include-dir.patch"))))
		  (native-inputs
		   (modify-inputs (package-native-inputs xgcc)
			 ;; Get rid of stock ISL
			 (delete "isl")
			 ;; Add additional dependencies that xgcc doesn't have
			 ;; including our special ISL
			 (prepend flex
					  perl
					  python-3
					  gmp
					  isl-0.15
					  texinfo
					  python
					  mpc
					  mpfr
					  zlib)))
		  (arguments
		   (substitute-keyword-arguments (package-arguments xgcc)
			 ((#:phases phases)
			  `(modify-phases ,phases
				 (add-after 'unpack 'fix-genmultilib
				   (lambda _
					 (substitute* "gcc/genmultilib"
					   (("#!/bin/sh") (string-append "#!" (which "sh"))))
					 #t))

				 (add-after 'set-paths 'augment-CPLUS_INCLUDE_PATH
				   (lambda* (#:key inputs #:allow-other-keys)
					 (let ((gcc (assoc-ref inputs  "gcc")))
					   ;; Remove the default compiler from CPLUS_INCLUDE_PATH to
					   ;; prevent header conflict with the GCC from native-inputs.
					   (setenv "CPLUS_INCLUDE_PATH"
							   (string-join
								(delete (string-append gcc "/include/c++")
										(string-split (getenv "CPLUS_INCLUDE_PATH")
													  #\:))
								":"))
					   (format #t
							   "environment variable `CPLUS_INCLUDE_PATH' changed to ~a~%"
							   (getenv "CPLUS_INCLUDE_PATH"))
					   #t)))))

			 ((#:configure-flags flags)
			  ;; The configure flags are largely identical to the flags used by the
			  ;; "GCC ARM embedded" project.
			  `(append (list "--enable-multilib"
							 "--with-newlib"
							 "--with-multilib-list=rmprofile"
							 "--with-host-libstdcxx=-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm"
							 "--enable-plugins"
							 "--disable-decimal-float"
							 "--disable-libffi"
							 "--disable-libgomp"
							 "--disable-libmudflap"
							 "--disable-libquadmath"
							 "--disable-libssp"
							 "--disable-libstdcxx-pch"
							 "--disable-nls"
							 "--disable-shared"
							 "--disable-threads"
							 "--disable-tls"
							 "--with-gnu-ld"
							 "--with-gnu-as"
							 "--enable-initfini-array")
					   (delete "--disable-multilib" ,flags)))))
		  (native-search-paths
		   (list (search-path-specification
				  (variable "CROSS_C_INCLUDE_PATH")
				  (files '("arm-zephyr-eabi/include")))
				 (search-path-specification
				  (variable "CROSS_CPLUS_INCLUDE_PATH")
				  (files '("arm-zephyr-eabi/include"
						   "arm-zephyr-eabi/c++"
						   "arm-zephyr-eabi/c++/arm-zephyr-eabi")))
				 (search-path-specification
				  (variable "CROSS_LIBRARY_PATH")
				  (files '("arm-zephyr-eabi/lib")))))
		  (home-page "https://zephyrproject.org")
		  (synopsis "GCC for zephyr RTOS"))))

This GCC can be built like so.

	guix build -L guix-zephyr gcc-cross-sans-libc-arm-zephyr-eabi

	/gnu/store/...-gcc-cross-sans-libc-arm-zephyr-eabi-12.1.0-lib
	/gnu/store/...-gcc-cross-sans-libc-arm-zephyr-eabi-12.1.0

Great! We now have our stage-1 compiler.


<a id="orgf6bafbc"></a>

## Newlib(-nano)

The newlib package package is quite straight forward (relatively).
It is mostly adding in the relevent configuration flags and patching
the files the `patch-shebangs` phase missed.

	  (define-public zephyr-newlib
	  (package
		(name "zephyr-newlib")
		(version "3.3")
		(source (origin
				  (method git-fetch)
				  (uri (git-reference
						(url "https://github.com/zephyrproject-rtos/newlib-cygwin")
						(commit "4e150303bcc1e44f4d90f3489a4417433980d5ff")))
				  (sha256
				   (base32 "08qwjpj5jhpc3p7a5mbl7n6z7rav5yqlydqanm6nny42qpa8kxij"))))
		(build-system gnu-build-system)
		(arguments
		 `(#:out-of-source? #t
		   #:configure-flags '("--target=arm-zephyr-eabi"
							   "--enable-newlib-io-long-long"
							   "--enable-newlib-io-float"
							   "--enable-newlib-io-c99-formats"
							   "--enable-newlib-retargetable-locking"
							   "--enable-newlib-lite-exit"
							   "--enable-newlib-multithread"
							   "--enable-newlib-register-fini"
							   "--enable-newlib-extra-sections"
							   "--disable-newlib-wide-orient"
							   "--disable-newlib-fseek-optimization"
							   "--disable-newlib-supplied-syscalls"
							   "--disable-newlib-target-optspace"
							   "--disable-nls")
		   #:phases
		   (modify-phases %standard-phases
			 (add-after 'unpack 'fix-references-to-/bin/sh
			   (lambda _
				 (substitute* '("libgloss/arm/cpu-init/Makefile.in"
								"libgloss/arm/Makefile.in"
								"libgloss/libnosys/Makefile.in"
								"libgloss/Makefile.in")
				   (("/bin/sh") (which "sh")))
				 #t)))))
		(native-inputs
		 `(("xbinutils" ,zephyr-binutils)
		   ("xgcc" ,gcc-arm-zephyr-eabi-12)
		   ("texinfo" ,texinfo)))
		(home-page "https://www.sourceware.org/newlib/")
		(synopsis "C library for use on embedded systems")
		(description "Newlib is a C library intended for use on embedded
	systems.  It is a conglomeration of several library parts that are easily
	usable on embedded products.")
		(license (license:non-copyleft
				  "https://www.sourceware.org/newlib/COPYING.NEWLIB"))))

And the build.

	guix build -L guix-zephyr zephyr-newlib

	/gnu/store/...-zephyr-newlib-3.3


<a id="org052f2a2"></a>

## Complete toolchain<sup><a id="fnr.3" class="footref" href="#fn.3" role="doc-backlink">3</a></sup>

Now that we've got the individual tools it's time to create our complete toolchain.
For this we need to do some package transformations.
Because these transformations are going to have to be done for every combination of
gcc/newlib it is best to create a function which we can reuse for every version
of the SDK.

	(define (arm-zephyr-eabi-toolchain xgcc newlib version)
	  "Produce a cross-compiler zephyr toolchain package with the compiler XGCC and the C
	library variant NEWLIB."
	  (let ((newlib-with-xgcc (package (inherit newlib)
									   (native-inputs
										(alist-replace "xgcc" (list xgcc)
													   (package-native-inputs newlib))))))
		(package
		  (name (string-append "arm-zephyr-eabi"
							   (if (string=? (package-name newlib-with-xgcc)
											 "newlib-nano")
								   "-nano" "")
							   "-toolchain"))
		  (version version)
		  (source #f)
		  (build-system trivial-build-system)
		  (arguments
		   '(#:modules ((guix build union)
						(guix build utils))
			 #:builder
			 (begin
			   (use-modules (ice-9 match)
							(guix build union)
							(guix build utils))
			   (let ((out (assoc-ref %outputs "out")))
				 (mkdir-p out)
				 (match %build-inputs
				   (((names . directories) ...)
					(union-build (string-append out "/arm-zephyr-eabi")
								 directories)
					#t))))))
		  (inputs
		   `(("binutils" ,zephyr-binutils)
			 ("gcc" ,xgcc)
			 ("newlib" ,newlib-with-xgcc)))
		  (synopsis "Complete GCC tool chain for ARM zephyrRTOS development")
		  (description "This package provides a complete GCC tool chain for ARM
	bare metal development with zephyr rtos.  This includes the GCC arm-zephyr-eabi cross compiler
	and newlib (or newlib-nano) as the C library.  The supported programming
	language is C.")
		  (home-page (package-home-page xgcc))
		  (license (package-license xgcc)))))

This function creates a special package which consists of the toolchain in a special directory hierarchy, i.e `arm-zephyr-eabi/`.
Our complete toolchain definition looks like this.

	(define-public arm-zephyr-eabi-toolchain-0.15.0
	  (arm-zephyr-eabi-toolchain
	   gcc-arm-zephyr-eabi-12
	   zephyr-newlib
	   "0.15.0"))

To build:

	guix build -L guix-zephyr arm-zephyr-eabi-toolchain

	/gnu/store/...-arm-zephyr-eabi-toolchain-0.15.0


<a id="orgc3f87f4"></a>

# Integrating with Zephyr Build System

Zephyr uses CMake as it's build system. It contains numerous CMake
files in both the so-called `ZEPHYR_BASE`, the zephyr source code
repository, as well as a handful in the SDK
which help select the correct toolchain for a given board.

There are standard locations the build system will look for the
SDK. We are not using any of them.  Our SDK lives in the store,
immutable forever. According to [this](https://docs.zephyrproject.org/latest/develop/west/without-west.html) the variable
`ZEPHYR_SDK_INSTALL_DIR` needs to point to our custom spot.

We also need to grab the cmake files from the [repository](https://github.com/zephyrproject-rtos/sdk-ng) and create a
file `sdk_version` which contains the version string `ZEPHYR_BASE`
uses to find a compatible SDK.

Along with the SDK proper we need to include a number of python
packages required by the build system<sup><a id="fnr.4" class="footref" href="#fn.4" role="doc-backlink">4</a></sup>.

	(define-public zephyr-sdk
	  (package
		(name "zephyr-sdk")
		(version "0.15.0")
		(home-page "https://zephyrproject.org")
		(source (origin (method git-fetch)
						(uri (git-reference
							  (url "https://github.com/zephyrproject-rtos/sdk-ng")
							  (commit "v0.15.0")))
						(file-name (git-file-name name version))
						(sha256 (base32 "04gsvh20y820dkv5lrwppbj7w3wdqvd8hcanm8hl4wi907lwlmwi"))))
		(build-system trivial-build-system)
		(arguments
		 `(#:modules ((guix build union)
					  (guix build utils))
		   #:builder
		   (begin
			 (use-modules (guix build union)
						  (ice-9 match)
						  (guix build utils))
			 (let* ((out (assoc-ref %outputs "out"))
					(cmake-scripts (string-append (assoc-ref %build-inputs "source")
												  "/cmake"))
					(sdk-out (string-append out "/zephyr-sdk-0.15.0")))
			   (mkdir-p out)

			   (match (assoc-remove! %build-inputs "source")
				 (((names . directories) ...)
				  (union-build sdk-out directories)))

			   (copy-recursively cmake-scripts
								 (string-append sdk-out "/cmake"))

			   (with-directory-excursion sdk-out
				 (call-with-output-file "sdk_version"
				   (lambda (p)
					 (format p "0.15.0")))
				 #t)))))
		(propagated-inputs
		 (list
		  arm-zephyr-eabi-toolchain-0.15.0
		  zephyr-binutils
		  python-3
		  python-pyelftools
		  python-pykwalify
		  python-pyyaml
		  python-packaging
		  dtc))
		(native-search-paths
		 (list (search-path-specification
				(variable "ZEPHYR_SDK_INSTALL_DIR")
				(files '("")))))
		(synopsis "SDK for zephyrRTOS")
		(description "zephyr-sdk contains bundles a complete gcc toolchain as well
		as host tools like dtc, openocd, qemu, and required python packages.")
		(license license:apsl2)))


<a id="org9f3c314"></a>

## Testing

In order to test we will need an environment with the SDK installed.
We can take advantage of `guix shell` to avoid installing test
packages into our home environment. This way if it causes problems we
can just exit the shell and try again.

	guix shell -L guix-zephyr zephyr-sdk cmake ninja git

`ZEPHYR_BASE` can be cloned into a temporary workspace to test our
toolchain functionality.<sup><a id="fnr.5" class="footref" href="#fn.5" role="doc-backlink">5</a></sup>

	mkdir /tmp/zephyr-project
	cd /tmp/zephyr-project
	git clone https://github.com/zephyrproject-rtos/zephyr
	export ZEPHYR_BASE=/tmp/zephyr-project/zephyr

In order to build for the test board (k64f in this case) we need to
get a hold of the vendor Hardware Abstraction Layers and CMSIS.<sup><a id="fnr.6" class="footref" href="#fn.6" role="doc-backlink">6</a></sup>

	git clone https://github.com/zephyrproject-rtos/hal_nxp &&
	git clone https://github.com/zephyrproject-rtos/cmsis

To inform the build system about this module we pass it in with
`-DZEPHYR_MODULES=` which is a semicolon separated list of paths
containing a module.yml file.

To build the hello world sample we use the following incantation.

	cmake -Bbuild $ZEPHYR_BASE/samples/hello_world \
		  -GNinja \
		  -DBOARD=frdm_k64f \
		  -DBUILD_VERSION=3.1.0 \
		  -DZEPHYR_MODULES="/tmp/zephyr-project/hal_nxp;/tmp/zephyr-project/cmsis" \
		&& ninja -Cbuild

If everything is set up correctly we will end up with a `./build`
directory with all our build artifacts. The SDK is installed correctly!


# Footnotes

<sup><a id="fn.1" href="#fnr.1">1</a></sup> I'm paraphrasing, but [not by much](https://github.com/zephyrproject-rtos/sdk-ng/tree/v0.14.2#build-process).

<sup><a id="fn.2" href="#fnr.2">2</a></sup> I got them from the SDK configuration scripts on the [sdk github](https://github.com/zephyrproject-rtos/sdk-ng) as well as the
commits to use for each of the tools.

<sup><a id="fn.3" href="#fnr.3">3</a></sup> *Mostly* complete. libstdc++ does not build because
\`arm-zephyr-eabi\` is not \`arm-none-eabi\` so a dynamic link check is
performed/failed. I cannot figure out how crosstool-ng handles this.

<sup><a id="fn.4" href="#fnr.4">4</a></sup> This is no longer the case. Now python packages are provided by the zephyr package
which was not available when this was being developed.

<sup><a id="fn.5" href="#fnr.5">5</a></sup> For now. Eventually we will need to create a package for `zephyr-base` that
our guix zephyr-build-system can use.

<sup><a id="fn.6" href="#fnr.6">6</a></sup> These will also need to become guix packages to allow the build system to compose modules.