Hey,

This is probably something I haven't spoken enough about, but my test
site (guix.cbaines.net) for building and providing substitutes has now
got to a point where I've tested most things I set out to test.

In particular, it's building the following things:

 - packages for:
   - x86_64-linux
   - i686-linux
   - aarch64-linux
   - armhf-linux
   - i586-gnu

 - cross build packages:
   - x86_64-linux -> arm-linux-gnueabihf
   - x86_64-linux -> aarch64-linux-gnu
   - x86_64-linux -> arm-linux-gnueabihf

 - channel instance derivations (guix pull)
   - x86_64-linux
   - i686-linux
   - aarch64-linux

I can't remember when this started, but I think it was mid 2020,
starting with a subset of the above, and I gradually got it building
more things.

Recent additions have been i586-gnu (the Hurd) support, as well as
mixing QEMU and native/compatible builds for aarch64-linux and
armhf-linux.

One of the things I wanted to try out with the Guix Build Coordinator
was retrying builds on failure, and I think that's a big part of why
guix.cbaines.net often will have higher substitute availability than
ci.guix.gnu.org.

I wanted to build on this to try mixing builds through QEMU for
aarch64-linux and armhf-linux with builds taking place on aarch64-linux
(with hardware that has compatibility with armhf). I don't think using
QEMU is a goal, as I think using QEMU can introduce problems, but
suplementing some ARM hardware with x86_64+QEMU can be useful for
building things for substitutes.

As the Guix Build Coordinator has the ability to tag builds and agents,
and match up tags so that specific builds only get allocated to specific
agents, this allows for spotting when aarch64-linux or armhf-linux
builds fail, and scheduling retry builds that are guarrenteed to take
place without using QEMU. Like the general approach of retrying failing
builds, this reduces the chance that a substitute won't be available
because of a specific failure, like using QEMU.

I think overall this experiment has been a success, specifically on the
aarch64-linux and armhf-linux builds, I think most things that can be
built, have been built (although I don't have a rigerous way of checking
this).

Back at the start of 2020, I think there was some optimism about using
the Guix Build Coordinator for building substitutes for ci.guix.gnu.org,
which hasn't happened yet, but I'll send a separate email about that.

Thanks,

Chris