Re: The Guix Build Coordinator in 2021

[Mathieu Othacehe <othacehe@gnu.org>]

Hello Chris,

> Near the beginning of 2020, things changed such that I suddenly had some
> time, and some of that time I spend putting idea's I'd had for a while
> around building derivations, including across multiple machines, in to
> practice [1].

With the Guix Build Coordinator, you made an impressive work, probably
based on the observation that the "guix-daemon" and its offloading
mechanism was too limited.

We are now in a situation where our continuous integration system,
while performing better and better is getting out of hand. Here are the
different software I'm keeping track of:

* Cuirass, deployed on ci.guix.gnu.org
* The Guix Build Coordinator, deployed on guix.cbaines.net
* The Guix Data Service, deployed on data.guix.gnu.org
* Patchwork, deployed on patchwork.cbaines.net

All those services have databases, using different DBMS on different
servers. Those databases are sometimes overlapping, in the same way as
some of the features of those software.

In particular I feel that what's implemented in the Guix Build
Coordinator can be seen as a subset of Cuirass functionalities. As you
know, I'm reluctant to the idea of connecting Cuirass to the Guix Build
Coordinator, because most of Cuirass PostgreSQL database content would
be duplicated in the SQLite database of the GBC.

On the other hand, maintaining those two software in separate ways seems
like a huge waste of time given the very limited number of people
contributing the maintenance of the CI system.

Furthermore, some of the features we are implementing here, should be
part of the "guix-daemon" itself, which makes me think that we should
not place too much effort in their development.

My proposition would be to make a listing of both Cuirass and the GBC
features, and see how we could merge them. By maintaining a single
software, with a single database, running on the same server, we could
spare some efforts, and quickly converge towards a better CI.

The new Cuirass architecture and the switch to PostgreSQL, make the
software way more modular, and should allow us to add new
functionalities without too much trouble.

What do you think of that proposition?

Thanks,

Mathieu