July update on qa.guix.gnu.org

[Christopher Baines <mail@cbaines.net>]

Hey,

It's been a while since I sent out the last update on QA things that
I've been working on, there was a small update back in May [1] and one
before that in March [2].

1: https://lists.gnu.org/archive/html/guix-devel/2023-05/msg00142.html
2: https://lists.gnu.org/archive/html/guix-devel/2023-03/msg00222.html

## Changes since the last update

The QA frontpage processing branches is now automatically handled
through creating issues. This seems to be working really well, both as a
process for making it clear what's going on with which branch, and also
feeding in to the automation and testing.

Old builds for patches and branches are now cancelled. This helps the
bordeaux build farm work on the most relevant builds.

Branch comparisons now are done against the merge base, rather than
master, this just helps give relevant feedback.

There's also a new package changes page which is linked to from the
package changes table on the issue and branch pages. This starts to
address the long standing issue where you can't see what the numbers in
the package changes table relate to.

## Current state

While the software is getting better, the service overall is struggling
a bit and that looks to continue for the short term at least.

The queue is very large at [3]. The branches for lots of patches could
have been rebased due to the tex-team-next merge, although it's possible
that the missing i686-linux and armhf-linux derivations recently are
causing some patches to be repeatedly rebased. It'll take a few weeks to
get back to a point where there's data on lots of patch series.

3: https://data.qa.guix.gnu.org/jobs/queue

Discussions ongoing about hosting for data.qa.guix.gnu.org going
forward. So far I've been organising and paying for a virtual machine to
run this, but I'd like to stop by the end of the year.

On the bordeaux build farm side, there's been quite a lot of work
recently on growing the available storage for nars. With the recent work
on branches though, I think it's easier to feel the impact of not having
many available machines. While it's possible to just wait longer for
builds to complete, that has it's own disadvantages. harbourfront (one
of the 3 x86_64 machines is going away at the end of August) which will
further reduce the already limited capacity for x86_64-linux and
i686-linux builds.

I'm in the progress of drafting a blog post about the qa-frontpage.

## Next steps

The qa-frontpage still could do with giving clearer and more actionable
feedback. The other big issue on my mind is getting the data service to
process revisions in less time, as that'll help give quicker feedback
and cope better when rebasing large numbers of patch branches. Apart
from that, I'd like to get back to actually using qa.guix.gnu.org to
review patches, that's a little difficult at the moment as data is
available for very view patches.

Also, as mentioned above, more work needs doing on longer term hosting
for some of the services, as well as growing the bordeaux build farm so
that it can keep up with the growing load.

Let me know if you have any questions or comments!

Thanks,

Chris