Mathieu Othacehe <othacehe@gnu.org> writes:

>> I think trying to change up how branches (staging/core-updates) are
>> tested is a good place to start. The concrete change I'm proposing is to
>> use an instance of the Guix Data Service plus an instance of the Guix
>> Build Coordinator to do the testing and builds, rather than Cuirass on
>> ci.guix.gnu.org which is the current approach.
>>
>> The main advantages of that would be the comparison support from the
>> Guix Data Service, and the build performance and reliability that the
>> Guix Build Coordinator brings. The main disadvantage is probably the
>> lack of an admin like interface similar to that of Cuirass (I think this
>> can be remedied in the medium term though).
>
> We indeed desperately need some more automation. For each new patch
> series, it would be great to have the following information:
>
> * Status of the linter.
> * Status of the depending derivations.
> * Status of the unit tests (in the tests/ directory).
> * Status of the system tests (in the gnu/tests/ directory).
>
> I would like to stay focused on the existing, well adopted solutions and
> build upon them.  With Cuirass we already have most of the machinery to
> provide those information.

I was thinking of using Cuirass for building derivations when testing
patches, but I gave up on that approach back in 2019 after trying to use
it (I discussed trying to use it here [1]).

1: https://lists.gnu.org/archive/html/guix-devel/2019-03/msg00010.html

I was specifically thinking about testing patches when I initially
designed both the Guix Data Service and Guix Build Coordinator. For me
at least, the focus has been on this direction for the last ~3 years.

I realise that Cuirass now has some of the functionality that the Guix
Data Service was written to provide, like tracking all
packages/derivations in each revision. But from my perspective, Cuirass
still lacks key things like being able to compare two revisions and
tracking lint warnings.

There's also things like testing derivations on different hardware,
regularly testing fixed output derivations and automatically retrying
failed builds that I think the Guix Build Coordinator is better setup to
do compared to Cuirass.

But this feedback is why I started this thread. I don't see the same
option as was found for improving substitutes by setting up a new
substitute server using the Guix Data Service and the Guix Build
Coordinator. There's a much stronger need to have one approach as a
project for testing changes, and if using the Guix Data Service and Guix
Build Coordinator isn't looking like a convincing option at this point,
that's better to know now, compared to later when more time and effort
has been put in.

Thanks,

Chris