Re: When is check-system run?

[Maxim Cournoyer <maxim.cournoyer@gmail.com>]

Hi Tomas,

Tomas Volf <~@wolfsden.cz> writes:

> Hello,
>
> I would like to inquire regarding system tests.  There is check-system make
> target, however it seems like at least some of them are broken.  Can someone
> check whether that is case even on their machine?  Do they all pass for you?

The system tests are run for new Guix revisions periodically by this job
set: https://ci.guix.gnu.org/jobset/tests

Otherwise we depend on people touching e.g. services running them
manually to ensure they pass.  Accumulated test failures that have
slipped in (we do not have a CI infrastructure gating merges on test
failures yet) should ideally be reported and fixed (thanks for working
on it!).

There is typically an effort to ensure all (or most) tests pass before a
new release is made.

> I already have patches fixing 4 of them, and debugging 5th, but I would like to
> ask when exactly are the those tests run and where are the failures
> reported to?

Thank you!  You can report the failures to bug-guix@gnu.org, if they
weren't already; these are Guix bugs.

> Since the breakage sneaked into the master, I assume they are not run as part of
> the regular patches?  Are there any measures the project is planning to take to
> prevent this from happening again?

It'd be nice to have our CI infrastructure act more like a CI (currently
it's more of a build farm than a CI), but I don't know of anyone working
on it (though Christopher may have long term plans toward that goal with
QA [0]).

[0] qa.guix.gnu.org/

-- 
Thanks,
Maxim