When is check-system run?

When is check-system run?

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 771 bytes --]

Hello,

I would like to inquire regarding system tests.  There is check-system make
target, however it seems like at least some of them are broken.  Can someone
check whether that is case even on their machine?  Do they all pass for you?

I already have patches fixing 4 of them, and debugging 5th, but I would like to
ask when exactly are the those tests run and where are the failures reported to?
Since the breakage sneaked into the master, I assume they are not run as part of
the regular patches?  Are there any measures the project is planning to take to
prevent this from happening again?

Thank you and have a nice day,
Tomas Volf

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: When is check-system run?

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Tomas,

On Fri, Jan 12 2024, Tomas Volf wrote:

> it seems like at least some of them are broken.  Can someone check
> whether that is case even on their machine?

Yes, some are broken.

> Do they all pass for you?

No.

> when exactly are the those tests run and where are the failures
> reported to?

I believe they run before 'guix pull' is finalized, but I cannot point
to how they are invoked.

So far I have written four of them. (Cachefilesd should pass; the other
three are pending and have not been accepted yet.) Sorry, that's all I
have.

Kind regards
Felix

Re: When is check-system run?

[Maxim Cournoyer]

Hi Tomas,

Tomas Volf <~@wolfsden.cz> writes:

> Hello,
>
> I would like to inquire regarding system tests.  There is check-system make
> target, however it seems like at least some of them are broken.  Can someone
> check whether that is case even on their machine?  Do they all pass for you?

The system tests are run for new Guix revisions periodically by this job
set: https://ci.guix.gnu.org/jobset/tests

Otherwise we depend on people touching e.g. services running them
manually to ensure they pass.  Accumulated test failures that have
slipped in (we do not have a CI infrastructure gating merges on test
failures yet) should ideally be reported and fixed (thanks for working
on it!).

There is typically an effort to ensure all (or most) tests pass before a
new release is made.

> I already have patches fixing 4 of them, and debugging 5th, but I would like to
> ask when exactly are the those tests run and where are the failures
> reported to?

Thank you!  You can report the failures to bug-guix@gnu.org, if they
weren't already; these are Guix bugs.

> Since the breakage sneaked into the master, I assume they are not run as part of
> the regular patches?  Are there any measures the project is planning to take to
> prevent this from happening again?

It'd be nice to have our CI infrastructure act more like a CI (currently
it's more of a build farm than a CI), but I don't know of anyone working
on it (though Christopher may have long term plans toward that goal with
QA [0]).

[0] qa.guix.gnu.org/

-- 
Thanks,
Maxim