package submission, unit test failures on Guix QA... policies?

package submission, unit test failures on Guix QA... policies?

[Andy Tai]

Hi, I have a patch submission that builds fine (in the GNU Guix sense,
that is, unit tests run and pass) locally but fails on Guix QA due to
unit tests failing.

I suspect that the failures are due to the build machine (or machines)
as plain x86-64 PC (fine as build server(s)) does not have exotic
hardware units like GPU that the package in question is meant to use.

This may become more common as GPU and NPU applications become more popular.

Curious how shall such cases be handled?  Maybe reviewers can build
and run the patch to verify the patch works, so the Guix QA failure
can be overridden?

Re: package submission, unit test failures on Guix QA... policies?

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Andy,

On Fri, Jun 07 2024, Andy Tai wrote:

> Curious how shall such cases be handled?

In your patch, I would disable the unit tests that require special
hardware.

The unit tests are not supposed to run at build time anyway, but Guix
does not have a separate testing infrastructure presently.  One day, our
infrastructure will select tests based on whichever test bed features
are available, like the special hardware you own.

Kind regards
Felix

Re: package submission, unit test failures on Guix QA... policies?

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1111 bytes --]

Andy Tai <atai@atai.org> writes:

> Hi, I have a patch submission that builds fine (in the GNU Guix sense,
> that is, unit tests run and pass) locally but fails on Guix QA due to
> unit tests failing.
>
> I suspect that the failures are due to the build machine (or machines)
> as plain x86-64 PC (fine as build server(s)) does not have exotic
> hardware units like GPU that the package in question is meant to use.
>
> This may become more common as GPU and NPU applications become more popular.
>
> Curious how shall such cases be handled?  Maybe reviewers can build
> and run the patch to verify the patch works, so the Guix QA failure
> can be overridden?

I'm not sure we have great guidance here, but viewing this through the
reproducible builds lens, we want packages to build identically across a
diverse range of setups.

While tests might not affect the outputs, I think the same premise still
holds. The set of tests that are run should be the same regardless of
the machine used.

Have you got some specific examples where you're seeing failing unit
tests, only on headless servers?

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]