system/package testing?

system/package testing?

[Simon Josefsson via Development of GNU Guix and the GNU System distribution.]

[-- Attachment #1: Type: text/plain, Size: 1265 bytes --]

Hi

Is there any infrastructure to run a system-wide test suite, or at least
a per-package test suite?  I'm not thinking of 'make check' that is run
during build, but of something similar to Debian's debci/autopkgtest to
test that an installed package works as expected (which may include
running the same self-checks as 'make check' would, but on the installed
package).

When designed for Guix, it could be even better than debci/autopkgtest,
it could be something the Guix user could invoke on an installed system
to verify that all the installed packages passes self-tests -- I miss
that functionality from debci/autopkgtest.

I'm considering packaging libgssglue for Guix and do some Kerberos
related work on other related packages, but having confidence of any
changes without proper testing of the resulting installed package is
fragile.  I'm sure other more complex parts of Guix could benefit from
similar testing too.  A system-wide testing of the published QEMU image
would be useful, to gain confidence that we don't introduce regressions
into a future release of the QEMU image.

Maybe this already exists, and if so my question/request should be
converted into a request for publicity around the feature and a pointer
to its documentation. :)

/Simon

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: system/package testing?

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 2410 bytes --]



On 06-10-2022 17:51, Simon Josefsson via Development of GNU Guix and the 
GNU System distribution. wrote:
> Hi
> 
> Is there any infrastructure to run a system-wide test suite, or at least
> a per-package test suite?

There's "make check-system", for system services (and implicitly, for 
the software behind those services), and "make check", for individual 
packages.

>  I'm not thinking of 'make check' that is run
> during build, but of something similar to Debian's debci/autopkgtest to
> test that an installed package works as expected (which may include
> running the same self-checks as 'make check' would, but on the installed
> package).

I do not understand the difference -- what's installed, is just what's 
built previously, on which "make check" is run, no?

If there are some additional tests beyond "make check", they can be run 
from the check phase (and if they require the package to be installed, 
the phase can be moved after 'install', as is done sometimes already for 
some Python packages).

> When designed for Guix, it could be even better than debci/autopkgtest,
> it could be something the Guix user could invoke on an installed system
> to verify that all the installed packages passes self-tests

This is done automatically, during the build.  If there is a concern it 
depends on the kernel version or CPU model, "guix build --check" can be 
used to redo the build.

  -- I miss
> that functionality from debci/autopkgtest.
> 
> I'm considering packaging libgssglue for Guix and do some Kerberos
> related work on other related packages, but having confidence of any
> changes without proper testing of the resulting installed package is
> fragile.

If you have additional tests for libgssglue, you can submit them 
upstream, or if they are somehow inappropriate for upstream, do them in 
a check phase.

> I'm sure other more complex parts of Guix could benefit from
> similar testing too.  A system-wide testing of the published QEMU image
> would be useful, to gain confidence that we don't introduce regressions
> into a future release of the QEMU image.

See: "make check-system".

> Maybe this already exists, and if so my question/request should be
> converted into a request for publicity around the feature and a pointer
> to its documentation. :)

Info: (guix)Running the Test Suite

Greetings,
Maxime

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: system/package testing?

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 136 bytes --]



On 06-10-2022 20:34, Maxime Devos wrote:
>   -- I miss

Nevermind this, just some writing error during composing the message

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]