bug#32539: Cuirass: a commit that updates doc will produce 86 tests

["Clément Lassieur" <clement@lassieur.org>]

Heya!

Ludovic Courtès <ludo@gnu.org> writes:

> Hi Clément,
>
> Clément Lassieur <clement@lassieur.org> skribis:
>
>> Evaluation 338[1] is triggered by commit 1d8d69c[2], which only updates
>> the documentation.  This evaluation produces 86 useless tests:
>> test.mcron.i686-linux, test.opensmtpd.i686-linux, etc.  I believe those
>> tests are triggered at each evaluation (almost), and it's a considerable
>> waste of resources.
>
> I claim that these tests are useful and that having them under CI is a
> good thing!
>
> AFAICS the result of test.basic.*, test.mcron.*, etc. should be
> unchanged when doc/*.texi are modified; if that’s not the case, we have
> a bug. :-)   So there’s no waste of resources in this case.

My point is that the derivations files are different at each commit,
despite the changes being only about documentation.  It seems to me that
documentation is not an input of the tests, therefore the inputs didn't
change and the test derivations shouldn't change.

> However the tests in (gnu tests install) do entail a full build of Guix
> from the checkout, among other things, at every commit, so they are
> expensive.  They are useful though, so I’d rather keep it this way if
> our build farm can keep up.
>
> Thoughts?
>
> Ludo’.