Re: Turning off tests leads to a different store item

[Csepp <raingloom@riseup.net>]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hi,
>
> Suhail <suhail@bayesians.ca> writes:
>
>> Felix Lechner via <help-guix@gnu.org> writes:
>>
>>> On Tue, Oct 31 2023, Julien Lepiller wrote:
>>>
>>>> You could skip tests, but that would build a different derivation
>>>> tree, so it might affect reproducibility
>>>
>>> Those shortfalls are a consequence of our packaging habits. In an ideal
>>> world, would running a test suite ever build a different derivation?
>
> You seem to be confusing 'derivation', which is the build recipe
> description that a package "compiles" to, with the package output, the
> result of executing the derivation.
>
> While I wouldn't expect the package output to change unless tests are
> installed when disabling the test suite, the derivation changes to
> account for the different build arguments used (e.g. that #:tests? #t vs
> #:tests? #f).
>
> Running tests *may* cause the output derivation to be different.  I
> remember a bug in Python that would cause some time stamps to be
> embedded in the .pyc when running the test suite (!).  I think it's
> safer to assume that a derivation with test enabled is not the same as a
> derivation with tests disabled.
>
> There was the "Intensional Store" RFC in Nix that tried to address that
> [0]; it may be worth reading but it looks like it comes with a lot of
> caveats and complexity.
>
> [0]  https://github.com/NixOS/rfcs/pull/17

Another way would be to run tests separately, but that might be even
more complicated.
I had the idea of simply exporting the build directory into a separate
output, or maybe only that would be an output and the regular package
just takes that as an input and only runs the install step, and the
test "package" just runs the test target.