On 2023-10-31, Felix Lechner via wrote:
> On Tue, Oct 31 2023, Julien Lepiller wrote:
>
>> You could skip tests, but that would build a different derivation
>> tree, so it might affect reproducibility
>
> Those shortfalls are a consequence of our packaging habits. In an ideal
> world, would running a test suite ever build a different derivation?

Ideally, no... and there are some things in a package, such as synopsis
and description, which do not affect the build; I suspect it would take
considerably more fiddling to exclude tests derivation hash
calculation...


> Should it ever affect a package's reproducibility?

Should build-time tests affect reproducibility? No, surely not!

Do they in practice? Most definitely yes. :(


> Please bear with me as I again belabor the same point without receiving
> public support. Building packages should be separate from testing them.

In general, I agree... sort of.

I do see value in build-time tests preventing a build from
succeeding... being a way to ensure that a broken build does not
actually get distributed.

You could completely separate out the tests, and set up some other
mechanism to prevent broken things from getting distributed, but that is
considerably more complicated.

I guess the question comes down to which corner-cases do you choose to
cater to?


live well,
  vagrant