Unable to build "Ten Years Reproducibility Challenge" paper

Unable to build "Ten Years Reproducibility Challenge" paper

[Suhail]

[-- Attachment #1: Type: text/plain, Size: 5078 bytes --]

Hi Guix,

I am relatively new to Guix. I recently came across this blog post:
<https://hpc.guix.info/blog/2020/06/reproducible-research-articles-from-source-code-to-pdf/>.

In trying to build and run the evaluation as documented in
<https://gitlab.inria.fr/lcourtes-phd/edcc-2006-redone>, I'm
encountering the following error (both on Guix package on OpenSUSE
distribution as well as on Guix system in a VM):

#+begin_example
  $> guix time-machine -C channels.scm -- build -f guix.scm
  Updating channel 'guix-past' from Git repository at 'https://gitlab.inria.fr/guix-hpc/guix-past.git'...
  Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...

  ...

  building /gnu/store/vsvpi0cy4g3s5w7p9rj3ixnw16jllmr1-guix-config.drv...
  building /gnu/store/2igl2vvjmvg2qwizy5w0gvf63vsvb44x-guix-core.drv...
  building /gnu/store/0y3m9wj8qziblk743fjjxmn1iybm2zif-guix-config-modules.drv...
  building /gnu/store/0ny9586ypfm7d7k9ly3s4i60hb3wczlj-guix-core-modules.drv...
  building /gnu/store/cscfsg5hmznn5rgk3q8affwc5445fqcd-openssl-1.1.1g.tar.xz.drv...
  applying 6 grafts for pango-1.42.4 ...
  building /gnu/store/vvhn7r1343mkyj291h13f3bk29n6m8j6-openssl-1.1.1g.drv...
  / 'check' phasebuilder for `/gnu/store/vvhn7r1343mkyj291h13f3bk29n6m8j6-openssl-1.1.1g.drv' failed with exit code 1
  build of /gnu/store/vvhn7r1343mkyj291h13f3bk29n6m8j6-openssl-1.1.1g.drv failed
  View build log at '/var/log/guix/drvs/vv/hn7r1343mkyj291h13f3bk29n6m8j6-openssl-1.1.1g.drv.gz'.
  applying 7 grafts for graphviz-2.40.1 ...
  cannot build derivation `/gnu/store/8adwgdgaf6xcyhyyc5z9nvrql7bkgvhy-libgit2-1.0.0.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/kml3x6f5arzsqbqcf9n77dpw1mwszybr-guile3.0-git-0.3.0.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/bgyz3mib0bvpmxl1j0jp9x4jqqhw6416-guix-cli.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/cjqbylp554mhwhl8895z5n74q5998waq-guix-extra.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/ql55k991hvz4x9splndhwyzq9apnasig-guix-module-union.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/s9j6wb6jzqhfcwb8isjbalf6msdqq83b-guix-packages-base.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/mf05ss7j1hh06vnrkywkswvlgdzv2i8k-guix-packages.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/x1ci1v82ncj679gipnk3s5sszshl66fi-guix-system-tests.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/pm8y6k5kzcddk170h0biarmqi278m0ww-guix-system.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/nnv3zac8hj40kdz5i4bwwbfzqq5j6bi9-guix-cli-modules.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/2xilwplb6mzqsh4p5yym52f1q4l14lmc-guix-command.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/9g1cw0wxwi0jm7akmrf48hx6rxswx4nn-guix-extra-modules.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/v9zjw4gphv900swc8z450hal3farykwd-guix-packages-base-modules.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/fjqv9nagdnda38l68x6f82qlradx18a7-guix-packages-modules.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/m5z9ba7gra3qml9gzl6vh4d8fcq4qn20-guix-system-modules.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/i37g4wlrgwcaq5v2c4zrkaysfnf95spy-guix-system-tests-modules.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/fp7isyvw7g8kr2vhqgw4p45yckz0srd9-guix-40fd909e3.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/x2fv84h6izjn8sh50nn55i2qp9h3hbzj-profile.drv': 1 dependencies couldn't be built
  guix time-machine: error: build of `/gnu/store/x2fv84h6izjn8sh50nn55i2qp9h3hbzj-profile.drv' failed
#+end_example

As can be seen in the attached build log, it seems some tests for
openssl are failing. I'm speculating that these tests are
non-deterministic and perhaps overly rigid.

Is there a way to tell Guix to either skip the 'check' build-phase for a
specific dependency (or perhaps for the entire build invocation)? Any
guidance?


-- 
Suhail

This email is not an offer capable of acceptance, does not evidence an intention
to enter into an agreement, has no operative effect until a definitive agreement
is signed in writing by both parties, and that no party should act in reliance
on the email or any representations of the sender until a definitive agreement
is signed in writing by both parties.

This email may contain information that is privileged, confidential and/or
exempt from disclosure.  No waiver whatsoever is intended by sending this e-mail
which is intended only for the named recipient(s).  Unauthorized use,
dissemination or copying is prohibited.  If you receive this email in error,
please notify the sender and destroy all copies of this email.

[-- Attachment #2: hn7r1343mkyj291h13f3bk29n6m8j6-openssl-1.1.1g.drv.gz --]
[-- Type: application/x-gzip, Size: 52294 bytes --]

Re: Unable to build "Ten Years Reproducibility Challenge" paper

[Julien Lepiller]

That's openssl failing due to a time-dependent test. Try making your system time something around 2010 and don't let ntpd set it back to today until openssl is built.

You could skip tests, but that would build a different derivation tree, so it might affect reproducibility (probably not): --without-checks=openssl

Le 29 octobre 2023 19:44:49 GMT+01:00, Suhail <suhail@bayesians.ca> a écrit :
>Hi Guix,
>
>I am relatively new to Guix. I recently came across this blog post:
><https://hpc.guix.info/blog/2020/06/reproducible-research-articles-from-source-code-to-pdf/>.
>
>In trying to build and run the evaluation as documented in
><https://gitlab.inria.fr/lcourtes-phd/edcc-2006-redone>, I'm
>encountering the following error (both on Guix package on OpenSUSE
>distribution as well as on Guix system in a VM):
>
>#+begin_example
>  $> guix time-machine -C channels.scm -- build -f guix.scm
>  Updating channel 'guix-past' from Git repository at 'https://gitlab.inria.fr/guix-hpc/guix-past.git'...
>  Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
>
>  ...
>
>  building /gnu/store/vsvpi0cy4g3s5w7p9rj3ixnw16jllmr1-guix-config.drv...
>  building /gnu/store/2igl2vvjmvg2qwizy5w0gvf63vsvb44x-guix-core.drv...
>  building /gnu/store/0y3m9wj8qziblk743fjjxmn1iybm2zif-guix-config-modules.drv...
>  building /gnu/store/0ny9586ypfm7d7k9ly3s4i60hb3wczlj-guix-core-modules.drv...
>  building /gnu/store/cscfsg5hmznn5rgk3q8affwc5445fqcd-openssl-1.1.1g.tar.xz.drv...
>  applying 6 grafts for pango-1.42.4 ...
>  building /gnu/store/vvhn7r1343mkyj291h13f3bk29n6m8j6-openssl-1.1.1g.drv...
>  / 'check' phasebuilder for `/gnu/store/vvhn7r1343mkyj291h13f3bk29n6m8j6-openssl-1.1.1g.drv' failed with exit code 1
>  build of /gnu/store/vvhn7r1343mkyj291h13f3bk29n6m8j6-openssl-1.1.1g.drv failed
>  View build log at '/var/log/guix/drvs/vv/hn7r1343mkyj291h13f3bk29n6m8j6-openssl-1.1.1g.drv.gz'.
>  applying 7 grafts for graphviz-2.40.1 ...
>  cannot build derivation `/gnu/store/8adwgdgaf6xcyhyyc5z9nvrql7bkgvhy-libgit2-1.0.0.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/kml3x6f5arzsqbqcf9n77dpw1mwszybr-guile3.0-git-0.3.0.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/bgyz3mib0bvpmxl1j0jp9x4jqqhw6416-guix-cli.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/cjqbylp554mhwhl8895z5n74q5998waq-guix-extra.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/ql55k991hvz4x9splndhwyzq9apnasig-guix-module-union.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/s9j6wb6jzqhfcwb8isjbalf6msdqq83b-guix-packages-base.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/mf05ss7j1hh06vnrkywkswvlgdzv2i8k-guix-packages.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/x1ci1v82ncj679gipnk3s5sszshl66fi-guix-system-tests.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/pm8y6k5kzcddk170h0biarmqi278m0ww-guix-system.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/nnv3zac8hj40kdz5i4bwwbfzqq5j6bi9-guix-cli-modules.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/2xilwplb6mzqsh4p5yym52f1q4l14lmc-guix-command.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/9g1cw0wxwi0jm7akmrf48hx6rxswx4nn-guix-extra-modules.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/v9zjw4gphv900swc8z450hal3farykwd-guix-packages-base-modules.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/fjqv9nagdnda38l68x6f82qlradx18a7-guix-packages-modules.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/m5z9ba7gra3qml9gzl6vh4d8fcq4qn20-guix-system-modules.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/i37g4wlrgwcaq5v2c4zrkaysfnf95spy-guix-system-tests-modules.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/fp7isyvw7g8kr2vhqgw4p45yckz0srd9-guix-40fd909e3.drv': 1 dependencies couldn't be built
>  cannot build derivation `/gnu/store/x2fv84h6izjn8sh50nn55i2qp9h3hbzj-profile.drv': 1 dependencies couldn't be built
>  guix time-machine: error: build of `/gnu/store/x2fv84h6izjn8sh50nn55i2qp9h3hbzj-profile.drv' failed
>#+end_example
>
>As can be seen in the attached build log, it seems some tests for
>openssl are failing. I'm speculating that these tests are
>non-deterministic and perhaps overly rigid.
>
>Is there a way to tell Guix to either skip the 'check' build-phase for a
>specific dependency (or perhaps for the entire build invocation)? Any
>guidance?
>
>

Re: Unable to build "Ten Years Reproducibility Challenge" paper

[Simon Tournier]

Hi,

Thanks for the attempt and the report.

On dim., 29 oct. 2023 at 18:44, Suhail <suhail@bayesians.ca> wrote:

> <https://hpc.guix.info/blog/2020/06/reproducible-research-articles-from-source-code-to-pdf/>.

[...]

>   $> guix time-machine -C channels.scm -- build -f guix.scm

[...]

> As can be seen in the attached build log, it seems some tests for
> openssl are failing.

Arf, it was still working end of June 2023.  The substitutes is probably
gone and yeah OpenSSL has a time-bomb in its test suite.

Sadly, it is a known issue:

     Note: Although it should technically be possible to travel to such
     an old commit, the ease to do so will largely depend on the
     availability of binary substitutes.  When traveling to a distant
     past, some packages may not easily build from source anymore.  One
     such example are old versions of Python 2 which had time bombs in
     its test suite, in the form of expiring SSL certificates.  This
     particular problem can be worked around by setting the hardware
     clock to a value in the past before attempting the build.

     https://guix.gnu.org/manual/devel/en/guix.html#Invoking-guix-time_002dmachine

Here, we are hitting some unexpected limitations about time-travel. :-)
For an immediate fix, you can do something along these lines:

        sudo timedatectl set-ntp false
        sudo timedatectl set-time '2020-06-23 00:00:00'

or adapt for OpenSuse.  This should make the OpenSSL test suite passes.


For a long-term fix, I have done a systematic review about what happens
in the worst case scenario:

    https://simon.tournier.info/posts/2023-06-23-hackathon-repro.html

And we are not yet bullet-proof.  Work in progress… :-)

Cheers,
simon

Re: Unable to build "Ten Years Reproducibility Challenge" paper

[Felix Lechner via]

Hi everyone,

On Tue, Oct 31 2023, Julien Lepiller wrote:

> You could skip tests, but that would build a different derivation
> tree, so it might affect reproducibility

Those shortfalls are a consequence of our packaging habits. In an ideal
world, would running a test suite ever build a different derivation?

Should it ever affect a package's reproducibility?

Please bear with me as I again belabor the same point without receiving
public support. Building packages should be separate from testing them.

Carthaginem esse delendam.

Kind regards
Felix

Re: Unable to build "Ten Years Reproducibility Challenge" paper

[Suhail]

Julien Lepiller <julien@lepiller.eu> writes:

> That's openssl failing due to a time-dependent test. Try making your system time
> something around 2010 and don't let ntpd set it back to today until openssl is
> built.

Thanks; that worked.

> You could skip tests, but that would build a different derivation tree, so it
> might affect reproducibility (probably not): --without-checks=openssl

Did you mean: --without-tests=openssl

-- 
Suhail

This email is not an offer capable of acceptance, does not evidence an intention
to enter into an agreement, has no operative effect until a definitive agreement
is signed in writing by both parties, and that no party should act in reliance
on the email or any representations of the sender until a definitive agreement
is signed in writing by both parties.

This email may contain information that is privileged, confidential and/or
exempt from disclosure.  No waiver whatsoever is intended by sending this e-mail
which is intended only for the named recipient(s).  Unauthorized use,
dissemination or copying is prohibited.  If you receive this email in error,
please notify the sender and destroy all copies of this email.

Re: Unable to build "Ten Years Reproducibility Challenge" paper

[Suhail]

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Sadly, it is a known issue:
>
> ...
>      https://guix.gnu.org/manual/devel/en/guix.html#Invoking-guix-time_002dmachine

Thank you for the reference; much appreciated!

> For an immediate fix, you can do something along these lines:
>
>         sudo timedatectl set-ntp false
>         sudo timedatectl set-time '2020-06-23 00:00:00'

Thank you; that resolved the issue.

> For a long-term fix, I have done a systematic review about what
> happens in the worst case scenario:
>
>     https://simon.tournier.info/posts/2023-06-23-hackathon-repro.html

That was an insightful read. Are there related discussions/plans to
better address some of the issues you point out in the post (over and
above those that are linked from the post)?

-- 
Suhail

This email is not an offer capable of acceptance, does not evidence an intention
to enter into an agreement, has no operative effect until a definitive agreement
is signed in writing by both parties, and that no party should act in reliance
on the email or any representations of the sender until a definitive agreement
is signed in writing by both parties.

This email may contain information that is privileged, confidential and/or
exempt from disclosure.  No waiver whatsoever is intended by sending this e-mail
which is intended only for the named recipient(s).  Unauthorized use,
dissemination or copying is prohibited.  If you receive this email in error,
please notify the sender and destroy all copies of this email.

Re: Unable to build "Ten Years Reproducibility Challenge" paper

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1345 bytes --]

On 2023-10-31, Felix Lechner via wrote:
> On Tue, Oct 31 2023, Julien Lepiller wrote:
>
>> You could skip tests, but that would build a different derivation
>> tree, so it might affect reproducibility
>
> Those shortfalls are a consequence of our packaging habits. In an ideal
> world, would running a test suite ever build a different derivation?

Ideally, no... and there are some things in a package, such as synopsis
and description, which do not affect the build; I suspect it would take
considerably more fiddling to exclude tests derivation hash
calculation...


> Should it ever affect a package's reproducibility?

Should build-time tests affect reproducibility? No, surely not!

Do they in practice? Most definitely yes. :(


> Please bear with me as I again belabor the same point without receiving
> public support. Building packages should be separate from testing them.

In general, I agree... sort of.

I do see value in build-time tests preventing a build from
succeeding... being a way to ensure that a broken build does not
actually get distributed.

You could completely separate out the tests, and set up some other
mechanism to prevent broken things from getting distributed, but that is
considerably more complicated.

I guess the question comes down to which corner-cases do you choose to
cater to?


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Unable to build "Ten Years Reproducibility Challenge" paper

[Simon Tournier]

Hi,

On Wed, 01 Nov 2023 at 17:06, Suhail <suhail@bayesians.ca> wrote:

>                              Are there related discussions/plans to
> better address some of the issues you point out in the post (over and
> above those that are linked from the post)?

The current work in progress is to improve the coverage by Software
Heritage; more robustness against vanishing source code.

About others as time-bomb and bootstrap, that’s the next steps. :-)

Cheers,
simon

Re: Unable to build "Ten Years Reproducibility Challenge" paper

[Simon Tournier]

Hi,

On Wed, 01 Nov 2023 at 11:09, Vagrant Cascadian <vagrant@reproducible-builds.org> wrote:

>> Please bear with me as I again belabor the same point without receiving
>> public support. Building packages should be separate from testing them.
>
> In general, I agree... sort of.
>
> I do see value in build-time tests preventing a build from
> succeeding... being a way to ensure that a broken build does not
> actually get distributed.
>
> You could completely separate out the tests, and set up some other
> mechanism to prevent broken things from getting distributed, but that is
> considerably more complicated.

I think the complication starts before. :-)

Well, hoping to not misunderstand something or miss a point.

Currently, the tests are one among many other phases of the build
system.  Therefore, this would need to be extracted as a separated
derivation.  Somehow, there is 3 derivations involved when building a
package: 

 + fetch source
 + run build system
 + graft (optionally)

And somehow you would like to split “run build system” (./configure &&
make && make check)) as two derivations:

 + run build (./configure && make)
 + test (make check)

If we are here, we could make all the phases as separated derivations.
A corollary is that a failure about one phases step would not require to
redo all the previous steps.

Well, if my understanding is correct, separating the tests would be a
piece of work, I guess. :-)

Cheers,
simon