bug#42009: package.cache not deterministic

[zimoun <zimon.toutoune@gmail.com>]

Hi Ludo,

On Sun, 28 Jun 2020 at 22:29, Ludovic Courtès <ludo@gnu.org> wrote:

> Most likely the problem with non-reproducible .go files is that the fix
> for <https://bugs.gnu.org/20272> was incomplete.  In particular, I think
> that gensyms are not reproducible when building things in parallel,
> because the gensym depends on what’s loaded vs. interpreted.

Thank you for the pointer.

How can I test the "hypothesis" of "building things in parallel"?

--8<---------------cut here---------------start------------->8---
guix describe -f channels > /tmp/chan.scm
guix pull -C /tmp/chan.scm --cores=1 -p /tmp/repull1

guix build --check --no-grafts --cores=1 \
     $(guix gc --derivers \
            $(readlink -f /tmp/repull1/lib/guix/package.cache))
The following profile hook will be built:
   /gnu/store/qbrgxbnx0hi13xm36a6a0zijzc1rcz22-guix-package-cache.drv
building package cache...
(repl-version 0 1 1)
Generating package cache for '/gnu/store/bgqy3mfpzbpyz3pysqxzkpch39q98yv3-profile'...
(values (value "/gnu/store/15nnwjqrmh5w9hqy9yp4ycxsyfbsr0wi-guix-package-cache/lib/guix/package.cache"))
guix build: error: derivation `/gnu/store/qbrgxbnx0hi13xm36a6a0zijzc1rcz22-guix-package-cache.drv' may not be deterministic: output `/gnu/store/15nnwjqrmh5w9hqy9yp4ycxsyfbsr0wi-guix-package-cache' differs
--8<---------------cut here---------------end--------------->8---

BTW, I do not understand why the derivations have different hashes,
containing derivations with different hashes and more importantly, why
it is not the same order.

--8<---------------cut here---------------start------------->8---
guix gc --derivers $(readlink -f ~/.config/guix/current/lib/guix/package.cache)
/gnu/store/0pmc85ni7zsd5jrflb0prrj7bhvn1m1y-guix-package-cache.drv

cat $(guix gc --derivers $(readlink -f ~/.config/guix/current/lib/guix/package.cache))
Derive
([("out","/gnu/store/pfpbh4v1m2dgn9dwiz6rsbqgx8lmd3ms-guix-package-cache","","")]
 ,[("/gnu/store/3pkfaqkdkaqy8khsfbsl0si3r9mydygl-profile.drv",["out"])
   ,("/gnu/store/nih4g42d2da8p2b5dmxqb081bbpv9ax4-inferior-script.scm.drv",["out"])
   ,("/gnu/store/x32cnfkd50fnxs10xp1jdn24h7ai2gxr-guile-3.0.2.drv",["out"])]
 ,["/gnu/store/50h7d8cx9k28gdbdzc9y615d1564m8ia-guix-package-cache-builder"]
 ,"x86_64-linux","/gnu/store/0m0vd873jp61lcm4xa3ljdgx381qa782-guile-3.0.2/bin/guile",["--no-auto-compile","/gnu/store/50h7d8cx9k28gdbdzc9y615d1564m8ia-guix-package-cache-builder"]
 ,[("guix properties","((type . profile-hook) (hook . package-cache))")
   ,("out","/gnu/store/pfpbh4v1m2dgn9dwiz6rsbqgx8lmd3ms-guix-package-cache")
   ,("preferLocalBuild","1")])
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
guix gc --derivers $(readlink -f /tmp/repull1/lib/guix/package.cache)
/gnu/store/qbrgxbnx0hi13xm36a6a0zijzc1rcz22-guix-package-cache.drv

cat $(guix gc --derivers $(readlink -f /tmp/repull1/lib/guix/package.cache))
Derive
([("out","/gnu/store/15nnwjqrmh5w9hqy9yp4ycxsyfbsr0wi-guix-package-cache","","")]
 ,[("/gnu/store/b4dcaccqli2zdfalrn0lc0cz94gd80sk-inferior-script.scm.drv",["out"])
   ,("/gnu/store/hm03mwl234lw43ivx33nsap0j4pjwqjp-profile.drv",["out"])
   ,("/gnu/store/x32cnfkd50fnxs10xp1jdn24h7ai2gxr-guile-3.0.2.drv",["out"])]
 ,["/gnu/store/251jkjnw9zza2zwr1k45x1049d1axl5q-guix-package-cache-builder"]
 ,"x86_64-linux","/gnu/store/0m0vd873jp61lcm4xa3ljdgx381qa782-guile-3.0.2/bin/guile",["--no-auto-compile","/gnu/store/251jkjnw9zza2zwr1k45x1049d1axl5q-guix-package-cache-builder"]
 ,[("guix properties","((type . profile-hook) (hook . package-cache))")
   ,("out","/gnu/store/15nnwjqrmh5w9hqy9yp4ycxsyfbsr0wi-guix-package-cache")
   ,("preferLocalBuild","1")])
--8<---------------cut here---------------end--------------->8---

And I am confused because if I repull again with '--cores=2', then,

--8<---------------cut here---------------start------------->8---
/tmp/repull1/bin/guix pull -C /tmp/chan.scm --cores=2 -p /tmp/repull2
md5sum \
  $(readlink -f /tmp/repull2/lib/guix/package.cache) \
  $(readlink -f /tmp/repull2/lib/guix/package.cache)
75f6feb9f52c312cc9cc8f73534926ba  /gnu/store/15nnwjqrmh5w9hqy9yp4ycxsyfbsr0wi-guix-package-cache/lib/guix/package.cache
75f6feb9f52c312cc9cc8f73534926ba  /gnu/store/15nnwjqrmh5w9hqy9yp4ycxsyfbsr0wi-guix-package-cache/lib/guix/package.cache
--8<---------------cut here---------------end--------------->8---

But '--check' fails in all cases.  What do I miss?


All the best,
simon