Re: Problem with guix copy

[Simon Tournier <zimon.toutoune@gmail.com>]

Hi Emmanuel,

On lun., 25 mars 2024 at 11:13, Emmanuel Medernach <Emmanuel.Medernach@iphc.cnrs.fr> wrote:

>>> Machine_A # guix time-machine -q -C ~/.config/guix/channels.scm -- build --no-grafts tensorflow -d
>>> /gnu/store/s6fm43503ra6yxclqvk80gfiw4zxbp91-tensorflow-2.13.1.drv

>>> Machine_B # guix time-machine -q -C ~/.config/guix/channels.scm -- build --no-grafts tensorflow -d
>>> /gnu/store/s6fm43503ra6yxclqvk80gfiw4zxbp91-tensorflow-2.13.1.drv

[...]

> Machine_A # guix time-machine -q -C ~/.config/guix/channels.scm -- build tensorflow -d --no-grafts
> /gnu/store/s6fm43503ra6yxclqvk80gfiw4zxbp91-tensorflow-2.13.1.drv

Well, If I open the derivation
/gnu/store/s6fm43503ra6yxclqvk80gfiw4zxbp91-tensorflow-2.13.1.drv the
header reads:

--8<---------------cut here---------------start------------->8---
Derive
([("out","/gnu/store/3mq3q11ripx53xg60shbshk7cr470yfx-tensorflow-2.13.1","","")
  ,("python","/gnu/store/qc234r07i3ndgrpa0fdkpkbg4d20h7i1-tensorflow-2.13.1-python","","")]
--8<---------------cut here---------------end--------------->8---

And you get:

> Machine_A # guix time-machine -q -C ~/.config/guix/channels.scm -- build tensorflow --no-grafts
> /gnu/store/3mq3q11ripx53xg60shbshk7cr470yfx-tensorflow-2.13.1
> /gnu/store/qc234r07i3ndgrpa0fdkpkbg4d20h7i1-tensorflow-2.13.1-python

Therefore, all seems consistent.  IIUC, it is also consistent on both
machines (see quote above from earlier message).

It means that the manual “--no-grafts” is OK.


The question is thus, is this derivation:

> Machine_A # guix time-machine -q -C ~/.config/guix/channels.scm -- build tensorflow -d
> /gnu/store/vivxpmrpkh1wq5pnw28vq9c51qd3vwf6-tensorflow-2.13.1.drv

built using
/gnu/store/s6fm43503ra6yxclqvk80gfiw4zxbp91-tensorflow-2.13.1.drv or
using another one?  Anyway. :-)


Now, if I run on my laptop:

--8<---------------cut here---------------start------------->8---
$ guix time-machine -q -C channels.scm -- build tensorflow@2.13 -n
[...]
The following derivations would be built:
  /gnu/store/4g2whbzgn5nqs55x9iqgg23jdapf1srk-tensorflow-2.13.1.drv
  /gnu/store/8jfr297mkpv698d4y1cq270a7f03wyh6-bazel-6.3.2.drv
  /gnu/store/g90vfnx128ifhgy0n1zmlscjrn57l8xm-tensorflow-2.13.1-bazel-deps.tar.xz.drv
  /gnu/store/h3zcigb52z7m6yyjl785w3a5ls1cjjc7-python-jax-0.4.20.drv
  /gnu/store/h9m0ix80glph19k4pfqh130nmkil8f39-python-jaxlib-0.4.20.drv
  /gnu/store/m4icdavpvbryliz34ai8817bb1d3l3xr-python-jaxlib-0.4.20.drv
  /gnu/store/ha0hl2j607g0wanxqbkwrcfy6yz3k2xq-python-jaxlib-0.4.20-bazel-deps.tar.xz.drv
--8<---------------cut here---------------end--------------->8---

the output mentions another no-grafts derivation, i.e.,
/gnu/store/4g2whbzgn5nqs55x9iqgg23jdapf1srk-tensorflow-2.13.1.drv.

This derivation 4g2whbzgn5nqs55x9iqgg23jdapf1srk-tensorflow-2.13.1.drv
is not the derivation for grafting but the derivation for building the
no-grafts.

Well, I think the issue with “guix copy” is coming from something
similar.  There is a mismatch between what is computed. Let give a look.

Let transform the one-line file to multi-lines:

--8<---------------cut here---------------start------------->8---
$ cat /gnu/store/s6fm43503ra6yxclqvk80gfiw4zxbp91-tensorflow-2.13.1.drv | sed 's/,/\n,/g' > /tmp/manual
--8<---------------cut here---------------end--------------->8---

And diff the both files (see complete raw below).  Inline comments…

        2c2
        < ,"/gnu/store/3mq3q11ripx53xg60shbshk7cr470yfx-tensorflow-2.13.1"
        ---
        > ,"/gnu/store/yhr2vxh26acz3pzpjlph4f82ffrw6icv-tensorflow-2.13.1"
        6c6
        < ,"/gnu/store/qc234r07i3ndgrpa0fdkpkbg4d20h7i1-tensorflow-2.13.1-python"
        ---
        > ,"/gnu/store/fyjpsvfdmkdimd1vx6d99ik01z3x3j93-tensorflow-2.13.1-python"

That’s expected.  Because this hash is the result of all the others,
i.e., something that comes next; for instance,

        90,91d89
        < ,("/gnu/store/50vi4ywi79irbg71l814xqkww4k1afh2-python-jaxlib-0.4.20.drv"
        < ,["out"])
        130,131d127
        < ,("/gnu/store/8avai81995rj5h7906nzgyrnrpvqi6if-tensorflow-2.13.1-bazel-deps.tar.xz.drv"
        < ,["out"])
        135a132,133
        > ,("/gnu/store/8jfr297mkpv698d4y1cq270a7f03wyh6-bazel-6.3.2.drv"
        > ,["out"])
        158,159d155
        < ,("/gnu/store/an1hq0y3npd66plwghgnccm4sl7lyvka-bazel-6.3.2.drv"
        < ,["out"])
        229a226,227
        > ,("/gnu/store/g90vfnx128ifhgy0n1zmlscjrn57l8xm-tensorflow-2.13.1-bazel-deps.tar.xz.drv"
        > ,["out"])

Hum, it seems:

 + the order is different;
 + the derivations are also different

Ok, the order might be one the problem.  And that could be a bug for
“guix copy”.  And because the derivations are not the same, then the
item is different.

Let consider tensorflow-2.13.1-bazel-deps.tar which is “fixed-output”:
it means the hash (NAR SHA256) is known beforehand.  Therefore the hash
of the store item should be same.  Well, let open these two derivations;
they both read:

        Derive
        ([("out","/gnu/store/rdpxkb8gl0ym0vrz8viklc0rwnjkn7cl-tensorflow-2.13.1-bazel-deps.tar.xz",
        "sha256","487a1b6685767e8d9fc0a23a2e82d2d7be305e1bd5aeaa4acd1599bc92e3142f")]

It means the derivations are different but their result is the exact
same item.


Well, I will do further tests. :-)


Cheers,
simon


--8<---------------cut here---------------start------------->8---
2c2
< ,"/gnu/store/3mq3q11ripx53xg60shbshk7cr470yfx-tensorflow-2.13.1"
---
> ,"/gnu/store/yhr2vxh26acz3pzpjlph4f82ffrw6icv-tensorflow-2.13.1"
6c6
< ,"/gnu/store/qc234r07i3ndgrpa0fdkpkbg4d20h7i1-tensorflow-2.13.1-python"
---
> ,"/gnu/store/fyjpsvfdmkdimd1vx6d99ik01z3x3j93-tensorflow-2.13.1-python"
90,91d89
< ,("/gnu/store/50vi4ywi79irbg71l814xqkww4k1afh2-python-jaxlib-0.4.20.drv"
< ,["out"])
130,131d127
< ,("/gnu/store/8avai81995rj5h7906nzgyrnrpvqi6if-tensorflow-2.13.1-bazel-deps.tar.xz.drv"
< ,["out"])
135a132,133
> ,("/gnu/store/8jfr297mkpv698d4y1cq270a7f03wyh6-bazel-6.3.2.drv"
> ,["out"])
158,159d155
< ,("/gnu/store/an1hq0y3npd66plwghgnccm4sl7lyvka-bazel-6.3.2.drv"
< ,["out"])
229a226,227
> ,("/gnu/store/g90vfnx128ifhgy0n1zmlscjrn57l8xm-tensorflow-2.13.1-bazel-deps.tar.xz.drv"
> ,["out"])
251a250,251
> ,("/gnu/store/h3zcigb52z7m6yyjl785w3a5ls1cjjc7-python-jax-0.4.20.drv"
> ,["out"])
257a258,259
> ,("/gnu/store/h9m0ix80glph19k4pfqh130nmkil8f39-python-jaxlib-0.4.20.drv"
> ,["out"])
338,339d339
< ,("/gnu/store/lvbjhnrdgl5i5z558b1zpvj4vjqzdm4g-python-jax-0.4.20.drv"
< ,["out"])
519c519
< ,["/gnu/store/r200hqk2jn68qr5wqj6762dpqbns3vqj-tensorflow-2.13.1-builder"
---
> ,["/gnu/store/a5pi7l07irynidrfkiw7x44s8angi2fh-tensorflow-2.13.1-builder"
528c528
< ,"/gnu/store/r200hqk2jn68qr5wqj6762dpqbns3vqj-tensorflow-2.13.1-builder"]
---
> ,"/gnu/store/a5pi7l07irynidrfkiw7x44s8angi2fh-tensorflow-2.13.1-builder"]
530c530
< ,"/gnu/store/3mq3q11ripx53xg60shbshk7cr470yfx-tensorflow-2.13.1")
---
> ,"/gnu/store/yhr2vxh26acz3pzpjlph4f82ffrw6icv-tensorflow-2.13.1")
532c532
< ,"/gnu/store/qc234r07i3ndgrpa0fdkpkbg4d20h7i1-tensorflow-2.13.1-python")])
\ No newline at end of file
---
> ,"/gnu/store/fyjpsvfdmkdimd1vx6d99ik01z3x3j93-tensorflow-2.13.1-python")])
\ No newline at end of file
--8<---------------cut here---------------end--------------->8---