bug#58419: Grafting order depends on store connection state

bug#58419: Grafting order depends on store connection state

[Josselin Poiret via Bug reports for GNU Guix]

Hi,

Someone reported yesterday on IRC [1] that they didn't get the same
canonical path for the pass (from `password-store`) binary if they built
it directly or in a profile with `fontconfig` added.  I managed to
isolate the issue down to the following:

--8<---------------cut here---------------start------------->8---
(let ((right (with-store store (run-with-store store (package->derivation
                                                      (specification->package
                                                       "password-store")))))
      (wrong (with-store store (run-with-store store (mbegin %store-monad
                                                       (package->derivation
                                                        (specification->package
                                                         "texlive-bin"))
                                                       (package->derivation
                                                        (specification->package
                                                         "password-store")))))))
  (pk right)
  (pk wrong))
--8<---------------cut here---------------end--------------->8---

Both derivations differ even though they ideally should be identical,
apparently git doesn't appear in the same place in the grafting
derivation.

I've tried to debug the grafting code but to no avail yet.  Does anyone
have any idea?

[1] https://logs.guix.gnu.org/guix/2022-10-09.log#203800

-- 
Josselin Poiret

bug#58419: Grafting order depends on store connection state

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1032 bytes --]

Hi,

Josselin Poiret <dev@jpoiret.xyz> skribis:

> (let ((right (with-store store (run-with-store store (package->derivation
>                                                       (specification->package
>                                                        "password-store")))))
>       (wrong (with-store store (run-with-store store (mbegin %store-monad
>                                                        (package->derivation
>                                                         (specification->package
>                                                          "texlive-bin"))
>                                                        (package->derivation
>                                                         (specification->package
>                                                          "password-store")))))))
>   (pk right)
>   (pk wrong))
>
> Both derivations differ even though they ideally should be identical,
> apparently git doesn't appear in the same place in the grafting
> derivation.

Right:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 2813 bytes --]

--- #<buffer a5dybalrk4xh8wfh2szja17g70x1d7l4-password-store-1.7.4-guile-builder>
+++ #<buffer fb7dvwb1v8bqyl1dspcq7db4qgw4qnm8-password-store-1.7.4-guile-builder>
@@ -19,17 +19,17 @@
       ("x" . "/gnu/store/3bpq5knfvzhxhqfwzqm9br917nz7r0yp-gnupg-2.2.32")
       ("x" . "/gnu/store/31ffp5lszf1g7h1zw750w621cm14hxlr-util-linux-2.37.2")
       ("x" . "/gnu/store/zpw3l0y7sq3ag3fmq001x22bdpalw1fy-xclip-0.13")
-      ("x" . "/gnu/store/f686n3snbkbbf41g7hqyb75dymnckr3z-git-2.37.3")
       ("x" . "/gnu/store/g8qm4vq1f9v81zg8aazkiaf1j3wb8w0s-dmenu-5.1")
       ("x" . "/gnu/store/mk4514a1rjf6mqp0z46kzh80z7j1mhbs-xdotool-3.20211022.1")
       ("x" . "/gnu/store/vmwvxj3ksnmck2cfwsgm9dfi9n41050x-wl-clipboard-2.0.0")
+      ("x" . "/gnu/store/f686n3snbkbbf41g7hqyb75dymnckr3z-git-2.37.3")
       ("x" . "/gnu/store/0jlw8kk0ll25lzbz939jaz4sbfkr8gqj-gnupg-2.2.32")
       ("x" . "/gnu/store/a8k9s0wpf0f3l7nwsscjhnbs5wrn2y1q-util-linux-2.37.2")
       ("x" . "/gnu/store/4vh3qdhsq6misl3vvgm39zdh4sflz4s0-xclip-0.13")
-      ("x" . "/gnu/store/svj9wb4jcb701g3fjf0cmi87rv85sx0x-git-2.37.3")
       ("x" . "/gnu/store/h647qh34g8afyy99gbkngavvlm2p14vn-dmenu-5.1")
       ("x" . "/gnu/store/n6gsqfcc51m4flr21p8szzic5yh1fpfb-xdotool-3.20211022.1")
-      ("x" . "/gnu/store/k28gncxkgxy3hn8qzwylsazc00pwr71s-wl-clipboard-2.0.0"))))
+      ("x" . "/gnu/store/k28gncxkgxy3hn8qzwylsazc00pwr71s-wl-clipboard-2.0.0")
+      ("x" . "/gnu/store/svj9wb4jcb701g3fjf0cmi87rv85sx0x-git-2.37.3"))))
   (unsetenv "GUILE_LOAD_COMPILED_PATH")
   (unsetenv "LD_LIBRARY_PATH"))
 (exit
@@ -44,10 +44,10 @@
 	   (("/gnu/store/3bpq5knfvzhxhqfwzqm9br917nz7r0yp-gnupg-2.2.32" . "/gnu/store/0jlw8kk0ll25lzbz939jaz4sbfkr8gqj-gnupg-2.2.32")
 	    ("/gnu/store/31ffp5lszf1g7h1zw750w621cm14hxlr-util-linux-2.37.2" . "/gnu/store/a8k9s0wpf0f3l7nwsscjhnbs5wrn2y1q-util-linux-2.37.2")
 	    ("/gnu/store/zpw3l0y7sq3ag3fmq001x22bdpalw1fy-xclip-0.13" . "/gnu/store/4vh3qdhsq6misl3vvgm39zdh4sflz4s0-xclip-0.13")
-	    ("/gnu/store/f686n3snbkbbf41g7hqyb75dymnckr3z-git-2.37.3" . "/gnu/store/svj9wb4jcb701g3fjf0cmi87rv85sx0x-git-2.37.3")
 	    ("/gnu/store/g8qm4vq1f9v81zg8aazkiaf1j3wb8w0s-dmenu-5.1" . "/gnu/store/h647qh34g8afyy99gbkngavvlm2p14vn-dmenu-5.1")
 	    ("/gnu/store/mk4514a1rjf6mqp0z46kzh80z7j1mhbs-xdotool-3.20211022.1" . "/gnu/store/n6gsqfcc51m4flr21p8szzic5yh1fpfb-xdotool-3.20211022.1")
-	    ("/gnu/store/vmwvxj3ksnmck2cfwsgm9dfi9n41050x-wl-clipboard-2.0.0" . "/gnu/store/k28gncxkgxy3hn8qzwylsazc00pwr71s-wl-clipboard-2.0.0")))
+	    ("/gnu/store/vmwvxj3ksnmck2cfwsgm9dfi9n41050x-wl-clipboard-2.0.0" . "/gnu/store/k28gncxkgxy3hn8qzwylsazc00pwr71s-wl-clipboard-2.0.0")
+	    ("/gnu/store/f686n3snbkbbf41g7hqyb75dymnckr3z-git-2.37.3" . "/gnu/store/svj9wb4jcb701g3fjf0cmi87rv85sx0x-git-2.37.3")))
 	  (map
 	   (match-lambda
 	     ((name . file)

[-- Attachment #3: Type: text/plain, Size: 512 bytes --]


If we squint a bit, we realize it’s the same thing but in a different
order, which is good news: it’s functionally equivalent.

The downside is obvious: it’s stupidly non-deterministic, and we can end
up building the same grafts multiple times.

The order differs in two places: in the definition of ‘%build-inputs’,
and in the definition of the ‘mapping’ variable.  This can be solved by
sorting things in the right place, but that needs some thought.

To be continued…

Ludo’.

bug#58419: Grafting order depends on store connection state

[Ludovic Courtès]

Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

> If we squint a bit, we realize it’s the same thing but in a different
> order, which is good news: it’s functionally equivalent.
>
> The downside is obvious: it’s stupidly non-deterministic, and we can end
> up building the same grafts multiple times.
>
> The order differs in two places: in the definition of ‘%build-inputs’,
> and in the definition of the ‘mapping’ variable.  This can be solved by
> sorting things in the right place, but that needs some thought.

I posted a patch series that fixes this as a side-effect of switching to
‘gexp->derivation’:

  https://issues.guix.gnu.org/58579

That eliminates ‘%build-inputs’, which was one source of differences,
and the ‘mapping’ variable is now populated in a deterministic fashion,
though I must say it’s not entirely clear to me why this part has
changed.

Feedback welcome!

Ludo’.