bug#47115: Redundant library grafts leads to breakage (was: Failure building grub-img.png when reconfiguring)

[Mark H Weaver <mhw@netris.org>]

retitle 47115 Redundant library grafts leads to breakage
thanks

Hi,

I looked a bit deeper, and now I think I finally know what's going on.
It turns out that the grafting process is creating two redundant
variants of the replacement guile-cairo.

All of the relevant information is in
/gnu/store/07xw2pp63xin4c4y8ndrcdn3n8z1vmx2-grub-image.png.drv and its
dependents, which fails to build if deduplication is disabled.

If you look through the output of "guix size
/gnu/store/07xw2pp63xin4c4y8ndrcdn3n8z1vmx2-grub-image.png.drv", you'll
find three distinct guile-cairo derivations:

(1) /gnu/store/vz4yw7zkm73diy95mdzywgixal3nf2s2-guile-cairo-1.11.2.drv,
    => /gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2
    (the original, ungrafted, cairo)

(2) /gnu/store/rcl324yiq7a56rwkqwgqx097dwc5mgni-guile-cairo-1.11.2.drv,
    => /gnu/store/vjn7ygzzqshvsfzck8hq5lp5pfrr2xp5-guile-cairo-1.11.2
    (the first graft)

(3) /gnu/store/9mha4bzbji8iql50prmq9br4j1c51sjn-guile-cairo-1.11.2.drv,
    => /gnu/store/j69k9n0g3h9ppqi7dmqypwy3lrhxvb97-guile-cairo-1.11.2
    (the second graft)

In the 'guile-builder' files referenced from the two graft derivations,
we see that they have the same inputs and perform the same rewrites, but
list them in a different order.  Graft 1 has this guile-builder:

--8<---------------cut here---------------start------------->8---
(begin
  (use-modules (guix build graft) (guix build utils) (ice-9 match))
  (define %output (getenv "out"))
  (define %outputs (map (lambda (o) (cons o (getenv o))) (quote ("out"))))
  (define %build-inputs
    (quote
     (("x" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2")
      ("x" . "/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10")
      ("x" . "/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0")
      ("x" . "/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10")
      ("x" . "/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4")
      ("x" . "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12")
      ("x" . "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0")
      ("x" . "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10")
      ("x" . "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4"))))
  (unsetenv "GUILE_LOAD_COMPILED_PATH")
  (unsetenv "LD_LIBRARY_PATH"))
(exit
 (begin
   (let* ((old-outputs
           (quote
            (("out" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2"))))
          (mapping
           (append (quote
                    (("/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10"
                      . "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12")
                     ("/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0"
                      . "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0")
                     ("/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10"
                      . "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10")
                     ("/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4"
                      . "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4")))
                   (map (match-lambda ((name . file)
                                       (cons (assoc-ref old-outputs name) file)))
                        %outputs))))
     (graft old-outputs %outputs mapping))))
--8<---------------cut here---------------end--------------->8---

Graft 2 has this guile-builder:

--8<---------------cut here---------------start------------->8---
(begin
  (use-modules (guix build graft) (guix build utils) (ice-9 match))
  (define %output (getenv "out"))
  (define %outputs (map (lambda (o) (cons o (getenv o))) (quote ("out"))))
  (define %build-inputs
    (quote (("x" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2")
            ("x" . "/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0")
            ("x" . "/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10")
            ("x" . "/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10")
            ("x" . "/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4")
            ("x" . "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0")
            ("x" . "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12")
            ("x" . "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10")
            ("x" . "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4"))))
  (unsetenv "GUILE_LOAD_COMPILED_PATH")
  (unsetenv "LD_LIBRARY_PATH"))
(exit
 (begin
   (let* ((old-outputs
           (quote
            (("out" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2"))))
          (mapping
           (append (quote
                    (("/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0"
                      . "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0")
                     ("/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10"
                      . "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12")
                     ("/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10"
                      . "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10")
                     ("/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4"
                      . "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4")))
                   (map (match-lambda ((name . file)
                                       (cons (assoc-ref old-outputs name) file)))
                        %outputs))))
     (graft old-outputs %outputs mapping))))
--8<---------------cut here---------------end--------------->8---

I think that my last hypothesis was on the right track, but not quite
right:

* Instead of 'libcairo' being loaded twice, I now suspect that
  "libguile-cairo.so" is being loaded twice.

* Instead of the original and replacement libraries being loaded, I now
  suspect that two different variants of the replacement "guile-cairo"
  are being loaded.

* Instead of libcairo type tags being duplicated, I now suspect that
  duplicated smob tags are being allocated.

However, *if* deduplication is enabled, two redundant replacements
created by grafting _should_ occupy the same inodes, assuming that the
replacement mappings are the same (modulo ordering), and assuming that
/gnu/store/.links doesn't hit a directory size limit (which can happen
on ext3/4, leading to missed deduplication opportunities).

I've known about these redundant replacements in Guix for many years,
but was not aware of any significant practical problems arising from
them until now.

       Mark