From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id gF0nCkt4UGAGWgAA0tVLHw (envelope-from ) for ; Tue, 16 Mar 2021 09:20:11 +0000 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id WErpBUt4UGC9DwAAB5/wlQ (envelope-from ) for ; Tue, 16 Mar 2021 09:20:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 59C1724E53 for ; Tue, 16 Mar 2021 10:20:10 +0100 (CET) Received: from localhost ([::1]:42684 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lM5s1-0001F2-IY for larch@yhetil.org; Tue, 16 Mar 2021 05:20:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53942) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lM5ru-0001EV-9S for bug-guix@gnu.org; Tue, 16 Mar 2021 05:20:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:54892) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lM5ru-0005lW-18 for bug-guix@gnu.org; Tue, 16 Mar 2021 05:20:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lM5rt-0005jR-Sb for bug-guix@gnu.org; Tue, 16 Mar 2021 05:20:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#47115: Redundant library grafts leads to breakage (was: Failure building grub-img.png when reconfiguring) Resent-From: Mark H Weaver Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 16 Mar 2021 09:20:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47115 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Jack Hill Received: via spool by 47115-submit@debbugs.gnu.org id=B47115.161588639722019 (code B ref 47115); Tue, 16 Mar 2021 09:20:01 +0000 Received: (at 47115) by debbugs.gnu.org; 16 Mar 2021 09:19:57 +0000 Received: from localhost ([127.0.0.1]:38205 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lM5ro-0005j0-Ud for submit@debbugs.gnu.org; Tue, 16 Mar 2021 05:19:57 -0400 Received: from world.peace.net ([64.112.178.59]:49600) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lM5rn-0005ig-DL; Tue, 16 Mar 2021 05:19:55 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lM5rg-0006Oo-Rf; Tue, 16 Mar 2021 05:19:49 -0400 From: Mark H Weaver In-Reply-To: <875z1rjyfn.fsf@netris.org> References: <87a6r7294k.fsf@netris.org> <877dmb1owa.fsf@netris.org> <874khds84o.fsf@netris.org> <87y2epqnq8.fsf@netris.org> <875z1rjyfn.fsf@netris.org> Date: Tue, 16 Mar 2021 05:18:15 -0400 Message-ID: <8735wvjw25.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 47115@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1615886410; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post; bh=9klXsUidQiJHyBCOFaRA2gJI4pPCd62rUxgaoFXg0IY=; b=mhgHeQL0RjZGoNdN4LdNWi0rG4fkj7N0LBjchuzx0YEoAWiTJnWqUFVKFXWeIc70Uv02XG A895azxpY0GCqhfOxjkBYBcBpRLxdwaweBchdepM9PhwkXYN2c978ql5sUGCInUo2daGhw 27BdFEx2J/XyiJZEneGuxXJZkM7aoC9aTziRdUpsSt1qoOr4r+nxrmPJDJhDXhWhHmq46x lmiXDH2WUD+9VVhaFbZPHu6WM7qvpqqg1g8GAKv63ieYmKCoTllcqJ4idntcS3pJP7Rn1H NzDAAwEZzVkBp3y6PAz6+I8OgCgrWxUAlwL1bjN5tv7XyGAHyy8NXCJwAAyFcA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1615886410; a=rsa-sha256; cv=none; b=T5qSrOLfUUhwtRTyo6Gp6Hs4qYS0pd/btkoy8y8TRWZZmmF99oxSrlqqMVgB8khAnxFKG4 dJXOpw90mplKbYV3GWjHpchD2omtI161zfky9VC9lhxQUlE61uat7uioXOs42rGPXrOSYs YUd//L3mcxZSFRJqLCSM6e7FawwLoCtgwWy5wJuwpi6GQT7RyCNkGdqTmWfa4Ry0QMvVCu HBZNh5qfMl0vqzGcxMvGxxqSOkizcr9bhusaLozEUoSmfQQJnxgUv/xu6oaHG2zGbWFt9/ zvY3ss3maiju2Xz1so9xo8Yo94TkCA5ZEoM6JHvYX/IXnIThsUF+zNkOCoIHrg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Spam-Score: -2.40 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 59C1724E53 X-Spam-Score: -2.40 X-Migadu-Scanner: scn0.migadu.com X-TUID: 3CO0/dxW8aUa retitle 47115 Redundant library grafts leads to breakage thanks Hi, I looked a bit deeper, and now I think I finally know what's going on. It turns out that the grafting process is creating two redundant variants of the replacement guile-cairo. All of the relevant information is in /gnu/store/07xw2pp63xin4c4y8ndrcdn3n8z1vmx2-grub-image.png.drv and its dependents, which fails to build if deduplication is disabled. If you look through the output of "guix size /gnu/store/07xw2pp63xin4c4y8ndrcdn3n8z1vmx2-grub-image.png.drv", you'll find three distinct guile-cairo derivations: (1) /gnu/store/vz4yw7zkm73diy95mdzywgixal3nf2s2-guile-cairo-1.11.2.drv, => /gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2 (the original, ungrafted, cairo) (2) /gnu/store/rcl324yiq7a56rwkqwgqx097dwc5mgni-guile-cairo-1.11.2.drv, => /gnu/store/vjn7ygzzqshvsfzck8hq5lp5pfrr2xp5-guile-cairo-1.11.2 (the first graft) (3) /gnu/store/9mha4bzbji8iql50prmq9br4j1c51sjn-guile-cairo-1.11.2.drv, => /gnu/store/j69k9n0g3h9ppqi7dmqypwy3lrhxvb97-guile-cairo-1.11.2 (the second graft) In the 'guile-builder' files referenced from the two graft derivations, we see that they have the same inputs and perform the same rewrites, but list them in a different order. Graft 1 has this guile-builder: --8<---------------cut here---------------start------------->8--- (begin (use-modules (guix build graft) (guix build utils) (ice-9 match)) (define %output (getenv "out")) (define %outputs (map (lambda (o) (cons o (getenv o))) (quote ("out")))) (define %build-inputs (quote (("x" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2") ("x" . "/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10") ("x" . "/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0") ("x" . "/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10") ("x" . "/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4") ("x" . "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12") ("x" . "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0") ("x" . "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10") ("x" . "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4")))) (unsetenv "GUILE_LOAD_COMPILED_PATH") (unsetenv "LD_LIBRARY_PATH")) (exit (begin (let* ((old-outputs (quote (("out" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2")))) (mapping (append (quote (("/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10" . "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12") ("/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0" . "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0") ("/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10" . "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10") ("/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4" . "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4"))) (map (match-lambda ((name . file) (cons (assoc-ref old-outputs name) file))) %outputs)))) (graft old-outputs %outputs mapping)))) --8<---------------cut here---------------end--------------->8--- Graft 2 has this guile-builder: --8<---------------cut here---------------start------------->8--- (begin (use-modules (guix build graft) (guix build utils) (ice-9 match)) (define %output (getenv "out")) (define %outputs (map (lambda (o) (cons o (getenv o))) (quote ("out")))) (define %build-inputs (quote (("x" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2") ("x" . "/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0") ("x" . "/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10") ("x" . "/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10") ("x" . "/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4") ("x" . "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0") ("x" . "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12") ("x" . "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10") ("x" . "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4")))) (unsetenv "GUILE_LOAD_COMPILED_PATH") (unsetenv "LD_LIBRARY_PATH")) (exit (begin (let* ((old-outputs (quote (("out" . "/gnu/store/5nmzfnxk8kp85xwma2r585fgyz3jfw56-guile-cairo-1.11.2")))) (mapping (append (quote (("/gnu/store/na0x00biq02fm5cyj5a8r67qwsnsskw8-cairo-1.16.0" . "/gnu/store/p51dv37zj24q8001zghc3wxhxz8i3c50-cairo-1.16.0") ("/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10" . "/gnu/store/rwkqxykm91a75w9afhb41saj0dmf30hw-libx11-1.6.12") ("/gnu/store/cw69is9wbbllwx95wky4pmbcsk4vvbpd-libxrender-0.9.10" . "/gnu/store/pzj036f1nmxdrbza6cqy419ddsn9bydp-libxrender-0.9.10") ("/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4" . "/gnu/store/3rmazp46f6g8w9qs8n3w7qcg8hhs1lig-libxext-1.3.4"))) (map (match-lambda ((name . file) (cons (assoc-ref old-outputs name) file))) %outputs)))) (graft old-outputs %outputs mapping)))) --8<---------------cut here---------------end--------------->8--- I think that my last hypothesis was on the right track, but not quite right: * Instead of 'libcairo' being loaded twice, I now suspect that "libguile-cairo.so" is being loaded twice. * Instead of the original and replacement libraries being loaded, I now suspect that two different variants of the replacement "guile-cairo" are being loaded. * Instead of libcairo type tags being duplicated, I now suspect that duplicated smob tags are being allocated. However, *if* deduplication is enabled, two redundant replacements created by grafting _should_ occupy the same inodes, assuming that the replacement mappings are the same (modulo ordering), and assuming that /gnu/store/.links doesn't hit a directory size limit (which can happen on ext3/4, leading to missed deduplication opportunities). I've known about these redundant replacements in Guix for many years, but was not aware of any significant practical problems arising from them until now. Mark