From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id 2DeODBlNLGXrsgAA9RJhRA:P1 (envelope-from ) for ; Sun, 15 Oct 2023 22:35:37 +0200 Received: from aspmx1.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id 2DeODBlNLGXrsgAA9RJhRA (envelope-from ) for ; Sun, 15 Oct 2023 22:35:37 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C39E23813D for ; Sun, 15 Oct 2023 22:35:36 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=protonmail.com header.s=protonmail3 header.b=FP6nwtfd; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1697402137; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=jYeiOwfmgEhB5zKw0Pyez4ODWX/KpYO5Zx2FVavkrpM=; b=DmU+ZS3IiFgB3iSwzlCWt/2Sp0qtQfjisHAXg8XjC44WIPrXnJz/WbTzQiRn8XnhG3wFsP EzYhd41kC2bPQ5LXRJ4hqXX4qt0qeu8qpeOpdkDV7XarYRyonQtwMcqPP7MFHrmuRWKW19 VWD3wFtNcNRSYgqfghwhvvxeSiJJzbcrOmtbS2k2zfFET5mVcc7RSulCqbJb31Gvw01wNn 5zEM3ALHfz7m75lPk5CcOesrgWYI/X5gDmOPz9QJon85ojzIZ2X3SxmStWOE5ZLOOBeQk2 DZNeuRKAHV1yg5pl6HfTZhhQWr/Ahcmn8nkP5S2bsszlmn0QNxKPBilCi1iRpg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=protonmail.com header.s=protonmail3 header.b=FP6nwtfd; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gnu.org ARC-Seal: i=1; s=key1; d=yhetil.org; t=1697402137; a=rsa-sha256; cv=none; b=b4KqB1e4EZJjsPIZ6l2PHv3gqN3hccZ93RHbqQdt2n/Pk3yJ1+CN/Rt7mgRtCYc1qvAWf1 ZyIBapc8vA/b1XjGZ3fsgDgKRmM/yT2KFvfiWtf1fBEzRuV8paYAwrb9DuJcYnVEj/5aI9 O0MPs3MNymfLunDp82Owa3B70RaKfoLVlMs/xs0loIZEWOR1/iex+avNztNRyooxHyDAD3 6RPZpMycqU+/KPbPrkScjVm56aFcfzJw43oo3GRQ9GVNltVGz7eKYAfvSJDa038n+aSasH oVsEXSAQMvsJaL3T5CaC4JLgO9xIkP9MM4oKhCjn0qamsn2rDg2L5azJzztmBA== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qs7VX-0007jU-U0; Sun, 15 Oct 2023 16:14:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qs7VW-0007jI-Ag for bug-guix@gnu.org; Sun, 15 Oct 2023 16:14:38 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qs7VW-0000By-2t for bug-guix@gnu.org; Sun, 15 Oct 2023 16:14:38 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qs7Vu-0005we-ET for bug-guix@gnu.org; Sun, 15 Oct 2023 16:15:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#53406: union-build incorrectly handles grafts Resent-From: John Kehayias Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 15 Oct 2023 20:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 53406 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 53406@debbugs.gnu.org Received: via spool by 53406-submit@debbugs.gnu.org id=B53406.169740084222741 (code B ref 53406); Sun, 15 Oct 2023 20:15:02 +0000 Received: (at 53406) by debbugs.gnu.org; 15 Oct 2023 20:14:02 +0000 Received: from localhost ([127.0.0.1]:54187 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qs7Uv-0005ub-GA for submit@debbugs.gnu.org; Sun, 15 Oct 2023 16:14:02 -0400 Received: from mail-4322.protonmail.ch ([185.70.43.22]:64675) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qs7Up-0005u6-Mp for 53406@debbugs.gnu.org; Sun, 15 Oct 2023 16:13:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1697400805; x=1697660005; bh=jYeiOwfmgEhB5zKw0Pyez4ODWX/KpYO5Zx2FVavkrpM=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=FP6nwtfd6t9fUW6bYI0WzMJnSc+/a/wn1SThsDMbYwfBdCDFEsdFvfBcbceeBMvYh uBDWh3qlxngQxUZ9G0T4oW4mfUI9AqhY/MqYZcHaKyLOngwupX6S1V9pPKrpzk4Cm2 4vKAtyXw1CgLmAwdLell4zryn7ZZBTOT30s7lKiwLj/r7wzTFLpLBMdb/k8vDalI0Q YrwQI6+YLtwAAmUs5bOYg4RXIwsEdkHsI+7pmHwjg4fN0EG7yLHMb/DKByq+6g/xx7 zbdW3YZuONgdpvYgIG1nBvhVimqWAKLVqmqRM2OhqO6xBQHmjr+MXcYu0KXXoquJAn CQwse/383D3FQ== Date: Sun, 15 Oct 2023 20:13:19 +0000 Message-ID: <877cnnbq5h.fsf@protonmail.com> In-Reply-To: <87bl00lyf6.fsf@gnu.org> References: <878rv5ushp.fsf@gnu.org> <87bl00lyf6.fsf@gnu.org> Feedback-ID: 7805494:user:proton MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: John Kehayias From: John Kehayias via Bug reports for GNU Guix Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -3.65 X-Spam-Score: -3.65 X-Migadu-Queue-Id: C39E23813D X-Migadu-Scanner: mx2.migadu.com X-TUID: +3ymAynq30a0 Hi Ludo=E2=80=99, Long time since I've thought about this bug, but with all the recent grafts I thought to return to it. I'll have to make a new example to dig again, but wanted to think through where we might look to see what's happening first. On Tue, Jan 25, 2022 at 02:47 PM, Ludovic Court=C3=A8s wrote: > Hi John, > > John Kehayias skribis: > [...] >> Perhaps I was too hasty in noting this "problem" which like I said >> was not the error I originally encountered. I was using a package >> that constructs both the 64- and 32-bit libraries to put in a >> container (say, a /lib32 and /lib64 or something similar to an FHS >> environment). A collision was happening between a file and >> directory, one being a good symlink and the other broken, rather >> than a "real" mismatch in file vs directory. Anyway, going back to >> that what I see is that one link is broken for the above reasons, >> but the good one is good because it is to the *ungrafted* library >> store path. I don't know now if these 2 things are connected other >> than one led me to the other, but I turn now to what demonstrates my >> original problem. >> >> I don't know why this happens or if it is something in this building >> process that is not correct, but I did come up with a minimal >> example (attached). The code is a bit odd in its stripped down form, >> though hopefully is clear in what way this would be used to do >> something useful (again, like an FHS environment or other >> container). Apologies for the old style and lack of gexps which I'm >> finally getting used to. The example package just tries to make a >> dummy package that has, for illustration, a "/lib64" and "/lib32" >> which link to the respective union-build inputs (of a single library >> for simplicity). I don't think the actual package being made matters >> so much, or how it is constructed, but that two inputs are >> union-builds of the same library (x86_64 and i686) which should have >> a graft of expat. Just my guess though. >> >> Doing: >> >> ls -la $(guix build -f graft-test.scm)/lib64/lib/libexpat* >> lrwxrwxrwx 1 root root 71 Dec 31 1969 >> /gnu/store/qbh16hfdv8pnfw01k6izbs3jkji6i978-test-pkg-0.0/lib64/lib/libex= pat.la >> -> >> /gnu/store/2q8wwhd3prib0swky68rbx9hl0xxs6hf-expat-2.4.3/lib/libexpat.la* >> lrwxrwxrwx 1 root root 71 Dec 31 1969 >> /gnu/store/qbh16hfdv8pnfw01k6izbs3jkji6i978-test-pkg-0.0/lib64/lib/libex= pat.so >> -> >> /gnu/store/2q8wwhd3prib0swky68rbx9hl0xxs6hf-expat-2.4.3/lib/libexpat.so* >> lrwxrwxrwx 1 root root 73 Dec 31 1969 >> /gnu/store/qbh16hfdv8pnfw01k6izbs3jkji6i978-test-pkg-0.0/lib64/lib/libex= pat.so.1 >> -> >> /gnu/store/2q8wwhd3prib0swky68rbx9hl0xxs6hf-expat-2.4.3/lib/libexpat.so.= 1* >> lrwxrwxrwx 1 root root 77 Dec 31 1969 >> /gnu/store/qbh16hfdv8pnfw01k6izbs3jkji6i978-test-pkg-0.0/lib64/lib/libex= pat.so.1.8.1 >> -> >> /gnu/store/2q8wwhd3prib0swky68rbx9hl0xxs6hf-expat-2.4.3/lib/libexpat.so.= 1.8.1 >> >> is what we saw already: libexpat is the newer (replacement, 2.4.3) >> version, with the full version symlink broken since the version >> number is wrong. Likewise in other pieces that have the version >> number, like share/doc. Okay, that's expected. But now, in the >> i686-linux union-build input: >> >> ls -la $(guix build -f graft-test.scm)/lib32/lib/libexpat* >> lrwxrwxrwx 1 root root 71 Dec 31 1969 >> /gnu/store/qbh16hfdv8pnfw01k6izbs3jkji6i978-test-pkg-0.0/lib32/lib/libex= pat.la >> -> >> /gnu/store/b0jns3vzhhpna7lim8bc3dr0payzx5yy-expat-2.4.1/lib/libexpat.la* >> lrwxrwxrwx 1 root root 71 Dec 31 1969 >> /gnu/store/qbh16hfdv8pnfw01k6izbs3jkji6i978-test-pkg-0.0/lib32/lib/libex= pat.so >> -> >> /gnu/store/b0jns3vzhhpna7lim8bc3dr0payzx5yy-expat-2.4.1/lib/libexpat.so* >> lrwxrwxrwx 1 root root 73 Dec 31 1969 >> /gnu/store/qbh16hfdv8pnfw01k6izbs3jkji6i978-test-pkg-0.0/lib32/lib/libex= pat.so.1 >> -> >> /gnu/store/b0jns3vzhhpna7lim8bc3dr0payzx5yy-expat-2.4.1/lib/libexpat.so.= 1* >> lrwxrwxrwx 1 root root 77 Dec 31 1969 >> /gnu/store/qbh16hfdv8pnfw01k6izbs3jkji6i978-test-pkg-0.0/lib32/lib/libex= pat.so.1.8.1 >> -> >> /gnu/store/b0jns3vzhhpna7lim8bc3dr0payzx5yy-expat-2.4.1/lib/libexpat.so.= 1.8.1* >> >> all the links are good and to the original (version 2.4.1) expat. In >> other words, the constructed union64 and union32 inputs (in the >> sample code) do not both get grafted, even though doing just the >> fhs-union command on it's own (not building both for another >> package) does graft for either architecture. At least that seems >> like the most obvious difference between the earlier example and >> this new one. >> >> Why? Does the grafting just happen "once" somehow and misses the >> "same" input again (but built for different system)? Is this >> expected or just a weird/wrong way to do this kind of build which is >> causing this? I'm not sure if this is just with union-build or if it >> would happen just with inputs of the same library but different >> architectures. I didn't know how to do that quickly off hand, so I >> haven't tried it yet. > > Woow, it=E2=80=99s a sophisticated example. :-) > This is potentially relevant again since I wanted to see about adding multi-arch support to guix shell --emulate-fhs. Likely there will be a collision in the profile created for this same reason. (These days from say libx11.) > I don=E2=80=99t actually have the Expat replacement we=E2=80=99re talking= about so I > can=E2=80=99t easily test. > > At first sight I=E2=80=99d say it should work (both lib32 and lib64 shoul= d refer > to the Expat replacement), but it could be that something somewhere > assumes all the packages are built for the system. That would need more > investigation work in (guix packages) in (guix grafts). > Taking a quick look, and not knowing much about graft internals, is the place to look more in (guix packages) bag-grafts, input-graft, package->derivation maybe? >From what you said, and my guess as well, is that mixed #:system in the inputs is somehow "missed" by grafting. Maybe just using the same #:system as the package? If that's the case, grafting needs to check the system of each input package rather than assuming it follows from the package, would that make sense? Maybe some clues in how cross-compiling works, or some package we have that does mix systems (like dxvk)? I'll reconstruct an example or maybe see if dxvk sheds some light with grafts, though looks like it is currently broken first... Thanks! John