From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms1.migadu.com with LMTPS id AJHoOvLiQGba3gAAqHPOHw:P1 (envelope-from ) for ; Sun, 12 May 2024 17:40:35 +0200 Received: from aspmx1.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id AJHoOvLiQGba3gAAqHPOHw (envelope-from ) for ; Sun, 12 May 2024 17:40:35 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b=PKihVCVm; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (strict)" header.from=posteo.net (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1715528434; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=lSguE+CZvNnwKOa0LLUMV0hT9dKRRNCXAfLjenBcf9w=; b=qqJ1HdHc9HoorlBdNA5WpH3MY6S2cRvsTsiZWEEdANrz1+aIAYTs7s14UTM2UGMK/Ush4k 5HOIoLyb+JSC8t1/78xRs0QDTESx8CxhlVaTMUjXDluMUjnkFO8xiJ1q9CZlTM5dM00IbU JkBXVImdNTTiAGXGzaBwKfl5egPZLYAbVuJPiCMkws9XFaE18EPsvD8i70ZMd+a8MY7zHZ XNgxFX3WRM/sd7OQ1yixroaa165KuzqJRXfOke0bAD/ehhXr3yfFjz3mH2zCV+GHQEFbb2 qqNmIYZTzzmis/iIztMIVWgjAy5Ze5fKmT+NCdpY9CjGTYTqjFQvO36UFDU3Fw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=posteo.net header.s=2017 header.b=PKihVCVm; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (strict)" header.from=posteo.net (policy=none) ARC-Seal: i=1; s=key1; d=yhetil.org; t=1715528434; a=rsa-sha256; cv=none; b=Fz6Fy1AMLhmxuBa3KO7zdYJvT4hj7QO0vya6kjtIdneBDTrdirZqs3pAr99CQarvaxAlsB Xn3RpbSAQyx4uw2svpL+k8WFj3DWAO9e9mfWgWmbZRI66lX0fE6P5R22i02IiwdIU7WdSi cS6wI/wUG7MLWlAZnMBrVc3XiEyilQ3WXwCLwQIyzhvMGmCAdSogC5+B+wonqR1x/KYlqd ZAK5vzBBPjhxkXlDbNmooWU0EZRSbG9yaBE9dFVk48vPjAPw/5y9AnQaDf/ZJEME0B8t8P Y++e3xsfMLjjP6m+wWS72ytO3EVHIBYzLHK2sgONzRndOXnLS8c0ACm1Y0t8Uw== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id BE84B507D8 for ; Sun, 12 May 2024 17:40:34 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s6BJ7-0001bI-9o; Sun, 12 May 2024 11:40:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s6BIx-0001YT-En for guix-patches@gnu.org; Sun, 12 May 2024 11:40:03 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s6BIx-0005n0-04 for guix-patches@gnu.org; Sun, 12 May 2024 11:40:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1s6BIw-0002C8-Rs for guix-patches@gnu.org; Sun, 12 May 2024 11:40:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#70492] bug#66866: Grafting breaks cross-compilation Resent-From: David Elsing Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 12 May 2024 15:40:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 70492 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= , dan Cc: dev@jpoiret.xyz, Philip McGrath , Mathieu Othacehe , efraim@flashner.co.il, 70492@debbugs.gnu.org, 66866@debbugs.gnu.org, janneke@gnu.org Received: via spool by 70492-submit@debbugs.gnu.org id=B70492.17155283848394 (code B ref 70492); Sun, 12 May 2024 15:40:02 +0000 Received: (at 70492) by debbugs.gnu.org; 12 May 2024 15:39:44 +0000 Received: from localhost ([127.0.0.1]:55037 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s6BIe-0002BJ-DH for submit@debbugs.gnu.org; Sun, 12 May 2024 11:39:44 -0400 Received: from mout01.posteo.de ([185.67.36.65]:46713) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1s6BIa-0002B7-Ru for 70492@debbugs.gnu.org; Sun, 12 May 2024 11:39:43 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 2704824002F for <70492@debbugs.gnu.org>; Sun, 12 May 2024 17:39:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1715528375; bh=lSguE+CZvNnwKOa0LLUMV0hT9dKRRNCXAfLjenBcf9w=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding:From; b=PKihVCVmgW//GKvj0gdfu7fPHJkBTvEXlVnIWBkcdKj0j5W+lbFmCYUdnzgOChcUh nXmCNQyPY6zf67TYQer3jjcLOtegWu6f5rLoI26bhySI5fgOvkIl3wrdE2QZzUtxfP kPGHSgsCqpEiSZeh88Q3IAt5+1k6LNXP+lNOxgNVklR8TI9NtUtSNJXHk/okA26rXl 41ZZ9knB6naIXzsKyz8S91xcdrW3SAgW/Xb7cVz+sjNojkOBwkyVD93Xnc+oEZZSp4 XMzyntL9Duvv9HsWdPxvJTQ//8+EWaf9q/BNerkVh2dhPvNxr0glhDa507OWOnVwlR /OFPVOIEsgRDg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4VcmyF21Zfz9rxL; Sun, 12 May 2024 17:39:32 +0200 (CEST) From: David Elsing In-Reply-To: <8734r143fv.fsf_-_@gnu.org> References: <87ttnh5b4g.fsf@gnu.org> <86h6ien6bg.fsf@posteo.net> <87frwn308k.fsf@dan.games> <8734r143fv.fsf_-_@gnu.org> Date: Sun, 12 May 2024 15:39:32 +0000 Message-ID: <86o79byr7v.fsf@posteo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: 2.43 X-Spam-Score: 2.43 X-Migadu-Queue-Id: BE84B507D8 X-Migadu-Scanner: mx13.migadu.com X-TUID: 3eIp3AK/wrnm Hello, I think I finally understand why the problem only occurs with grafts if a dependency uses copy-build-system. It is actually somewhat complicated. When a package is lowered into a derivation by 'package->derivation' in guix/packages.scm, a list of potentially applicable grafts is created by the 'bag-grafts' procedure. This works by first traversing the bag for native packages (where bag-build-inputs and bag-target-inputs are followed recursively; and also bag-host-inputs if not cross building) and building the native derivations of the grafted versions of the packages which have a replacement (such as glibc at the time of writing). Building the grafted package (in the 'input-graft' procedure) is done again with 'package->derivation', which eventually calls 'graft-derivation' in guix/grafts.scm, which in turn calls 'non-self-references', where the ungrafted package is actually built (ignoring the store monad indirection), not just its derivation. In the case of cross builds, the bag is also traversed for the target dependencies, where the bag-host-inputs are followed recursively. Analogously, this causes the ungrafted packages and their ungrafted replacement to be cross built to compute the grafted derivations. As dan found out, the 'lower' procedure in guix/build-system/copy.scm incorrectly puts standard-packages into the host inputs. They contain the glibc-final package in gnu/packages/commencement.scm (which inherits the replacement of glibc). Because copy-build-system puts them into bag-host-inputs, the package replacement of the glibc-final package is then cross built without grafts when calculating its grafted derivation, which it does not support. Note that the glibc-final package and its package replacement are distinct from the glibc package and its replacement in gnu/packages/base.scm. I confirmed this by running --8<---------------cut here---------------start------------->8--- ,use (guix) (with-store store (run-with-store store (package->cross-derivation (package= -replacement (@@ (gnu packages commencement) glibc-final)) "i686-linux-gnu"= #:graft? #f))) --8<---------------cut here---------------end--------------->8--- in the REPL, which returns the same incorrect glibc derivation that is attempted to be built as a dependency when running `guix build alsa-lib --target=3Di686-linux-gnu`. It is actually possible to compute the graft derivations only when needed by a store reference (which does not change the derivation of the grafted package). Then, this problem does not occur even without the fix for the copy-build-system, as the invalid derivations are never actually built. With substitutes or after running GC, this may even prevent building a package with replacement when the requested package output which (transitively) depends on it has no (transitive) store reference to it. I made a patch for that here: https://issues.guix.gnu.org/70895 Regarding the changes to the copy-build-system: Ludovic Court=C3=A8s writes: > But still, there seem to be some expectation that =E2=80=98copy-build-sys= tem=E2=80=99 > can support cross-compilation for real, so maybe we should add a > =E2=80=98copy-cross-build=E2=80=99 procedure in addition to the patch abo= ve. In guix/build-system/copy.scm, it is described as an extension of gnu-build-system and the manual says "It adds much of the gnu-build-system packages to the set of inputs. Because of this, the copy-build-system does not require all the boilerplate code often needed for the trivial-build-system." Therefore, I think it makes sense to add the copy-cross-build procedure so that copy-build-system actually behaves like gnu-build-system. Cheers, David