From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id gA4qJDiRK2WGKAAAauVa8A:P1 (envelope-from ) for ; Sun, 15 Oct 2023 09:14:00 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id gA4qJDiRK2WGKAAAauVa8A (envelope-from ) for ; Sun, 15 Oct 2023 09:14:00 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id CECE74355D for ; Sun, 15 Oct 2023 09:13:59 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=tilde.club header.s=mail header.b=xTxLfxfl; dmarc=fail reason="SPF not aligned (relaxed)" header.from=tilde.club (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1697354040; a=rsa-sha256; cv=none; b=cezOUeg4pSiR4+mOQKYj8/fioXkdYSSig7uFxZw9PFnSJVTIrQIU+9raxwgD34RL+SiJrW CjdWIta85llYDRtKFHUZj4BFtCnV7z/QFxEaQANbNMXJrRw1WsCoAW4wZaeybR99DQE5qu zLUWJ6CaMeVbHlDwKtFzYAbmX05qd5G/2J4AVSq54qoxAxVdwH4anabF8+e6KwphFVCIIC 1Wj3jx0M61aQX5DodWCU8JxucxRQy1f8ljvF7DMaJ4hieqhCO46OfPeEgNioZOXn196Lvv 6AEqx+C0zY4biK3l8+pLTi9cfBnRLj3HJiR7Qu7T/YJNFgNiK/IN4nj9nUEQcw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=tilde.club header.s=mail header.b=xTxLfxfl; dmarc=fail reason="SPF not aligned (relaxed)" header.from=tilde.club (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1697354040; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=yfiiDrQNrcFMbo/n88R6IGK+8hUaPb44US0wDUiiWL8=; b=OySWpeHFimy2JhpzgWfKPimPrygIk8U6akWVnuIOCqL89PinvTdTZ0+kSi9ff90SK+cdlp 12IByf7RWOlXQmSoUMwIwlPJcX9IAlxxsbzY/NFSDSKHGmGKcbAquDOK8qp6hme1XG9d5I tAse/Vml+YtqNFvT4GADSnuk4v8qy0eSoziTGXm2hUiEMXOnLzwHftGOrd2NKsRQGED9xi 7Jx9jmZoC30zyUczSWuVYPG95eBKPTbBqeDiJpkhIZKn7XXpmxFrpNlt5LepB/ny+WwL7j 9OYpoTVwunXsJh3S2KoTn2rrhYLfNExj5lQ1J21FPBCw9MxEXWKhZaq0w3vDxw== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qrvJk-000899-Hx; Sun, 15 Oct 2023 03:13:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qrvJi-00088n-Cj for bug-guix@gnu.org; Sun, 15 Oct 2023 03:13:38 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qrvJi-0003kn-51 for bug-guix@gnu.org; Sun, 15 Oct 2023 03:13:38 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qrvK6-0008R7-57 for bug-guix@gnu.org; Sun, 15 Oct 2023 03:14:02 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#65665: package-mapping with #:deep? #t doesn't get all the implicit inputs Resent-From: Ulf Herrman Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 15 Oct 2023 07:14:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 65665 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Ulf Herrman , 65665@debbugs.gnu.org Received: via spool by 65665-submit@debbugs.gnu.org id=B65665.169735401832389 (code B ref 65665); Sun, 15 Oct 2023 07:14:02 +0000 Received: (at 65665) by debbugs.gnu.org; 15 Oct 2023 07:13:38 +0000 Received: from localhost ([127.0.0.1]:51142 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qrvJh-0008QK-Is for submit@debbugs.gnu.org; Sun, 15 Oct 2023 03:13:38 -0400 Received: from tilde.club ([142.44.150.184]:39290 ident=postfix) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qrvJe-0008QB-S1 for 65665@debbugs.gnu.org; Sun, 15 Oct 2023 03:13:36 -0400 Received: by tilde.club (Postfix, from userid 5378) id E6F0522047744; Sun, 15 Oct 2023 07:13:09 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 tilde.club E6F0522047744 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tilde.club; s=mail; t=1697353989; bh=ute/3mQLNMQY6rR5Xu8eZqUo30BSnp7FM3SNEdP21HQ=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=xTxLfxflcCr9LSE/SLu7Txl92Br0Vh38YVfDBOr396vnott6zBF3PS62vKqpUcxaS uS6g4ca5iZS7fOrZp4JEO0SRkcw7it0oQVAR/Y3Y46o5ICkP6k6VGSsQH1iTHG+vQB O8U8tgNNgdq8S+Ieg1qNlY3F/E3PE6nY3ArBzugE= From: Ulf Herrman References: <87h6ofufy5.fsf@tilde.club> <87msxmqwng.fsf@tilde.club> <871qe0lytk.fsf_-_@gnu.org> <87wmvrdxnf.fsf@tilde.club> <87y1g5fd1e.fsf@gnu.org> Date: Sun, 15 Oct 2023 02:12:43 -0500 In-Reply-To: <87y1g5fd1e.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Sat, 14 Oct 2023 17:18:21 +0200") Message-ID: <87o7h0e4us.fsf@tilde.club> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: mx1.migadu.com X-Migadu-Spam-Score: 3.71 X-Spam-Score: 3.71 X-Migadu-Queue-Id: CECE74355D X-TUID: GOGVVbkI7qLo --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Hello, > > Ulf Herrman skribis: > >> That and a growing thirst for a nuclear option for package rewriting >> brought about by trying to debug deep transformations while >> simultaneously experimenting with rewriting a manifest of around 270 >> packages. > > On that topic of a catch-all option for rewriting: there=E2=80=99s an unu= sed > procedure called =E2=80=98map-derivation=E2=80=99, which performs rewriti= ng at the level > of derivations. It=E2=80=99s harder to use, more expensive, but who know= s, > perhaps we=E2=80=99ll find a motivating use case=E2=80=A6 (Allowing for = graph rewriting > has been a major goal for me since the beginning.) > >> There are really several distinct issues at play here: >> 1. The case of #:qtbase and #:guile being invisible to package-mapping. >> This is what I first noticed, and cannot be fixed without modifying >> the build systems. This is what prompted looking for packages in >> package and bag arguments, and recursing into lists therein (just in >> case an argument took a list of packages). > > How are #:qtbase and #:guile invisible to package mapping? > > They appear in the bag inputs and thus are definitely visible to > =E2=80=98package-mapping=E2=80=99, as a I showed with the CMake and Pytho= n examples in > this thread. "Invisible to package-mapping" was perhaps not the clearest phrasing; "not completely replaced by package-mapping" might be better. qtbase does indeed go in the bag inputs, but replacing the bag inputs has no effect on the bag arguments, and even if we replaced it in the bag arguments as well, it might not even be in there to begin with, in which case the bag-builder would introduce a default that is only visible at the level of derivations. > > For good measure :-) here=E2=80=99s an example with #:qtbase: > > $ ./pre-inst-env guix build qgit -n > substitute: updating substitutes from 'http://192.168.1.48:8123'... 100.0% > substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% > 0.4 MB would be downloaded: > /gnu/store/7b20q17yg90b62404chgbnwgvd6ry1qf-qgit-2.10 > $ ./pre-inst-env guix build qgit -n --with-latest=3Dqtbase > following redirection to `https://mirrors.ocf.berkeley.edu/qt/official_re= leases/qt/'... > following redirection to `https://mirrors.ocf.berkeley.edu/qt/official_re= leases/qt/6.6/'... > guix build: warning: cannot authenticate source of 'qtbase', version 6.6.0 > > Starting download of /tmp/guix-file.CTehnY > From https://mirrors.ocf.berkeley.edu/qt/official_releases/qt/6.6/6.6.0/s= ubmodules/qtbase-everywhere-src-6.6.0.tar.xz... > =E2=80=A6-src-6.6.0.tar.xz 46.1MiB = = 12.9MiB/s 00:04 =E2=96=95=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96= =88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88= =E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=88=E2=96=8F 100.0% > substitute: updating substitutes from 'http://192.168.1.48:8123'... 100.0% > substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% > substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... = 100.0% > substitute: updating substitutes from 'https://guix.bordeaux.inria.fr'...= 100.0% > The following derivations would be built: > /gnu/store/paixxkdaakv55bffggxx4l9hiknl8i5r-qgit-2.10.drv > /gnu/store/f9fdjk1g1s1aqmlmi4clla2kqns7283v-qtbase-6.6.0.drv > 0.4 MB would be downloaded: > /gnu/store/nl9dadzfmjm9wg7v3r31jkx773dl683x-module-import-compiled > /gnu/store/6zryxmypw0wygayc9pvhyxkx47w0lyci-gperf-3.1 > /gnu/store/a57n7wy8mdi7l52pr4zg07132blgj5xp-qgit-2.10-checkout And if you do a quick 'cat /gnu/store/paixxkdaakv55bffggxx4l9hiknl8i5r-qgit-2.10.drv'... =2D----------------------------- Derive ([("out","/gnu/store/z8qrhfmicylxy2mwvcvh9sizfhd3x4d3-qgit-2.10","","")] ,[("/gnu/store/0jk33gpzyicyppbnh458213007v0qjhs-mesa-23.1.4.drv",["out"]) ,("/gnu/store/0njvvgjlb52abhqnmb4rx22sfkxm2h9c-gcc-11.3.0.drv",["out"]) ,("/gnu/store/0p1sns81qbgr8ayiv02fv4rm5drcycd7-libxdamage-1.1.5.drv",["o= ut"]) ,("/gnu/store/14a2ban238fng3c8632lrfkmz54y7m2c-binutils-2.38.drv",["out"= ]) ,("/gnu/store/1n39zcbr528b7rh9bf1pwfrm7mv8nr8m-bzip2-1.0.8.drv",["out"]) ,("/gnu/store/2pv3mjjiwh37b0m3m1hijxifnchrw76i-libpciaccess-0.16.drv",["= out"]) ,("/gnu/store/3fy3bf7wysi1n1qz9jz8xzx11sgy8m6d-git-2.41.0.drv",["out"]) ,("/gnu/store/3r0r8j76l0qvxasmb7rgn7lvpikjdyn1-libxdmcp-1.1.3.drv",["out= "]) ,("/gnu/store/4jfy1ca1d5772z15jcyk1v8wdwdcllbi-gzip-1.12.drv",["out"]) ,("/gnu/store/5nvwagz2hphvlax2bnj93smr1rgrzr8l-libx11-1.8.1.drv",["out"]) ,("/gnu/store/64vwaah2spd7q66hji6sm1j2fl6pd1rn-diffutils-3.8.drv",["out"= ]) ,("/gnu/store/6k4xxkp725r09vkn7rz2gc50asjjhpkk-xorgproto-2022.2.drv",["o= ut"]) ,("/gnu/store/91b6yraa6qax7lq7riqg1ag6lql2gfzi-tar-1.34.drv",["out"]) ,("/gnu/store/9kcv1x0lrf6fdck2j42zarxrvjzxxznv-coreutils-9.1.drv",["out"= ]) ,("/gnu/store/9piaq0aaf202r1gq7crig1cr131kx8zn-file-5.44.drv",["out"]) ,("/gnu/store/9y28bf3ywai2ybhr92c904s3cxsc8apx-libpthread-stubs-0.4.drv"= ,["out"]) ,("/gnu/store/ak17xsjb4zcw7sf0r0lxxiy4xmh57i2h-findutils-4.9.0.drv",["ou= t"]) ,("/gnu/store/bmc1hqlb207n1mnf9rs7wy8zh9hhf0br-qgit-2.10-checkout.drv",[= "out"]) ,("/gnu/store/cmd13zzya808ca8siibj8ib3llpkrr3f-sed-4.8.drv",["out"]) ,("/gnu/store/cr84vpnszpy3hkwsm8ijwpnkknxvx8k4-grep-3.8.drv",["out"]) ,("/gnu/store/cx6ws98i67qf6ba9msz7n63x78ky812c-libxxf86vm-1.1.4.drv",["o= ut"]) ,("/gnu/store/f9fdjk1g1s1aqmlmi4clla2kqns7283v-qtbase-6.6.0.drv",["out"]) ,("/gnu/store/fpprrvfiqw6g0mn5gb7ac87nbmrsq64l-make-4.3.drv",["out"]) ,("/gnu/store/g2z0fvzkh5xl4f00aixchkwxrwbbljlr-libxext-1.3.4.drv",["out"= ]) ,("/gnu/store/gb247cil5nlnx175dhqmgg67q7ng7n2h-which-2.21.drv",["out"]) ,("/gnu/store/hb2y5axynnpwqpx1znjsn5azspc9a4lw-module-import-compiled.dr= v",["out"]) ,("/gnu/store/hjhr64r5x3bhdw63zz3a3v09vfrlkhrh-qtbase-5.15.10.drv",["out= "]) ,("/gnu/store/hvqfl21wx397k58jpn5dpn2l5y5k1dl0-patch-2.7.6.drv",["out"]) ,("/gnu/store/kn7kcpkkcvy6gmrc74mf4mq0290dsf94-glibc-2.35.drv",["out","s= tatic"]) ,("/gnu/store/l3ns54f8vgqmycwi50p5cwbr6l466kj3-libdrm-2.4.114.drv",["out= "]) ,("/gnu/store/lfsv077ggq8w8cbw86wf1075zhb3bhrw-xz-5.2.8.drv",["out"]) ,("/gnu/store/lssiz5ppxfybwmr638x3bcmn4b6ixk89-linux-libre-headers-5.15.= 49.drv",["out"]) ,("/gnu/store/n4p8vzp14k1pkgxyfb5mak12rm1yiwnr-cmake-minimal-3.24.2.drv"= ,["out"]) ,("/gnu/store/nldpb7xl1slkiigvr9rax7qhcky5flnc-util-macros-1.19.3.drv",[= "out"]) ,("/gnu/store/nvaxgm6jvjy6agmp89ivahz7z9vl3ldh-libxcb-1.15.drv",["out"]) ,("/gnu/store/pdkcmkk5h3zq82ws9fps4bxnfmsmz0kx-libxshmfence-1.3.drv",["o= ut"]) ,("/gnu/store/s5nyfh0cd3z9wsk69c9blj5y850kvvcm-libvdpau-1.5.drv",["out"]) ,("/gnu/store/swblndkq2c4rzyv3xfkmsa4cjf6abphf-ld-wrapper-0.drv",["out"]) ,("/gnu/store/vkns1vij5hjamh7dpd74zs5203526747-libxfixes-6.0.0.drv",["ou= t"]) ,("/gnu/store/w2mnryfds1bd7wyyqmdi0kblz0dym1bx-glibc-utf8-locales-2.35.d= rv",["out"]) ,("/gnu/store/wwmk0rrnb5q8f48fm7h6grzar5qmslgi-libxau-1.0.10.drv",["out"= ]) ,("/gnu/store/xwckz4hbfydkfiiaa7bgslbh317gdkiq-bash-minimal-5.1.16.drv",= ["out"]) ,("/gnu/store/z7fxw9jj7avcr1ng88pak3ds3kxbfy15-gawk-5.2.1.drv",["out"]) ,("/gnu/store/zraigp7miin3vzr5dcbr4i9rvds0i07r-guile-3.0.9.drv",["out"])] ,["/gnu/store/2lg0ibv3vw01xc83advvrhvmc8yxbhh0-qgit-2.10-builder","/gnu/st= ore/p173g6kxa69qaypf8dvnw1ismd8g4k8q-module-import"] ,"x86_64-linux","/gnu/store/g8p09w6r78hhkl2rv1747pcp9zbk6fxv-guile-3.0.9/b= in/guile",["--no-auto-compile","-L","/gnu/store/p173g6kxa69qaypf8dvnw1ismd8= g4k8q-module-import","-C","/gnu/store/nl9dadzfmjm9wg7v3r31jkx773dl683x-modu= le-import-compiled","/gnu/store/2lg0ibv3vw01xc83advvrhvmc8yxbhh0-qgit-2.10-= builder"] ,[("out","/gnu/store/z8qrhfmicylxy2mwvcvh9sizfhd3x4d3-qgit-2.10")]) =2D-------------------------- you'll see we have both a ("/gnu/store/f9fdjk1g1s1aqmlmi4clla2kqns7283v-qtbase-6.6.0.drv",["out"]) and a ("/gnu/store/hjhr64r5x3bhdw63zz3a3v09vfrlkhrh-qtbase-5.15.10.drv",["out"]) in use. (and if this were a transformation that applied to all packages, it would be using two variants of the entire world beneath qtbase also) >> 2. The (perceived) case of packages hiding inside arguments. In >> hindsight, this was probably actually (3) causing this, though it's >> hard to tell because I discovered it last. I attempted to resolve >> this by recursing through s and s. > > Yeah. I think we have to understand that =E2=80=9Chiding=E2=80=9D is to = some extent > inevitable; is more concise than , which is more concise > than . There are extra =E2=80=9Cinputs=E2=80=9D showing up w= hen we go from > one abstraction to the one below. Ideally no new packages would be introduced when going from bags to derivations - there is also information at the package level that is missing at the derivation level, like properties and arguments, that transformations should be able to access. > >> 3. `this-package' referring to the inherited package in thunked fields, >> rather than the package inheriting them. This is what prompted the >> use of package-{inputs,arguments,etc}-with-package. > > Ah yes, I agree; I reported it here: > > https://issues.guix.gnu.org/50335 > > I think we can discuss it separately though, in that bug report > probably. =20 Ah, in the meantime I created issue #66510. I suppose I should merge that. I think the syntactic option presented there seems promising - extending record constructors so that (inherit parent) can be replaced with (inherit parent (fieldname variable-like-macro-name) ...) to cause uses of variable-like-macro-name within the constructor to refer to the parent's value of fieldname, with any field thunk of the parent evaluated while `this-package' is bound to the package being defined. =2D Ulf --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQHIBAEBCAAyFiEEn6BUn0yca1D9JsMa1lV76sJM9mgFAmUrkOwUHHN0cmluZXNz QHRpbGRlLmNsdWIACgkQ1lV76sJM9mgTWAv+M3SCZPO5+RvL+fOfa1hVFhHkbwM7 JURJ43VQadC/I1VlPCHfAb/gJ+8aXG+Up/fkEU5tDEg2HLxwSsGRR6Ser/iAjtIC Z508AZ2Y85bdwMk+8NAJIIfrQEZb6pJJKXcH6Nls6/sVkZFgtmjxjW6aDlLC/mkg ETZQwjfcF5e5ZsimzFkLBT+B558gQDpvAtjDD42jQ10NZJOJlc0mRpeR1I3YzbMk QBy1VjdqiBFYJ1hO5VRZ3egNki6m0aTOYrPNR57ogyJ1HQHg55sPqY/ZcjKhNWAz SJHs+jX0Xvx33G9AGnS5/c71e7TxI6YDN7ANZlQMJlkUm+BBxb7yJJ+YxDflXI/A eZzCsrSRg0R+IS/QeNwmrP4f3KNcHkii8Njn7HdngZJ2SINUWa0OsXRVGN+vYwVD Ql8dMuCsigPGaObe4BgszTJdh/D7IJ066HlXfuZNTYbjTRLk5k+g0p+slfooZjQr VBakEtN4co4UQjokOM/JDzUqM+PcOehcUlZJ =NZ9m -----END PGP SIGNATURE----- --=-=-=--