From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:8:6d80::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 2jTKDpgsfWAaWQEAgWs5BA (envelope-from ) for ; Mon, 19 Apr 2021 09:09:12 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id iEuXCJgsfWCMOQAAbx9fmQ (envelope-from ) for ; Mon, 19 Apr 2021 07:09:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9E3442CBD7 for ; Mon, 19 Apr 2021 09:09:11 +0200 (CEST) Received: from localhost ([::1]:59476 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lYO1u-0005aF-Mv for larch@yhetil.org; Mon, 19 Apr 2021 03:09:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50028) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lYO1l-0005Zc-JQ for guix-devel@gnu.org; Mon, 19 Apr 2021 03:09:01 -0400 Received: from out4-smtp.messagingengine.com ([66.111.4.28]:55721) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lYO1j-0002hy-6i; Mon, 19 Apr 2021 03:09:01 -0400 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id DB8A55C05CA; Mon, 19 Apr 2021 03:08:56 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Mon, 19 Apr 2021 03:08:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.net; h= from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-type; s=fm3; bh=1kJndCvM6Rx8+8PQgND9bbga2m w+8/Mdb0rtplawIjg=; b=swkKF3VSVHrWSNMCBfXtA4zQ1FP+c9ytWt8bCEjGSN hBcHnlbWZ50S42IjxkBWK9FQiYktiuY6mYq68lu8aGqhwFXlJSWTjkmxvFLglY/F iu3VxjUHO15GjLkDxoF2+cDe/HXakC0URKMKq2F9bz9YQGrFgGfL7lAFf1RpMfCk i4bpDiR8XJpfyAYDEX1IxCc9KQaPyn8iK7gi3KCnP1HPW6Bgtq6RvNdg1HI6iEyr 5vf27VfpmAO6ssJQNn9FKVwz4w5pu53H7rsjKlB/oSiTrvgmO/54ZwUkjfSQkm0n lJohUJbTeZRXAe+ifV/Z/xryCqPSCvtzF5M8/j9PjvRA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=1kJndC vM6Rx8+8PQgND9bbga2mw+8/Mdb0rtplawIjg=; b=PY557kfjhnSTxWXBNxA1n8 xhhM8WfyJe7XY5i54TJNoGw7UQYCTM3HrhVlqezPMeWWh0LsJ3K8LTYkq62fyyNn A49m1aaPkNRaqnxfPpyOE4fSiQx6ew0uLWjKL326OdIfv3Ns94lDxXMAoOX0yzga j1f8tJrKzOGtA9R75LG93cg0PuIGhC2bH0lTSQz9ogUEgQvfXmBp9TU9KdhYLuvR pOuRcWW2r3KiXWHfOOniuaOH13QAJYKhvsIxGNcTGxQPcWBBaTyMcVZyMe98iG37 dq6vK8E6P5tIFO6rNmHMOmL7nNLocfpjiHsQ9gjJRN/GCHYiMBmW8DZGkH4TvCuQ == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvddtfedguddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufgjfhffkfggtgesmhdtreertddtjeenucfhrhhomhepmfhonhhrrggu ucfjihhnshgvnhcuoehkohhnrhgrugdrhhhinhhsvghnsehfrghsthhmrghilhdrnhgvth eqnecuggftrfgrthhtvghrnhepffehueehieeiueetheeifeegffekgfejhedtieehhefg ffelfeevueejueelieegnecuffhomhgrihhnpehgnhhurdhorhhgnecukfhppeekvddrud dvfedrfeegrdduudejnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghi lhhfrhhomhepkhhonhhrrggurdhhihhnshgvnhesfhgrshhtmhgrihhlrdhnvght X-ME-Proxy: Received: from ordinateur-de-catherine--konrad.home (lfbn-idf2-1-313-117.w82-123.abo.wanadoo.fr [82.123.34.117]) by mail.messagingengine.com (Postfix) with ESMTPA id 068D8108005F; Mon, 19 Apr 2021 03:08:55 -0400 (EDT) From: Konrad Hinsen To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: Finding the store path of a package In-Reply-To: <8735wnxiok.fsf@gnu.org> References: <864kh9ws7z.fsf@gmail.com> <8735wq7xb4.fsf@gnu.org> <8735wnxiok.fsf@gnu.org> Date: Mon, 19 Apr 2021 09:08:54 +0200 Message-ID: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=66.111.4.28; envelope-from=konrad.hinsen@fastmail.net; helo=out4-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Guix Devel , zimoun Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1618816151; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=1kJndCvM6Rx8+8PQgND9bbga2mw+8/Mdb0rtplawIjg=; b=JygZFwl+LqvLwIPlbxNNuJAswqsZ8CmeMmv+H4vRUtbWwTk/N+m4zgL0IKz4qM6q++x9XB 0gSbc+TV2LskDbCZPrwaPoByOUEp95cpiMqtK3AgUFh82fvwRq869Z46SmPMSNh7akZhq4 PVKNhFGqYULR7/67Fn8L4vHE+F3O7ZfXgu/ancjHV32nCt41DEqTinZ017p5fy2Ep4Xm8K obDOS38YlKMT16yzKbNwqKWQH1PE0ktCcqkYBu3Vd6SXp7/GHmnvR66Y0wQ6DV0B3ivPib BeWZT/K52LoxnY8KeZueTj7CTZVMX0zkggPffooGmguX8ZWWfK4aD6giU1pDvw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1618816151; a=rsa-sha256; cv=none; b=D8g+SMXypwjcKqo+2xzqLcukD1lERymWzLQPuA8JFmIT9kyRLMqYV8YOxRlhgi2UhYx8+s 34Dp8dDngb4mvybQFkYpONSHRZ3ry+TGqg+kp2AC/owDpCuu1PB0LXXilGRwD1OUyyowT0 g6yZz0x5L9g8s6Btt+UcpHqoTFnFpmtodrsGoR4tK8srSJnUDyThC1D1pPZnR4ON6ANa1j h6EOns7h9FPSRld7CYdDHekY6tTbHT0vrY4WzVtc/2ckWMJJX8GXRgKeL9qjeVFAWhFVSk bTTqMFOAKIRjV2xEB9VMmYCw8C1qcBdjkmFJyvh7Co8GLAg2XtYQ6dPLhNUU9g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=fastmail.net header.s=fm3 header.b=swkKF3VS; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b=PY557kfj; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -1.34 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=fastmail.net header.s=fm3 header.b=swkKF3VS; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b=PY557kfj; dmarc=fail reason="SPF not aligned (relaxed)" header.from=fastmail.net (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 9E3442CBD7 X-Spam-Score: -1.34 X-Migadu-Scanner: scn0.migadu.com X-TUID: SDIwjA6YQAZ5 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo, > Why #:graft? #f? Because if you enable graft, you=E2=80=99ll potentially= have > to build/download the thing, and that wouldn=E2=80=99t buy you anything b= ecause > the set of file names is the same in the grafted package. Four weeks later: this mostly works, but sometimes fails (by downloading/building package) and it rarely fails completely (error message). And I have no idea what is going on. As an experiment, I ran the attached script via "guix repl". It requests and displays the store paths for all non-hidden packages. Using "guix gc" before and after, I found that it downloads/builds more than 50000 store entries with a total size of 7.5 GB. One small but interesting example: ABCL ################################################## Store paths for abcl@1.8.0: fetching path `/gnu/store/rz42ba0my9vrgbkjpkzr2drmnjk5ah50-python-3.8.2'... Downloading https://ci.guix.gnu.org/nar/lzip/rz42ba0my9vrgbkjpkzr2drmnjk5ah= 50-python-3.8.2... python-3.8.2 12.0MiB 1.8MiB/s 00:07 [##################] 1= 00.0% fetching path `/gnu/store/bvd09gb8ka642jzgxd2lpqlpdp160gn0-python-wrapper-3= .8.2'... Downloading https://ci.guix.gnu.org/nar/lzip/bvd09gb8ka642jzgxd2lpqlpdp160g= n0-python-wrapper-3.8.2... python-wrapper-3.8.2 347B 177KiB/s 00:00 [##################] 1= 00.0% fetching path `/gnu/store/h8z924ip7ialjhd3sc2id5yh0jy7cj20-python-nose-1.3.= 7'... Downloading https://ci.guix.gnu.org/nar/lzip/h8z924ip7ialjhd3sc2id5yh0jy7cj= 20-python-nose-1.3.7... python-nose-1.3.7 197KiB 1.4MiB/s 00:00 [##################] 1= 00.0% fetching path `/gnu/store/cshy0265w5ifh12v9dcmlrdd3cwb61db-mercurial-5.6.1'= ... Downloading https://ci.guix.gnu.org/nar/lzip/cshy0265w5ifh12v9dcmlrdd3cwb61= db-mercurial-5.6.1... mercurial-5.6.1 3.3MiB 3.1MiB/s 00:01 [##################] 1= 00.0% ((out . "/gnu/store/nz8xi3x7n3v2k4s4piiw7r8plvxam9qz-abcl-1.8.0")) ################################################## Why does computing the store path for a Java package require fetching Mercurial (and thus Python)? The only link I see is icedtea-6, whose build procedure apparently uses Mercurial to download the OpenJDK sources. Probably this gets inherited by icedtea-8, which is used for building ABCL. But... why does it take Mercurial to just compute the store path??? If my understanding of Guix hashes is correct (which I am beginning to doubt), it shouldn't require more than the hash of the Mercurial package. As an example for a hard failure, see sunxi-tools@1.4.2. Asking for its store paths downloads plenty of stuff, before failing with: while setting up the build environment: a `armhf-linux' is required to bu= ild `/gnu/store/4blcfrrzc3yx6xjvw31mhnr1r9lpw4mj-Python-3.8.2.tar.xz.drv', = but I am a `x86_64-linux' That makes sense, of course, and I don't mind getting an error in such a case, but I'd prefer it to fail without first downloading tons of stuff, including the heavyweight gcc-7.5.0. Cheers, Konrad. --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=display-store-paths.scm Content-Transfer-Encoding: base64 KHVzZS1tb2R1bGVzCiAoZ251IHBhY2thZ2VzKQogKGd1aXggcGFja2FnZXMpCiAoZ3VpeCBkZXJp dmF0aW9ucykKIChndWl4IHN0b3JlKSkKCihkZWZpbmUgKGRpc3BsYXktc3RvcmUtcGF0aHMgc3Bl Y2lmaWNhdGlvbikKICAoZm9ybWF0IChjdXJyZW50LWVycm9yLXBvcnQpCiAgICAgICAgICAiU3Rv cmUgcGF0aHMgZm9yIH5hOlxuIiBzcGVjaWZpY2F0aW9uKQogIChsZXQqCiAgICAgICgocGFja2Fn ZSAoc3BlY2lmaWNhdGlvbi0+cGFja2FnZSBzcGVjaWZpY2F0aW9uKSkKICAgICAgIChkcnYgKHdp dGgtc3RvcmUgc3RvcmUKICAgICAgICAgICAgICAgICAgICAgICAgKHBhY2thZ2UtZGVyaXZhdGlv biBzdG9yZSBwYWNrYWdlICM6Z3JhZnQ/ICNmKSkpCiAgICAgICAocGF0aHMgKGRlcml2YXRpb24t Pm91dHB1dC1wYXRocyBkcnYpKQogICAgICAgKHN0b3JlLXBhdGhzIChtYXAgKGxhbWJkYSAoZW50 cnkpCiAgICAgICAgICAgICAgICAgICAgICAgICAgIChjb25zIChzdHJpbmctPnN5bWJvbCAoY2Fy IGVudHJ5KSkgKGNkciBlbnRyeSkpKQogICAgICAgICAgICAgICAgICAgICAgICAgcGF0aHMpKSkK ICAgIChmb3JtYXQgKGN1cnJlbnQtZXJyb3ItcG9ydCkKICAgICAgICAgICAgIn5zXG4iIHN0b3Jl LXBhdGhzKSkpCgooZm9sZC1hdmFpbGFibGUtcGFja2FnZXMKIChsYW1iZGEqIChuYW1lIHZlcnNp b24gcmVzdWx0CgkJIzprZXkgb3V0cHV0cyBsb2NhdGlvbgoJCXN1cHBvcnRlZD8gZGVwcmVjYXRl ZD8KCQkjOmFsbG93LW90aGVyLWtleXMpCiAgIChpZiAoYW5kIHN1cHBvcnRlZD8gKG5vdCBkZXBy ZWNhdGVkPykpCiAgICAgICAoZGlzcGxheS1zdG9yZS1wYXRocyAoZm9ybWF0ICNmICJ+YUB+YSIg bmFtZSB2ZXJzaW9uKSkKICAgICAgICNmKSkKICcoKSkK --=-=-=--