From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id QOXRK7XVsl+gdwAA0tVLHw (envelope-from ) for ; Mon, 16 Nov 2020 19:40:37 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id cCifJ7XVsl+gUAAAbx9fmQ (envelope-from ) for ; Mon, 16 Nov 2020 19:40:37 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id AF363940415 for ; Mon, 16 Nov 2020 19:40:36 +0000 (UTC) Received: from localhost ([::1]:45950 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kekMd-0007jJ-6V for larch@yhetil.org; Mon, 16 Nov 2020 14:40:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52046) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kekDO-00065S-A4 for bug-guix@gnu.org; Mon, 16 Nov 2020 14:31:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:45822) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kekDN-0000rM-NM for bug-guix@gnu.org; Mon, 16 Nov 2020 14:31:01 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kekDN-0004St-Ia for bug-guix@gnu.org; Mon, 16 Nov 2020 14:31:01 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#44574: guix weather not checking all substitute urls Resent-From: Efraim Flashner Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 16 Nov 2020 19:31:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 44574 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: zimoun Received: via spool by 44574-submit@debbugs.gnu.org id=B44574.160555505816938 (code B ref 44574); Mon, 16 Nov 2020 19:31:01 +0000 Received: (at 44574) by debbugs.gnu.org; 16 Nov 2020 19:30:58 +0000 Received: from localhost ([127.0.0.1]:57368 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kekDJ-0004OW-J6 for submit@debbugs.gnu.org; Mon, 16 Nov 2020 14:30:58 -0500 Received: from flashner.co.il ([178.62.234.194]:52046) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kekDH-0004HR-A3 for 44574@debbugs.gnu.org; Mon, 16 Nov 2020 14:30:56 -0500 Received: from localhost (unknown [141.226.9.166]) by flashner.co.il (Postfix) with ESMTPSA id 45F63403D2; Mon, 16 Nov 2020 19:30:49 +0000 (UTC) Date: Mon, 16 Nov 2020 21:30:11 +0200 From: Efraim Flashner Message-ID: <20201116193011.GP987@E5400> References: <20201111113005.GB12414@E5400> <86ft59ql7t.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="K8zN2sh9fO5jmbe4" Content-Disposition: inline In-Reply-To: <86ft59ql7t.fsf@gmail.com> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 44574@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: ns3122888.ip-94-23-21.eu Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: -2.61 X-TUID: DTg7+EWBlswP --K8zN2sh9fO5jmbe4 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Nov 16, 2020 at 06:12:54PM +0100, zimoun wrote: > Hi Efraim, >=20 > On Wed, 11 Nov 2020 at 13:30, Efraim Flashner wro= te: > > I would have assumed that it would check for all of the substitute urls > > I listed and not just the first one. Most of the packages listed below > > I've test built on my pine64 so they shouldn't appear in the list > > anymore. >=20 > I think #44688 [1] fixes it. Please let me know. >=20 > 1: I had hoped for a unified list of the missing packages between the two or more build machines but the separate output is also useful. With a unified list it's possible to see which packages are missing overall and need fixing/building. >=20 > Examples: >=20 > --8<---------------cut here---------------start------------->8--- > $ ./pre-inst-env guix weather --substitute-urls=3D"https://bayfront.guix.= gnu.org https://ci.guix.gnu.org" > computing 15,391 package derivations for x86_64-linux... > looking for 16,707 store items on https://bayfront.guix.gnu.org... > updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0% > https://bayfront.guix.gnu.org > 55.6% substitutes available (9,289 out of 16,707) > at least 51,869.2 MiB of nars (compressed) > 82,986.2 MiB on disk (uncompressed) > 0.001 seconds per request (24.6 seconds in total) > 679.0 requests per second >=20 > 0.3% (20 out of 7,418) of the missing items are queued > 32 queued builds > x86_64-linux: 32 (100.0%) > build rate: 13.83 builds per hour > x86_64-linux: 13.83 builds per hour > looking for 16,707 store items on https://ci.guix.gnu.org... > updating substitutes from 'https://ci.guix.gnu.org'... 100.0% > https://ci.guix.gnu.org > 88.9% substitutes available (14,860 out of 16,707) > at least 78,929.4 MiB of nars (compressed) > 130,797.2 MiB on disk (uncompressed) > 0.002 seconds per request (26.9 seconds in total) > 620.2 requests per second >=20 > 0.5% (10 out of 1,847) of the missing items are queued > at least 1,000 queued builds > x86_64-linux: 493 (49.3%) > aarch64-linux: 140 (14.0%) > i686-linux: 346 (34.6%) > armhf-linux: 21 (2.1%) > build rate: 38.98 builds per hour > x86_64-linux: 13.29 builds per hour > i686-linux: 12.32 builds per hour > aarch64-linux: 6.78 builds per hour > armhf-linux: 6.59 builds per hour > --8<---------------cut here---------------end--------------->8--- > =20 > or >=20 > --8<---------------cut here---------------start------------->8--- > $ ./pre-inst-env guix weather --substitute-urls=3D"https://guix.tobias.gr= https://ci.guix.gnu.org" -c 100 > computing 15,391 package derivations for x86_64-linux... > looking for 16,707 store items on https://guix.tobias.gr... > updating substitutes from 'https://guix.tobias.gr'... 100.0% > https://guix.tobias.gr > 35.0% substitutes available (5,855 out of 16,707) > unknown substitute sizes > 52,230.7 MiB on disk (uncompressed) > 0.017 seconds per request (291.8 seconds in total) > 57.3 requests per second > (continuous integration information unavailable) > updating substitutes from 'https://guix.tobias.gr'... 100.0% > 6777 packages are missing from 'https://guix.tobias.gr' for 'x86_64-linux= ', among which: > 16564 bootar@1a /gnu/store/x2iy3ka299sk1x97an783jl4pfbn60pj-bootar-1a= =20 > 16561 bootstrap-mes-rewired@0.19 /gnu/store/a4m8c80mrys8w3vlm6vdnq05c7a= 1warl-bootstrap-mes-rewired-0.19=20 > 16495 ld-wrapper-boot3@0 /gnu/store/v3wwjj9i9zqglq4g2yczf6cxsh028qfb-ld= -wrapper-boot3-0=20 > 9743 bison@3.5.3 /gnu/store/rxli27pbpr9x9c8mk9cbp6masnwaszrs-bison-3.5.= 3=20 > 1831 xkbcomp-intermediate@1.4.3 /gnu/store/fjwjb14b5irngvv9r06dmkhhds56= 6vw2-xkbcomp-intermediate-1.4.3=20 > 788 bootar@1a /gnu/store/x2iy3ka299sk1x97an783jl4pfbn60pj-bootar-1a=20 > 785 bootstrap-mes-rewired@0.19 /gnu/store/a4m8c80mrys8w3vlm6vdnq05c7a1= warl-bootstrap-mes-rewired-0.19=20 > 719 ld-wrapper-boot3@0 /gnu/store/v3wwjj9i9zqglq4g2yczf6cxsh028qfb-ld-= wrapper-boot3-0=20 > 595 bootar@1a /gnu/store/x2iy3ka299sk1x97an783jl4pfbn60pj-bootar-1a=20 > 592 bootstrap-mes-rewired@0.19 /gnu/store/a4m8c80mrys8w3vlm6vdnq05c7a1= warl-bootstrap-mes-rewired-0.19=20 > 526 ld-wrapper-boot3@0 /gnu/store/v3wwjj9i9zqglq4g2yczf6cxsh028qfb-ld-= wrapper-boot3-0=20 > 339 bison@3.5.3 /gnu/store/rxli27pbpr9x9c8mk9cbp6masnwaszrs-bison-3.5.= 3=20 > 305 go@1.4-bootstrap-20171003 /gnu/store/asw6676iy7i0a8azpra2870w5hkq7= bvv-go-1.4-bootstrap-20171003-doc /gnu/store/j20pik1y8psi78v7zb5ksqybn198in= x7-go-1.4-bootstrap-20171003 /gnu/store/csfqsyyysbg9afnck405wmkhrlrblmza-go= -1.4-bootstrap-20171003-tests=20 > 211 xkbcomp-intermediate@1.4.3 /gnu/store/fjwjb14b5irngvv9r06dmkhhds56= 6vw2-xkbcomp-intermediate-1.4.3=20 > 121 bison@3.5.3 /gnu/store/rxli27pbpr9x9c8mk9cbp6masnwaszrs-bison-3.5.= 3=20 > 113 r-codetools@0.2-16 /gnu/store/k217j4c4ra2hfdldva075dmvq89ww1dn-r-c= odetools-0.2-16=20 > looking for 16,707 store items on https://ci.guix.gnu.org... > https://ci.guix.gnu.org > 88.9% substitutes available (14,860 out of 16,707) > at least 78,929.4 MiB of nars (compressed) > 130,797.2 MiB on disk (uncompressed) > 0.001 seconds per request (10.1 seconds in total) > 1,650.0 requests per second >=20 > 0.5% (10 out of 1,847) of the missing items are queued > at least 1,000 queued builds > x86_64-linux: 493 (49.3%) > aarch64-linux: 140 (14.0%) > i686-linux: 346 (34.6%) > armhf-linux: 21 (2.1%) > build rate: 38.70 builds per hour > x86_64-linux: 13.20 builds per hour > i686-linux: 12.23 builds per hour > aarch64-linux: 6.73 builds per hour > armhf-linux: 6.54 builds per hour > updating substitutes from 'https://ci.guix.gnu.org'... 100.0% > 1251 packages are missing from 'https://ci.guix.gnu.org' for 'x86_64-linu= x', among which: > 112 java-snappy@1.1.7.5 /gnu/store/mq1xagq5sn9mfhpfg7j2cn1dn4vsds72-ja= va-snappy-1.1.7.5=20 > --8<---------------cut here---------------end--------------->8--- >=20 >=20 >=20 > compared to the current behaviour: >=20 > --8<---------------cut here---------------start------------->8--- > $ guix weather --substitute-urls=3D"https://bayfront.guix.gnu.org https:/= /ci.guix.gnu.org" > computing 15,287 package derivations for x86_64-linux... > looking for 16,584 store items on https://bayfront.guix.gnu.org... > updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0% > https://bayfront.guix.gnu.org > 54.4% substitutes available (9,026 out of 16,584) > at least 49,365.8 MiB of nars (compressed) > 81,792.9 MiB on disk (uncompressed) > 0.003 seconds per request (41.6 seconds in total) > 398.0 requests per second >=20 > 0.1% (10 out of 7,558) of the missing items are queued > 32 queued builds > x86_64-linux: 32 (100.0%) > build rate: 13.84 builds per hour > x86_64-linux: 13.84 builds per hour > --8<---------------cut here---------------end--------------->8--- >=20 > (Note that Bayfront is working hard. ;-)) >=20 > All the best, > simon --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --K8zN2sh9fO5jmbe4 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl+y0z8ACgkQQarn3Mo9 g1HUvQ/8CYH2z88DMi/3P54c1cTRftIgWQo+bKGCXVcw1ndeLTvGUol+oFTzxmr8 SzRPZt892RdUy76BH6/d/sUz3fehIlz7/EYC3tjenSWBibISx/33+C/101xvn1EK Q72SROsIo9lAhPFcjl76KY5OeICfY318dT+dpsBTTN+l/6nH0sD503pVfwcPMQFX xbGLnk4n0cMP9n2D7/SRQe7R3theEcEfGAiOmvd01tO4DiVWLKgByrt8DCSZT04c 1bmd8r3XLlf1e6Fsdp/hSirTEu4yjdHm7EcQUs5i/iPkjqO7Mauh84hwZT7HeCeN khP9DUHRLoBS8cfcsp04frSPX549yxjt+Pn8Bx/zN4b6ELONXpAVM5L4UjctDpQk 7FhzkzNeFwoAaZWVATSNcd1JSQW9qHrRifOMhLDFFyDgp9GP5UWV3yrA8Xd0ZXOj Iub23ED6tmm3YJBZCj1gr0hvOH9Y0sGKB4cBapeEgG3FQ//gKMYeCgE6j60mb3J4 DOoH+HL2PsD0gTuqPQVb8KtVMvcB9nr1XeiN5o5dh+DYEBFCR52XuqA0ee8DeYLC lEfdjTeOgCLl3LIO9U5hQrZYt9J86kAFwvMSrW0dZOm3J6hNmFJBrigE/qLkOuEF HvxDqily2UU5+a77eknEq1Jts63Md3fUzIjk5BcENVEheCnzslk= =8lah -----END PGP SIGNATURE----- --K8zN2sh9fO5jmbe4--