From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id SOCdAkicfWQTLwAASxT56A (envelope-from ) for ; Mon, 05 Jun 2023 10:26:48 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id OGPVAkicfWT0QQEAauVa8A (envelope-from ) for ; Mon, 05 Jun 2023 10:26:48 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 6FF9522AAA for ; Mon, 5 Jun 2023 10:26:47 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q65AN-0007H9-VU; Mon, 05 Jun 2023 04:02:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q65A6-0007DK-BV for guix-devel@gnu.org; Mon, 05 Jun 2023 04:01:59 -0400 Received: from mail-lf1-x12f.google.com ([2a00:1450:4864:20::12f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q65A4-00023F-81 for guix-devel@gnu.org; Mon, 05 Jun 2023 04:01:58 -0400 Received: by mail-lf1-x12f.google.com with SMTP id 2adb3069b0e04-4f50470d77cso5482223e87.0 for ; Mon, 05 Jun 2023 01:01:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685952113; x=1688544113; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender:from:to :cc:subject:date:message-id:reply-to; bh=ROVrLnwLGumEam78N3MtpBg0rO+bdwMRTh40CTM1trw=; b=IVXDXCQrUVxJEWjkM5S2e6sZxt+7JUxHS9tduvfwvyD06w8bUCEUG7Nb8aJdwzbElv i0P6AtSYzM8Hz96lxA3TqweuiUcHvxTRLY1gm8w0EpJzjNglvrTNs2NM8VSWdHfzTNw0 uLTYaGrxduHBoAeyRAchLlJZrQBxu4DyeEbTqiqPJsTiGrJn+TbAGMQSHzC/tlI4t4dX 6DxijA7F8UDRcRZgGfMa2DTFUmgALHQOd2JPXKPr6PvQbYELDN94/rd+pypykeLgoJga gS5mxis7RH5YZz/BGScv+xSBm+RR4mf9jHx3Sc7Dtc8ujfCk+R8m9Aoe1KFr6TgAKFQ/ fQGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685952113; x=1688544113; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ROVrLnwLGumEam78N3MtpBg0rO+bdwMRTh40CTM1trw=; b=PjuZOSimrlk5RJj7WqGVuHnj9TaUzjEYIGMtEPR0zJujah8Z0ZgwDGZkLCyJicBsn3 Cr3XdlPqpXePRBYWFMMK+mTeOBunRT+6T7qyoqmumoL/bD2RBWhOgpWWxpE43Fh8Og0h tfra93YDL9BaawwZDB328OIrZsnsSmz7Miw0z336Xr8HjQIGFgIe01qU7145FxEgBbSS GpJ1NtZOqyXFiXs7qHx9liqqeKYgnpnBMF1DIOfLZdHpdXsY4uvAL1JMqiOaaOTEIQhR QKhcMxt//LYQAmGlahQAQm7YZzV7OJMXP73G8k5hGlwemAqES0fYjxJ0RxM/EL0cJ0pc QpsQ== X-Gm-Message-State: AC+VfDxUc9WYCTZ++K5L1GenpfTVAiFuYkwHDjq0vdTGd+nUJ2y3KbaE graF9FIi+sQbPS7Zy2pKA45RhIB/j1IFKA== X-Google-Smtp-Source: ACHHUZ6zlpyiZmzRTAv9fxu6ompNgIo0L3ODfU22VrXf16I3UOMtYn2NpJlUhJZeAKAbzJLamQNLig== X-Received: by 2002:ac2:4c18:0:b0:4ef:eda6:c14 with SMTP id t24-20020ac24c18000000b004efeda60c14mr4725956lfq.35.1685952113436; Mon, 05 Jun 2023 01:01:53 -0700 (PDT) Received: from localhost ([37.46.46.3]) by smtp.gmail.com with ESMTPSA id u4-20020a7bc044000000b003f70a7b4537sm13464316wmc.36.2023.06.05.01.01.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Jun 2023 01:01:52 -0700 (PDT) Date: Mon, 5 Jun 2023 11:01:57 +0300 From: Efraim Flashner To: Christopher Baines Cc: guix-devel@gnu.org Subject: Re: 02/03: gnu: openblas: Update architectures we provide substitutes for. Message-ID: Mail-Followup-To: Christopher Baines , guix-devel@gnu.org References: <168560801502.2735.15747711991420829133@vcs2.savannah.gnu.org> <20230601082655.B4449C23EC5@vcs2.savannah.gnu.org> <87wn0l5xkk.fsf@cbaines.net> <87ttvo49wh.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="yjE9QnCWniboDWQj" Content-Disposition: inline In-Reply-To: <87ttvo49wh.fsf@cbaines.net> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Received-SPF: pass client-ip=2a00:1450:4864:20::12f; envelope-from=efraim.flashner@gmail.com; helo=mail-lf1-x12f.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1685953607; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=ROVrLnwLGumEam78N3MtpBg0rO+bdwMRTh40CTM1trw=; b=P0+6tg90LYtKPwiDCL+SpMb+cMtsqm6iyWViJ4OeWLwnxoN6vR18gWnRKzHwDL8s0BLd+I WqGC+2Kjp8WarKIaSKwCPKAKSwmWAO1Fol+Y7scb6gr2lX35Mjre8B84Ys1Ebe0qXWiXAA kWrAzxC42CdlC8pTy0geAGwm3HV7bcZiW5bDOJgvQ9w4hTTwICgfWnfgFVW/UObUbdzXxJ bdd0RpbumQwIJMLBgHD+ld0GR5gnwNvLI+Gswryj8jNJILrjYsmTJamModmXT6MUlBWyWj CyM2UidlqlJ63n24/4NLWw9f5gJdtpX/0wSS6TijO/xdrlBdYAVlojaVEUzheg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=IVXDXCQr; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1685953607; a=rsa-sha256; cv=none; b=Rhj1mHgO3aDtifQR//cNj/DwZJP7tl2ajBJ+FEsc1V0nGaIiiTM68yuHDkveVv+tGIxmsx aPlxBp7+RJQ6mBji2PMgrli4KLOMFnqY+t8lVXC6FCRBh0fQm2lizTSUY1hs0ssjzgS6y8 1LE0o3sf10glcysnSR8NSrYnKdFPAM5TIz1LHXtbSmkZzK0igige0Rephurfz4YwHdLggW HYcrb/HvBksFrWW5+OYG4FxIiXJw3OtptHre1NsbKIJ6Cah17RU+FVHkur/MZb7lwCi7cT bsij4251+tipqgziTcgz7UCRrHuD/F9uyorh4TpuIVaCgubLr0LrxopN3eWx3Q== X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -2.09 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20221208 header.b=IVXDXCQr; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 6FF9522AAA X-Spam-Score: -2.09 X-TUID: hm+ZyHFqutfj --yjE9QnCWniboDWQj Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jun 03, 2023 at 08:12:48PM +0100, Christopher Baines wrote: >=20 > Efraim Flashner writes: >=20 > > On Fri, Jun 02, 2023 at 11:03:42PM +0100, Christopher Baines wrote: > >>=20 > >> guix-commits@gnu.org writes: > >>=20 > >> > efraim pushed a commit to branch master > >> > in repository guix. > >> > > >> > commit 076688fa1e41a09f034a80e1a593bac43f1f1482 > >> > Author: Efraim Flashner > >> > AuthorDate: Thu Jun 1 11:06:00 2023 +0300 > >> > > >> > gnu: openblas: Update architectures we provide substitutes for. > >> > > >> > * gnu/packages/maths.scm (openblas)[arguments]: Adjust the subst= itutable? > >> > flag to only not provide substitutes when building for powerpc-l= inux. > >> > Adjust the comment accordingly. > >> > --- > >> > gnu/packages/maths.scm | 11 ++--------- > >> > 1 file changed, 2 insertions(+), 9 deletions(-) > >>=20 > >> I've been looking at why armhf-linux substitute availability has been > >> dropping recently, and I think this change triggered a lot of > >> rebuilds. Could this have gone to core-updates? > >>=20 > >> =E2=86=92 guix refresh -l openblas > >> Building the following 2282 packages would ensure 5596 dependent packa= ges are rebuilt: ... > > > > It's not that it's triggered rebuilds, but that it's triggered builds. > > It's also triggered builds on powerpc64le and riscv64. Before any > > package which had openblas as a transitive dependency wasn't built by > > the CI because it wasn't substitutable=C2=B9. People still have the opt= ion of > > using package transformations to use openblas tuned for the cortex a7 or > > a15 on armhf, but in reality this just unlocks substitutes for those > > ~5600 packages which wasn't available before. > > > > =C2=B9 We saw this in the past briefly in the past when openzfs made it= s way > > as a dependency to qemu and through that to Gnome. >=20 > Ok, so the documentation does mention "rebuilding", and I do see that > indeed ci.guix.gnu.org doesn't build not substitutable things. >=20 > Although I think it doesn't apply recursively. Take qjson, guix refresh > -l tells me it's dependent on openblas, and looking back at say this > output [1] for powerpc64le-linux, that's available from both > ci.guix.gnu.org. Which makes sense, as that derivation is substitutable, > even though one of it's inputs isn't. >=20 > 1: https://data.guix.gnu.org/gnu/store/fibiwzyz8s899ccpix5zs6r2pcdpxk5b-q= json-0.9.0 >=20 > Maybe on the client side this works differently, and guix won't > substitute things which have a non substitutable input? >=20 > Assuming ci.guix.gnu.org was building things for armhf-linux, I think > this would have still caused ~5596 rebuilds, and as I say, I think for > systems like powerpc64le-linux, I think it did cause ~5596 > rebuilds. I looked into it more. First I ran on master: './pre-inst-env guix build --no-grafts --system=3Darmhf-linux openblas -d' /gnu/store/whi4yhiw2b0c0i3n6l8s0qfcphkvbzg4-openblas-0.3.20.drv Then I locally reverted the patch expanding the architectures where we provided substitutes: /gnu/store/1m57z8jkbf6gz7qlbw3ws4ayl0ln9602-openblas-0.3.20.drv Then I locally reverted the patch adjusting the make-flags: /gnu/store/1m57z8jkbf6gz7qlbw3ws4ayl0ln9602-openblas-0.3.20.drv It seems I was wrong, changing the #:substitutable? flag _does_ change the derivation of the package. I also checked the substitutes and saw that bordeaux did (and does) have substitutes for the non-substitutable version, showing that it was built before. I also checked for powerpc64le, to see if perhaps cuirass worked a different way and honored the #:substitutable flag by not building it there, and it too has substitutes for both versions of openblas. I didn't check for riscv64 but I assume the case is the same with a changing derivation. So obviously if I had realized this would cause ~5596 rebuilds per affected arch I wouldn't have pushed the patch. I should've checked the derivation before and after to make sure it didn't change. --=20 Efraim Flashner =D7=A8=D7=A0=D7=A9=D7=9C=D7=A4 = =D7=9D=D7=99=D7=A8=D7=A4=D7=90 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --yjE9QnCWniboDWQj Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmR9lnIACgkQQarn3Mo9 g1Ekig/+I/uoSML2hq1owZW4g0ayFTRQ8/BfsDso830PT2Re+9IoMmUtuajJWZa+ 2u+VW0y113d81VRQL9qOtvZUvoIbqkEdSCxoJ2FSdcQ7G1x1QyQvvAT8w6DFrVAO PmoTSgB9Hk3NO1iGCUlHjhHN6ATNtBgQLWje1wpQr2kmsDh4IPL8vu6M5IAa9QSn jty8uCOnFYto0A/sncPvjRkDp5vcykjEvuHnI36WBJ8HqOKG6jDzuFe4EQupI8O4 E/PvfUuHgqnrSnzcFKV91MTJg0eiHFzRrQUkwe5zAIuzuhFDPEtAOyd3G5UPiQTq trSMGIEteRaxyy6TwDi1dJG3gNJDwhz8I3H3mPQkYhB6rmiWUPl7JaOXetcLXWof c3I1U2FebEup9nKlMIoSfgIQW35IR771FWyQHbyZS9z9TUr66sarLX2PQzZHYqKn boTq8G7K8OLI84NthcYOq8HMivhY3lKOfKAospdXLQh4xTdNFMoHGyUaInlFNYWT Z0Xo7LdFWdmb+zwoaN6J/jhut5PCWV6tkIcT4KsXSS6Pwkn7h+TIvwAVghDkxnJx DBtNxIfeYkaplUWyYlYI1+MMIURSH/XvI1eVYm3Pgn5nkXlYV/Em+0M87N3N1lLT 8xdYe5X2aKOVQqy0nOgTbvUbXqR0MshnGY7YjysWXHVFqJpP6S4= =9jnR -----END PGP SIGNATURE----- --yjE9QnCWniboDWQj--