Re: 02/03: gnu: openblas: Update architectures we provide substitutes for.

[Efraim Flashner <efraim@flashner.co.il>]

[-- Attachment #1: Type: text/plain, Size: 4286 bytes --]

On Sat, Jun 03, 2023 at 08:12:48PM +0100, Christopher Baines wrote:
> 
> Efraim Flashner <efraim@flashner.co.il> writes:
> 
> > On Fri, Jun 02, 2023 at 11:03:42PM +0100, Christopher Baines wrote:
> >> 
> >> guix-commits@gnu.org writes:
> >> 
> >> > efraim pushed a commit to branch master
> >> > in repository guix.
> >> >
> >> > commit 076688fa1e41a09f034a80e1a593bac43f1f1482
> >> > Author: Efraim Flashner <efraim@flashner.co.il>
> >> > AuthorDate: Thu Jun 1 11:06:00 2023 +0300
> >> >
> >> >     gnu: openblas: Update architectures we provide substitutes for.
> >> >
> >> >     * gnu/packages/maths.scm (openblas)[arguments]: Adjust the substitutable?
> >> >     flag to only not provide substitutes when building for powerpc-linux.
> >> >     Adjust the comment accordingly.
> >> > ---
> >> >  gnu/packages/maths.scm | 11 ++---------
> >> >  1 file changed, 2 insertions(+), 9 deletions(-)
> >> 
> >> I've been looking at why armhf-linux substitute availability has been
> >> dropping recently, and I think this change triggered a lot of
> >> rebuilds. Could this have gone to core-updates?
> >> 
> >> → guix refresh -l openblas
> >> Building the following 2282 packages would ensure 5596 dependent packages are rebuilt: ...
> >
> > It's not that it's triggered rebuilds, but that it's triggered builds.
> > It's also triggered builds on powerpc64le and riscv64. Before any
> > package which had openblas as a transitive dependency wasn't built by
> > the CI because it wasn't substitutable¹. People still have the option of
> > using package transformations to use openblas tuned for the cortex a7 or
> > a15 on armhf, but in reality this just unlocks substitutes for those
> > ~5600 packages which wasn't available before.
> >
> > ¹ We saw this in the past briefly in the past when openzfs made its way
> > as a dependency to qemu and through that to Gnome.
> 
> Ok, so the documentation does mention "rebuilding", and I do see that
> indeed ci.guix.gnu.org doesn't build not substitutable things.
> 
> Although I think it doesn't apply recursively. Take qjson, guix refresh
> -l tells me it's dependent on openblas, and looking back at say this
> output [1] for powerpc64le-linux, that's available from both
> ci.guix.gnu.org. Which makes sense, as that derivation is substitutable,
> even though one of it's inputs isn't.
> 
> 1: https://data.guix.gnu.org/gnu/store/fibiwzyz8s899ccpix5zs6r2pcdpxk5b-qjson-0.9.0
> 
> Maybe on the client side this works differently, and guix won't
> substitute things which have a non substitutable input?
> 
> Assuming ci.guix.gnu.org was building things for armhf-linux, I think
> this would have still caused ~5596 rebuilds, and as I say, I think for
> systems like powerpc64le-linux, I think it did cause ~5596
> rebuilds.

I looked into it more. First I ran on master:
'./pre-inst-env guix build --no-grafts --system=armhf-linux openblas -d'
/gnu/store/whi4yhiw2b0c0i3n6l8s0qfcphkvbzg4-openblas-0.3.20.drv

Then I locally reverted the patch expanding the architectures where we
provided substitutes:
/gnu/store/1m57z8jkbf6gz7qlbw3ws4ayl0ln9602-openblas-0.3.20.drv

Then I locally reverted the patch adjusting the make-flags:
/gnu/store/1m57z8jkbf6gz7qlbw3ws4ayl0ln9602-openblas-0.3.20.drv

It seems I was wrong, changing the #:substitutable? flag _does_ change
the derivation of the package. I also checked the substitutes and saw
that bordeaux did (and does) have substitutes for the non-substitutable
version, showing that it was built before. I also checked for
powerpc64le, to see if perhaps cuirass worked a different way and
honored the #:substitutable flag by not building it there, and it too
has substitutes for both versions of openblas.

I didn't check for riscv64 but I assume the case is the same with a
changing derivation.

So obviously if I had realized this would cause ~5596 rebuilds per
affected arch I wouldn't have pushed the patch. I should've checked the
derivation before and after to make sure it didn't change.

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]