Efraim Flashner writes: > On Sat, Jun 03, 2023 at 08:12:48PM +0100, Christopher Baines wrote: >> >> Efraim Flashner writes: >> >> > On Fri, Jun 02, 2023 at 11:03:42PM +0100, Christopher Baines wrote: >> >> >> >> guix-commits@gnu.org writes: >> >> >> >> > efraim pushed a commit to branch master >> >> > in repository guix. >> >> > >> >> > commit 076688fa1e41a09f034a80e1a593bac43f1f1482 >> >> > Author: Efraim Flashner >> >> > AuthorDate: Thu Jun 1 11:06:00 2023 +0300 >> >> > >> >> > gnu: openblas: Update architectures we provide substitutes for. >> >> > >> >> > * gnu/packages/maths.scm (openblas)[arguments]: Adjust the substitutable? >> >> > flag to only not provide substitutes when building for powerpc-linux. >> >> > Adjust the comment accordingly. >> >> > --- >> >> > gnu/packages/maths.scm | 11 ++--------- >> >> > 1 file changed, 2 insertions(+), 9 deletions(-) >> >> >> >> I've been looking at why armhf-linux substitute availability has been >> >> dropping recently, and I think this change triggered a lot of >> >> rebuilds. Could this have gone to core-updates? >> >> >> >> → guix refresh -l openblas >> >> Building the following 2282 packages would ensure 5596 dependent packages are rebuilt: ... >> > >> > It's not that it's triggered rebuilds, but that it's triggered builds. >> > It's also triggered builds on powerpc64le and riscv64. Before any >> > package which had openblas as a transitive dependency wasn't built by >> > the CI because it wasn't substitutable¹. People still have the option of >> > using package transformations to use openblas tuned for the cortex a7 or >> > a15 on armhf, but in reality this just unlocks substitutes for those >> > ~5600 packages which wasn't available before. >> > >> > ¹ We saw this in the past briefly in the past when openzfs made its way >> > as a dependency to qemu and through that to Gnome. >> >> Ok, so the documentation does mention "rebuilding", and I do see that >> indeed ci.guix.gnu.org doesn't build not substitutable things. >> >> Although I think it doesn't apply recursively. Take qjson, guix refresh >> -l tells me it's dependent on openblas, and looking back at say this >> output [1] for powerpc64le-linux, that's available from both >> ci.guix.gnu.org. Which makes sense, as that derivation is substitutable, >> even though one of it's inputs isn't. >> >> 1: https://data.guix.gnu.org/gnu/store/fibiwzyz8s899ccpix5zs6r2pcdpxk5b-qjson-0.9.0 >> >> Maybe on the client side this works differently, and guix won't >> substitute things which have a non substitutable input? >> >> Assuming ci.guix.gnu.org was building things for armhf-linux, I think >> this would have still caused ~5596 rebuilds, and as I say, I think for >> systems like powerpc64le-linux, I think it did cause ~5596 >> rebuilds. > > I looked into it more. First I ran on master: > './pre-inst-env guix build --no-grafts --system=armhf-linux openblas -d' > /gnu/store/whi4yhiw2b0c0i3n6l8s0qfcphkvbzg4-openblas-0.3.20.drv > > Then I locally reverted the patch expanding the architectures where we > provided substitutes: > /gnu/store/1m57z8jkbf6gz7qlbw3ws4ayl0ln9602-openblas-0.3.20.drv > > Then I locally reverted the patch adjusting the make-flags: > /gnu/store/1m57z8jkbf6gz7qlbw3ws4ayl0ln9602-openblas-0.3.20.drv > > It seems I was wrong, changing the #:substitutable? flag _does_ change > the derivation of the package. I also checked the substitutes and saw > that bordeaux did (and does) have substitutes for the non-substitutable > version, showing that it was built before. I also checked for > powerpc64le, to see if perhaps cuirass worked a different way and > honored the #:substitutable flag by not building it there, and it too > has substitutes for both versions of openblas. > > I didn't check for riscv64 but I assume the case is the same with a > changing derivation. > > So obviously if I had realized this would cause ~5596 rebuilds per > affected arch I wouldn't have pushed the patch. I should've checked the > derivation before and after to make sure it didn't change. Thanks for investigating further. There's maybe a secondary issue here about why changing the substitutibility of a package affects it's outputs. That doesn't make much sense to me, but maybe there's a reason.