* source tarballs potentially built for each derivation
[not found] ` <87k0ngg6zn.fsf@yucca>
@ 2021-08-01 20:45 ` Vagrant Cascadian
2021-08-01 21:04 ` Vagrant Cascadian
0 siblings, 1 reply; 2+ messages in thread
From: Vagrant Cascadian @ 2021-08-01 20:45 UTC (permalink / raw)
To: bug-guix; +Cc: guix-devel
[-- Attachment #1: Type: text/plain, Size: 3206 bytes --]
Turning this conversation into a bug, original thread around here:
https://lists.gnu.org/archive/html/guix-devel/2021-05/threads.html#00427
On 2021-05-29, Vagrant Cascadian wrote:
> On 2021-05-01, Leo Famulari wrote:
>> On Sat, May 01, 2021 at 06:45:32PM -0700, Vagrant Cascadian wrote:
>>> Pragmatically speaking, on slower platforms this is a huge resource
>>> overhead. So much so that ci.guix.gnu.org *usually* times out when
>>> generating the linux-libre aarch64 tarballs:
>>>
>>> https://ci.guix.gnu.org/search?query=system%3Aaarch64-linux+linux-libre-arm64-generic
>>
>> Thanks for letting me know. I didn't know this was happening.
>>
>> The immediate solution is for me to make sure the tarballs have built
>> before committing the updates. I already do this for x86_64 and I can
>> start doing it for aarch64 too.
>
> This has definitely helped sometimes, thanks! I even saw a substitute of
> linux-libre for aarch64 earlier today! :)
>
> Still, I'm noticing another problem with the way way the tarballs are
> generated on ci.guix.gnu.org ...
>
> When it generates a tarball, all the various packages independently try
> to recreate the source tarball; so you have at least fours jobs
> ("linux-libre", "linux-libre-arm64-generic", "linux-libre-headers",
> "linux-libre-bpf") all concurrently trying to build the very same
> very-slow-to-build tarball on ci.guix.gnu.org. Sometimes one of them
> might succeed, but the others may not, and even though one of them
> succeeded, none of the failing ones retry...
>
> Not knowing exactly how ci.guix.gnu.org works, would it make sense to
> create a tarball package instead of the ... computed origin(?) tarball,
> so it could be better represented in the package dependency graph, and
> the various linux-libre-* packages can wait till it is available rather
> than all trying to recreate the same thing?
>
> That still requires the tarball generation to not time out in the first
> place, but maybe it would help with the resource limitations a bit to
> only build the source tarball once per architecture?
This seems to still be an issue for ci.guix.gnu.org, but the
linux-libre* substitutes for aarch64 seem to be available on
https://bordeaux.guix.gnu.org ...
$ guix weather linux-libre linux-libre-arm64-generic
computing 2 package derivations for aarch64-linux...
looking for 2 store items on https://ci.guix.gnu.org...
https://ci.guix.gnu.org
0.0% substitutes available (0 out of 2)
unknown substitute sizes
0.0 MiB on disk (uncompressed)
0.740 seconds per request (0.7 seconds in total)
1.4 requests per second
0.0% (0 out of 2) of the missing items are queued
1 queued builds
aarch64-linux: 1 (100.0%)
build rate: .00 builds per hour
x86_64-linux: 0.00 builds per hour
aarch64-linux: 0.00 builds per hour
i686-linux: 0.00 builds per hour
looking for 2 store items on https://bordeaux.guix.gnu.org...
https://bordeaux.guix.gnu.org
100.0% substitutes available (2 out of 2)
83.9 MiB of nars (compressed)
202.2 MiB on disk (uncompressed)
(continuous integration information unavailable)
live well,
vagrant
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread