unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
From: Vagrant Cascadian <vagrant@debian.org>
To: bug-guix@gnu.org
Cc: guix-devel <guix-devel@gnu.org>
Subject: source tarballs potentially built for each derivation
Date: Sun, 01 Aug 2021 13:45:41 -0700	[thread overview]
Message-ID: <87o8ag6gi2.fsf@yucca> (raw)
In-Reply-To: <87k0ngg6zn.fsf@yucca>

[-- Attachment #1: Type: text/plain, Size: 3206 bytes --]

Turning this conversation into a bug, original thread around here:

  https://lists.gnu.org/archive/html/guix-devel/2021-05/threads.html#00427

On 2021-05-29, Vagrant Cascadian wrote:
> On 2021-05-01, Leo Famulari wrote:
>> On Sat, May 01, 2021 at 06:45:32PM -0700, Vagrant Cascadian wrote:
>>> Pragmatically speaking, on slower platforms this is a huge resource
>>> overhead. So much so that ci.guix.gnu.org *usually* times out when
>>> generating the linux-libre aarch64 tarballs:
>>> 
>>>   https://ci.guix.gnu.org/search?query=system%3Aaarch64-linux+linux-libre-arm64-generic
>>
>> Thanks for letting me know. I didn't know this was happening.
>>
>> The immediate solution is for me to make sure the tarballs have built
>> before committing the updates. I already do this for x86_64 and I can
>> start doing it for aarch64 too.
>
> This has definitely helped sometimes, thanks! I even saw a substitute of
> linux-libre for aarch64 earlier today! :)
>
> Still, I'm noticing another problem with the way way the tarballs are
> generated on ci.guix.gnu.org ...
>
> When it generates a tarball, all the various packages independently try
> to recreate the source tarball; so you have at least fours jobs
> ("linux-libre", "linux-libre-arm64-generic", "linux-libre-headers",
> "linux-libre-bpf") all concurrently trying to build the very same
> very-slow-to-build tarball on ci.guix.gnu.org. Sometimes one of them
> might succeed, but the others may not, and even though one of them
> succeeded, none of the failing ones retry...
>
> Not knowing exactly how ci.guix.gnu.org works, would it make sense to
> create a tarball package instead of the ... computed origin(?) tarball,
> so it could be better represented in the package dependency graph, and
> the various linux-libre-* packages can wait till it is available rather
> than all trying to recreate the same thing?
>
> That still requires the tarball generation to not time out in the first
> place, but maybe it would help with the resource limitations a bit to
> only build the source tarball once per architecture?

This seems to still be an issue for ci.guix.gnu.org, but the
linux-libre* substitutes for aarch64 seem to be available on
https://bordeaux.guix.gnu.org ...

$ guix weather linux-libre linux-libre-arm64-generic
computing 2 package derivations for aarch64-linux...
looking for 2 store items on https://ci.guix.gnu.org...
https://ci.guix.gnu.org
  0.0% substitutes available (0 out of 2)
  unknown substitute sizes
  0.0 MiB on disk (uncompressed)
  0.740 seconds per request (0.7 seconds in total)
  1.4 requests per second

  0.0% (0 out of 2) of the missing items are queued
  1 queued builds
      aarch64-linux: 1 (100.0%)
  build rate: .00 builds per hour
      x86_64-linux: 0.00 builds per hour
      aarch64-linux: 0.00 builds per hour
      i686-linux: 0.00 builds per hour
looking for 2 store items on https://bordeaux.guix.gnu.org...
https://bordeaux.guix.gnu.org
  100.0% substitutes available (2 out of 2)
  83.9 MiB of nars (compressed)
  202.2 MiB on disk (uncompressed)
  (continuous integration information unavailable)


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

  parent reply	other threads:[~2021-08-01 20:47 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-02  1:45 linux-libre source tarballs Vagrant Cascadian
2021-05-02  2:45 ` Leo Famulari
2021-05-02  4:42   ` Leo Famulari
2021-05-02 21:08     ` Ludovic Courtès
2021-05-02 22:26       ` Leo Famulari
2021-05-02 22:38         ` Leo Famulari
2021-05-03 15:44   ` linux-libre source tarballs (disable "deblob-check"?) Leo Famulari
2021-05-03 16:39   ` linux-libre source tarballs Alexandre Oliva
2021-05-03 17:13     ` Leo Famulari
2021-05-06  4:10       ` Alexandre Oliva
2021-05-06 21:23         ` Leo Famulari
2021-05-06 20:30     ` Maxim Cournoyer
2021-05-09  3:27       ` Alexandre Oliva
2021-05-30  4:50   ` Vagrant Cascadian
2021-05-31  3:42     ` Leo Famulari
2021-05-31 19:57       ` Mathieu Othacehe
2021-08-01 20:45     ` Vagrant Cascadian [this message]
2021-08-01 21:04       ` source tarballs potentially built for each derivation Vagrant Cascadian

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87o8ag6gi2.fsf@yucca \
    --to=vagrant@debian.org \
    --cc=bug-guix@gnu.org \
    --cc=guix-devel@gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).