all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Christopher Baines <mail@cbaines.net>
To: zimoun <zimon.toutoune@gmail.com>
Cc: guix-devel@gnu.org
Subject: Re: guix weather -m etc/sources-manifest.scm and CI?
Date: Fri, 17 Sep 2021 10:49:02 +0100	[thread overview]
Message-ID: <87sfy3y0nn.fsf@cbaines.net> (raw)
In-Reply-To: <CAJ3okZ0Zs3f8o0LV95H0SSOJ9vfWHEpRgHvwNNH3hcO8LODsNw@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 8583 bytes --]


zimoun <zimon.toutoune@gmail.com> writes:

> Playing with the new 'etc/sources-manifest.scm', using fb32a38, I get:
>
> $ guix weather -m ~/src/guix/guix/etc/source-manifest.scm
> computing 16,831 package derivations for x86_64-linux...
> looking for 16,831 store items on https://ci.guix.gnu.org...
> https://ci.guix.gnu.org
>   74.6% substitutes available (12,556 out of 16,831)
>   at least 65,367.1 MiB of nars (compressed)
>   81,988.1 MiB on disk (uncompressed)
>   0.095 seconds per request (1,606.8 seconds in total)
>   10.5 requests per second
>
>   0.0% (0 out of 4,275) of the missing items are queued
>   5 queued builds
>       aarch64-linux: 4 (80.0%)
>       powerpc64le-linux: 1 (20.0%)
>   build rate: .00 builds per hour
>       powerpc64le-linux: 0.00 builds per hour
>       aarch64-linux: 0.00 builds per hour
>       i686-linux: 0.00 builds per hour
>       x86_64-linux: 0.00 builds per hour
> looking for 16,831 store items on https://bordeaux.guix.gnu.org...
> https://bordeaux.guix.gnu.org
>   99.8% substitutes available (16,804 out of 16,831)
>   62,195.0 MiB of nars (compressed)
>   108,212.7 MiB on disk (uncompressed)
>   0.049 seconds per request (829.2 seconds in total)
>   20.3 requests per second
>   (continuous integration information unavailable)
>
>
> The questions are:
>
> Why ci.guix.gnu.org contains only 75%?  And bordeaux almost everything?
> (I guess the missing ones on bordeaux are corner cases as icecat, linux-libre).

bordeaux.guix.gnu.org is hopefully only missing substitutes for things
where there's actually an issue building them. It's possible not to
guess though and instead ask guix weather what is missing.

I used time-machine as well for the latest commit processed by
data.guix.gnu.org, just so that the list doesn't include builds which
haven't started yet.

→ guix time-machine --commit=33bc3fb2a5f30a6e21f1b8d6d43867d921bd951c -- weather --substitute-urls=https://bordeaux.guix.gnu.org --display-missing -m ./etc/source-manifest.scm
computing 16,850 package derivations for x86_64-linux...
looking for 16,850 store items on https://bordeaux.guix.gnu.org...
https://bordeaux.guix.gnu.org
  99.8% substitutes available (16,822 out of 16,850)
  62,377.9 MiB of nars (compressed)
  108,593.9 MiB on disk (uncompressed)
  0.160 seconds per request (4.5 seconds in total)
  6.3 requests per second
  (continuous integration information unavailable)

Substitutes are missing for the following items:
  /gnu/store/42knh9b75m6kc30m8v247sswhdfqnn8i-xpp3-1.1.4_src.tgz                                                                                         i686-linux
  /gnu/store/9fgglvi13vhpc63knf15dipzmmck6ia9-mini-os-git-checkout                                                                                       i586-gnu
  /gnu/store/pb5jmi9zalg6xylzsjmrskwxs0kar97l-fossil-src-2.11.tar.gz                                                                                     armhf-linux
  /gnu/store/8s5d2hjd6nhzf6m9dlxrykk8ijkf62pi-texlive-marginnote-51265-checkout                                                                          armhf-linux
  /gnu/store/cp3ka40bhb28rrmyj4mzf9xjhi0ssxjx-CombBLAS_beta_16_2.tgz                                                                                     x86_64-linux
  /gnu/store/f1h94axw82id0k8c2lippg73sqlibqs8-dovecot-trees-2.1.0.tar.gz                                                                                 aarch64-linux
  /gnu/store/f12l7dg2z1xyf2wcw9g3v7jwpgd8m5zv-tla2tools-1.8.0-checkout                                                                                   i686-linux
  /gnu/store/53j9996hdgnmhgzswjjggdz9wnv29p5b-jpegsrc.v9d.tar.gz                                                                                         i586-gnu
  /gnu/store/80s1m4q2hnjfbqzw3fhywvsyim2b00cd-gcc-4.7.4.tar.bz2                                                                                          x86_64-linux
  /gnu/store/csv5xca0p8w5jqqx53szy2dja8lwxma2-unicode-blocks.txt                                                                                         i686-linux
  /gnu/store/xn62dzq9hw3qnvmbxyxjkvhlacs72rz7-canada1500.zip                                                                                             armhf-linux
  /gnu/store/6vqin3by3nkn0sxhgwnzi9l7gflpfw1q-gcc-vc4-6.5.0-checkout                                                                                     x86_64-linux
  /gnu/store/zxfkf2bzq7pp7nhmbdgzvmjp0iv46wds-propeller-gcc-b4f45a4725e0b6d0af59e594c4e3e35ca4105867-checkout                                            i686-linux
  /gnu/store/g98l0m8qsfdqybak2jz3ma2miv9bki4j-emacs-danneskjold-theme-0.0.0-2.e4d1f2c-checkout                                                           x86_64-linux
  /gnu/store/bznj81ls02r7kwld6338dhba0pql7nik-Rserve_1.8-6.tar.gz                                                                                        i686-linux
  /gnu/store/fb0rqvv38pf02qv3zjvjsindjdhlbpca-rda_1.0.2-2.1.tar.gz                                                                                       x86_64-linux
  /gnu/store/m5jn626l55nwxhgs405byka7pzkg1arz-d3.v6.js                                                                                                   powerpc64le-linux
  /gnu/store/bf496ni94ik19gdxkgp0vddkfxbid2vf-d3.v5.js                                                                                                   mips64el-linux
  /gnu/store/c0kig783lv56vv0yc6i8rsvhaxdp52sd-paml4.9e.tgz                                                                                               mips64el-linux
  /gnu/store/17pi83jai4wlnnv1dilqwv1xv6g35d7x-plink-1.07-src.zip                                                                                         aarch64-linux
  /gnu/store/c3k6gxx7yha95f4avia11ls71ndzfx34-ggtree_3.0.3.tar.gz                                                                                        x86_64-linux
  /gnu/store/g2fhcc16yk5asszzmih0cpkwqb3b9bb3-clusterProfiler_4.0.4.tar.gz                                                                               x86_64-linux
  /gnu/store/7n2rk9n46am8qx03ppycvkcdbnh6qzkq-eye-1.4.1.tar.gz                                                                                           aarch64-linux
  /gnu/store/izhvfxx0iiwv8p1xr8ag5mpvsi86x9kx-weightwatcher-1.12.tar.gz                                                                                  mips64el-linux
  /gnu/store/l3r5j8r4f6fh6bxa9va3681n3bclxpdq-stuff-1.26.0.tar.gz                                                                                        i686-linux
  /gnu/store/ymip20qmlz320r2zx0hhfxn70428n76z-skymaker-3.10.5.tar.gz                                                                                     armhf-linux
  /gnu/store/xqc7n8fsq5zxag37a2ay88cx6c3zsm51-mpfrcx-0.6.3.tar.gz                                                                                        x86_64-linux
  /gnu/store/yf2dgn6jcdwvbnzcjwcadc8lwr9ks0wr-eclib-20190909-checkout                                                                                    mips64el-linux


If you visit https://data.guix.gnu.org and add the store path to the
URL, you can get links to the failed builds ([1] for example).

1: https://data.guix.gnu.org/gnu/store/42knh9b75m6kc30m8v247sswhdfqnn8i-xpp3-1.1.4_src.tgz

> Does it make sense to duplicate the storage of all these origins?
>
> Using extensively "guix time-machine", I note that a lot of
> derivations are missing and thus they are built locally, which is
> costly on poor machine.  Could we reduce the duplication and so save
> some space in order to systematically keep these derivations?  It
> would greatly ease the Guix experience for "guix time-machine" users.
> :-)

In terms of duplication, I still think that an argument can be made that
bordeaux.guix.gnu.org is providing value, even if it's doing some of the
same stuff as ci.guix.gnu.org.

As for keeping build results, everything that's ever been built for
bordeaux.guix.gnu.org (that's only ~337739 things totalling ~1.4TiBs) is
still around. In some ways, this is because deleting things is a bit
more difficult, as the files aren't in the store, you can't just do a
guix gc.

However, I do want to try to never delete anything. That's going to
require a bit of work as the local storage on the bayfront machine will
be all used up at some point, but I do have the beginnings of a plan to
avoid this an keep building things.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

  reply	other threads:[~2021-09-17 10:09 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-15 17:03 guix weather -m etc/sources-manifest.scm and CI? zimoun
2021-09-17  9:49 ` Christopher Baines [this message]
2021-09-21  8:32   ` zimoun
2021-09-21 20:36     ` Christopher Baines
2021-09-23 20:18 ` Ludovic Courtès
2021-09-23 20:36   ` zimoun
2021-09-28 12:28     ` Ludovic Courtès
2021-10-01  9:38       ` Mathieu Othacehe
2021-10-02 14:48         ` Ludovic Courtès
2021-10-02 17:08           ` Mathieu Othacehe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87sfy3y0nn.fsf@cbaines.net \
    --to=mail@cbaines.net \
    --cc=guix-devel@gnu.org \
    --cc=zimon.toutoune@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.