From: Christopher Baines <mail@cbaines.net>
To: zimoun <zimon.toutoune@gmail.com>
Cc: guix-devel@gnu.org
Subject: Re: guix weather -m etc/sources-manifest.scm and CI?
Date: Fri, 17 Sep 2021 10:49:02 +0100 [thread overview]
Message-ID: <87sfy3y0nn.fsf@cbaines.net> (raw)
In-Reply-To: <CAJ3okZ0Zs3f8o0LV95H0SSOJ9vfWHEpRgHvwNNH3hcO8LODsNw@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 8583 bytes --]
zimoun <zimon.toutoune@gmail.com> writes:
> Playing with the new 'etc/sources-manifest.scm', using fb32a38, I get:
>
> $ guix weather -m ~/src/guix/guix/etc/source-manifest.scm
> computing 16,831 package derivations for x86_64-linux...
> looking for 16,831 store items on https://ci.guix.gnu.org...
> https://ci.guix.gnu.org
> 74.6% substitutes available (12,556 out of 16,831)
> at least 65,367.1 MiB of nars (compressed)
> 81,988.1 MiB on disk (uncompressed)
> 0.095 seconds per request (1,606.8 seconds in total)
> 10.5 requests per second
>
> 0.0% (0 out of 4,275) of the missing items are queued
> 5 queued builds
> aarch64-linux: 4 (80.0%)
> powerpc64le-linux: 1 (20.0%)
> build rate: .00 builds per hour
> powerpc64le-linux: 0.00 builds per hour
> aarch64-linux: 0.00 builds per hour
> i686-linux: 0.00 builds per hour
> x86_64-linux: 0.00 builds per hour
> looking for 16,831 store items on https://bordeaux.guix.gnu.org...
> https://bordeaux.guix.gnu.org
> 99.8% substitutes available (16,804 out of 16,831)
> 62,195.0 MiB of nars (compressed)
> 108,212.7 MiB on disk (uncompressed)
> 0.049 seconds per request (829.2 seconds in total)
> 20.3 requests per second
> (continuous integration information unavailable)
>
>
> The questions are:
>
> Why ci.guix.gnu.org contains only 75%? And bordeaux almost everything?
> (I guess the missing ones on bordeaux are corner cases as icecat, linux-libre).
bordeaux.guix.gnu.org is hopefully only missing substitutes for things
where there's actually an issue building them. It's possible not to
guess though and instead ask guix weather what is missing.
I used time-machine as well for the latest commit processed by
data.guix.gnu.org, just so that the list doesn't include builds which
haven't started yet.
→ guix time-machine --commit=33bc3fb2a5f30a6e21f1b8d6d43867d921bd951c -- weather --substitute-urls=https://bordeaux.guix.gnu.org --display-missing -m ./etc/source-manifest.scm
computing 16,850 package derivations for x86_64-linux...
looking for 16,850 store items on https://bordeaux.guix.gnu.org...
https://bordeaux.guix.gnu.org
99.8% substitutes available (16,822 out of 16,850)
62,377.9 MiB of nars (compressed)
108,593.9 MiB on disk (uncompressed)
0.160 seconds per request (4.5 seconds in total)
6.3 requests per second
(continuous integration information unavailable)
Substitutes are missing for the following items:
/gnu/store/42knh9b75m6kc30m8v247sswhdfqnn8i-xpp3-1.1.4_src.tgz i686-linux
/gnu/store/9fgglvi13vhpc63knf15dipzmmck6ia9-mini-os-git-checkout i586-gnu
/gnu/store/pb5jmi9zalg6xylzsjmrskwxs0kar97l-fossil-src-2.11.tar.gz armhf-linux
/gnu/store/8s5d2hjd6nhzf6m9dlxrykk8ijkf62pi-texlive-marginnote-51265-checkout armhf-linux
/gnu/store/cp3ka40bhb28rrmyj4mzf9xjhi0ssxjx-CombBLAS_beta_16_2.tgz x86_64-linux
/gnu/store/f1h94axw82id0k8c2lippg73sqlibqs8-dovecot-trees-2.1.0.tar.gz aarch64-linux
/gnu/store/f12l7dg2z1xyf2wcw9g3v7jwpgd8m5zv-tla2tools-1.8.0-checkout i686-linux
/gnu/store/53j9996hdgnmhgzswjjggdz9wnv29p5b-jpegsrc.v9d.tar.gz i586-gnu
/gnu/store/80s1m4q2hnjfbqzw3fhywvsyim2b00cd-gcc-4.7.4.tar.bz2 x86_64-linux
/gnu/store/csv5xca0p8w5jqqx53szy2dja8lwxma2-unicode-blocks.txt i686-linux
/gnu/store/xn62dzq9hw3qnvmbxyxjkvhlacs72rz7-canada1500.zip armhf-linux
/gnu/store/6vqin3by3nkn0sxhgwnzi9l7gflpfw1q-gcc-vc4-6.5.0-checkout x86_64-linux
/gnu/store/zxfkf2bzq7pp7nhmbdgzvmjp0iv46wds-propeller-gcc-b4f45a4725e0b6d0af59e594c4e3e35ca4105867-checkout i686-linux
/gnu/store/g98l0m8qsfdqybak2jz3ma2miv9bki4j-emacs-danneskjold-theme-0.0.0-2.e4d1f2c-checkout x86_64-linux
/gnu/store/bznj81ls02r7kwld6338dhba0pql7nik-Rserve_1.8-6.tar.gz i686-linux
/gnu/store/fb0rqvv38pf02qv3zjvjsindjdhlbpca-rda_1.0.2-2.1.tar.gz x86_64-linux
/gnu/store/m5jn626l55nwxhgs405byka7pzkg1arz-d3.v6.js powerpc64le-linux
/gnu/store/bf496ni94ik19gdxkgp0vddkfxbid2vf-d3.v5.js mips64el-linux
/gnu/store/c0kig783lv56vv0yc6i8rsvhaxdp52sd-paml4.9e.tgz mips64el-linux
/gnu/store/17pi83jai4wlnnv1dilqwv1xv6g35d7x-plink-1.07-src.zip aarch64-linux
/gnu/store/c3k6gxx7yha95f4avia11ls71ndzfx34-ggtree_3.0.3.tar.gz x86_64-linux
/gnu/store/g2fhcc16yk5asszzmih0cpkwqb3b9bb3-clusterProfiler_4.0.4.tar.gz x86_64-linux
/gnu/store/7n2rk9n46am8qx03ppycvkcdbnh6qzkq-eye-1.4.1.tar.gz aarch64-linux
/gnu/store/izhvfxx0iiwv8p1xr8ag5mpvsi86x9kx-weightwatcher-1.12.tar.gz mips64el-linux
/gnu/store/l3r5j8r4f6fh6bxa9va3681n3bclxpdq-stuff-1.26.0.tar.gz i686-linux
/gnu/store/ymip20qmlz320r2zx0hhfxn70428n76z-skymaker-3.10.5.tar.gz armhf-linux
/gnu/store/xqc7n8fsq5zxag37a2ay88cx6c3zsm51-mpfrcx-0.6.3.tar.gz x86_64-linux
/gnu/store/yf2dgn6jcdwvbnzcjwcadc8lwr9ks0wr-eclib-20190909-checkout mips64el-linux
If you visit https://data.guix.gnu.org and add the store path to the
URL, you can get links to the failed builds ([1] for example).
1: https://data.guix.gnu.org/gnu/store/42knh9b75m6kc30m8v247sswhdfqnn8i-xpp3-1.1.4_src.tgz
> Does it make sense to duplicate the storage of all these origins?
>
> Using extensively "guix time-machine", I note that a lot of
> derivations are missing and thus they are built locally, which is
> costly on poor machine. Could we reduce the duplication and so save
> some space in order to systematically keep these derivations? It
> would greatly ease the Guix experience for "guix time-machine" users.
> :-)
In terms of duplication, I still think that an argument can be made that
bordeaux.guix.gnu.org is providing value, even if it's doing some of the
same stuff as ci.guix.gnu.org.
As for keeping build results, everything that's ever been built for
bordeaux.guix.gnu.org (that's only ~337739 things totalling ~1.4TiBs) is
still around. In some ways, this is because deleting things is a bit
more difficult, as the files aren't in the store, you can't just do a
guix gc.
However, I do want to try to never delete anything. That's going to
require a bit of work as the local storage on the bayfront machine will
be all used up at some point, but I do have the beginnings of a plan to
avoid this an keep building things.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]
next prev parent reply other threads:[~2021-09-17 10:09 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-15 17:03 guix weather -m etc/sources-manifest.scm and CI? zimoun
2021-09-17 9:49 ` Christopher Baines [this message]
2021-09-21 8:32 ` zimoun
2021-09-21 20:36 ` Christopher Baines
2021-09-23 20:18 ` Ludovic Courtès
2021-09-23 20:36 ` zimoun
2021-09-28 12:28 ` Ludovic Courtès
2021-10-01 9:38 ` Mathieu Othacehe
2021-10-02 14:48 ` Ludovic Courtès
2021-10-02 17:08 ` Mathieu Othacehe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87sfy3y0nn.fsf@cbaines.net \
--to=mail@cbaines.net \
--cc=guix-devel@gnu.org \
--cc=zimon.toutoune@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.