Re: guix weather -m etc/sources-manifest.scm and CI?

["Ludovic Courtès" <ludo@gnu.org>]

Hello!

zimoun <zimon.toutoune@gmail.com> skribis:

> Playing with the new 'etc/sources-manifest.scm', using fb32a38, I get:
>
> $ guix weather -m ~/src/guix/guix/etc/source-manifest.scm
> computing 16,831 package derivations for x86_64-linux...
> looking for 16,831 store items on https://ci.guix.gnu.org...
> https://ci.guix.gnu.org
>   74.6% substitutes available (12,556 out of 16,831)
>   at least 65,367.1 MiB of nars (compressed)
>   81,988.1 MiB on disk (uncompressed)
>   0.095 seconds per request (1,606.8 seconds in total)
>   10.5 requests per second
>
>   0.0% (0 out of 4,275) of the missing items are queued
>   5 queued builds
>       aarch64-linux: 4 (80.0%)
>       powerpc64le-linux: 1 (20.0%)
>   build rate: .00 builds per hour
>       powerpc64le-linux: 0.00 builds per hour
>       aarch64-linux: 0.00 builds per hour
>       i686-linux: 0.00 builds per hour
>       x86_64-linux: 0.00 builds per hour
> looking for 16,831 store items on https://bordeaux.guix.gnu.org...
> https://bordeaux.guix.gnu.org
>   99.8% substitutes available (16,804 out of 16,831)
>   62,195.0 MiB of nars (compressed)
>   108,212.7 MiB on disk (uncompressed)
>   0.049 seconds per request (829.2 seconds in total)
>   20.3 requests per second
>   (continuous integration information unavailable)
>
>
> The questions are:
>
> Why ci.guix.gnu.org contains only 75%?  And bordeaux almost everything?
> (I guess the missing ones on bordeaux are corner cases as icecat, linux-libre).

I wonder too!

BTW, I meant to send a message here about these two manifests, and then
got caught in a crazy patch-review/bug-fix/work loop.  The goal of this
manifest is to run ‘weather’ and ‘build’ so we can get all the source
and check the status at once, as you found out.

I set up a Cuirass job the other day¹, but due to my aforementioned
disorganization, I haven’t yet taken the time to investigate its
failure.

¹ https://ci.guix.gnu.org/jobset/source

> Does it make sense to duplicate the storage of all these origins?

It surely does.

> PS: about etc/disarchive-manifest.scm, I guess 'all-origins' is missing:
>
> (let ((origins (all-origins)))
>   (manifest
>    (list (manifest-entry
>            (name "disarchive-collection")
>            (version (length origins))
>            (item (disarchive-collection origins))))))

No, there’s:

  (include "source-manifest.scm")

at the top.

So the goal here was to move forward on this front by (1) setting up a
CI jobset, and (2) having an mcron job or something that would
periodically copy the “disarchives” in a directory served by nginx.  We
have had all the code and most of the infrastructure for several months,
so I thought it’s high time to get our act together.  Help welcome!

Thanks,
Ludo’.