unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
From: "Ludovic Courtès" <ludo@gnu.org>
To: Christopher Baines <mail@cbaines.net>
Cc: guix-devel@gnu.org
Subject: Re: Bringing substitutes from the Guix Build Coordinator to users
Date: Sun, 02 May 2021 23:51:10 +0200	[thread overview]
Message-ID: <87eeeo24a9.fsf@gnu.org> (raw)
In-Reply-To: <878s4ye116.fsf@cbaines.net> (Christopher Baines's message of "Sat, 01 May 2021 19:56:05 +0100")

Hi Christopher,

Thanks for your message!  It’s great to see a summary of what’s been
cooking in the Coordinator and your vision around it.

Christopher Baines <mail@cbaines.net> skribis:

> More specifically, while the architecture is similar to daemon
> offloading, there are some practical advantages. Since the switch to
> Cuirass remote workers for ci.guix.gnu.org, the advantages of building
> things across multiple machines in a coordinated manor have also become
> clear. Additionally, there are things that the Guix Build Coordinator
> enables, like automated retries, with the option to target specific
> agents, which can help with avoiding issues with non-reproducible
> failures, as well as helping to avoid issues when mixing QEMU emulated
> and native builds.

These are nice examples of current QA blind spots that would be nice to
address.

[...]

> This is a topic I haven't got directly involved in, until now. I'm just
> a volunteer, in some ways the most involved I am is that I host an idle
> ARM build machine, I don't have any particular connection in the default
> approach to substitutes for users, or the hardware currently
> involved.

It would be great to have that OverDrive plugged in behind ci.guix
because we’re still short on CPU power for ARM.  (Not great we had
forgotten about this one!)

It could be reconfigured with a config similar to
<https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/overdrive1.scm>,
with a different host name and IP.  We can discuss the details in a
separate thread and/or on guix-sysadmin, let me know!

> However, I think some of the stuff I've been working on could be
> helpful, as I say, I think the Guix Build Coordinator is a step
> forward in terms of building things for substitutes, and that shows in
> the substitute availability percentages.
>
> Is there still a path to bring some of these benefits to users, and if
> so, what things need doing?

My understanding is that, to build substitutes, we need more than the
Coordinator; we also need something to perform evaluations, similar to
the “top half” of Cuirass and the Data Service, right?  How would you
envision setting it up?

To me, the way forward is not necessarily substitutes; it could be QA,
as you showed that it has helpful features.  You already have access to
bayfront and it’d be great to experiment there.  Do you have ideas on
how to do make progress on that front?


Besides, I feel that the Data Service has a huge role to play with a
variety of applications, including QA, but also way beyond as you showed
in the blog post¹.  I think it’s time take advantage of it.  :-)

Specifically, as far as QA is concerned, I think we should place the
Data Service at the center of the stage.  It already does things that
Cuirass cannot and will not do, notably: linting, challenging substitute
servers to estimate reproducibility, and recording the history of all
this.  Couldn’t it gather these other QA metrics mentioned above,
possibly from a Coordinator-powered bayfront?

What I’d love to see is a set of “skins” for the Data Service:
purpose-specific interfaces to the service.  There’s already the
reproducibility one, the one for package versions, etc.  It would be
great to have a qa.guix.gnu.org entry point with a simplified interface
that would only show (say) reproducibility data, lint warnings, and so
forth.  (Then we can also think of other skins, like Guix Weekly News, a
page dedicated to browsing package version history, a security-related
page, etc.  But now I feel like a spoiled kid asking for yet more
presents.  :-))

How does that sound?

Thanks!

Ludo’.

¹ https://guix.gnu.org/en/blog/2020/introduction-to-the-guix-data-service-the-missing-blog-post/


  reply	other threads:[~2021-05-02 21:51 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-01 18:56 Bringing substitutes from the Guix Build Coordinator to users Christopher Baines
2021-05-02 21:51 ` Ludovic Courtès [this message]
2021-05-03 10:30   ` Christopher Baines
2021-05-04  8:27     ` Ludovic Courtès
2021-05-04 19:22       ` Christopher Baines
2021-05-11 20:18         ` Ludovic Courtès
2021-05-04 18:38 ` Andreas Enge
2021-05-04 19:29   ` Christopher Baines
2021-05-12 22:58     ` Christopher Baines
2021-05-15 16:38       ` Ludovic Courtès
2021-05-15 17:24         ` Christopher Baines
2021-05-17 20:28           ` Ludovic Courtès
2021-05-18  8:26             ` Christopher Baines
2021-05-06 16:26   ` Ludovic Courtès
2021-05-18 19:45 ` [bug#48435] " Christopher Baines
2021-05-18 21:24   ` Ludovic Courtès
2021-05-18 22:29     ` Christopher Baines
2021-05-19  6:54       ` Mathieu Othacehe
2021-05-19  7:57         ` Christopher Baines
2021-06-07 14:53   ` Christopher Baines

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87eeeo24a9.fsf@gnu.org \
    --to=ludo@gnu.org \
    --cc=guix-devel@gnu.org \
    --cc=mail@cbaines.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).