all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* Bringing substitutes from the Guix Build Coordinator to users
@ 2021-05-01 18:56 Christopher Baines
  2021-05-02 21:51 ` Ludovic Courtès
                   ` (2 more replies)
  0 siblings, 3 replies; 20+ messages in thread
From: Christopher Baines @ 2021-05-01 18:56 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 3202 bytes --]

Hey,

The Guix Build Coordinator has been around for a while, it's been
available in Guix for more than 6 months now. The setup I've been using
to test the Guix Build Coordinator for building things for substitutes
(guix.cbaines.net) has been running since mid 2020.

Anecdotally, the test setup I've been running (guix.cbaines.net) has for
a while now, generally had higher percentage substitute availability
compared to ci.guix.gnu.org, as reported by guix weather. I think it
might also be building more things since cross derivations are built
plus i586-gnu builds.

I think there are some benefits for using the Guix Build Coordinator to
build things for substitutes, and it would be good to work out how to
get those benefits to users of Guix generally.

More specifically, while the architecture is similar to daemon
offloading, there are some practical advantages. Since the switch to
Cuirass remote workers for ci.guix.gnu.org, the advantages of building
things across multiple machines in a coordinated manor have also become
clear. Additionally, there are things that the Guix Build Coordinator
enables, like automated retries, with the option to target specific
agents, which can help with avoiding issues with non-reproducible
failures, as well as helping to avoid issues when mixing QEMU emulated
and native builds.

Going back to discussions in 2020, I think there was a path for starting
to experiment with the Guix Build Coordinator, trying it on a few
machines in the berlin build farm, but as I say, this was last year and
before any work on implementing similar functionality in Cuirass as far
as I'm aware. Since then, I've also got an instance of the Guix Build
Coordinator running on bayfront with a couple of extra machines also
involved for performing builds, but this is just starting off, and
doesn't bring any benefit in terms of substitutes, at least not yet.

This is a small part of the wider puzzle, when I was designing the Guix
Build Coordinator, it was meant for much more than building things for
substitutes, and those things are also happening. The Guix Build
Coordinator has enabled things like building packages affected by
patches, building fixed output derivations regularly, and will hopefully
enable a whole load of quality assurance things. But my hope was also to
try and tackle building things for substitutes, such that there's not
more software to maintain, and also to inform future work on the
guix-daemon itself, like whether it would be good for it to have an
asynchronous API, and how that might work.

This is a topic I haven't got directly involved in, until now. I'm just
a volunteer, in some ways the most involved I am is that I host an idle
ARM build machine, I don't have any particular connection in the default
approach to substitutes for users, or the hardware currently
involved. However, I think some of the stuff I've been working on could
be helpful, as I say, I think the Guix Build Coordinator is a step
forward in terms of building things for substitutes, and that shows in
the substitute availability percentages.

Is there still a path to bring some of these benefits to users, and if
so, what things need doing?

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2021-06-07 14:53 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-01 18:56 Bringing substitutes from the Guix Build Coordinator to users Christopher Baines
2021-05-02 21:51 ` Ludovic Courtès
2021-05-03 10:30   ` Christopher Baines
2021-05-04  8:27     ` Ludovic Courtès
2021-05-04 19:22       ` Christopher Baines
2021-05-11 20:18         ` Ludovic Courtès
2021-05-04 18:38 ` Andreas Enge
2021-05-04 19:29   ` Christopher Baines
2021-05-12 22:58     ` Christopher Baines
2021-05-15 16:38       ` Ludovic Courtès
2021-05-15 17:24         ` Christopher Baines
2021-05-17 20:28           ` Ludovic Courtès
2021-05-18  8:26             ` Christopher Baines
2021-05-06 16:26   ` Ludovic Courtès
2021-05-18 19:45 ` [bug#48435] " Christopher Baines
2021-05-18 21:24   ` Ludovic Courtès
2021-05-18 22:29     ` Christopher Baines
2021-05-19  6:54       ` Mathieu Othacehe
2021-05-19  7:57         ` Christopher Baines
2021-06-07 14:53   ` Christopher Baines

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.