unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* New blog post on the Guix Build Coordinator: Building derivations, how complicated can it be?
@ 2021-04-24  6:28 Christopher Baines
  2021-04-24  8:19 ` Vincent Legoll
  0 siblings, 1 reply; 7+ messages in thread
From: Christopher Baines @ 2021-04-24  6:28 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 522 bytes --]

Hey,

With some prompting, there's now a blog post about the Guix Build
Coordinator: 

  https://guix.gnu.org/en/blog/2021/building-derivations-how-complicated-can-it-be/

Since it doesn't have a web interface like the Guix Data Service, and
doesn't directly meet a widespread need, I think it's probably harder
than usual to understand the intent and design , but hopefully this
helps.

Thanks to Ludo for his review and feedback.

Do let me know if you notice any mistakes or have any questions!

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: New blog post on the Guix Build Coordinator: Building derivations, how complicated can it be?
  2021-04-24  6:28 New blog post on the Guix Build Coordinator: Building derivations, how complicated can it be? Christopher Baines
@ 2021-04-24  8:19 ` Vincent Legoll
  2021-04-24  9:10   ` Christopher Baines
  0 siblings, 1 reply; 7+ messages in thread
From: Vincent Legoll @ 2021-04-24  8:19 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guix-devel

Hello,

On Sat, Apr 24, 2021 at 8:28 AM Christopher Baines <mail@cbaines.net> wrote:
> With some prompting, there's now a blog post about the Guix Build
> Coordinator

Nice post that explains a lot, but I'm still not so sure about the
relations to cuirass. I'd have liked a small paragraph explaining
what the differences are, how can they complement each other,
kind of the CI envisionned big picture...

Regards

-- 
Vincent Legoll


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: New blog post on the Guix Build Coordinator: Building derivations, how complicated can it be?
  2021-04-24  8:19 ` Vincent Legoll
@ 2021-04-24  9:10   ` Christopher Baines
  2021-04-24  9:21     ` Vincent Legoll
  0 siblings, 1 reply; 7+ messages in thread
From: Christopher Baines @ 2021-04-24  9:10 UTC (permalink / raw)
  To: Vincent Legoll; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 1760 bytes --]


Vincent Legoll <vincent.legoll@gmail.com> writes:

> Hello,
>
> On Sat, Apr 24, 2021 at 8:28 AM Christopher Baines <mail@cbaines.net> wrote:
>> With some prompting, there's now a blog post about the Guix Build
>> Coordinator
>
> Nice post that explains a lot, but I'm still not so sure about the
> relations to cuirass. I'd have liked a small paragraph explaining
> what the differences are, how can they complement each other,
> kind of the CI envisionned big picture...

Thanks for the feedback Vincent :)

I did think about trying to include something about Cuirass, but I don't
have a clear picture of it's scope or purpose, so I'm not really the
right person to attempt to write authoritatively about it.

On a technical level, there's no connection, although there was talk
over the last 6 months or so about trying to have or allow Cuirass to
benefit from the Guix Build Coordinator's ability to perform builds in a
methodical manor across multiple machines. That hasn't happened yet
though, and in the mean time, Cuirass has gained it's own mechanism of
running builds on other machines.

I try to avoid using the CI (Continuous Integration) term as I'm not
sure there's a shared understanding of the term (I think it's the
practice of multiple people frequently merging their changes to some
software they're all working on).

In terms of building things for substitutes, that's one of the use cases
I had in mind when I designed the Guix Build Coordinator, and I think
that design has worked out well, so I'm still interested in trying to
get some benefits for Guix users through using the Guix Build
Coordinator to produce substitutes in a faster and more reliable way.

Does that make any sense? Do say if you have more questions.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: New blog post on the Guix Build Coordinator: Building derivations, how complicated can it be?
  2021-04-24  9:10   ` Christopher Baines
@ 2021-04-24  9:21     ` Vincent Legoll
  2021-04-25 10:17       ` Mathieu Othacehe
  0 siblings, 1 reply; 7+ messages in thread
From: Vincent Legoll @ 2021-04-24  9:21 UTC (permalink / raw)
  To: Christopher Baines; +Cc: guix-devel

On Sat, Apr 24, 2021 at 11:10 AM Christopher Baines <mail@cbaines.net> wrote:
> I did think about trying to include something about Cuirass, but I don't
> have a clear picture of it's scope or purpose, so I'm not really the
> right person to attempt to write authoritatively about it.

OK, fair enough, Matthieu, Ludo, or anyone else can shed some light
here ?

> I try to avoid using the CI (Continuous Integration) term as I'm not
> sure there's a shared understanding of the term (I think it's the
> practice of multiple people frequently merging their changes to some
> software they're all working on).

Yes, I know the term is overloaded, but it's easy and conveys at least
a bit of the subject at hands, so...

> Does that make any sense? Do say if you have more questions.

Yes, and I will, thanks

-- 
Vincent Legoll


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: New blog post on the Guix Build Coordinator: Building derivations, how complicated can it be?
  2021-04-24  9:21     ` Vincent Legoll
@ 2021-04-25 10:17       ` Mathieu Othacehe
  2021-04-30 14:48         ` Ludovic Courtès
  0 siblings, 1 reply; 7+ messages in thread
From: Mathieu Othacehe @ 2021-04-25 10:17 UTC (permalink / raw)
  To: Vincent Legoll; +Cc: guix-devel


Hello,

> On Sat, Apr 24, 2021 at 11:10 AM Christopher Baines <mail@cbaines.net> wrote:
>> I did think about trying to include something about Cuirass, but I don't
>> have a clear picture of it's scope or purpose, so I'm not really the
>> right person to attempt to write authoritatively about it.
>
> OK, fair enough, Matthieu, Ludo, or anyone else can shed some light
> here ?

You can learn more about Cuirass by reading its online manual[1].
The Cuirass server at https://ci.guix.gnu.org currently:

* Evaluates new Guix derivations on several branches (master,
  core-updates, ...).

* Builds those derivations on a build farm of 29 machines, some of them
  connected using a Wireguard VPN.

* Reports the build status on the Web interface, by email at
  guix-ci@gnu.org, though a Web API used by "guix pull" and "guix
  weather".
  
Maintaining and improving this whole architecture has been keeping me
busy for the last year. It involved a constant monitoring of the build
farm, upgrading the database, and rewriting Cuirass almost completely,
between other things.

The situation is now much better. Cuirass offers a nice substitutes
coverage, at least for Intel architectures, it is stable, well covered
by unit tests and documented.

We have already discussed the Cuirass vs Build coordinator situation in
the past. I haven't changed by mind on that subject. The Guix Build
Coordinator is more or less the equivalent of the Cuirass remote build
mechanism[2].

Integrating the Build coordinator in the Berlin build farm would mean
hooking in to Cuirass as an alternative to the remote build mechanism. I
don't think it would be wise because:

* It wouldn't bring any new features as far as I can tell.

* It would mean maintaining a new SQLite database.

  When Cuirass was using an SQLite database, maintaining it was a
  nightmare. I had to dive into SQLite sources, apply a wide variety of
  hacks[3] to make it scale.

  Even if the Build coordinator is updated to use a PostgreSQL database,
  that would mean having two databases, sitting next to each other,
  with a very similar content, which is a nonsense in my opinion.

* The Build coordinator has no documentation, no unit tests and a large
  code base that would drastically increase the system administrator
  burden.

It really makes me sad that we have two pieces of software that are
stepping on each other toes. The Build coordinator has a nice concept
and represents a huge amount of work. However, integrating it to Berlin
is not an option for me.

I think that the next challenges on the CI front are:

* Increase the number of ARM machines in the build farm.

* Work on substitutes mirrors.

* Find a way to make Cuirass and the Data Service work together.

and we should face those issues together, rather than having competing
software sharing the few build machines we own.

Thanks,

Mathieu

[1]: https://guix.gnu.org/cuirass/manual/html_node/index.html
[2]: https://guix.gnu.org/cuirass/manual/html_node/Build-modes.html#With-the-remote-build-mechanism_002e
[3]: https://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: New blog post on the Guix Build Coordinator: Building derivations, how complicated can it be?
  2021-04-25 10:17       ` Mathieu Othacehe
@ 2021-04-30 14:48         ` Ludovic Courtès
  2021-04-30 20:59           ` Vincent Legoll
  0 siblings, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2021-04-30 14:48 UTC (permalink / raw)
  To: Mathieu Othacehe; +Cc: guix-devel

Hello!

Mathieu Othacehe <othacehe@gnu.org> skribis:

> We have already discussed the Cuirass vs Build coordinator situation in
> the past. I haven't changed by mind on that subject. The Guix Build
> Coordinator is more or less the equivalent of the Cuirass remote build
> mechanism[2].

Yes.  So to answer Vincent’s question with my own words and as an
outsider: the Guix Build Coordinator focuses on distributing builds
across several machines that may come and go dynamically.  This is
similar to cuirass-remote-worker.

One difference I see is that cuirass-remote-worker uses ZMQ for
communication while the Coordinator uses HTTP, which makes it easier to
use in a more distributed and dynamic context (HTTP goes through all
firewalls).  Cuirass-remote-worker can discover the central server via
Avahi discovery, which makes it easy to add new workers but is limited
to LANs.  On berlin, a WireGuard VPN was set up to address this (the
non-x86 build nodes are far away from the rest of the build farm).

Another one is that, because the Coordinator focuses on builds, it
brings specific features, such as retrying failed builds.

I like that the Coordinator is quite orthogonal; you can use it together
with the Data Service, or you can use it as an alternative to the
daemon’s offload mechanism.

Conversely, Cuirass is more of an all-in-one solution.  Depending on how
you look at it, it can be a weakness, but it’s also a strength when it
comes to deploying a “build farm” kind of service (I think it’s a good
fit for ci.guix).  Its monitoring features and dashboards have become
very nice and well adapted to someone willing to build packages from a
bunch of channels.

> Integrating the Build coordinator in the Berlin build farm would mean
> hooking in to Cuirass as an alternative to the remote build mechanism. I
> don't think it would be wise because:

[...]

> It really makes me sad that we have two pieces of software that are
> stepping on each other toes. The Build coordinator has a nice concept
> and represents a huge amount of work. However, integrating it to Berlin
> is not an option for me.
>
> I think that the next challenges on the CI front are:
>
> * Increase the number of ARM machines in the build farm.
>
> * Work on substitutes mirrors.
>
> * Find a way to make Cuirass and the Data Service work together.
>
> and we should face those issues together, rather than having competing
> software sharing the few build machines we own.

Right, I think the fruitful way to frame it is not “which one is
better”, but rather how can we take the good things from each approach.

For example, I believe the Guix Data Service relied on the former
Cuirass notification mechanism to learn about build status.  That
mechanism is gone, so “we” (i.e., you :-) Chris & Mathieu) should make
sure the Data Service can take advantage of the new notification
mechanism, possibly adjusting it so there’s a good match.

There are certainly other opportunities for cooperation in this area.
The Data Service does things that Cuirass doesn’t do, such as linting.
It wouldn’t make sense IMO to extend Cuirass to do these things; instead
we should find ways to aggregate and present all this information in the
Data Service.  It already does that to a large extent, but maybe we can
think of tighter integration, such as providing QA-oriented pages
instead of the more generic interface it currently provides.

Thoughts?

Ludo’.


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: New blog post on the Guix Build Coordinator: Building derivations, how complicated can it be?
  2021-04-30 14:48         ` Ludovic Courtès
@ 2021-04-30 20:59           ` Vincent Legoll
  0 siblings, 0 replies; 7+ messages in thread
From: Vincent Legoll @ 2021-04-30 20:59 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, Mathieu Othacehe

Thanks Christopher, Mathieu & Ludo to help us understand what's going on

-- 
Vincent Legoll


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2021-04-30 21:33 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-24  6:28 New blog post on the Guix Build Coordinator: Building derivations, how complicated can it be? Christopher Baines
2021-04-24  8:19 ` Vincent Legoll
2021-04-24  9:10   ` Christopher Baines
2021-04-24  9:21     ` Vincent Legoll
2021-04-25 10:17       ` Mathieu Othacehe
2021-04-30 14:48         ` Ludovic Courtès
2021-04-30 20:59           ` Vincent Legoll

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).