unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Building the core-updates branch.
@ 2021-02-06  8:06 Mathieu Othacehe
  2021-02-06 11:11 ` Mathieu Othacehe
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Mathieu Othacehe @ 2021-02-06  8:06 UTC (permalink / raw)
  To: guix-devel


Hello,

The evaluation of the core-updates branch on Cuirass has been broken for
a long time. It now appears to be fixed[1]. This means that around 45k
packages must be built. Each change on this branch can possibly trigger
the same amount of builds.

Even if the core-updates builds have to the lowest priority on the CI,
and the CI is performing a bit better, I'm not sure building this branch
is reasonable for now. I feel like we should first focus on providing
substitutes for the master, guix-modular-master and staging branches
consistently and on all architectures before trying to build
core-updates.

WDYT?

Thanks,

Mathieu

[1]: https://ci.guix.gnu.org/jobset/core-updates-core-updates


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Building the core-updates branch.
  2021-02-06  8:06 Building the core-updates branch Mathieu Othacehe
@ 2021-02-06 11:11 ` Mathieu Othacehe
  2021-02-06 21:48   ` Ludovic Courtès
  2021-02-06 22:21   ` Ricardo Wurmus
  2021-02-06 13:40 ` zimoun
  2021-02-06 21:50 ` Ludovic Courtès
  2 siblings, 2 replies; 9+ messages in thread
From: Mathieu Othacehe @ 2021-02-06 11:11 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 608 bytes --]


> Even if the core-updates builds have to the lowest priority on the CI,
> and the CI is performing a bit better, I'm not sure building this branch
> is reasonable for now. I feel like we should first focus on providing
> substitutes for the master, guix-modular-master and staging branches
> consistently and on all architectures before trying to build
> core-updates.

Here's a small graph to illustrate the CI throughput improvement: the
CPU idle time on all the CI build nodes for the last 6 months. The build
remote mechanism that has been deployed in early January shows clear
improvements.

Mathieu


[-- Attachment #2: idle.png --]
[-- Type: image/png, Size: 145255 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Building the core-updates branch.
  2021-02-06  8:06 Building the core-updates branch Mathieu Othacehe
  2021-02-06 11:11 ` Mathieu Othacehe
@ 2021-02-06 13:40 ` zimoun
  2021-02-06 21:50 ` Ludovic Courtès
  2 siblings, 0 replies; 9+ messages in thread
From: zimoun @ 2021-02-06 13:40 UTC (permalink / raw)
  To: Mathieu Othacehe, guix-devel

Hi Mathieu,

On Sat, 06 Feb 2021 at 09:06, Mathieu Othacehe <othacehe@gnu.org> wrote:

> Even if the core-updates builds have to the lowest priority on the CI,
> and the CI is performing a bit better, I'm not sure building this branch
> is reasonable for now. I feel like we should first focus on providing
> substitutes for the master, guix-modular-master and staging branches
> consistently and on all architectures before trying to build
> core-updates.

Back on October, Chris proposed this:

    <https://yhetil.org/guix/877drg3lmi.fsf@cbaines.net/>

and it seems similar.  Roughly speaking, let only build core-updates (on
Berlin) when the freeze starts just before the merge.

About the CI, the “production” (= substitutes of master for all the
architectures) and the “testing” (= substitutes of core-updates and
wip-*) should be clearly 2 different things.  Maybe on Berlin and
Bayfront.


All the best,
simon


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Building the core-updates branch.
  2021-02-06 11:11 ` Mathieu Othacehe
@ 2021-02-06 21:48   ` Ludovic Courtès
  2021-02-06 22:21   ` Ricardo Wurmus
  1 sibling, 0 replies; 9+ messages in thread
From: Ludovic Courtès @ 2021-02-06 21:48 UTC (permalink / raw)
  To: Mathieu Othacehe; +Cc: guix-devel

Mathieu Othacehe <othacehe@gnu.org> skribis:

> Here's a small graph to illustrate the CI throughput improvement: the
> CPU idle time on all the CI build nodes for the last 6 months. The build
> remote mechanism that has been deployed in early January shows clear
> improvements.

Woow, impressive!

Ludo’.


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Building the core-updates branch.
  2021-02-06  8:06 Building the core-updates branch Mathieu Othacehe
  2021-02-06 11:11 ` Mathieu Othacehe
  2021-02-06 13:40 ` zimoun
@ 2021-02-06 21:50 ` Ludovic Courtès
  2021-02-07 18:22   ` Mathieu Othacehe
  2 siblings, 1 reply; 9+ messages in thread
From: Ludovic Courtès @ 2021-02-06 21:50 UTC (permalink / raw)
  To: Mathieu Othacehe; +Cc: guix-devel

Mathieu Othacehe <othacehe@gnu.org> skribis:

> The evaluation of the core-updates branch on Cuirass has been broken for
> a long time. It now appears to be fixed[1]. This means that around 45k
> packages must be built. Each change on this branch can possibly trigger
> the same amount of builds.
>
> Even if the core-updates builds have to the lowest priority on the CI,
> and the CI is performing a bit better, I'm not sure building this branch
> is reasonable for now. I feel like we should first focus on providing
> substitutes for the master, guix-modular-master and staging branches
> consistently and on all architectures before trying to build
> core-updates.

Agreed, I think that was the intent.  Can we make it focus on the ‘core’
subset?  How does one do that nowadays?  Looks like my sqlite3/Cuirass
foo is no longer helpful.  :-)

Ludo’.


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Building the core-updates branch.
  2021-02-06 11:11 ` Mathieu Othacehe
  2021-02-06 21:48   ` Ludovic Courtès
@ 2021-02-06 22:21   ` Ricardo Wurmus
  1 sibling, 0 replies; 9+ messages in thread
From: Ricardo Wurmus @ 2021-02-06 22:21 UTC (permalink / raw)
  To: Mathieu Othacehe; +Cc: guix-devel


Mathieu Othacehe <othacehe@gnu.org> writes:

>> Even if the core-updates builds have to the lowest priority on the CI,
>> and the CI is performing a bit better, I'm not sure building this branch
>> is reasonable for now. I feel like we should first focus on providing
>> substitutes for the master, guix-modular-master and staging branches
>> consistently and on all architectures before trying to build
>> core-updates.
>
> Here's a small graph to illustrate the CI throughput improvement: the
> CPU idle time on all the CI build nodes for the last 6 months. The build
> remote mechanism that has been deployed in early January shows clear
> improvements.

Wow, this is wonderful!  I’m glad these machines finally get a workout.
Thank you!

-- 
Ricardo


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Building the core-updates branch.
  2021-02-06 21:50 ` Ludovic Courtès
@ 2021-02-07 18:22   ` Mathieu Othacehe
  2021-02-07 18:43     ` Mathieu Othacehe
  0 siblings, 1 reply; 9+ messages in thread
From: Mathieu Othacehe @ 2021-02-07 18:22 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel


Hey Ludo,

> Agreed, I think that was the intent.  Can we make it focus on the ‘core’
> subset?  How does one do that nowadays?  Looks like my sqlite3/Cuirass
> foo is no longer helpful.  :-)

Turns out "core-updates" is still configured to build the "core" subset,
I don't really understand why the evaluation 74281 triggered the build
of so many packages.

Regarding the sqlite3 -> psql transition, there are not that much
differences. I fire up psql this way:

--8<---------------cut here---------------start------------->8---
sudo -u cuirass -s /bin/sh -c 'psql cuirass -h /var/run/postgresql'
--8<---------------cut here---------------end--------------->8---

The commands that are listed here[1] are still (mostly) relevant, even
if this document would really need to be updated. I plan to improve the
Cuirass documentation for a future 1.0 release anyways.

Thanks,

Mathieu

[1]: https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/cuirass-howto.org


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Building the core-updates branch.
  2021-02-07 18:22   ` Mathieu Othacehe
@ 2021-02-07 18:43     ` Mathieu Othacehe
  2021-02-07 19:29       ` Leo Famulari
  0 siblings, 1 reply; 9+ messages in thread
From: Mathieu Othacehe @ 2021-02-07 18:43 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel


> Turns out "core-updates" is still configured to build the "core" subset,
> I don't really understand why the evaluation 74281 triggered the build
> of so many packages.

Just fixed it with 2c30440f89244202bddebe76f90c6b0b8145ba5d on
maintenance repository.

We will now be building 87 jobs instead of 44290 on core-updates.

Mathieu


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Building the core-updates branch.
  2021-02-07 18:43     ` Mathieu Othacehe
@ 2021-02-07 19:29       ` Leo Famulari
  0 siblings, 0 replies; 9+ messages in thread
From: Leo Famulari @ 2021-02-07 19:29 UTC (permalink / raw)
  To: Mathieu Othacehe; +Cc: guix-devel

On Sun, Feb 07, 2021 at 07:43:50PM +0100, Mathieu Othacehe wrote:
> 
> > Turns out "core-updates" is still configured to build the "core" subset,
> > I don't really understand why the evaluation 74281 triggered the build
> > of so many packages.
> 
> Just fixed it with 2c30440f89244202bddebe76f90c6b0b8145ba5d on
> maintenance repository.
> 
> We will now be building 87 jobs instead of 44290 on core-updates.

Awesome!


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2021-02-07 19:29 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-06  8:06 Building the core-updates branch Mathieu Othacehe
2021-02-06 11:11 ` Mathieu Othacehe
2021-02-06 21:48   ` Ludovic Courtès
2021-02-06 22:21   ` Ricardo Wurmus
2021-02-06 13:40 ` zimoun
2021-02-06 21:50 ` Ludovic Courtès
2021-02-07 18:22   ` Mathieu Othacehe
2021-02-07 18:43     ` Mathieu Othacehe
2021-02-07 19:29       ` Leo Famulari

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).