* Building the core-updates branch.
@ 2021-02-06 8:06 Mathieu Othacehe
2021-02-06 11:11 ` Mathieu Othacehe
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: Mathieu Othacehe @ 2021-02-06 8:06 UTC (permalink / raw)
To: guix-devel
Hello,
The evaluation of the core-updates branch on Cuirass has been broken for
a long time. It now appears to be fixed[1]. This means that around 45k
packages must be built. Each change on this branch can possibly trigger
the same amount of builds.
Even if the core-updates builds have to the lowest priority on the CI,
and the CI is performing a bit better, I'm not sure building this branch
is reasonable for now. I feel like we should first focus on providing
substitutes for the master, guix-modular-master and staging branches
consistently and on all architectures before trying to build
core-updates.
WDYT?
Thanks,
Mathieu
[1]: https://ci.guix.gnu.org/jobset/core-updates-core-updates
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Building the core-updates branch.
2021-02-06 8:06 Building the core-updates branch Mathieu Othacehe
@ 2021-02-06 11:11 ` Mathieu Othacehe
2021-02-06 21:48 ` Ludovic Courtès
2021-02-06 22:21 ` Ricardo Wurmus
2021-02-06 13:40 ` zimoun
2021-02-06 21:50 ` Ludovic Courtès
2 siblings, 2 replies; 9+ messages in thread
From: Mathieu Othacehe @ 2021-02-06 11:11 UTC (permalink / raw)
To: guix-devel
[-- Attachment #1: Type: text/plain, Size: 608 bytes --]
> Even if the core-updates builds have to the lowest priority on the CI,
> and the CI is performing a bit better, I'm not sure building this branch
> is reasonable for now. I feel like we should first focus on providing
> substitutes for the master, guix-modular-master and staging branches
> consistently and on all architectures before trying to build
> core-updates.
Here's a small graph to illustrate the CI throughput improvement: the
CPU idle time on all the CI build nodes for the last 6 months. The build
remote mechanism that has been deployed in early January shows clear
improvements.
Mathieu
[-- Attachment #2: idle.png --]
[-- Type: image/png, Size: 145255 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Building the core-updates branch.
2021-02-06 8:06 Building the core-updates branch Mathieu Othacehe
2021-02-06 11:11 ` Mathieu Othacehe
@ 2021-02-06 13:40 ` zimoun
2021-02-06 21:50 ` Ludovic Courtès
2 siblings, 0 replies; 9+ messages in thread
From: zimoun @ 2021-02-06 13:40 UTC (permalink / raw)
To: Mathieu Othacehe, guix-devel
Hi Mathieu,
On Sat, 06 Feb 2021 at 09:06, Mathieu Othacehe <othacehe@gnu.org> wrote:
> Even if the core-updates builds have to the lowest priority on the CI,
> and the CI is performing a bit better, I'm not sure building this branch
> is reasonable for now. I feel like we should first focus on providing
> substitutes for the master, guix-modular-master and staging branches
> consistently and on all architectures before trying to build
> core-updates.
Back on October, Chris proposed this:
<https://yhetil.org/guix/877drg3lmi.fsf@cbaines.net/>
and it seems similar. Roughly speaking, let only build core-updates (on
Berlin) when the freeze starts just before the merge.
About the CI, the “production” (= substitutes of master for all the
architectures) and the “testing” (= substitutes of core-updates and
wip-*) should be clearly 2 different things. Maybe on Berlin and
Bayfront.
All the best,
simon
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Building the core-updates branch.
2021-02-06 11:11 ` Mathieu Othacehe
@ 2021-02-06 21:48 ` Ludovic Courtès
2021-02-06 22:21 ` Ricardo Wurmus
1 sibling, 0 replies; 9+ messages in thread
From: Ludovic Courtès @ 2021-02-06 21:48 UTC (permalink / raw)
To: Mathieu Othacehe; +Cc: guix-devel
Mathieu Othacehe <othacehe@gnu.org> skribis:
> Here's a small graph to illustrate the CI throughput improvement: the
> CPU idle time on all the CI build nodes for the last 6 months. The build
> remote mechanism that has been deployed in early January shows clear
> improvements.
Woow, impressive!
Ludo’.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Building the core-updates branch.
2021-02-06 8:06 Building the core-updates branch Mathieu Othacehe
2021-02-06 11:11 ` Mathieu Othacehe
2021-02-06 13:40 ` zimoun
@ 2021-02-06 21:50 ` Ludovic Courtès
2021-02-07 18:22 ` Mathieu Othacehe
2 siblings, 1 reply; 9+ messages in thread
From: Ludovic Courtès @ 2021-02-06 21:50 UTC (permalink / raw)
To: Mathieu Othacehe; +Cc: guix-devel
Mathieu Othacehe <othacehe@gnu.org> skribis:
> The evaluation of the core-updates branch on Cuirass has been broken for
> a long time. It now appears to be fixed[1]. This means that around 45k
> packages must be built. Each change on this branch can possibly trigger
> the same amount of builds.
>
> Even if the core-updates builds have to the lowest priority on the CI,
> and the CI is performing a bit better, I'm not sure building this branch
> is reasonable for now. I feel like we should first focus on providing
> substitutes for the master, guix-modular-master and staging branches
> consistently and on all architectures before trying to build
> core-updates.
Agreed, I think that was the intent. Can we make it focus on the ‘core’
subset? How does one do that nowadays? Looks like my sqlite3/Cuirass
foo is no longer helpful. :-)
Ludo’.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Building the core-updates branch.
2021-02-06 11:11 ` Mathieu Othacehe
2021-02-06 21:48 ` Ludovic Courtès
@ 2021-02-06 22:21 ` Ricardo Wurmus
1 sibling, 0 replies; 9+ messages in thread
From: Ricardo Wurmus @ 2021-02-06 22:21 UTC (permalink / raw)
To: Mathieu Othacehe; +Cc: guix-devel
Mathieu Othacehe <othacehe@gnu.org> writes:
>> Even if the core-updates builds have to the lowest priority on the CI,
>> and the CI is performing a bit better, I'm not sure building this branch
>> is reasonable for now. I feel like we should first focus on providing
>> substitutes for the master, guix-modular-master and staging branches
>> consistently and on all architectures before trying to build
>> core-updates.
>
> Here's a small graph to illustrate the CI throughput improvement: the
> CPU idle time on all the CI build nodes for the last 6 months. The build
> remote mechanism that has been deployed in early January shows clear
> improvements.
Wow, this is wonderful! I’m glad these machines finally get a workout.
Thank you!
--
Ricardo
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Building the core-updates branch.
2021-02-06 21:50 ` Ludovic Courtès
@ 2021-02-07 18:22 ` Mathieu Othacehe
2021-02-07 18:43 ` Mathieu Othacehe
0 siblings, 1 reply; 9+ messages in thread
From: Mathieu Othacehe @ 2021-02-07 18:22 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel
Hey Ludo,
> Agreed, I think that was the intent. Can we make it focus on the ‘core’
> subset? How does one do that nowadays? Looks like my sqlite3/Cuirass
> foo is no longer helpful. :-)
Turns out "core-updates" is still configured to build the "core" subset,
I don't really understand why the evaluation 74281 triggered the build
of so many packages.
Regarding the sqlite3 -> psql transition, there are not that much
differences. I fire up psql this way:
--8<---------------cut here---------------start------------->8---
sudo -u cuirass -s /bin/sh -c 'psql cuirass -h /var/run/postgresql'
--8<---------------cut here---------------end--------------->8---
The commands that are listed here[1] are still (mostly) relevant, even
if this document would really need to be updated. I plan to improve the
Cuirass documentation for a future 1.0 release anyways.
Thanks,
Mathieu
[1]: https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/doc/cuirass-howto.org
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Building the core-updates branch.
2021-02-07 18:22 ` Mathieu Othacehe
@ 2021-02-07 18:43 ` Mathieu Othacehe
2021-02-07 19:29 ` Leo Famulari
0 siblings, 1 reply; 9+ messages in thread
From: Mathieu Othacehe @ 2021-02-07 18:43 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel
> Turns out "core-updates" is still configured to build the "core" subset,
> I don't really understand why the evaluation 74281 triggered the build
> of so many packages.
Just fixed it with 2c30440f89244202bddebe76f90c6b0b8145ba5d on
maintenance repository.
We will now be building 87 jobs instead of 44290 on core-updates.
Mathieu
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Building the core-updates branch.
2021-02-07 18:43 ` Mathieu Othacehe
@ 2021-02-07 19:29 ` Leo Famulari
0 siblings, 0 replies; 9+ messages in thread
From: Leo Famulari @ 2021-02-07 19:29 UTC (permalink / raw)
To: Mathieu Othacehe; +Cc: guix-devel
On Sun, Feb 07, 2021 at 07:43:50PM +0100, Mathieu Othacehe wrote:
>
> > Turns out "core-updates" is still configured to build the "core" subset,
> > I don't really understand why the evaluation 74281 triggered the build
> > of so many packages.
>
> Just fixed it with 2c30440f89244202bddebe76f90c6b0b8145ba5d on
> maintenance repository.
>
> We will now be building 87 jobs instead of 44290 on core-updates.
Awesome!
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-02-07 19:29 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-06 8:06 Building the core-updates branch Mathieu Othacehe
2021-02-06 11:11 ` Mathieu Othacehe
2021-02-06 21:48 ` Ludovic Courtès
2021-02-06 22:21 ` Ricardo Wurmus
2021-02-06 13:40 ` zimoun
2021-02-06 21:50 ` Ludovic Courtès
2021-02-07 18:22 ` Mathieu Othacehe
2021-02-07 18:43 ` Mathieu Othacehe
2021-02-07 19:29 ` Leo Famulari
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).