* core-updates-frozen: Planning for the last world rebuild
@ 2021-09-28 11:57 Ludovic Courtès
2021-09-28 12:20 ` Mathieu Othacehe
` (2 more replies)
0 siblings, 3 replies; 14+ messages in thread
From: Ludovic Courtès @ 2021-09-28 11:57 UTC (permalink / raw)
To: guix-devel; +Cc: Maxim Cournoyer
Hello Guix!
Wouldn’t it be nice to get ‘core-updates-frozen’ merged? :-)
What about merging the patches blocked by
<https://issues.guix.gnu.org/50358>? We could even have them on a
separate branch, have Cuirass build it, and merge it 24–36h later, at
which point substitute coverage should be reasonably good.
I haven’t followed the new ports, POWER9 and RISC-V. If there’s
anything needed in this area, like localized toolchain changes, please
send a heads-up!
As I say every time someone proposes a new port, the main issue is not
the port itself but maintaining it longer-term. And here it’s not even
clear to me what the status of POWER9 is on ‘core-updates-frozen’, so
it’d be great to get feedback on this, ensure Cuirass builds it, and so
on.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: core-updates-frozen: Planning for the last world rebuild
2021-09-28 11:57 core-updates-frozen: Planning for the last world rebuild Ludovic Courtès
@ 2021-09-28 12:20 ` Mathieu Othacehe
2021-09-29 19:54 ` Mathieu Othacehe
2021-09-30 1:32 ` Thiago Jung Bauermann
2021-09-30 8:55 ` Efraim Flashner
2 siblings, 1 reply; 14+ messages in thread
From: Mathieu Othacehe @ 2021-09-28 12:20 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel, Maxim Cournoyer
Hey!
> Wouldn’t it be nice to get ‘core-updates-frozen’ merged? :-)
That would be great :) Before merging to master, it would be nice to
have more system tests passing. Currently 16 out of 87 are
broken. Except for the nfs-root-fs that has always been broken, the
other tests probably cover user configurations out there.
I'm currently working on the installation tests.
Thanks,
Mathieu
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: core-updates-frozen: Planning for the last world rebuild
2021-09-28 11:57 core-updates-frozen: Planning for the last world rebuild Ludovic Courtès
2021-09-28 12:20 ` Mathieu Othacehe
@ 2021-09-30 1:32 ` Thiago Jung Bauermann
2021-09-30 9:03 ` Mathieu Othacehe
2021-09-30 8:55 ` Efraim Flashner
2 siblings, 1 reply; 14+ messages in thread
From: Thiago Jung Bauermann @ 2021-09-30 1:32 UTC (permalink / raw)
To: guix-devel; +Cc: Maxim Cournoyer, Ludovic Courtès
Hello Ludo,
Em terça-feira, 28 de setembro de 2021, às 08:57:45 -03, Ludovic Courtès
escreveu:
> Hello Guix!
>
> Wouldn’t it be nice to get ‘core-updates-frozen’ merged? :-)
>
> What about merging the patches blocked by
> <https://issues.guix.gnu.org/50358>? We could even have them on a
> separate branch, have Cuirass build it, and merge it 24–36h later, at
> which point substitute coverage should be reasonably good.
>
> I haven’t followed the new ports, POWER9 and RISC-V. If there’s
> anything needed in this area, like localized toolchain changes, please
> send a heads-up!
There are two things I’m aware of for POWER9:
• diffutils test fix: https://issues.guix.gnu.org/50239
• gcc-5 build with GCC 10 fix: https://issues.guix.gnu.org/49896
In the case of diffutils, IMHO ideally we would apply the v2 patch there for
the world rebuild, but as Ludo suggested it’s also possible to make the
change only for powerpc64le-linux and avoid affecting other platforms.
In the case of gcc-5, it’s the same situation actually. But in addition to
that, `guix refresh --list-dependent` says that modifying gcc-5 would only
cause 32 dependent packages to be rebuilt so it’s not necessary to address
it for the world rebuild.
> As I say every time someone proposes a new port, the main issue is not
> the port itself but maintaining it longer-term. And here it’s not even
> clear to me what the status of POWER9 is on ‘core-updates-frozen’, so
> it’d be great to get feedback on this, ensure Cuirass builds it, and so
> on.
Yeah, the status of POWER9 on ‘core-updates-frozen’ is obfuscated by some
build failures early in the dependency chain that were caused by QEMU bugs.
My impression is that the same is also true for aarch64-linux. For
instance, QEMU can’t run libsigsegv’s testsuite (among other things), and
that blocks a ton of stuff. Ditto for util-linux’s testsuite.
I was waiting for this world rebuild to happen to see the results in
Cuirass now that powerpc64le-linux builds are only done on guixp9, but in
hindsight it would have been better if I requested that those builds were
restarted instead. Restarting this build should remove the main roadblock
though:
https://ci.guix.gnu.org/build/703194/details
When that build is restarted and succeeds, does Cuirass also restart the
builds that failed because they depended on it, or do we need to explicitly
restart those as well?
What I can say is that IMHO POWER9 support isn’t too bad at least for the
base packages given that I can build e.g., emacs (assuming that the patch
for issue 50521 fixing a problem in gtk+-2 is applied), which depends on
many base packages including X11 and GTK+.
--
Thanks,
Thiago
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: core-updates-frozen: Planning for the last world rebuild
2021-09-30 1:32 ` Thiago Jung Bauermann
@ 2021-09-30 9:03 ` Mathieu Othacehe
2021-09-30 15:29 ` Thiago Jung Bauermann
0 siblings, 1 reply; 14+ messages in thread
From: Mathieu Othacehe @ 2021-09-30 9:03 UTC (permalink / raw)
To: Thiago Jung Bauermann; +Cc: guix-devel, Maxim Cournoyer
Hey Thiago,
> I was waiting for this world rebuild to happen to see the results in
> Cuirass now that powerpc64le-linux builds are only done on guixp9, but in
> hindsight it would have been better if I requested that those builds were
> restarted instead. Restarting this build should remove the main roadblock
> though:
>
> https://ci.guix.gnu.org/build/703194/details
Now restarted!
>
> When that build is restarted and succeeds, does Cuirass also restart the
> builds that failed because they depended on it, or do we need to explicitly
> restart those as well?
Yes, on successful build completion Cuirass looks if some builds that
were marked as "failed-dependency" (failing because some of the
dependencies are also failing) can be resumed.
That being said, Cuirass is not performing nicely these days because of:
this issue https://issues.guix.gnu.org/48468.
Thanks,
Mathieu
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: core-updates-frozen: Planning for the last world rebuild
2021-09-30 9:03 ` Mathieu Othacehe
@ 2021-09-30 15:29 ` Thiago Jung Bauermann
2021-10-01 6:21 ` Mathieu Othacehe
0 siblings, 1 reply; 14+ messages in thread
From: Thiago Jung Bauermann @ 2021-09-30 15:29 UTC (permalink / raw)
To: Mathieu Othacehe; +Cc: guix-devel, Maxim Cournoyer, Ludovic Courtès
Hello Mathieu,
Em quinta-feira, 30 de setembro de 2021, às 06:03:52 -03, Mathieu Othacehe
escreveu:
> Hey Thiago,
>
> > I was waiting for this world rebuild to happen to see the results in
> > Cuirass now that powerpc64le-linux builds are only done on guixp9, but
> > in hindsight it would have been better if I requested that those
> > builds were restarted instead. Restarting this build should remove the
> > main roadblock though:
> >
> > https://ci.guix.gnu.org/build/703194/details
>
> Now restarted!
Thank you! Unfortunately that build is now affected by an issue that I only
noticed yesterday: powerpc64le-linux builds for core-updates-frozen get
permanently stuck in the ‘Scheduled’ state, even though guixp9 is
completely idle (as it is right now).
For example, this evaluation from Sep 17 has thousands of scheduled
evaluations:
https://ci.guix.gnu.org/eval/19271?status=pending
This is also happening with at least one other platform as well. This
evaluation from Sep 25 has a lot of scheduled evaluations for i586-gnu (as
well as powerpc64le-linux):
https://ci.guix.gnu.org/eval/23187?status=pending
> > When that build is restarted and succeeds, does Cuirass also restart
> > the
> > builds that failed because they depended on it, or do we need to
> > explicitly restart those as well?
>
> Yes, on successful build completion Cuirass looks if some builds that
> were marked as "failed-dependency" (failing because some of the
> dependencies are also failing) can be resumed.
That is awesome!
> That being said, Cuirass is not performing nicely these days because of:
> this issue https://issues.guix.gnu.org/48468.
Ah, ok. Thank you for the heads up.
--
Thanks,
Thiago
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: core-updates-frozen: Planning for the last world rebuild
2021-09-30 15:29 ` Thiago Jung Bauermann
@ 2021-10-01 6:21 ` Mathieu Othacehe
2021-10-01 21:51 ` Thiago Jung Bauermann
0 siblings, 1 reply; 14+ messages in thread
From: Mathieu Othacehe @ 2021-10-01 6:21 UTC (permalink / raw)
To: Thiago Jung Bauermann; +Cc: guix-devel, Maxim Cournoyer
Hey,
> For example, this evaluation from Sep 17 has thousands of scheduled
> evaluations:
>
> https://ci.guix.gnu.org/eval/19271?status=pending
Mmmh, not nice. Any chance you could share the cuirass-remote-worker
logs on the guixp9 machine?
Thanks,
Mathieu
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: core-updates-frozen: Planning for the last world rebuild
2021-10-01 6:21 ` Mathieu Othacehe
@ 2021-10-01 21:51 ` Thiago Jung Bauermann
0 siblings, 0 replies; 14+ messages in thread
From: Thiago Jung Bauermann @ 2021-10-01 21:51 UTC (permalink / raw)
To: Mathieu Othacehe; +Cc: guix-devel, Maxim Cournoyer, Ludovic Courtès
[-- Attachment #1: Type: text/plain, Size: 699 bytes --]
Em sexta-feira, 1 de outubro de 2021, às 03:21:16 -03, Mathieu Othacehe
escreveu:
> Hey,
>
> > For example, this evaluation from Sep 17 has thousands of scheduled
> > evaluations:
> >
> > https://ci.guix.gnu.org/eval/19271?status=pending
>
> Mmmh, not nice. Any chance you could share the cuirass-remote-worker
> logs on the guixp9 machine?
Of course. Here it is. For some reason there’s no log before Sep 14, but at
least it covers the Cuirass evaluations I mentioned.
I restarted the Cuirass worker service a couple of times to adjust the
number of workers. Perhaps it is related? I tried to do it at times when
guixp9 wasn’t running CI jobs.
--
Thanks,
Thiago
[-- Attachment #2: cuirass.log.xz --]
[-- Type: application/x-xz, Size: 161508 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: core-updates-frozen: Planning for the last world rebuild
2021-09-28 11:57 core-updates-frozen: Planning for the last world rebuild Ludovic Courtès
2021-09-28 12:20 ` Mathieu Othacehe
2021-09-30 1:32 ` Thiago Jung Bauermann
@ 2021-09-30 8:55 ` Efraim Flashner
2 siblings, 0 replies; 14+ messages in thread
From: Efraim Flashner @ 2021-09-30 8:55 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel, Maxim Cournoyer
[-- Attachment #1: Type: text/plain, Size: 1578 bytes --]
On Tue, Sep 28, 2021 at 01:57:45PM +0200, Ludovic Courtès wrote:
> Hello Guix!
>
> Wouldn’t it be nice to get ‘core-updates-frozen’ merged? :-)
>
> What about merging the patches blocked by
> <https://issues.guix.gnu.org/50358>? We could even have them on a
> separate branch, have Cuirass build it, and merge it 24–36h later, at
> which point substitute coverage should be reasonably good.
>
> I haven’t followed the new ports, POWER9 and RISC-V. If there’s
> anything needed in this area, like localized toolchain changes, please
> send a heads-up!
>
> As I say every time someone proposes a new port, the main issue is not
> the port itself but maintaining it longer-term. And here it’s not even
> clear to me what the status of POWER9 is on ‘core-updates-frozen’, so
> it’d be great to get feedback on this, ensure Cuirass builds it, and so
> on.
I'm still building away on riscv64-linux. I have some minor changes,
namely renaming target-riscv? to target-riscv64? As far as cuirass
support I have it officially unsupported in %hydra-supported-systems (or
whatever that variable is called). I've tried to keep the patches so
that they can be applied without interfering with the other
architectures. At this point as far as I know it's waiting for another
pair of eyes or so to say it looks OK to merge.
--
Efraim Flashner <efraim@flashner.co.il> רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: core-updates-frozen: Planning for the last world rebuild
@ 2021-09-28 19:18 John Kehayias
0 siblings, 0 replies; 14+ messages in thread
From: John Kehayias @ 2021-09-28 19:18 UTC (permalink / raw)
To: guix-devel@gnu.org, Ludovic Courtès
Hi Ludo’ and Guixers,
Thanks for the good progress on core-updates-frozen (where I'm currently writing from and have nearly everything working as before). Seems like we have a few different bug numbers and threads for the world rebuild. Sorry for repeating from #50860, which I will paste below.
In summary, I also noticed an older bug for Flatpak with p11-kit, as well as p11-kit being out of date. Locally I've tested it builds with the patch and version update and fixes the bug. The configuration change is the same as nix does for p11-kit for the same reason, for what that is worth.
Less critical is Mesa continuing to put out updates, both on 21.1 and now moving to 21.2 (considered stable I believe). I thought we might sneak that in if we're doing lots of rebuilding.
Thanks! Original message with more details below:
Is there anything else with huge rebuilds to push together? I don't want to keep finding and adding things, but two possibilities come to mind that I've just noticed:
1a. p11-kit #49957 https://issues.guix.gnu.org/49957
I've just hit this bug on core-updates-frozen as well, though was originally reported on master. As I noted there, I tried to test with just grafts but didn't fix it for me (I'm guessing grafting won't work with that configure flag change). The patch matches how nix configures p11-kit as well, due to this bug.
1b. p11-kit is also now out of date. The changelog doesn't look substantial or serious, but given the nature of p11-kit I wonder if we want to update it now too. https://github.com/p11-glue/p11-kit/releases/tag/0.24.0
2. (minor) Mesa has had a few more bugfix and major releases since my initial patch for core-updates-frozen. Now at 21.1.8 for the 21.1 branch (we have 21.1.6 currently), but 21.2 has also had stable releases, with 21.2.2. I previously built 21.2.1 and sent a patch for it, and could test 21.2.2 if we want to do that too.
I'm aware this could continue forever, and #2 is likely lower priority. #1, though, should we consider p11-kit a critical update (version, at least) since we're still fixing core-updates-frozen?
John
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2021-10-05 19:25 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-09-28 11:57 core-updates-frozen: Planning for the last world rebuild Ludovic Courtès
2021-09-28 12:20 ` Mathieu Othacehe
2021-09-29 19:54 ` Mathieu Othacehe
2021-09-30 17:44 ` Ricardo Wurmus
2021-10-01 7:16 ` Mathieu Othacehe
2021-10-01 20:13 ` Ricardo Wurmus
2021-10-05 19:24 ` Mathieu Othacehe
2021-09-30 1:32 ` Thiago Jung Bauermann
2021-09-30 9:03 ` Mathieu Othacehe
2021-09-30 15:29 ` Thiago Jung Bauermann
2021-10-01 6:21 ` Mathieu Othacehe
2021-10-01 21:51 ` Thiago Jung Bauermann
2021-09-30 8:55 ` Efraim Flashner
-- strict thread matches above, loose matches on Subject: below --
2021-09-28 19:18 John Kehayias
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).