bug#74142: [rust-team] rust-bootstrap fails parallel build on x86-64

[Efraim Flashner <efraim@flashner.co.il>]

[-- Attachment #1: Type: text/plain, Size: 3640 bytes --]

On Sat, Dec 07, 2024 at 10:45:32PM +0900, Maxim Cournoyer wrote:
> Hi,
> 
> Greg Hogan <code@greghogan.com> writes:
> 
> > On Sun, Nov 3, 2024 at 2:49 AM Efraim Flashner <efraim@flashner.co.il> wrote:
> >>
> >> On Thu, Oct 31, 2024 at 03:13:40PM -0400, Greg Hogan wrote:
> >> > derivation: 83qbfg0xxsfr5lsc81m5a770c7c7b55k-rust-1.54.0.drv
> >> > commit: 6e50b0c56a8cc767bd3acb26638f78c450bde718
> >> >
> >> > Build log attached, tail portion below. Has consistently failed to
> >> > build unless cores=1.
> >> >
> >> > The package definition has "#:parallel-build? ,(target-x86-64?)" but
> >> > the noted build issue ("race conditions between various dependent
> >> > crates") is also seen on x86-64.
> >> >
> >> > It is only the 1.54 build with mrustc that fails in this manner, the
> >> > following rust packages in the chain build fine with parallelism
> >> > enabled.
> >> >
> >> > Greg
> >>
> >> I see the note there says something about ppc64le having a race
> >> condition too. My machine has "only" 24 cores, so I didn't run into
> >> anything, but I see you're building with 32 cores. IIRC the ppc64le
> >> machine was building with at least 32 cores also.
> >>
> >> Can you try building rust-1.54 with only 24 cores and see if that works
> >> for you? If it does then I think we can switch the (job-count) variable
> >> to (min 24 (parallel-job-count))
> >
> > Log attached from what looks to be a similar failure on an 8 core instance.
> 
> I encountered the same problem on a 32 "cores" (aka threads) machine.
> I've reported it here [0].
> 
> [0]  https://github.com/thepowersgang/mrustc/issues/343

I followed up with Mutabah (the main author) on IRC and a bit on the bug
tracker, and their suggestion is to use a later commit from mrustc.  I
used the latest commit from the master branch but there I ran into
issues on aarch64 not building correctly. From there they basically said
that they were focused on getting 1.74 bootstrapped, and they weren't
going to be circling back to the master branch until 1.74 was merged.

I went and tried out the rust_1.74 branch which is nearing completion
and that one built without any issues on x86_64 but failed for different
reasons on aarch64, powerpc64le and riscv64.

Someone from Gentoo was also asking around, they're interested in using
mrustc to bootstrap rust-1.74 and continuing on from there, so it's
likely we'll get some help going forward with architecture support.

I think in the meantime while it's not ready I'll change the
parallel-builds? flag to false for everyone, and then I'll continue
working on getting the bootstrap to work on the other architectures. The
good news is it only takes about 15 minutes to fail on aarch64 and
powerpc64le, so those should go pretty quickly through each iteration. I
might need to try some by-hand builds on riscv64; that one was closer to
2 hours.

Mutabah also said that their primary target is x86_64 and they don't
have the hardware to test other architectures, so its more hit-or-miss
there, and they rely on patches from others to get those ports working
as well as they do.  I don't love the answer, but I don't blame them.

I'll probably focus on aarch64 first since I can offload that easily and
I don't need to use a remote GUIX_DAEMON_SOCKET to get it to build, but
unfortunately it looks like we're sticking with 1.54 for a while.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]