bug#41669: Cross-compiled powerpc64-linux bootstrap-tarballs not reproducible

[Efraim Flashner <efraim@flashner.co.il>]

[-- Attachment #1: Type: text/plain, Size: 6552 bytes --]

On Sun, Dec 13, 2020 at 03:36:58PM -0800, Chris Marusich wrote:
> Hi,
> 
> I tried to do some experiments to see if this problem happens with the
> current GCC (version 10).
> 
> I built GCC 10 (not cross-compiling) on an x86_64 system using Guix with
> substitutes on Debian.  (I tried without substitutes, too, but some of
> the dependencies failed to be built for unrelated reasons.)  I then
> manually copied the /gnu/store and related files (except for the GCC 10
> output paths) from Debian onto a Fedora machine, and I rebuilt GCC 10
> there using Guix (again, not cross-compiling).  The output on Fedora was
> identical to that of Debian.  Of course, the configuration Guix uses to
> build GCC 10 is a bit different from the one used to (cross-)build the
> powerpc64-linux bootstrap GCC, but it's still an interesting data point.
> In particular, GCC 10's libstdc++.a was identical on Debian and Fedora,
> so I suppose maybe they've fixed that issue in the more recent versions.
> 
> I also tried to use Guix (the current version, from master branch - I
> ran guix pull today) to cross-build gcc-10 for the powerpc64-linux-gnu
> target on both Debian and Fedora x86_64 systems, starting from scratch
> with substitutes enabled:
> 
> guix build --target=powerpc64-linux-gnu -e '(@ (gnu packages gcc) gcc-10)'
> 
> On both Debian and Fedora, the build of gcc-10.2.0.drv failed with the
> following error:
> 
> checking for -fPIC -shared... yes
> configure: error: 
>    Building GCC with plugin support requires a host that supports
>    -fPIC, -shared, -ldl and -rdynamic.
> 
> This basically just means that we can't cross-build gcc-10 for
> powerpc64-linux-gnu out of the box on x86_64 with current Guix.  I was
> hoping that the builds would succeed, and I would be able to find out if
> cross-building gcc-10 in this way would create non-reproducible
> artifacts.  I was hoping maybe I could ask for help from the GCC
> community if that were the case.  But since it doesn't even build, the
> results of that experiment were not very useful.
> 
> It's been almost half a year now, and we're not really any closer to
> figuring out why the cross-built GCC bootstrap binary is
> non-reproducible.  It seems counter-productive to obsess about making
> this specific binary reproducible, although I wish it could be so.
> 
> What do you think about using the bootstrap binaries we built half a
> year ago, and proceed with bootstrapping efforts?  To be totally honest,
> I'm feeling pretty exhausted by this bug, since I have spent so many
> days trying to unravel it, and I haven't made any significant progress.
> With no clear end in sight, I would really prefer to move on instead of
> blocking the entire bootstrapping effort on this reproducibility bug.
> The reproducibility of the bootstrap binaries is important, but simply
> having any bootstrap binaries at all is also important.  I think I have
> done my due diligence to try making them reproducible.  Most of them
> are, but I just can't figure out why GCC isn't.  I think it would be
> best to proceed with the binaries we have.
> 
> Ludovic Courtès <ludo@gnu.org> writes:
> 
> > Hi Chris,
> >
> > Chris Marusich <cmmarusich@gmail.com> skribis:
> >
> >> From e3d1778a86dfd171d59d91eb01417faaf63dfa17 Mon Sep 17 00:00:00 2001
> >> From: Chris Marusich <cmmarusich@gmail.com>
> >> Date: Sat, 19 Sep 2020 14:25:43 -0700
> >> Subject: [PATCH] gnu: Disable libstdc++ in bootstrap GCC.
> >>
> >> Fixes part of: <https://bugs.gnu.org/41669>.
> >>
> >> * gnu/packages/make-bootstrap.scm (%gcc-static) [#:configure-flags]: Add
> >> --disable-libstdcxx to disable building the libstdc++-v3 directory.
> >
> > [...]
> >
> >> +                   ;; In this GCC version, libstdc++.a is not reproducible:
> >> +                   ;; https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41669
> >> +                   "--disable-libstdcxx"
> >
> > Does it have any effect with GCC > 4.7?  My understanding is that it
> > builds its libstdc++ no matter what.
> >
> > Also, if it’s just libstdc++.a that’s problematic (ordering issue in the
> > .a archive?), perhaps we can use --disable-shared?
> >
> > My 2¢ (I didn’t follow the whole discussion),
> > Ludo’.
> 
> Actually, --disable-shared is already present in the configure options.
> My understanding is that libstdc++.a is a statically linked library
> (perhaps I am mistaken...?), so I don't see why the presence or absence
> of --disable-shared would affect it.  I thought that option was just
> supposed to control whether or not to build shared libraries.
> 
> Efraim Flashner <efraim@flashner.co.il> writes:
> 
> > On Fri, Sep 25, 2020 at 11:52:48PM -0700, Chris Marusich wrote:
> >> Hi everyone,
> >> 
> >> Efraim Flashner <efraim@flashner.co.il> writes:
> >> 
> >> > Is this a file we actually need during the bootstrap process? Can we
> >> > "work around it" by just deleting it?
> 
> I've spent all of my spare Guix time trying to debug this
> reproducibility issue first, and half a year has passed without progress
> as a result.  I think we should use the bootstrap binaries we built half
> a year ago, and move on with life.
> 
> At this point, it might even make more sense to try bootstrapping for
> powerpc64le instead of powerpc64, since the rest of the world seems to
> be gravitating toward the little-endian variant on POWER9 hardware, and
> thus various programs out there are more likely to be better tested on
> powerpc64le than powerpc64.
> 
> In any case, I don't think we should wait any longer.

As far as powerpc64 vs powerpc64le, I'll let those with the hardware
have more of a say, they'll be the ones using it. As far as the
bootstrap binaries go, I don't remember having this much pushback with
my binaries for aarch64 (just a request to rebuild with guile-2.0.14
since it was reproducible), and I'm not sure how much Janneke had with
the Hurd binaries but I don't think it was this much. The ultimate goal
anyway is to replace them with artisanally crafted mes binaries, and I
understand we want to have them as reproducible as possible, but I don't
think it's fair to keep this architecture out when we've let other ones
in with similar reproducible problems.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]