From mboxrd@z Thu Jan 1 00:00:00 1970 From: Carl Dong Subject: Re: Bootstrappable bitcoin release builds with Guix Date: Tue, 23 Apr 2019 20:56:20 +0000 Message-ID: References: <874l6w8i2r.fsf@gnu.org> Reply-To: Carl Dong Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([209.51.188.92]:56109) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hJ2TF-0004cn-3p for help-guix@gnu.org; Tue, 23 Apr 2019 16:56:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hJ2TC-0008U6-PB for help-guix@gnu.org; Tue, 23 Apr 2019 16:56:53 -0400 Received: from mail1.protonmail.ch ([185.70.40.18]:16169) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hJ2TA-0008Ba-E0 for help-guix@gnu.org; Tue, 23 Apr 2019 16:56:50 -0400 In-Reply-To: <874l6w8i2r.fsf@gnu.org> List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+gcggh-help-guix=m.gmane.org@gnu.org Sender: "Help-Guix" To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= Cc: "help-guix\\@gnu.org" Hi Ludovic! > OK. You could store the output of =E2=80=98guix describe -f channels= =E2=80=99 in > addition to the manifest, and then do =E2=80=98guix pull -C=E2=80=99 of t= his file. That > way, you=E2=80=99d be pinning a specific package set. Ah... things like this make me more fall in love with Guix everyday. > > 1. libstdc++ would not link statically even with "-static-libstdc++". = The hack > > was to remove the .la file under $LIBRARY_PATH. > > > > Weird. Could you send a small test case for this tobug-guix@gnu.org? I realized afterwards that this is actually another problem. The real probl= em is that configure insists on adding `-lstdc++` to `config.status` and `l= ibtool` even when `-static-libstdc++` is in `LDFLAGS`. I will do some more = digging but it seems to only happen in Guix environments. I would appreciat= e any insight. > > 2. Upon inspection of the binaries produced at the end of this process= , they all > > had rpaths. The hack was to use patchelf --remove-rpath on them. > > > > Yes, =E2=80=98ld-wrapper=E2=80=99 and our =E2=80=98gcc=E2=80=99 packages = add those on purpose; they=E2=80=99re > required for dynamically-linked binaries. But you=E2=80=99re producing > statically-linked executables in the end, right? We're producing a executable that's dynamically linked in the end, we perfo= rm a few checks to make sure that only the libraries we want to dynamically= link to are dynamically linked to: https://github.com/bitcoin/bitcoin/blob= /40a720acb8472f6e8afdf8b8d1897f35d58daf1f/contrib/devtools/symbol-check.py#= L57. I have resorted to modifying the default gcc package to omit the `GNU_= USER_TARGET_LIB_SPEC` substitute which is how the rpaths sneak in. I suspec= t that using a specfile would have also worked? > > 3. Upon inspection of the binaries produced at the end of this process= , their > > interpreters all had a `/gnu/store/blahblah-glibc-2.28' prefix. The= hack was > > to use patchelf --set-interpreter on them. > > > > To use /lib/ld-linux-x86-64.so.2 instead? > > You can do that, but there=E2=80=99s a risk: this is assuming that the lo= ader > and libc on the user=E2=80=99s machine are compatible with those you buil= t > against. Right. This is why we check our libc version is one that is reasonably old = to work on old systems: https://github.com/bitcoin/bitcoin/blob/40a720acb84= 72f6e8afdf8b8d1897f35d58daf1f/contrib/devtools/symbol-check.py#L38. I'm not= familiar with how the loader might be incompatible? > Can you achieve this by simply taking the ouptut of =E2=80=9Cguix build > bitcoin-core-alt=E2=80=9D (where =E2=80=98bitcoin-core-alt=E2=80=99 is a = variant that statically > links against libstdc++ etc.), patching the ELF interpreter, and then > copying it in a tarball? > > You=E2=80=99d also get cross-compilation by adding =E2=80=98--target= =E2=80=99. Yes, I think you're right that that would be a little easier. I might trans= ition to this method once I'm ready, and I don't think patching the ELF int= erpreter is that bad. Cheers, Carl Dong contact@carldong.me "I fight for the users" =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Original Me= ssage =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 On Wednesday, April 17, 2019 4:58 PM, Ludovic Court=C3=A8s w= rote: > Hi Carl, > > Carl Dong contact@carldong.me skribis: > > > I've been on a quest to use Guix for Bitcoin Core's reproducible builds= as I > > believe that Guix's focus on bootstrappability, and Guile's simplicity = and > > flexibility are very desirable qualities in building an auditable, secu= re, and > > reliable build process. My pull request (very short thanks to Guix's > > infrastructure) can be found here: > > https://github.com/bitcoin/bitcoin/pull/15277/files > > I=E2=80=99m late to the party, but this is awesome! The goals you= =E2=80=99re pursuing > are in line with those we=E2=80=99re after in Guix, Reproducible Builds, = and > Bootstrappable, so it=E2=80=99s great to see that Guix can help others he= re. > > > I've submitted patches for the Guix bitcoin-core package to make it > > reproducible, which seems to work fine. However, for easier acceptance = into the > > bitcoin core process, I need to produce tarballs like the ones we have = on our > > servers today: https://bitcoincore.org/bin/bitcoin-core-0.17.1/ > > For some context, we have a "mini-guix" of sorts seen in our "depends t= ree" > > here: https://github.com/bitcoin/bitcoin/tree/master/depends. This buil= ds all > > the dependencies for bitcoin just the way we want them, in preparation = for > > getting linked into bitcoin itself. > > My current approach for the build process is to produce a Guix containe= r in > > which I execute a build of our "depends tree" followed by a build of bi= tcoin > > itself. See the Guix manifest and scripts here: > > https://github.com/bitcoin/bitcoin/pull/15277/files > > OK. You could store the output of =E2=80=98guix describe -f channels= =E2=80=99 in > addition to the manifest, and then do =E2=80=98guix pull -C=E2=80=99 of t= his file. That > way, you=E2=80=99d be pinning a specific package set. > > > However, there were three hiccups that I had to hack my way around: > > > > 1. libstdc++ would not link statically even with "-static-libstdc++". = The hack > > was to remove the .la file under $LIBRARY_PATH. > > > > Weird. Could you send a small test case for this tobug-guix@gnu.org? > > > 2. Upon inspection of the binaries produced at the end of this process= , they all > > had rpaths. The hack was to use patchelf --remove-rpath on them. > > > > Yes, =E2=80=98ld-wrapper=E2=80=99 and our =E2=80=98gcc=E2=80=99 packages = add those on purpose; they=E2=80=99re > required for dynamically-linked binaries. But you=E2=80=99re producing > statically-linked executables in the end, right? > > > 3. Upon inspection of the binaries produced at the end of this process= , their > > interpreters all had a `/gnu/store/blahblah-glibc-2.28' prefix. The= hack was > > to use patchelf --set-interpreter on them. > > > > To use /lib/ld-linux-x86-64.so.2 instead? > > You can do that, but there=E2=80=99s a risk: this is assuming that the lo= ader > and libc on the user=E2=80=99s machine are compatible with those you buil= t > against. > > > My questions are: > > > > 1. Is there a way to avoid the hacks that I listed above? I understand= that it > > might mean writing custom gcc packages and I'm 100% okay with that. > > > > 2. Is there an easier way of achieving the same thing? > > > > I think the difficulty here lies in producing non-self-contained binary > tarballs, paradoxically. :-) > > By that I mean that the tarball at > https://bitcoincore.org/bin/bitcoin-core-0.17.1/bitcoin-0.17.1-x86_64-lin= ux-gnu.tar.gz > makes assumptions on the user=E2=80=99s machine: > > --8<---------------cut here---------------start------------->8--- > $ file /tmp/bitcoin-0.17.1/bin/bitcoin-cli > /tmp/bitcoin-0.17.1/bin/bitcoin-cli: ELF 64-bit LSB pie executable x86-64= , version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x8= 6-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=3Daced8cd0e28adb7a9e0c15f2e79= 4aae7150d348e, stripped > $ objdump -x /tmp/bitcoin-0.17.1/bin/bitcoin-cli |grep NEED > NEEDED librt.so.1 > NEEDED libm.so.6 > NEEDED libgcc_s.so.1 > NEEDED libpthread.so.0 > NEEDED libc.so.6 > NEEDED ld-linux-x86-64.so.2 > VERNEED 0x0000000000002850 > VERNEEDNUM 0x0000000000000006 > --8<---------------cut here---------------end--------------->8--- > > The binary assumes that /lib64/ld-linux-x86-64.so.2 exists, that > libgcc_s.so & co. will be found in the =E2=80=9Cstandard locations= =E2=80=9D, etc. These > assumptions do not hold on a standalone Guix system or on NixOS or > Gobolinux. They are probably reasonable for FHS distros, though. > > Can you achieve this by simply taking the ouptut of =E2=80=9Cguix build > bitcoin-core-alt=E2=80=9D (where =E2=80=98bitcoin-core-alt=E2=80=99 is a = variant that statically > links against libstdc++ etc.), patching the ELF interpreter, and then > copying it in a tarball? > > You=E2=80=99d also get cross-compilation by adding =E2=80=98--target= =E2=80=99. > > Thanks, > Ludo=E2=80=99.