Re: Bootstrappable bitcoin release builds with Guix

["Ludovic Courtès" <ludo@gnu.org>]

Hi Carl,

Carl Dong <contact@carldong.me> skribis:

> I've been on a quest to use Guix for Bitcoin Core's reproducible builds as I
> believe that Guix's focus on bootstrappability, and Guile's simplicity and
> flexibility are very desirable qualities in building an auditable, secure, and
> reliable build process. My pull request (very short thanks to Guix's
> infrastructure) can be found here:
> https://github.com/bitcoin/bitcoin/pull/15277/files

I’m late to the party, but this is awesome!  The goals you’re pursuing
are in line with those we’re after in Guix, Reproducible Builds, and
Bootstrappable, so it’s great to see that Guix can help others here.

> I've submitted patches for the Guix bitcoin-core package to make it
> reproducible, which seems to work fine. However, for easier acceptance into the
> bitcoin core process, I need to produce tarballs like the ones we have on our
> servers today: https://bitcoincore.org/bin/bitcoin-core-0.17.1/
>
> For some context, we have a "mini-guix" of sorts seen in our "depends tree"
> here: https://github.com/bitcoin/bitcoin/tree/master/depends. This builds all
> the dependencies for bitcoin just the way we want them, in preparation for
> getting linked into bitcoin itself.
>
> My current approach for the build process is to produce a Guix container in
> which I execute a build of our "depends tree" followed by a build of bitcoin
> itself. See the Guix manifest and scripts here:
> https://github.com/bitcoin/bitcoin/pull/15277/files

OK.  You could store the output of ‘guix describe -f channels’ in
addition to the manifest, and then do ‘guix pull -C’ of this file.  That
way, you’d be pinning a specific package set.

> However, there were three hiccups that I had to hack my way around:
>
> 1. libstdc++ would not link statically even with "-static-libstdc++". The hack
>    was to remove the .la file under $LIBRARY_PATH.

Weird.  Could you send a small test case for this to bug-guix@gnu.org?

> 2. Upon inspection of the binaries produced at the end of this process, they all
>    had rpaths. The hack was to use patchelf --remove-rpath on them.

Yes, ‘ld-wrapper’ and our ‘gcc’ packages add those on purpose; they’re
required for dynamically-linked binaries.  But you’re producing
statically-linked executables in the end, right?

> 3. Upon inspection of the binaries produced at the end of this process, their
>    interpreters all had a `/gnu/store/blahblah-glibc-2.28' prefix. The hack was
>    to use patchelf --set-interpreter on them.

To use /lib/ld-linux-x86-64.so.2 instead?

You can do that, but there’s a risk: this is assuming that the loader
and libc on the user’s machine are compatible with those you built
against.

> My questions are:
>
> 1. Is there a way to avoid the hacks that I listed above? I understand that it
>    might mean writing custom gcc packages and I'm 100% okay with that.
>
> 2. Is there an easier way of achieving the same thing?

I think the difficulty here lies in producing non-self-contained binary
tarballs, paradoxically.  :-)

By that I mean that the tarball at
<https://bitcoincore.org/bin/bitcoin-core-0.17.1/bitcoin-0.17.1-x86_64-linux-gnu.tar.gz>
makes assumptions on the user’s machine:

--8<---------------cut here---------------start------------->8---
$ file /tmp/bitcoin-0.17.1/bin/bitcoin-cli 
/tmp/bitcoin-0.17.1/bin/bitcoin-cli: ELF 64-bit LSB pie executable x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=aced8cd0e28adb7a9e0c15f2e794aae7150d348e, stripped
$ objdump -x /tmp/bitcoin-0.17.1/bin/bitcoin-cli |grep NEED
  NEEDED               librt.so.1
  NEEDED               libm.so.6
  NEEDED               libgcc_s.so.1
  NEEDED               libpthread.so.0
  NEEDED               libc.so.6
  NEEDED               ld-linux-x86-64.so.2
  VERNEED              0x0000000000002850
  VERNEEDNUM           0x0000000000000006
--8<---------------cut here---------------end--------------->8---

The binary assumes that /lib64/ld-linux-x86-64.so.2 exists, that
libgcc_s.so & co. will be found in the “standard locations”, etc.  These
assumptions do not hold on a standalone Guix system or on NixOS or
Gobolinux.  They are probably reasonable for FHS distros, though.

Can you achieve this by simply taking the ouptut of “guix build
bitcoin-core-alt” (where ‘bitcoin-core-alt’ is a variant that statically
links against libstdc++ etc.), patching the ELF interpreter, and then
copying it in a tarball?

You’d also get cross-compilation by adding ‘--target’.

Thanks,
Ludo’.