From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Thompson, David" Subject: Building "special" binary bundles (i.e. not using guix pack) Date: Mon, 4 Nov 2019 09:53:41 -0500 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:37391) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iRdjv-0001es-Q1 for help-guix@gnu.org; Mon, 04 Nov 2019 09:53:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iRdju-0004Ek-50 for help-guix@gnu.org; Mon, 04 Nov 2019 09:53:55 -0500 Received: from mail-ua1-x942.google.com ([2607:f8b0:4864:20::942]:42678) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iRdjt-0004Cn-Pv for help-guix@gnu.org; Mon, 04 Nov 2019 09:53:54 -0500 Received: by mail-ua1-x942.google.com with SMTP id 31so1325343uas.9 for ; Mon, 04 Nov 2019 06:53:53 -0800 (PST) List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+gcggh-help-guix=m.gmane.org@gnu.org Sender: "Help-Guix" To: help-guix Hello all, First, context and parameters: I have a project for which I'd like to release pre-built binaries that will "just work" on major FHS-based distributions such as Ubuntu and Fedora. As far as native executables go, this bundle requires Guile and roughly a dozen C shared libraries. The goal is to compile guile and the additional libraries in such a way that there is *no* runpath information embedded in the ELF files. The goal is *not* to ship a complete graph, down to glibc, like 'guix pack' would do. Instead, I'm taking the more traditional approach of bundling only the libraries that I cannot assume will be on the user's system (say, libvorbis) and assuming that others (like libx11) will be there. Everything included in the bundle needs to be built against a glibc version that is the most compatible across other distros right now. I believe that version is 2.27. Sure, I could just grab a bunch of pre-built stuff from Debian's repositories and stuff it in a tarball, but I want to use Guix in order to have a reproducible toolchain and because this bundle will include several Guile libraries that would be a pain to build without Guix. I want to be able to say "Here, use this convenient binary bundle. If you're curious what's Furthermore, I really want to see Guix fill this particular niche because it could lead to an Appimage-like feature in Guix itself. Now, onto my issues. I have experimented with a few different approaches for building the bundle but haven't had success with any: 1) Using Guix packages as-is and using patchelf to strip runpath information. This honestly came very close to working for me, and I think I could probably make it work with a little more debugging, but it's very hacky and unsatisfying. The biggest issue is that because Guix builds everything with glibc 2.29 these days, I have to bundle glibc 2.29, *including the dynamic linker* so I can actually run Guile. I sent a bundle to a friend to test and it didn't work, but they couldn't tell me what the backtrace was because their system locales were incompatible and thus Guile couldn't failed to print a backtrace. I could ship glibc 2.29 locales in the bundle, too, but that was the last straw for me. How much more stuff would I have to shove into this bundle before it worked reliably? When the easy but hacky solution starts to lose the easy part, it's time to move on. 2) Use a special GCC toolchain sans ld-wrapper and with a few other tweaks to keep runpaths away. This seemed promising but, because there's no runpaths, configure scripts would fail. Although the scripts could compile test programs, running those test programs failed because they were unable to find libraries. Setting LD_LIBRARY_PATH doesn't work because then the configure script doesn't work at all because now binaries built against glibc 2.29 are trying to link against libraries built against 2.27 and everything explodes. I was able to hack around this for a little while by passing "LDFLAGS=-static-libgcc" to configure scripts, but eventually I ran into a configure script that linked against more than libgcc in its test programs (if you're curious, libvorbis builds a program linked against libogg when running ./configure) and I was out of luck. When autoconf-generated configure scripts cannot run test executables, they say something to effect of "are you cross-compiling? use the --host flag". So, I got to thinking: Should I reframe this as a cross-compilation? It certainly seems like I'm dealing with a special target system. Which leads me to... 3) Performing a cross-build. It seems a little weird to cross-compile for x86_64 on a machine that is natively x86_64, but I suppose it makes sense because I'm targeting a foreign type of system. However, I was unable to get this to work because Guix was trying to build tons of stuff from source, from gcc to Linux headers. I assumed I was doing something wrong and aborted this process. I'm not so good at navigating the cross-compilation code in Guix right now. So, now I'd like to ask: Does anyone which approach I *should* be taking here? Does anyone have any experience they can offer? Carl Dong shared his approach to building bitcoin using Guix with me (see: https://github.com/dongcarl/bitcoin-guix/blob/master/packages.scm) which enabled me to try out approach #2 and #3, but it doesn't seem like an exact fit, though I could just be misunderstanding something. My hunch is that #3 is the correct approach, but it's unclear how I setup the proper cross-compilation environment with my special gcc toolchain. I hope I've explained myself somewhat clearly. Any guidance appreciated! If you've read this far, thank you! - Dave