Bootstrappable Release Build of Rust Application

Bootstrappable Release Build of Rust Application

[Antoine Poinsot]

Good morning all,

I've been working on using Guix for the reproducible builds of my Rust software. My main question is about linking my program against an older glibc version (or better yet, statically linking it against musl). But first let me give a bit of context.

We are developing a Rust software and releasing binaries. For Linux binaries we use Guix for reproducible release builds. We need to link against a reasonable glibc version to support older systems. For the past year i've been using a `guix shell` container in a `guix time-machine` in which i would `cargo rustc` my project. This allowed us to target an older glibc, at the expense of requiring compatibility with older rustc compilers. As new rustc releases are frequent (and Rust libraries tend to require bleeding edge versions of the compiler...) i knew this was only a short-term solution. Well, short term lasted a year and a half! Not bad. Now i need to do a proper Guix packaging of my Rust program.

To write the backbone of my package with the insane number of dependencies a Rust project may have, i hacked a small program which would translate a `Cargo.lock` into a `mypackage.scm`. Some of our dependencies are git repos, and surprisingly `cargo-build-system` doesn't support them correctly (they don't get unpacked in the vendor directory). This is only a small annoyance, i can package them and upload them to crates.io myself. So far so good.

Now this links against glibc 2.35. I need to target ideally 2.29 or at most 2.31. That's what i'm seeking help for. I've tried the obvious:
```
guix build -f test2.scm --with-input=glibc=glibc@2.29
```

But i immediately hit a dependency cycle on perl:
```
gnu/packages/perl.scm:106:2: error: perl@5.36.0: dependency cycle detected:
  perl@5.36.0 -> linux-libre-headers@5.15.49 -> glibc@2.29 -> glibc-utf8-locales@2.35 -> coreutils@9.1 -> tar@1.34 -> perl@5.36.0 -> texinfo@6.8 -> gdb@12.1 -> rust@1.75.0 -> liana_gui@5.0.0
```

For what it's worth you can simply reproduce this with other crates. For instance trying to build the `log` crate will give the same error:
```
guix build rust-log --with-input=glibc=glibc@2.29
```

Then i've tried using `with-c-toolchain` and `make-gcc-toolchain` (not sure it would even help but i tried anyways):
```
(define gcc-toolchain-2.29
  (make-gcc-toolchain gcc-7 glibc-2.29))

(package-with-c-toolchain rust-liana_gui-5.0.0 `(("toolchain" ,gcc-toolchain-2.29)))
```

This one keeps failing to compile no matter the flavour (different glibc / gcc versions). For instance for gcc-7 + glibc-2.29:
```
g++ -std=gnu++98 -no-pie   -g -DIN_GCC     -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wno-format -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common  -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc -Wl,-rpath=/gnu/store/wry6srbz3wr35kdqnwyls9ypkkfq172j-glibc-2.29/lib -Wl,-dynamic
-linker -Wl,/gnu/store/wry6srbz3wr35kdqnwyls9ypkkfq172j-glibc-2.29/lib/ld-linux-x86-64.so.2 gcov-dump.o \                                                                                                                                                                                                                                                                                                                               
        hash-table.o ggc-none.o\                                                                                                                                                                                                                                                                                                                                                                                                        
        libcommon.a ../libcpp/libcpp.a   ../libbacktrace/.libs/libbacktrace.a ../libiberty/libiberty.a ../libdecnumber/libdecnumber.a  -o gcov-dump                                                                                                                                                                                                                                                                                     
ld: /gnu/store/930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../libstdc++.a(stdexcept.o): in function `std::logic_error::~logic_error()':                                                                                                                                                                                                                                               
(.text._ZNSt11logic_errorD2Ev+0x43): undefined reference to `__libc_single_threaded'                                                                                                                                                                                                                                                                                                                                                    
ld: /gnu/store/930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../libstdc++.a(stdexcept.o): in function `std::runtime_error::~runtime_error()':                                                                                                                                                                                                                                           
(.text._ZNSt13runtime_errorD2Ev+0x43): undefined reference to `__libc_single_threaded'                                                                                                                                                                                                                                                                                                                                                  
ld: /gnu/store/930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../libstdc++.a(stdexcept.o): in function `std::logic_error::logic_error(std::string const&)':                                                                                                                                                                                                                              
(.text._ZNSt11logic_errorC2ERKSs+0x63): undefined reference to `__libc_single_threaded'                                                                                                                                                                                                                                                                                                                                                 
ld: /gnu/store/930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../libstdc++.a(stdexcept.o): in function `std::runtime_error::runtime_error(std::string const&)':                                                                                                                                                                                                                          
(.text._ZNSt13runtime_errorC2ERKSs+0x63): undefined reference to `__libc_single_threaded'                                                                                                                                                                                                                                                                                                                                               
ld: /gnu/store/930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../libstdc++.a(cow-stdexcept.o): in function `_txnal_cow_string_D1_commit(void*)':                                                                                                                                                                                                                                         
(.text._Z27_txnal_cow_string_D1_commitPv+0x13): undefined reference to `__libc_single_threaded'                                                                                                                                                                                                                                                                                                                                         
ld: /gnu/store/930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../libstdc++.a(cow-stdexcept.o):(.text._ZNSt11logic_errorC2ERKS_+0x63): more undefined references to `__libc_single_threaded' follow                                                                                                                                                                                       
g++ -std=gnu++98 -no-pie   -g -DIN_GCC     -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wno-format -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -fno-common  -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc -Wl,-rpath=/gnu/store/wry6srbz3wr35kdqnwyls9ypkkfq172j-glibc-2.29/lib -Wl,-dynamic
-linker -Wl,/gnu/store/wry6srbz3wr35kdqnwyls9ypkkfq172j-glibc-2.29/lib/ld-linux-x86-64.so.2 gcov-tool.o libgcov-util.o libgcov-driver-tool.o libgcov-merge-tool.o libcommon.a ../libcpp/libcpp.a   ../libbacktrace/.libs/libbacktrace.a ../libiberty/libiberty.a ../libdecnumber/libdecnumber.a  -o gcov-tool                                                                                                                           
collect2: error: ld returned 1 exit status                                                                                                                                                                                                                                                                                                                                                                                              
make[3]: *** [Makefile:2851: gcov] Error 1
```

But anyways would this help at all to have a gcc-toolchain with an older glibc? Isn't this just building gcc against an older glibc? Would this also make cargo-build-system link against this glibc?

Finally, i've tried both with musl as well. In both cases it complains that the musl package doesn't provide a "static" output.

I'm running out of options to try here. Does anyone have any pointer on how to link against an older glibc in Guix? Is it at all supposed to be supported without using the time-machine? Is my approach misguided? Any pointer or partial answer is appreciated, even if it's only "you clearly don't understand how Guix works go read resource X and Y and come back".

Thanks,
Antoine

Re: Bootstrappable Release Build of Rust Application

[Philip McGrath]

Hi,

On Wed, Apr 17, 2024, at 6:28 AM, Antoine Poinsot wrote:
> Good morning all,
>
> I've been working on using Guix for the reproducible builds of my Rust 
> software. My main question is about linking my program against an older 
> glibc version (or better yet, statically linking it against musl).
>
> [...]
>
> Now this links against glibc 2.35. I need to target ideally 2.29 or at 
> most 2.31. That's what i'm seeking help for. I've tried the obvious:
> ```
> guix build -f test2.scm --with-input=glibc=glibc@2.29
> ```
>

My knowledge in this area is very shallow, but I also have wanted to link against old glibc versions, for somewhat similar purposes. The best I've got so far is at [1], in which I successfully use `make-gcc-toolchain` to use `gcc` with `glibc-2.31` and define a helper function `package-with-old-stable-libc` to supply it to `package-with-c-toolchain`. I use that in [2] to compile the library I'm interested in against the old glibc version. While this is the best solution I've found so far, I'm still not entirely happy with it! While the code I'm pointing you to isn't too bad, other parts of the overall setup are overly complex (in fairness, largely to also build for Apple platforms), and I haven't managed to build against Glibc 2.23 to fix [3]. But hopefully some of this might be useful, or, better yet, someone might suggest a better solution!

Philip

[1]: https://github.com/libgit2-racket/native-libs/blob/bd400daed8d4e38db6fc472d4ec82b99a884303c/guix/old-stable-libc.scm
[2]: https://github.com/libgit2-racket/native-libs/blob/bd400daed8d4e38db6fc472d4ec82b99a884303c/guix/extracted/non-apple.scm
[3]: https://github.com/libgit2-racket/libgit2/issues/2

Re: Bootstrappable Release Build of Rust Application

[Antoine Poinsot]

Thanks for sharing.

Unfortunately the very same `make-gcc-toolchain` keeps failing to compile for me, even with the same gcc and glibc versions as you:
```
In file included from ../include/pthread.h:1,                                                                                                                                                                                                                                                                                                                                                                                           
                 from ../sysdeps/nptl/thread_db.h:25,                                                                                                                                                                                                                                                                                                                                                                                   
                 from ../nptl/descr.h:32,                                                                                                                                                                                                                                                                                                                                                                                               
                 from ../sysdeps/x86_64/nptl/tls.h:130,                                                                                                                                                                                                                                                                                                                                                                                 
                 from ../include/link.h:51,                                                                                                                                                                                                                                                                                                                                                                                             
                 from ../include/dlfcn.h:4,                                                                                                                                                                                                                                                                                                                                                                                             
                 from ../sysdeps/generic/ldsodefs.h:32,                                                                                                                                                                                                                                                                                                                                                                                 
                 from ../sysdeps/x86/ldsodefs.h:64,                                                                                                                                                                                                                                                                                                                                                                                     
                 from ../sysdeps/gnu/ldsodefs.h:46,                                                                                                                                                                                                                                                                                                                                                                                     
                 from ../sysdeps/unix/sysv/linux/ldsodefs.h:25,                                                                                                                                                                                                                                                                                                                                                                         
                 from <stdin>:2:                                                                                                                                                                                                                                                                                                                                                                                                        
../sysdeps/nptl/pthread.h:719:47: error: argument 1 of type ‘struct __jmp_buf_tag *’ declared as a pointer [-Werror=array-parameter=]                                                                                                                                                                                                                                                                                                   
  719 | extern int __sigsetjmp (struct __jmp_buf_tag *__env, int __savemask) __THROWNL;                                                                                                                                                                                                                                                                                                                                                 
      |                         ~~~~~~~~~~~~~~~~~~~~~~^~~~~
```

Code is basically the same as yours:
```
(define gcc-toolchain-2.31
  (make-gcc-toolchain gcc glibc-2.31))

(package-with-c-toolchain rust-liana_gui-5.0.0 `(("gcc-toolchain" ,gcc-toolchain-2.31)))
```


On Wednesday, April 17th, 2024 at 5:17 PM, Philip McGrath <philip@philipmcgrath.com> wrote:

> Hi,
> 
> On Wed, Apr 17, 2024, at 6:28 AM, Antoine Poinsot wrote:
> 
> > Good morning all,
> > 
> > I've been working on using Guix for the reproducible builds of my Rust
> > software. My main question is about linking my program against an older
> > glibc version (or better yet, statically linking it against musl).
> > 
> > [...]
> > 
> > Now this links against glibc 2.35. I need to target ideally 2.29 or at
> > most 2.31. That's what i'm seeking help for. I've tried the obvious:
> > `guix build -f test2.scm --with-input=glibc=glibc@2.29`
> 
> 
> My knowledge in this area is very shallow, but I also have wanted to link against old glibc versions, for somewhat similar purposes. The best I've got so far is at [1], in which I successfully use `make-gcc-toolchain` to use `gcc` with `glibc-2.31` and define a helper function `package-with-old-stable-libc` to supply it to `package-with-c-toolchain`. I use that in [2] to compile the library I'm interested in against the old glibc version. While this is the best solution I've found so far, I'm still not entirely happy with it! While the code I'm pointing you to isn't too bad, other parts of the overall setup are overly complex (in fairness, largely to also build for Apple platforms), and I haven't managed to build against Glibc 2.23 to fix [3]. But hopefully some of this might be useful, or, better yet, someone might suggest a better solution!
> 
> Philip
> 
> [1]: https://github.com/libgit2-racket/native-libs/blob/bd400daed8d4e38db6fc472d4ec82b99a884303c/guix/old-stable-libc.scm
> [2]: https://github.com/libgit2-racket/native-libs/blob/bd400daed8d4e38db6fc472d4ec82b99a884303c/guix/extracted/non-apple.scm
> [3]: https://github.com/libgit2-racket/libgit2/issues/2