On Thu, Nov 14, 2024 at 12:46:59AM +0800, Hilton Chain wrote: > Hello everyone, > > With a new forced push (and new rebuilds, sorry), wip-zig-bootstrap is mostly > ready (for me) now. Please take a look at the first few commits, as I'm > changing Zig's behavior there, here are some additional notes: > > To Efraim: Can adding pkg-config to native-inputs avoid the ncdu snippet? Unfortunately no. The code reads: exe.root_module.linkSystemLibrary("ncursesw", .{}); exe.root_module.linkSystemLibrary("libzstd", .{}); and so it searches (something like the following, I lost the log): /usr/lib/liblibzstd.so /usr/lib/liblibzstd.a ... /gnu/store/...-glibc.../lib/liblibzstd.so /gnu/store/...-glibc.../lib/liblibzstd.a ... /gnu/store/...-zstd.../lib/liblibzstd.so /gnu/store/...-zstd.../lib/liblibzstd.a so it looks like it automatically adds the 'lib' at the front. When I've built out to ncdu again I'll check it again. > I'm building this branch on my personal Cuirass instance[1][2], for x86_64-linux > and aarch64-linux (qemu-binfmt), in previous revisions I have indentified > reproduciblity issue, and aarch64 builds timed-out. I haven't investigated them > yet. I've also been having berlin build zig on the wip-zig-bootstrap branch for x86_64. Unfortunately there's quite a bit to go on aarch64 for it to get there, but I can confirm from my machine that it's working. > Then for what's the RUNPATH issue I have mentioned in commits: > + For current zig@0.10 on Guix master: glibc is missing from RUNPATH, which > fails the validate-runpath check. > + For zig@0.11, some other inputs are missing, making the binary failing to run on Guix[3]. > + dan also mentioned privately to me that they needed to add paths from > LIBRARY_PATH to RUNPATH for their own projects, so programs built by Zig is also > affected. I didn't test it. ncdu@2.3 builds with zig-0.11, so that's an option for testing it out. https://dev.yorhel.nl/ncdu/changes2 > I think this is due to Zig's implemention of its own linking logic, which > bypasses our ld-wrapper. I wonder if switching from lld to make-lld-wrapper would make a difference here. > I'm not going to implement ld-wrapper within Zig. :) So my proposed workaround > in wip-zig-bootstrap is to patch the handling logic added for Guix: > > (In lib/std/zig/system/NativePaths.zig) > --8<---------------cut here---------------start------------->8--- > // Distros like guix don't use FHS, so they rely on environment > // variables to search for headers and libraries. > // We use os.getenv here since this part won't be executed on > // windows, to get rid of unnecessary error handling. > - if (std.posix.getenv("C_INCLUDE_PATH")) |c_include_path| { > + if (std.posix.getenv("CROSS_C_INCLUDE_PATH") orelse std.posix.getenv("C_INCLUDE_PATH")) |c_include_path| { > var it = mem.tokenizeScalar(u8, c_include_path, ':'); > while (it.next()) |dir| { > try self.addIncludeDir(dir); > } > } > > - if (std.posix.getenv("CPLUS_INCLUDE_PATH")) |cplus_include_path| { > + if (std.posix.getenv("CROSS_CPLUS_INCLUDE_PATH") orelse std.posix.getenv("CPLUS_INCLUDE_PATH")) |cplus_include_path| { > var it = mem.tokenizeScalar(u8, cplus_include_path, ':'); > while (it.next()) |dir| { > try self.addIncludeDir(dir); > } > } > > - if (std.posix.getenv("LIBRARY_PATH")) |library_path| { > + if (std.posix.getenv("CROSS_LIBRARY_PATH") orelse std.posix.getenv("LIBRARY_PATH")) |library_path| { > var it = mem.tokenizeScalar(u8, library_path, ':'); > while (it.next()) |dir| { > try self.addLibDir(dir); > + try self.addRPath(dir); > } > } > } > --8<---------------cut here---------------end--------------->8--- > > Adding directories from CROSS_LIBRARY_PATH or LIBRARY_PATH to RUNPATH, "CROSS_" > part is for our cross toolchain, I haven't tested it yet. I like this, and it seems like it should make it work for cross compiling zig programs. That's part of why I added the updated ncdu commit, to use it for testing. > I think this behavior change is reasonable since the search path > (CROSS_)?LIBRARY_PATH is only automatically set by our compilers. > > I added this change to 0.9 as well to make all Zigs behave consistently. I also > used shrink-runpath phase from meson-build-system in Zig and zig-build-system. > > I want to move shrink-runpath to (guix build utils) and export it too, so that > it can be used easier. But I'm not sure if this change will trigger rebuilds of > other packages, so I didn't do it. I found that there was still the full LIBRARY_PATH embedded in the ncdu binary as a string. So with that I'm not sure about using RPath instead of LibDir for the LIBRARY_PATH. > Thanks to Guile, for builds not managed by guix-daemon, something like the > following script can be used, we can ship a program-file if we agree on this > workaround. > --8<---------------cut here---------------start------------->8--- > (use-modules (guix build meson-build-system)) > > (define shrink-runpath > (assoc-ref %standard-phases 'shrink-runpath)) > > (define (main directories) > (for-each (lambda (dir) > (false-if-exception > (shrink-runpath > #:elf-directories '(".") > #:outputs `(("out" . ,dir))))) > directories)) > > (main (cdr (command-line))) > --8<---------------cut here---------------end--------------->8--- > Usage: guile DIRECTORY... > > > Thanks > --- > [1]: https://ci.boiledscript.com/jobset/guix-zig > [2]: https://substitute.boiledscript.com, if you want to challenge it. > [3]: https://github.com/ziglang/zig/issues/18434 I spent a bunch of time trying to get zig-0.10 or 0.10.0-610 to build on riscv64 and ppc64le but haven't been able to crack it yet. -- Efraim Flashner אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted