bug#66699: zig: fail to link SDL2

bug#66699: zig: fail to link SDL2

[Nguyễn Gia Phong via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]

I am trying to package Mepo, an OpenStreetMap frontend written in Zig.
The currently packaged Zig fails to find transitive dependencies of SDL2:
https://paste.sr.ht/~cnx/db21e1ef9a6828f3ec898db43b790d71b5ef213a

Here's a minimal reproducible Zig project:

// main.zig
const c = @cImport(@cInclude("SDL2/SDL.h"));
pub fn main() !void {
    if (c.SDL_Init(c.SDL_INIT_VIDEO) != 0)
        return error.SDLInitializationFailed;
    c.SDL_Quit();
}

// build.zig
pub fn build(b: *@import("std").build.Builder) void {
    const exe = b.addExecutable("sdl2-smoke-test", "main.zig");
    exe.linkSystemLibrary("SDL2");
    exe.setTarget(b.standardTargetOptions(.{}));
    exe.setBuildMode(b.standardReleaseOptions());
    exe.install();
}

For convenience, here goes a manifest for use with guix build -f:

(use-modules (guix build-system zig)
             (guix gexp)
             (guix licenses)
             (guix packages)
             (gnu packages pkg-config)
             (gnu packages sdl))
(package
  (name "sdl2-smoke-test")
  (version "0.0.0")
  (source (local-file "." "sdl2-smoke-test" #:recursive? #t))
  (build-system zig-build-system)
  (arguments (list #:tests? #f
                   #:zig-release-type "safe"))
  (native-inputs (list pkg-config))
  (inputs (list sdl2))
  (synopsis "Zig build system SDL2 linkage failure reproducer")
  (description
    (string-append "This is a reproducer of zig-build-system"
                   " failing to link SDL2's transitive dependencies,"
                   " as seen when trying to package mepo."))
  (home-page "https://larkspur.one/notice/Ab24w1IMGVoZ72J9xg")
  (license zlib))

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

bug#66699: zig: fail to link SDL2

[Ekaitz Zarraga]

------- Original Message -------
On Monday, October 23rd, 2023 at 09:34, Nguyễn Gia Phong <cnx@loang.net> wrote:


> I am trying to package Mepo, an OpenStreetMap frontend written in Zig.
> The currently packaged Zig fails to find transitive dependencies of SDL2:
> https://paste.sr.ht/~cnx/db21e1ef9a6828f3ec898db43b790d71b5ef213a
> 
> Here's a minimal reproducible Zig project:
> 
> // main.zig
> const c = @cImport(@cInclude("SDL2/SDL.h"));
> pub fn main() !void {
> if (c.SDL_Init(c.SDL_INIT_VIDEO) != 0)
> return error.SDLInitializationFailed;
> c.SDL_Quit();
> }
> 
> // build.zig
> pub fn build(b: *@import("std").build.Builder) void {
> const exe = b.addExecutable("sdl2-smoke-test", "main.zig");
> exe.linkSystemLibrary("SDL2");
> exe.setTarget(b.standardTargetOptions(.{}));
> exe.setBuildMode(b.standardReleaseOptions());
> exe.install();
> }
> 
> For convenience, here goes a manifest for use with guix build -f:
> 
> (use-modules (guix build-system zig)
> (guix gexp)
> (guix licenses)
> (guix packages)
> (gnu packages pkg-config)
> (gnu packages sdl))
> (package
> (name "sdl2-smoke-test")
> (version "0.0.0")
> (source (local-file "." "sdl2-smoke-test" #:recursive? #t))
> (build-system zig-build-system)
> (arguments (list #:tests? #f
> #:zig-release-type "safe"))
> (native-inputs (list pkg-config))
> (inputs (list sdl2))
> (synopsis "Zig build system SDL2 linkage failure reproducer")
> (description
> (string-append "This is a reproducer of zig-build-system"
> " failing to link SDL2's transitive dependencies,"
> " as seen when trying to package mepo."))
> (home-page "https://larkspur.one/notice/Ab24w1IMGVoZ72J9xg")
> (license zlib))


Hi,

I have good and bad news for you.

I tried to replicate the problem, and I certainly did. I also managed to mitigate the problem adding `exe.linkSystemLibrary()` lines to the build.zig with the transitive dependencies.
And later I wondered how did this all work in GCC, just in case.

(NOTE: take in account you also need to link with libc! for that you can add it as a linkSystemLibrary("c"))

The problem is not Zig specific, look:

I tried a random example from a repo in Github using SDL2 and SDL2-Image: 
https://github.com/aminosbh/sdl2-image-sample/blob/master/src/main.c

Made a shell for it:

guix shell sdl2 sdl2-image gcc-toolchain glibc:static

And tried to build:

$ gcc a.c -static -lSDL2 -lSDL2_image -lm -lc 2>&1 | head
ld: /gnu/store/0mhh6ycg8dq6n43qwpvpvr2smbxlixpi-profile/lib/libSDL2.a(SDL_dynapi.o):(.text+0x27d8): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
ld: /gnu/store/0mhh6ycg8dq6n43qwpvpvr2smbxlixpi-profile/lib/libSDL2.a(SDL_waylandvideo.o):(.text+0x48f): undefined reference to `wl_proxy_get_version'
ld: /gnu/store/0mhh6ycg8dq6n43qwpvpvr2smbxlixpi-profile/lib/libSDL2.a(SDL_waylandvideo.o):(.text+0xd3e): undefined reference to `wl_proxy_get_version'
ld: /gnu/store/0mhh6ycg8dq6n43qwpvpvr2smbxlixpi-profile/lib/libSDL2.a(SDL_waylandvideo.o):(.text+0xd5a): undefined reference to `wl_proxy_marshal_flags'
ld: /gnu/store/0mhh6ycg8dq6n43qwpvpvr2smbxlixpi-profile/lib/libSDL2.a(SDL_waylandvideo.o):(.text+0xd73): undefined reference to `wl_proxy_add_listener'
ld: /gnu/store/0mhh6ycg8dq6n43qwpvpvr2smbxlixpi-profile/lib/libSDL2.a(SDL_waylandvideo.o):(.text+0xeb9): undefined reference to `wl_proxy_get_version'
ld: /gnu/store/0mhh6ycg8dq6n43qwpvpvr2smbxlixpi-profile/lib/libSDL2.a(SDL_waylandvideo.o):(.text+0xfa8): undefined reference to `wl_proxy_marshal_flags'
ld: /gnu/store/0mhh6ycg8dq6n43qwpvpvr2smbxlixpi-profile/lib/libSDL2.a(SDL_waylandvideo.o):(.text+0x1002): undefined reference to `wl_proxy_marshal_flags'
ld: /gnu/store/0mhh6ycg8dq6n43qwpvpvr2smbxlixpi-profile/lib/libSDL2.a(SDL_waylandvideo.o):(.text+0x1071): undefined reference to `wl_proxy_add_listener'
ld: /gnu/store/0mhh6ycg8dq6n43qwpvpvr2smbxlixpi-profile/lib/libSDL2.a(SDL_waylandvideo.o):(.text+0x107e): undefined reference to `wl_proxy_set_tag'

The list of errors are basically the same you have.
Zig builds statically if I'm not mistaken so it's triggering the same problem GCC would for that case.

We need to research this further...

I also checked Mepo, how do they manage to build directly?

I don't have the Zig knowledge to know how does all this work, but I know it's not just the build-system's fault because it happens in GCC too, and also with a normal shell with `zig` available, and running the command by hand. The build-system just replicates that behavior.

We need to dig on this further. Are you, Nguyễn, a Zig programmer? Maybe you can help me figure out this?

Thanks for the report, the patience and the help!

All the best,
Ekaitz

bug#66699: zig: fail to link SDL2

[Ekaitz Zarraga]

Hi,


On Friday, November 10th, 2023 at 06:01, Nguyễn Gia Phong <cnx@loang.net> wrote:

> On 2023-11-01 at 20:52+00:00, Ekaitz Zarraga wrote:
> 
> > On Monday, October 23rd, 2023 at 09:34, Nguyễn Gia Phong wrote:
> > 
> > > I am trying to package Mepo, an OpenStreetMap frontend written in Zig.
> > > The currently packaged Zig fails to find transitive dependencies of SDL2:
> > > https://paste.sr.ht/~cnx/db21e1ef9a6828f3ec898db43b790d71b5ef213a
> > 
> > The problem is not Zig specific, look: [...]
> > 
> > guix shell sdl2 sdl2-image gcc-toolchain glibc:static
> > 
> > And tried to build: [...] The list of errors are basically the same
> > you have. Zig builds statically if I'm not mistaken
> > so it's triggering the same problem GCC would for that case.
> > 
> > We need to research this further...
> > 
> > I don't have the Zig knowledge to know how does all this work,
> > but I know it's not just the build-system's fault
> > because it happens in GCC too, and also with a normal shell
> > with `zig` available, and running the command by hand.
> > The build-system just replicates that behavior.
> 
> 
> Thanks, this clears it up for me. So far Zig does not offer a way
> to prefer dynamic or static and just search alphabetically:
> https://github.com/ziglang/zig/issues/14111

Maybe there are more flags for this, but I'm not sure.
 
> On 2023-05-23 at 17:11Z, Michael Dusan wrote:
> 
> > A workaround is if you are only using the zig cc driver,
> > the syntax -l:<FULLNAME> will find the first fully-named library file
> > that matches against search dirs. For example:
> > 
> > * -lz will search dirs, prefer lib<BASENAME>.a over lib<BASENAME>.so
> > * -l:libz.so will search directly for libz.so
> > and not care about any other name permutations
> 
> 
> I do wonder though, if it's a generally good idea have separate
> shared and static outputs for e.g. SDL2. Obviously Zig programs
> would benefit from this, and specifying linkage preference
> under GCC is via the same -l:lib*.* flag.

I'm not very versed on this so I can't give you a proper answer.

> On 2023-11-01 at 20:52+00:00, Ekaitz Zarraga wrote:
> 
> > I also checked Mepo, how do they manage to build directly?
> 
> Sorry, could you rephrase this question?
> I don't get what you meant by directly here.

Yes, it was pretty badly phrased.
The question here was about how do the Mepo devs build mepo in other
linux distributions... They should have a similar problem to what we
have. Or other systems like Nix, how do they build?
In Nix the problem should appear too.

With *directly* I meant the Mepo repository doesn't contain the
transitive dependencies so... It would never work for us, but it
should be working for them.

Reading your message before seems like they manage to link against
the SDL .so so nothing of this happens...

I'll try to check all this with the Zig people to see if we can
fix.

Thanks!

bug#66699: zig: fail to link SDL2

[Ekaitz Zarraga]

Hi,

I tracked this pretty far:

This is a related issue that mentions the `mepo` package:
https://github.com/ziglang/zig/issues/12170

And I realized the problem comes from `pkg-config`. It's telling zig to use the static library first.
Removing `pkg-config` from the `native-inputs` in your proposed package make it build but the `runpath` validation phase fails.

I'm not sure about what to do now.

In the issue the people working on freebsd-ports mention they did this:
https://github.com/freebsd/freebsd-ports/blob/16ada4d8d14a222bd92c4bf332d28b7c604af457/astro/mepo/Makefile#L31
But I don't know if we can replicate that.

I'll keep researching on this a little bit further.

Best,
Ekaitz