Re: crash on macOS with dlsym RTLD_LOCAL

["Aleix Conchillo Flaqué" <aconchillo@gmail.com>]

Alright. Problem found and fixed. The issue was the way libgcrypt was
linked in macOS 11.x.

See: https://dev.gnupg.org/T5610

A libgcrypt's libtool.m4 patch has been applied upstream. The patch
already existed in Homebrew's libtool, so I just provided it and they
have applied it to libgcrypt (and libgpg-error). In the meantime I
have sent patches to Homebrew's libgcrypt so macOS users can enjoy
guile-gcrypt :-).

Aleix

On Tue, Sep 14, 2021 at 10:06 AM Aleix Conchillo Flaqué
<aconchillo@gmail.com> wrote:
>
> I have posted a message to Apple's forums to see if there's any luck:
>
> https://developer.apple.com/forums/thread/689991
>
> Aleix
>
> On Sat, Sep 11, 2021 at 10:14 PM Aleix Conchillo Flaqué
> <aconchillo@gmail.com> wrote:
> >
> > I did a little bit more research. The combination that actually fails
> > is RTLD_LAZY | RTLD_GLOBAL (what guile uses). So, you can actually
> > use:
> >
> > - RTLD_LAZY | RTLD_GLOBAL
> > - RTLD_NOW | RTLD_GLOBAL
> > - RTLD_NOW | RTLD_LOCAL
> >
> > I found the source code of dyld
> >
> >    https://opensource.apple.com/source/dyld/dyld-852.2/
> >
> > But I have no clue how to build this, I tried it but failed with
> > missing dependencies. In any case, I provided a patch so at least we
> > can use Guile on macOS.
> >
> > Best,
> >
> > Aleix
> >
> > On Sat, Sep 4, 2021 at 10:48 PM Aleix Conchillo Flaqué
> > <aconchillo@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > I'm trying to load guile-grcypt with guile 3.0.7 and after my previous
> > > fix (https://git.savannah.gnu.org/cgit/guile.git/commit/?id=1f100a4f20c3a6e57922fb26fce212997e2a03cb)
> > > guile-gcrypt still does not load properly.
> > >
> > > TLDR; Using RTLD_LOCAL instead of RTLD_GLOBAL (default) causes issues on macOS.
> > >
> > > https://git.savannah.gnu.org/cgit/guile.git/tree/module/system/foreign-library.scm#n181
> > >
> > > In the case of guile-gcrypt this is what I'm getting:
> > >
> > > -------------------------
> > > scheme@(guile-user)> (use-modules (gcrypt hmac))
> > > dyld: lazy symbol binding failed: Symbol not found: __gcry_check_version
> > >   Referenced from: /usr/local/lib/libgcrypt.dylib
> > >   Expected in: flat namespace
> > >
> > > dyld: Symbol not found: __gcry_check_version
> > >   Referenced from: /usr/local/lib/libgcrypt.dylib
> > >   Expected in: flat namespace
> > > -------------------------
> > >
> > > Looking at gcrypt symbols I can see:
> > >
> > > ❯ nm -gU /usr/local/lib/libgcrypt.dylib | grep gcry_check_version
> > > 0000000000005954 T __gcry_check_version
> > > 0000000000002aa7 T _gcry_check_version
> > >
> > > Actually in libgcrypt the public functions are: gcry_check_version and
> > > _gcry_check_version (no extra underscore). I think extra underscores
> > > are automatically added when building the library but I'm not sure why
> > > and when and what systems. And when you call dlsym() you don't need to
> > > add those extra underscores.
> > >
> > > The following code (which uses RTLD_GLOBAL) works fine:
> > >
> > > ./a.out gcry_check_version
> > >
> > > -------------------------
> > > #include <dlfcn.h>
> > > #include <stdio.h>
> > >
> > > int
> > > main(int argc, char *argv[])
> > > {
> > >   char* (*fptr)(char *);
> > >   void *handle = dlopen("/usr/local/lib/libgcrypt.dylib", RTLD_LAZY |
> > > RTLD_GLOBAL);
> > >   if (handle == NULL) {
> > >     printf("OOOPS: %s\n", dlerror());
> > >   } else {
> > >     *(void **)(&fptr) = dlsym(handle, argv[1]);
> > >     if (fptr == NULL) {
> > >        printf("NOT FOUND: %s : %s\n", argv[1], dlerror());
> > >     } else {
> > >        printf("FOUND: %s %s\n", argv[1], (*fptr)(NULL));
> > >     }
> > >   }
> > >   return 0;
> > > }
> > > -------------------------
> > >
> > > But if we change to RTLD_LOCAL we get a crash like in guile's case.
> > >
> > > Sorry for being too vague but I'm not familiar with how all this
> > > works. All I know is that using RTLD_GLOBAL fixes the issue.
> > >
> > > Also, from dlopen man page I read:
> > >
> > > -------------------------
> > >     RTLD_GLOBAL  Symbols exported from this image (dynamic library or
> > > bundle) will be available to any images build with -flat_namespace
> > > option to
> > >                   ld(1) or to calls to dlsym() when using a special handle.
> > >
> > >      RTLD_LOCAL   Symbols exported from this image (dynamic library or
> > > bundle) are generally hidden and only availble to dlsym() when
> > > directly using
> > >                   the handle returned by this call to dlopen().
> > > -------------------------
> > >
> > > But I don't fully get what this means.
> > >
> > > Any help would be really appreciated.
> > >
> > > Thank you,
> > >
> > > Aleix