unofficial mirror of guile-devel@gnu.org 
 help / color / mirror / Atom feed
* crash on macOS with dlsym RTLD_LOCAL
@ 2021-09-05  5:48 Aleix Conchillo Flaqué
  2021-09-12  5:14 ` Aleix Conchillo Flaqué
  0 siblings, 1 reply; 4+ messages in thread
From: Aleix Conchillo Flaqué @ 2021-09-05  5:48 UTC (permalink / raw)
  To: guile-devel

Hi,

I'm trying to load guile-grcypt with guile 3.0.7 and after my previous
fix (https://git.savannah.gnu.org/cgit/guile.git/commit/?id=1f100a4f20c3a6e57922fb26fce212997e2a03cb)
guile-gcrypt still does not load properly.

TLDR; Using RTLD_LOCAL instead of RTLD_GLOBAL (default) causes issues on macOS.

https://git.savannah.gnu.org/cgit/guile.git/tree/module/system/foreign-library.scm#n181

In the case of guile-gcrypt this is what I'm getting:

-------------------------
scheme@(guile-user)> (use-modules (gcrypt hmac))
dyld: lazy symbol binding failed: Symbol not found: __gcry_check_version
  Referenced from: /usr/local/lib/libgcrypt.dylib
  Expected in: flat namespace

dyld: Symbol not found: __gcry_check_version
  Referenced from: /usr/local/lib/libgcrypt.dylib
  Expected in: flat namespace
-------------------------

Looking at gcrypt symbols I can see:

❯ nm -gU /usr/local/lib/libgcrypt.dylib | grep gcry_check_version
0000000000005954 T __gcry_check_version
0000000000002aa7 T _gcry_check_version

Actually in libgcrypt the public functions are: gcry_check_version and
_gcry_check_version (no extra underscore). I think extra underscores
are automatically added when building the library but I'm not sure why
and when and what systems. And when you call dlsym() you don't need to
add those extra underscores.

The following code (which uses RTLD_GLOBAL) works fine:

./a.out gcry_check_version

-------------------------
#include <dlfcn.h>
#include <stdio.h>

int
main(int argc, char *argv[])
{
  char* (*fptr)(char *);
  void *handle = dlopen("/usr/local/lib/libgcrypt.dylib", RTLD_LAZY |
RTLD_GLOBAL);
  if (handle == NULL) {
    printf("OOOPS: %s\n", dlerror());
  } else {
    *(void **)(&fptr) = dlsym(handle, argv[1]);
    if (fptr == NULL) {
       printf("NOT FOUND: %s : %s\n", argv[1], dlerror());
    } else {
       printf("FOUND: %s %s\n", argv[1], (*fptr)(NULL));
    }
  }
  return 0;
}
-------------------------

But if we change to RTLD_LOCAL we get a crash like in guile's case.

Sorry for being too vague but I'm not familiar with how all this
works. All I know is that using RTLD_GLOBAL fixes the issue.

Also, from dlopen man page I read:

-------------------------
    RTLD_GLOBAL  Symbols exported from this image (dynamic library or
bundle) will be available to any images build with -flat_namespace
option to
                  ld(1) or to calls to dlsym() when using a special handle.

     RTLD_LOCAL   Symbols exported from this image (dynamic library or
bundle) are generally hidden and only availble to dlsym() when
directly using
                  the handle returned by this call to dlopen().
-------------------------

But I don't fully get what this means.

Any help would be really appreciated.

Thank you,

Aleix



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: crash on macOS with dlsym RTLD_LOCAL
  2021-09-05  5:48 crash on macOS with dlsym RTLD_LOCAL Aleix Conchillo Flaqué
@ 2021-09-12  5:14 ` Aleix Conchillo Flaqué
  2021-09-14 17:06   ` Aleix Conchillo Flaqué
  0 siblings, 1 reply; 4+ messages in thread
From: Aleix Conchillo Flaqué @ 2021-09-12  5:14 UTC (permalink / raw)
  To: guile-devel

I did a little bit more research. The combination that actually fails
is RTLD_LAZY | RTLD_GLOBAL (what guile uses). So, you can actually
use:

- RTLD_LAZY | RTLD_GLOBAL
- RTLD_NOW | RTLD_GLOBAL
- RTLD_NOW | RTLD_LOCAL

I found the source code of dyld

   https://opensource.apple.com/source/dyld/dyld-852.2/

But I have no clue how to build this, I tried it but failed with
missing dependencies. In any case, I provided a patch so at least we
can use Guile on macOS.

Best,

Aleix

On Sat, Sep 4, 2021 at 10:48 PM Aleix Conchillo Flaqué
<aconchillo@gmail.com> wrote:
>
> Hi,
>
> I'm trying to load guile-grcypt with guile 3.0.7 and after my previous
> fix (https://git.savannah.gnu.org/cgit/guile.git/commit/?id=1f100a4f20c3a6e57922fb26fce212997e2a03cb)
> guile-gcrypt still does not load properly.
>
> TLDR; Using RTLD_LOCAL instead of RTLD_GLOBAL (default) causes issues on macOS.
>
> https://git.savannah.gnu.org/cgit/guile.git/tree/module/system/foreign-library.scm#n181
>
> In the case of guile-gcrypt this is what I'm getting:
>
> -------------------------
> scheme@(guile-user)> (use-modules (gcrypt hmac))
> dyld: lazy symbol binding failed: Symbol not found: __gcry_check_version
>   Referenced from: /usr/local/lib/libgcrypt.dylib
>   Expected in: flat namespace
>
> dyld: Symbol not found: __gcry_check_version
>   Referenced from: /usr/local/lib/libgcrypt.dylib
>   Expected in: flat namespace
> -------------------------
>
> Looking at gcrypt symbols I can see:
>
> ❯ nm -gU /usr/local/lib/libgcrypt.dylib | grep gcry_check_version
> 0000000000005954 T __gcry_check_version
> 0000000000002aa7 T _gcry_check_version
>
> Actually in libgcrypt the public functions are: gcry_check_version and
> _gcry_check_version (no extra underscore). I think extra underscores
> are automatically added when building the library but I'm not sure why
> and when and what systems. And when you call dlsym() you don't need to
> add those extra underscores.
>
> The following code (which uses RTLD_GLOBAL) works fine:
>
> ./a.out gcry_check_version
>
> -------------------------
> #include <dlfcn.h>
> #include <stdio.h>
>
> int
> main(int argc, char *argv[])
> {
>   char* (*fptr)(char *);
>   void *handle = dlopen("/usr/local/lib/libgcrypt.dylib", RTLD_LAZY |
> RTLD_GLOBAL);
>   if (handle == NULL) {
>     printf("OOOPS: %s\n", dlerror());
>   } else {
>     *(void **)(&fptr) = dlsym(handle, argv[1]);
>     if (fptr == NULL) {
>        printf("NOT FOUND: %s : %s\n", argv[1], dlerror());
>     } else {
>        printf("FOUND: %s %s\n", argv[1], (*fptr)(NULL));
>     }
>   }
>   return 0;
> }
> -------------------------
>
> But if we change to RTLD_LOCAL we get a crash like in guile's case.
>
> Sorry for being too vague but I'm not familiar with how all this
> works. All I know is that using RTLD_GLOBAL fixes the issue.
>
> Also, from dlopen man page I read:
>
> -------------------------
>     RTLD_GLOBAL  Symbols exported from this image (dynamic library or
> bundle) will be available to any images build with -flat_namespace
> option to
>                   ld(1) or to calls to dlsym() when using a special handle.
>
>      RTLD_LOCAL   Symbols exported from this image (dynamic library or
> bundle) are generally hidden and only availble to dlsym() when
> directly using
>                   the handle returned by this call to dlopen().
> -------------------------
>
> But I don't fully get what this means.
>
> Any help would be really appreciated.
>
> Thank you,
>
> Aleix



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: crash on macOS with dlsym RTLD_LOCAL
  2021-09-12  5:14 ` Aleix Conchillo Flaqué
@ 2021-09-14 17:06   ` Aleix Conchillo Flaqué
  2021-09-27  4:49     ` Aleix Conchillo Flaqué
  0 siblings, 1 reply; 4+ messages in thread
From: Aleix Conchillo Flaqué @ 2021-09-14 17:06 UTC (permalink / raw)
  To: guile-devel

I have posted a message to Apple's forums to see if there's any luck:

https://developer.apple.com/forums/thread/689991

Aleix

On Sat, Sep 11, 2021 at 10:14 PM Aleix Conchillo Flaqué
<aconchillo@gmail.com> wrote:
>
> I did a little bit more research. The combination that actually fails
> is RTLD_LAZY | RTLD_GLOBAL (what guile uses). So, you can actually
> use:
>
> - RTLD_LAZY | RTLD_GLOBAL
> - RTLD_NOW | RTLD_GLOBAL
> - RTLD_NOW | RTLD_LOCAL
>
> I found the source code of dyld
>
>    https://opensource.apple.com/source/dyld/dyld-852.2/
>
> But I have no clue how to build this, I tried it but failed with
> missing dependencies. In any case, I provided a patch so at least we
> can use Guile on macOS.
>
> Best,
>
> Aleix
>
> On Sat, Sep 4, 2021 at 10:48 PM Aleix Conchillo Flaqué
> <aconchillo@gmail.com> wrote:
> >
> > Hi,
> >
> > I'm trying to load guile-grcypt with guile 3.0.7 and after my previous
> > fix (https://git.savannah.gnu.org/cgit/guile.git/commit/?id=1f100a4f20c3a6e57922fb26fce212997e2a03cb)
> > guile-gcrypt still does not load properly.
> >
> > TLDR; Using RTLD_LOCAL instead of RTLD_GLOBAL (default) causes issues on macOS.
> >
> > https://git.savannah.gnu.org/cgit/guile.git/tree/module/system/foreign-library.scm#n181
> >
> > In the case of guile-gcrypt this is what I'm getting:
> >
> > -------------------------
> > scheme@(guile-user)> (use-modules (gcrypt hmac))
> > dyld: lazy symbol binding failed: Symbol not found: __gcry_check_version
> >   Referenced from: /usr/local/lib/libgcrypt.dylib
> >   Expected in: flat namespace
> >
> > dyld: Symbol not found: __gcry_check_version
> >   Referenced from: /usr/local/lib/libgcrypt.dylib
> >   Expected in: flat namespace
> > -------------------------
> >
> > Looking at gcrypt symbols I can see:
> >
> > ❯ nm -gU /usr/local/lib/libgcrypt.dylib | grep gcry_check_version
> > 0000000000005954 T __gcry_check_version
> > 0000000000002aa7 T _gcry_check_version
> >
> > Actually in libgcrypt the public functions are: gcry_check_version and
> > _gcry_check_version (no extra underscore). I think extra underscores
> > are automatically added when building the library but I'm not sure why
> > and when and what systems. And when you call dlsym() you don't need to
> > add those extra underscores.
> >
> > The following code (which uses RTLD_GLOBAL) works fine:
> >
> > ./a.out gcry_check_version
> >
> > -------------------------
> > #include <dlfcn.h>
> > #include <stdio.h>
> >
> > int
> > main(int argc, char *argv[])
> > {
> >   char* (*fptr)(char *);
> >   void *handle = dlopen("/usr/local/lib/libgcrypt.dylib", RTLD_LAZY |
> > RTLD_GLOBAL);
> >   if (handle == NULL) {
> >     printf("OOOPS: %s\n", dlerror());
> >   } else {
> >     *(void **)(&fptr) = dlsym(handle, argv[1]);
> >     if (fptr == NULL) {
> >        printf("NOT FOUND: %s : %s\n", argv[1], dlerror());
> >     } else {
> >        printf("FOUND: %s %s\n", argv[1], (*fptr)(NULL));
> >     }
> >   }
> >   return 0;
> > }
> > -------------------------
> >
> > But if we change to RTLD_LOCAL we get a crash like in guile's case.
> >
> > Sorry for being too vague but I'm not familiar with how all this
> > works. All I know is that using RTLD_GLOBAL fixes the issue.
> >
> > Also, from dlopen man page I read:
> >
> > -------------------------
> >     RTLD_GLOBAL  Symbols exported from this image (dynamic library or
> > bundle) will be available to any images build with -flat_namespace
> > option to
> >                   ld(1) or to calls to dlsym() when using a special handle.
> >
> >      RTLD_LOCAL   Symbols exported from this image (dynamic library or
> > bundle) are generally hidden and only availble to dlsym() when
> > directly using
> >                   the handle returned by this call to dlopen().
> > -------------------------
> >
> > But I don't fully get what this means.
> >
> > Any help would be really appreciated.
> >
> > Thank you,
> >
> > Aleix



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: crash on macOS with dlsym RTLD_LOCAL
  2021-09-14 17:06   ` Aleix Conchillo Flaqué
@ 2021-09-27  4:49     ` Aleix Conchillo Flaqué
  0 siblings, 0 replies; 4+ messages in thread
From: Aleix Conchillo Flaqué @ 2021-09-27  4:49 UTC (permalink / raw)
  To: guile-devel

Alright. Problem found and fixed. The issue was the way libgcrypt was
linked in macOS 11.x.

See: https://dev.gnupg.org/T5610

A libgcrypt's libtool.m4 patch has been applied upstream. The patch
already existed in Homebrew's libtool, so I just provided it and they
have applied it to libgcrypt (and libgpg-error). In the meantime I
have sent patches to Homebrew's libgcrypt so macOS users can enjoy
guile-gcrypt :-).

Aleix

On Tue, Sep 14, 2021 at 10:06 AM Aleix Conchillo Flaqué
<aconchillo@gmail.com> wrote:
>
> I have posted a message to Apple's forums to see if there's any luck:
>
> https://developer.apple.com/forums/thread/689991
>
> Aleix
>
> On Sat, Sep 11, 2021 at 10:14 PM Aleix Conchillo Flaqué
> <aconchillo@gmail.com> wrote:
> >
> > I did a little bit more research. The combination that actually fails
> > is RTLD_LAZY | RTLD_GLOBAL (what guile uses). So, you can actually
> > use:
> >
> > - RTLD_LAZY | RTLD_GLOBAL
> > - RTLD_NOW | RTLD_GLOBAL
> > - RTLD_NOW | RTLD_LOCAL
> >
> > I found the source code of dyld
> >
> >    https://opensource.apple.com/source/dyld/dyld-852.2/
> >
> > But I have no clue how to build this, I tried it but failed with
> > missing dependencies. In any case, I provided a patch so at least we
> > can use Guile on macOS.
> >
> > Best,
> >
> > Aleix
> >
> > On Sat, Sep 4, 2021 at 10:48 PM Aleix Conchillo Flaqué
> > <aconchillo@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > I'm trying to load guile-grcypt with guile 3.0.7 and after my previous
> > > fix (https://git.savannah.gnu.org/cgit/guile.git/commit/?id=1f100a4f20c3a6e57922fb26fce212997e2a03cb)
> > > guile-gcrypt still does not load properly.
> > >
> > > TLDR; Using RTLD_LOCAL instead of RTLD_GLOBAL (default) causes issues on macOS.
> > >
> > > https://git.savannah.gnu.org/cgit/guile.git/tree/module/system/foreign-library.scm#n181
> > >
> > > In the case of guile-gcrypt this is what I'm getting:
> > >
> > > -------------------------
> > > scheme@(guile-user)> (use-modules (gcrypt hmac))
> > > dyld: lazy symbol binding failed: Symbol not found: __gcry_check_version
> > >   Referenced from: /usr/local/lib/libgcrypt.dylib
> > >   Expected in: flat namespace
> > >
> > > dyld: Symbol not found: __gcry_check_version
> > >   Referenced from: /usr/local/lib/libgcrypt.dylib
> > >   Expected in: flat namespace
> > > -------------------------
> > >
> > > Looking at gcrypt symbols I can see:
> > >
> > > ❯ nm -gU /usr/local/lib/libgcrypt.dylib | grep gcry_check_version
> > > 0000000000005954 T __gcry_check_version
> > > 0000000000002aa7 T _gcry_check_version
> > >
> > > Actually in libgcrypt the public functions are: gcry_check_version and
> > > _gcry_check_version (no extra underscore). I think extra underscores
> > > are automatically added when building the library but I'm not sure why
> > > and when and what systems. And when you call dlsym() you don't need to
> > > add those extra underscores.
> > >
> > > The following code (which uses RTLD_GLOBAL) works fine:
> > >
> > > ./a.out gcry_check_version
> > >
> > > -------------------------
> > > #include <dlfcn.h>
> > > #include <stdio.h>
> > >
> > > int
> > > main(int argc, char *argv[])
> > > {
> > >   char* (*fptr)(char *);
> > >   void *handle = dlopen("/usr/local/lib/libgcrypt.dylib", RTLD_LAZY |
> > > RTLD_GLOBAL);
> > >   if (handle == NULL) {
> > >     printf("OOOPS: %s\n", dlerror());
> > >   } else {
> > >     *(void **)(&fptr) = dlsym(handle, argv[1]);
> > >     if (fptr == NULL) {
> > >        printf("NOT FOUND: %s : %s\n", argv[1], dlerror());
> > >     } else {
> > >        printf("FOUND: %s %s\n", argv[1], (*fptr)(NULL));
> > >     }
> > >   }
> > >   return 0;
> > > }
> > > -------------------------
> > >
> > > But if we change to RTLD_LOCAL we get a crash like in guile's case.
> > >
> > > Sorry for being too vague but I'm not familiar with how all this
> > > works. All I know is that using RTLD_GLOBAL fixes the issue.
> > >
> > > Also, from dlopen man page I read:
> > >
> > > -------------------------
> > >     RTLD_GLOBAL  Symbols exported from this image (dynamic library or
> > > bundle) will be available to any images build with -flat_namespace
> > > option to
> > >                   ld(1) or to calls to dlsym() when using a special handle.
> > >
> > >      RTLD_LOCAL   Symbols exported from this image (dynamic library or
> > > bundle) are generally hidden and only availble to dlsym() when
> > > directly using
> > >                   the handle returned by this call to dlopen().
> > > -------------------------
> > >
> > > But I don't fully get what this means.
> > >
> > > Any help would be really appreciated.
> > >
> > > Thank you,
> > >
> > > Aleix



^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2021-09-27  4:49 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-05  5:48 crash on macOS with dlsym RTLD_LOCAL Aleix Conchillo Flaqué
2021-09-12  5:14 ` Aleix Conchillo Flaqué
2021-09-14 17:06   ` Aleix Conchillo Flaqué
2021-09-27  4:49     ` Aleix Conchillo Flaqué

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).