From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?Q?Aleix_Conchillo_Flaqu=C3=A9?= Newsgroups: gmane.lisp.guile.devel Subject: Re: crash on macOS with dlsym RTLD_LOCAL Date: Sun, 26 Sep 2021 21:49:26 -0700 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="35870"; mail-complaints-to="usenet@ciao.gmane.io" To: guile-devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Mon Sep 27 07:13:25 2021 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mUixB-0009Ay-6N for guile-devel@m.gmane-mx.org; Mon, 27 Sep 2021 07:13:25 +0200 Original-Received: from localhost ([::1]:58468 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mUix9-0007Dn-TW for guile-devel@m.gmane-mx.org; Mon, 27 Sep 2021 01:13:23 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:58388) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mUiaH-00024D-VR for guile-devel@gnu.org; Mon, 27 Sep 2021 00:49:47 -0400 Original-Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]:33539) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mUiaB-0001Ku-1D for guile-devel@gnu.org; Mon, 27 Sep 2021 00:49:45 -0400 Original-Received: by mail-pf1-x431.google.com with SMTP id s16so14738171pfk.0 for ; Sun, 26 Sep 2021 21:49:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-transfer-encoding; bh=5Em4+O4dYiYBq2o6pB1LLSf97I23eH64vVTrxAQ1zjI=; b=ZIciQVx7mvzl3WmRNMg0JigcKNn8eACMoM84zwyNgi9MIX/vlLvTXO1Q1kvOm40V1u +KWVORkBB34MhXr8gUOUE8CTyOaox9kjDRjBs00BKs8fCuIRcVMprYpEEFO72WbwA7Hr 2Y3XlbXj2/C8YV8DzHVoeufkNgyoOwrvV3EXSuEeDhWgx24kCvv99uFludgTnNhY9LIB 1iV3X0t2TYH+Dpbs3kaI87YPWJnL5B1b0H9H2Ym5q3Zj/r4fpTGVj7X2qD4PtH7Apg67 phA9FNczM6CV+pSgpK2uJXtKB8spVRhW1+yAQz2G+33vHpjKKHKCwjBqb9GwOee38if0 pIkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:content-transfer-encoding; bh=5Em4+O4dYiYBq2o6pB1LLSf97I23eH64vVTrxAQ1zjI=; b=MOx93ACJ/GMzLOVtsF9WGujnGnvGvs6tTqCRIAAXxFLl/Y6hKpRwSjoUji60RAAbvB x2twUAC4fJBaei1ZivV2XeY6OMxJfNWvGC60Q1masivCGZgnFOb4xTXZzpHEt4oWTVAR yB5S9+c8dm5cJFW1fg2rwTHhMgf2k+ROH/5HapLxkK9teomQX1dLKvML8RxUfPWjbYQx k49Uar9+2yJ2FqM5cmjER03ToWUxOIWKfsQ9l0RoLoNf5uTNubpynkWGgSihXTT4cb4o IAv0fl578u5dgBHCiSoAyyXAJtog3CKUOn4UBQymD8CZxM1ICNBuu4z40JphGfEO3Juz Kyfw== X-Gm-Message-State: AOAM531QJxDKWWR+bG5pmV6pNUNZ3mt3Uq4Deg571f9rXU/wP72Hw5/t pE3YCuZPHVcZc7yRoBfjLXg4XXUjoc3eWIiJaZs300hmBvQ= X-Google-Smtp-Source: ABdhPJwvVFy+VJrt0f+lRTuDguA93WaBYK6tHO3AocSqfkEU4LYlkWPuaGUa68E0MqyYrVTCiQsP2Y1flwnbBmRdXPo= X-Received: by 2002:aa7:9aee:0:b0:440:cd9d:405e with SMTP id y14-20020aa79aee000000b00440cd9d405emr21769828pfp.47.1632718176974; Sun, 26 Sep 2021 21:49:36 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=aconchillo@gmail.com; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.io gmane.lisp.guile.devel:20888 Archived-At: Alright. Problem found and fixed. The issue was the way libgcrypt was linked in macOS 11.x. See: https://dev.gnupg.org/T5610 A libgcrypt's libtool.m4 patch has been applied upstream. The patch already existed in Homebrew's libtool, so I just provided it and they have applied it to libgcrypt (and libgpg-error). In the meantime I have sent patches to Homebrew's libgcrypt so macOS users can enjoy guile-gcrypt :-). Aleix On Tue, Sep 14, 2021 at 10:06 AM Aleix Conchillo Flaqu=C3=A9 wrote: > > I have posted a message to Apple's forums to see if there's any luck: > > https://developer.apple.com/forums/thread/689991 > > Aleix > > On Sat, Sep 11, 2021 at 10:14 PM Aleix Conchillo Flaqu=C3=A9 > wrote: > > > > I did a little bit more research. The combination that actually fails > > is RTLD_LAZY | RTLD_GLOBAL (what guile uses). So, you can actually > > use: > > > > - RTLD_LAZY | RTLD_GLOBAL > > - RTLD_NOW | RTLD_GLOBAL > > - RTLD_NOW | RTLD_LOCAL > > > > I found the source code of dyld > > > > https://opensource.apple.com/source/dyld/dyld-852.2/ > > > > But I have no clue how to build this, I tried it but failed with > > missing dependencies. In any case, I provided a patch so at least we > > can use Guile on macOS. > > > > Best, > > > > Aleix > > > > On Sat, Sep 4, 2021 at 10:48 PM Aleix Conchillo Flaqu=C3=A9 > > wrote: > > > > > > Hi, > > > > > > I'm trying to load guile-grcypt with guile 3.0.7 and after my previou= s > > > fix (https://git.savannah.gnu.org/cgit/guile.git/commit/?id=3D1f100a4= f20c3a6e57922fb26fce212997e2a03cb) > > > guile-gcrypt still does not load properly. > > > > > > TLDR; Using RTLD_LOCAL instead of RTLD_GLOBAL (default) causes issues= on macOS. > > > > > > https://git.savannah.gnu.org/cgit/guile.git/tree/module/system/foreig= n-library.scm#n181 > > > > > > In the case of guile-gcrypt this is what I'm getting: > > > > > > ------------------------- > > > scheme@(guile-user)> (use-modules (gcrypt hmac)) > > > dyld: lazy symbol binding failed: Symbol not found: __gcry_check_vers= ion > > > Referenced from: /usr/local/lib/libgcrypt.dylib > > > Expected in: flat namespace > > > > > > dyld: Symbol not found: __gcry_check_version > > > Referenced from: /usr/local/lib/libgcrypt.dylib > > > Expected in: flat namespace > > > ------------------------- > > > > > > Looking at gcrypt symbols I can see: > > > > > > =E2=9D=AF nm -gU /usr/local/lib/libgcrypt.dylib | grep gcry_check_ver= sion > > > 0000000000005954 T __gcry_check_version > > > 0000000000002aa7 T _gcry_check_version > > > > > > Actually in libgcrypt the public functions are: gcry_check_version an= d > > > _gcry_check_version (no extra underscore). I think extra underscores > > > are automatically added when building the library but I'm not sure wh= y > > > and when and what systems. And when you call dlsym() you don't need t= o > > > add those extra underscores. > > > > > > The following code (which uses RTLD_GLOBAL) works fine: > > > > > > ./a.out gcry_check_version > > > > > > ------------------------- > > > #include > > > #include > > > > > > int > > > main(int argc, char *argv[]) > > > { > > > char* (*fptr)(char *); > > > void *handle =3D dlopen("/usr/local/lib/libgcrypt.dylib", RTLD_LAZY= | > > > RTLD_GLOBAL); > > > if (handle =3D=3D NULL) { > > > printf("OOOPS: %s\n", dlerror()); > > > } else { > > > *(void **)(&fptr) =3D dlsym(handle, argv[1]); > > > if (fptr =3D=3D NULL) { > > > printf("NOT FOUND: %s : %s\n", argv[1], dlerror()); > > > } else { > > > printf("FOUND: %s %s\n", argv[1], (*fptr)(NULL)); > > > } > > > } > > > return 0; > > > } > > > ------------------------- > > > > > > But if we change to RTLD_LOCAL we get a crash like in guile's case. > > > > > > Sorry for being too vague but I'm not familiar with how all this > > > works. All I know is that using RTLD_GLOBAL fixes the issue. > > > > > > Also, from dlopen man page I read: > > > > > > ------------------------- > > > RTLD_GLOBAL Symbols exported from this image (dynamic library or > > > bundle) will be available to any images build with -flat_namespace > > > option to > > > ld(1) or to calls to dlsym() when using a special h= andle. > > > > > > RTLD_LOCAL Symbols exported from this image (dynamic library o= r > > > bundle) are generally hidden and only availble to dlsym() when > > > directly using > > > the handle returned by this call to dlopen(). > > > ------------------------- > > > > > > But I don't fully get what this means. > > > > > > Any help would be really appreciated. > > > > > > Thank you, > > > > > > Aleix