From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id YI1TFBm+n2FzhQAAgWs5BA (envelope-from ) for ; Thu, 25 Nov 2021 17:47:21 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id CGsEEBm+n2EnSwAA1q6Kng (envelope-from ) for ; Thu, 25 Nov 2021 16:47:21 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5308B21FC2 for ; Thu, 25 Nov 2021 17:47:20 +0100 (CET) Received: from localhost ([::1]:41754 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mqHu3-0002Xp-FV for larch@yhetil.org; Thu, 25 Nov 2021 11:47:19 -0500 Received: from eggs.gnu.org ([209.51.188.92]:36698) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mqHtq-0002Uc-Um for help-guix@gnu.org; Thu, 25 Nov 2021 11:47:06 -0500 Received: from mail-4317.proton.ch ([185.70.43.17]:48267) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mqHtm-0002Bk-D2 for help-guix@gnu.org; Thu, 25 Nov 2021 11:47:05 -0500 Date: Thu, 25 Nov 2021 16:46:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=elenq.tech; s=protonmail; t=1637858810; bh=INBSUybyXO3566DhO2XjTkOdme+bd1m0MFVWq9i++uI=; h=Date:To:From:Reply-To:Subject:From; b=d63EU8VJq+FkB2ubeqLZWYO8bfZ8ijz7PE2LFtpq4Ni4dADTwTLLuHuzSOb6CPvwc vZhRLhOM5XqFxjjgXiWswlV+aIsKLXWvcx/EXVrQSJucj/bgU8BBqKAvwZPYOGQG9B TPqVQIqE8CcNBHIDK64ucqrQnkspYR4tZ/YK+plVImQSymQdyT3C8riSMAoOxWlkRe Yd2xXILe4x148EIsGbMI2YgPUBPbSFZ7ixIh8dXbfj52ftdzu6tIPUZJqiGFNT0URN 6BXzgZsDu55CP7l+97XmCachK+1inCipX996NrmHmRCYTyt/ypucOjN54ql+bZFKyQ +Z6zFpOKqxGuA== To: "help-guix\\@gnu.org" From: Ekaitz Zarraga Subject: LuaJIT ffi issue with dynamic libraries Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.70.43.17; envelope-from=ekaitz@elenq.tech; helo=mail-4317.proton.ch X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Ekaitz Zarraga Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1637858840; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=INBSUybyXO3566DhO2XjTkOdme+bd1m0MFVWq9i++uI=; b=icXVxVOH0VWfBvpn7d5gQpPSRYfhgsfCxsJof5VBhR3NcMzg/L/8Gcr7LmVnR4RdgCOv2W 2vUEDf2hgXBq7cPPVwbhvgT1GS1XUBYRHZXIhr03e9fL21HE3i+OLEopBOpwgwTiJgPZL6 TA0vQkdpMscaelQ9SKg188KcGdbJZTwvbMsdsJF869Nnv5S1xAZ5C5VHatwxcTAg6q5F6y 7T4oce/BPhsqqhBFOI39jpefZ7JNC1vVbKgo6PtOYJVx0C1zf5iBuIgAo4rC/OcF3lUSYA OolGXvMU68dtwtMJYPRV139u/jxjW51RSWTdBreGgxnwkC8JZCXuXbj7SlRjfQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1637858840; a=rsa-sha256; cv=none; b=unB/m8byPUrILuLx0JeNGpCxVifkv3imL2BXP3A1CmFb7s2CcMCnr6wZNse7vo/4bJwr4C iOTcMVDjrVlAuoq8QECpOx5yhLPVPdL3dO6nH9r06LBa321omRNTy/BWOZuhuvka7857/2 Nz23rSX8AJPrT9pvMWsjJu3e6WSsiWNN/k2uw2lUXP+E0MbYESagDyddutAjW7GwiUzN0G wlT/ly0my9pARcoUVg10O0mkrz1wg5V2/jcGERgrkLOZ0k5nAardPEK4PZCkGUcxa0HDG6 hFQiUqOZrxX4aODB+b8xQIcIQ6/9ecTs2uNt/UP7KbepR9Gil5KfnmBDumuM+g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=elenq.tech header.s=protonmail header.b=d63EU8VJ; dmarc=pass (policy=none) header.from=elenq.tech; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.09 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=elenq.tech header.s=protonmail header.b=d63EU8VJ; dmarc=pass (policy=none) header.from=elenq.tech; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 5308B21FC2 X-Spam-Score: -4.09 X-Migadu-Scanner: scn0.migadu.com X-TUID: nJ2dj3hD/k4C Hi all, I've been having some issues with LuaJIT's FFI. More specifically with the `load` function, that loads a library in the "search path for dynamic libraries". The LuaJIT documentation says the following: > On POSIX systems, if the name contains no dot, the extension .so is appen= ded. > Also, the lib prefix is prepended if necessary. So ffi.load("z") looks fo= r > "libz.so" in the default shared library search path. In my case I'm unable to load a library, in this case `librec.so` simply by calling `ffi.load` even if both are part of the same environment. I read a little bit of the LuaJIT code and I found that it is doing a `dlopen` insid= e so I made a test: First I made a simple C file with a call to `dlopen` and straced its result= . It simply finds the library after proving some folders. I also noted that the environment automatically set the `LIBRARY_PATH` variable for me (even if t= he value of the variable does not match the patch where librec.so is being searched): --8<---------------cut here---------------start------------->8--- $ guix shell gcc-toolchain strace recutils $ cat < dl.c > #include > > int main(int argc, char *argv[]){ > void * i =3D dlopen("librec.so", RTLD_LAZY); > return (int)i; > } EOF $ gcc dl.c -ldl $ strace ./a.out 2>&1 | grep librec.so openat(AT_FDCWD, "/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/li= b/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/gnu/store/28bcmy08ki5krvr2g9hbm3bhys822fvn-gcc-11.2.0-li= b/lib/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or direct= ory) openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/t= ls/haswell/x86_64/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such fi= le or directory) openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/t= ls/haswell/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or d= irectory) openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/t= ls/x86_64/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or di= rectory) openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/t= ls/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or directory= ) openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/h= aswell/x86_64/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file o= r directory) openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/h= aswell/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or direc= tory) openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/x= 86_64/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or direct= ory) openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/l= ibrec.so", O_RDONLY|O_CLOEXEC) =3D 3 $ echo $LIBRARY_PATH /gnu/store/hiiyy2b8pw2cq9mw8iik9xi9lpclcv7b-profile/lib --8<---------------cut here---------------end--------------->8--- When I do a similar approach with LuaJIT I get the following: - The paths the `dlopen` call is proving are different to the previous case= and they are pretty weird too. - There's no `LIBRARY_PATH` set. - Even if I set `LIBRARY_PATH` myself it's ignored. - It works correctly when I set `LD_LIBRARY_PATH`, but should I? --8<---------------cut here---------------start------------->8--- $ guix shell luajit strace recutils $ cat < dl.lua > ffi =3D require "ffi" > ffi.load "rec" > EOF $ strace luajit dl.lua 2>&1 | grep librec.so openat(AT_FDCWD, "/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/li= b/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib= /lib/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or directo= ry) openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib= /lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell/x86_64/librec.= so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib= /lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell/librec.so", O_= RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib= /lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/x86_64/librec.so", O_R= DONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib= /lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/librec.so", O_RDONLY|O= _CLOEXEC) =3D -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib= /lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell/x86_64/librec.so",= O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib= /lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell/librec.so", O_RDON= LY|O_CLOEXEC) =3D -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib= /lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../x86_64/librec.so", O_RDONL= Y|O_CLOEXEC) =3D -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib= /lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../librec.so", O_RDONLY|O_CLO= EXEC) =3D -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/li= b/librec.so", O_RDONLY|O_CLOEXEC) =3D -1 ENOENT (No such file or directory) $ echo $LIBRARY_PATH $ export LIBRARY_PATH=3D$GUIX_ENVIRONMENT/lib $ strace luajit dl.lua 2>&1 | grep librec.so #### SAME AS BEFORE #### $ export LD_LIBRARY_PATH=3D$GUIX_ENVIRONMENT/lib $ echo $LD_LIBRARY_PATH /gnu/store/5v09kbdda7966x263cgnva2z15yrdbfb-profile/lib $ strace luajit dl.lua 2>&1 | grep librec.so openat(AT_FDCWD, "/gnu/store/5v09kbdda7966x263cgnva2z15yrdbfb-profile/lib/l= ibrec.so", O_RDONLY|O_CLOEXEC) =3D 3 --8<---------------cut here---------------end--------------->8--- Does any of this make sense? The easiest conclusion is that LuaJIT is not working out the box and we sho= uld fix it, but I'm not sure where to start because I'm assuming both examples = here come from the same concept: making a similar call to `dlopen`. There must b= e some context I'm missing here to solve this. Any ideas? ElenQ Technology Ethical Innovation