all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* LuaJIT ffi issue with dynamic libraries
@ 2021-11-25 16:46 Ekaitz Zarraga
  2022-01-05 22:32 ` Ekaitz Zarraga
  0 siblings, 1 reply; 3+ messages in thread
From: Ekaitz Zarraga @ 2021-11-25 16:46 UTC (permalink / raw)
  To: help-guix\@gnu.org

Hi all,

I've been having some issues with LuaJIT's FFI. More specifically with the
`load` function, that loads a library in the "search path for dynamic
libraries".

The LuaJIT documentation says the following:

> On POSIX systems, if the name contains no dot, the extension .so is appended.
> Also, the lib prefix is prepended if necessary. So ffi.load("z") looks for
> "libz.so" in the default shared library search path.


In my case I'm unable to load a library, in this case `librec.so` simply by
calling `ffi.load` even if both are part of the same environment. I read a
little bit of the LuaJIT code and I found that it is doing a `dlopen` inside so
I made a test:

First I made a simple C file with a call to `dlopen` and straced its result. It
simply finds the library after proving some folders. I also noted that the
environment automatically set the `LIBRARY_PATH` variable for me (even if the
value of the variable does not match the patch where librec.so is being
searched):

--8<---------------cut here---------------start------------->8---
$ guix shell gcc-toolchain strace recutils
$ cat <<EOF > dl.c
> #include<dlfcn.h>
>
> int main(int argc, char *argv[]){
>     void * i = dlopen("librec.so", RTLD_LAZY);
>     return (int)i;
> }
EOF
$ gcc dl.c -ldl
$ strace ./a.out 2>&1 | grep librec.so
openat(AT_FDCWD, "/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/28bcmy08ki5krvr2g9hbm3bhys822fvn-gcc-11.2.0-lib/lib/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/haswell/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/haswell/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/haswell/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/haswell/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/librec.so", O_RDONLY|O_CLOEXEC) = 3
$ echo $LIBRARY_PATH
/gnu/store/hiiyy2b8pw2cq9mw8iik9xi9lpclcv7b-profile/lib
--8<---------------cut here---------------end--------------->8---

When I do a similar approach with LuaJIT I get the following:

- The paths the `dlopen` call is proving are different to the previous case and
  they are pretty weird too.
- There's no `LIBRARY_PATH` set.
- Even if I set `LIBRARY_PATH` myself it's ignored.
- It works correctly when I set `LD_LIBRARY_PATH`, but should I?


--8<---------------cut here---------------start------------->8---
$ guix shell luajit strace recutils
$ cat <<EOF > dl.lua
> ffi = require "ffi"
> ffi.load "rec"
> EOF
$ strace luajit dl.lua 2>&1  | grep librec.so
openat(AT_FDCWD, "/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
$ echo $LIBRARY_PATH

$ export LIBRARY_PATH=$GUIX_ENVIRONMENT/lib
$ strace luajit dl.lua 2>&1  | grep librec.so
#### SAME AS BEFORE ####
$ export LD_LIBRARY_PATH=$GUIX_ENVIRONMENT/lib
$ echo $LD_LIBRARY_PATH
/gnu/store/5v09kbdda7966x263cgnva2z15yrdbfb-profile/lib
$ strace luajit dl.lua 2>&1  | grep librec.so
openat(AT_FDCWD, "/gnu/store/5v09kbdda7966x263cgnva2z15yrdbfb-profile/lib/librec.so", O_RDONLY|O_CLOEXEC) = 3
--8<---------------cut here---------------end--------------->8---

Does any of this make sense?

The easiest conclusion is that LuaJIT is not working out the box and we should
fix it, but I'm not sure where to start because I'm assuming both examples here
come from the same concept: making a similar call to `dlopen`. There must be
some context I'm missing here to solve this.

Any ideas?


ElenQ Technology
Ethical Innovation


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: LuaJIT ffi issue with dynamic libraries
  2021-11-25 16:46 LuaJIT ffi issue with dynamic libraries Ekaitz Zarraga
@ 2022-01-05 22:32 ` Ekaitz Zarraga
  2022-01-05 23:33   ` zimoun
  0 siblings, 1 reply; 3+ messages in thread
From: Ekaitz Zarraga @ 2022-01-05 22:32 UTC (permalink / raw)
  To: Ekaitz Zarraga; +Cc: guix-devel\@gnu.org, help-guix\\@gnu.org

Hi,

I'm pushing this again just in case anyone has an answer.

Thanks!

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Thursday, November 25th, 2021 at 5:46 PM, Ekaitz Zarraga <ekaitz@elenq.tech> wrote:

> Hi all,
>
> I've been having some issues with LuaJIT's FFI. More specifically with the
> `load` function, that loads a library in the "search path for dynamic
> libraries".
>
> The LuaJIT documentation says the following:
>
> > On POSIX systems, if the name contains no dot, the extension .so is appended.
> > Also, the lib prefix is prepended if necessary. So ffi.load("z") looks for
> > "libz.so" in the default shared library search path.
>
>
> In my case I'm unable to load a library, in this case `librec.so` simply by
> calling `ffi.load` even if both are part of the same environment. I read a
> little bit of the LuaJIT code and I found that it is doing a `dlopen` inside so
> I made a test:
>
> First I made a simple C file with a call to `dlopen` and straced its result. It
> simply finds the library after proving some folders. I also noted that the
> environment automatically set the `LIBRARY_PATH` variable for me (even if the
> value of the variable does not match the patch where librec.so is being
> searched):
>
> --8<---------------cut here---------------start------------->8---
> $ guix shell gcc-toolchain strace recutils
> $ cat <<EOF > dl.c
> > #include<dlfcn.h>
> >
> > int main(int argc, char *argv[]){
> >     void * i = dlopen("librec.so", RTLD_LAZY);
> >     return (int)i;
> > }
> EOF
> $ gcc dl.c -ldl
> $ strace ./a.out 2>&1 | grep librec.so
> openat(AT_FDCWD, "/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/28bcmy08ki5krvr2g9hbm3bhys822fvn-gcc-11.2.0-lib/lib/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/haswell/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/haswell/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/tls/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/haswell/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/haswell/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/fz52krs37spvca6q0g0zmx6jmc1n388g-profile/lib/librec.so", O_RDONLY|O_CLOEXEC) = 3
> $ echo $LIBRARY_PATH
> /gnu/store/hiiyy2b8pw2cq9mw8iik9xi9lpclcv7b-profile/lib
> --8<---------------cut here---------------end--------------->8---
>
> When I do a similar approach with LuaJIT I get the following:
>
> - The paths the `dlopen` call is proving are different to the previous case and
>   they are pretty weird too.
> - There's no `LIBRARY_PATH` set.
> - Even if I set `LIBRARY_PATH` myself it's ignored.
> - It works correctly when I set `LD_LIBRARY_PATH`, but should I?
>
>
> --8<---------------cut here---------------start------------->8---
> $ guix shell luajit strace recutils
> $ cat <<EOF > dl.lua
> > ffi = require "ffi"
> > ffi.load "rec"
> > EOF
> $ strace luajit dl.lua 2>&1  | grep librec.so
> openat(AT_FDCWD, "/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/haswell/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../tls/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell/x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../haswell/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../x86_64/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/01b4w3m6mp55y531kyi1g8shh722kwqm-gcc-7.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/7.5.0/../../../librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> openat(AT_FDCWD, "/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/librec.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
> $ echo $LIBRARY_PATH
>
> $ export LIBRARY_PATH=$GUIX_ENVIRONMENT/lib
> $ strace luajit dl.lua 2>&1  | grep librec.so
> #### SAME AS BEFORE ####
> $ export LD_LIBRARY_PATH=$GUIX_ENVIRONMENT/lib
> $ echo $LD_LIBRARY_PATH
> /gnu/store/5v09kbdda7966x263cgnva2z15yrdbfb-profile/lib
> $ strace luajit dl.lua 2>&1  | grep librec.so
> openat(AT_FDCWD, "/gnu/store/5v09kbdda7966x263cgnva2z15yrdbfb-profile/lib/librec.so", O_RDONLY|O_CLOEXEC) = 3
> --8<---------------cut here---------------end--------------->8---
>
> Does any of this make sense?
>
> The easiest conclusion is that LuaJIT is not working out the box and we should
> fix it, but I'm not sure where to start because I'm assuming both examples here
> come from the same concept: making a similar call to `dlopen`. There must be
> some context I'm missing here to solve this.
>
> Any ideas?
>
>
> ElenQ Technology
> Ethical Innovation



^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: LuaJIT ffi issue with dynamic libraries
  2022-01-05 22:32 ` Ekaitz Zarraga
@ 2022-01-05 23:33   ` zimoun
  0 siblings, 0 replies; 3+ messages in thread
From: zimoun @ 2022-01-05 23:33 UTC (permalink / raw)
  To: Ekaitz Zarraga, Ekaitz Zarraga; +Cc: guix-devel\@gnu.org, help-guix\\@gnu.org

Hi,

On Wed, 05 Jan 2022 at 22:32, Ekaitz Zarraga <ekaitz@elenq.tech> wrote:

> I'm pushing this again just in case anyone has an answer.

Well, I do not have the answer and do not know either if it related but
there is this really old bug:

    <http://issues.guix.gnu.org/issue/25425>


Cheers,
simon



^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2022-01-05 23:39 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-25 16:46 LuaJIT ffi issue with dynamic libraries Ekaitz Zarraga
2022-01-05 22:32 ` Ekaitz Zarraga
2022-01-05 23:33   ` zimoun

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.