bug#39079: SBCL CFFI from Guix unable to find dynamic libraries

[Evan Straw <evan.straw99@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 10408 bytes --]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> What are "dependencies" here?
> Can you provide a full recipe of what you are trying to do, including
> your .sbclrc and the Lisp code snippets you've run to test this?

Absolutely. Apologies for the wall of text that is about to follow:

My system is Ubuntu 19.04, with Guix 09bad3c, as shown by `guix describe:`

--8<---------------cut here---------------start------------->8---
guix 09bad3c
  repository URL: https://git.savannah.gnu.org/git/guix.git
  branch: master
  commit: 09bad3cd42ca51974f3697f04ce841bf2fe6786c
--8<---------------cut here---------------end--------------->8---

I am using SBCL 2.0.0 and sbcl-cffi 0.19.0, both from Guix. My .sbclrc
file is as follows:

--8<---------------cut here---------------start------------->8---
;;; The following lines added by ql:add-to-init-file:
#-quicklisp
(let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp"
                                       (user-homedir-pathname))))
  (when (probe-file quicklisp-init)
    (load quicklisp-init)))

(when (ignore-errors (asdf:load-system "cffi"))
  ;; Warning: Trailing "/" matters!
  (dolist (guix-profile
           `(,(when (uiop:directory-exists-p "~/.guix-profile")
                (format nil "~a/.guix-profile/lib/" (uiop:getenv "HOME")))
              ,@(mapcar (lambda (d)
                          (format nil "~a~a/lib/"
                                  (namestring d)
                                  (first (last (pathname-directory d)))))
                        (uiop:subdirectories "~/.guix-extra-profiles/"))
	      "/usr/lib/x86_64-linux-gnu/"))
    (push guix-profile
          (symbol-value (find-symbol (string '*foreign-library-directories*)
                                     (find-package 'cffi))))))
--8<---------------cut here---------------end--------------->8---

I am attempting to load "/usr/lib/x86_64-linux-gnu/libcurl.so.4" since
the tutorial for CFFI starts by loading libcurl. This library depends on
the following other libraries, as reported by `ldd
/usr/lib/x86_64-linux-gnu/libcurl.so.4`:

--8<---------------cut here---------------start------------->8---
linux-vdso.so.1 (0x00007ffd5f393000)
libnghttp2.so.14 => /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007f941aae8000)
libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f941aac8000)
librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f941a8a8000)
libssh.so.4 => /usr/lib/x86_64-linux-gnu/libssh.so.4 (0x00007f941a828000)
libpsl.so.5 => /usr/lib/x86_64-linux-gnu/libpsl.so.5 (0x00007f941a810000)
libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f941a780000)
libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f941a4b0000)
libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f941a460000)
libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f941a408000)
liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f941a3f0000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f941a3d0000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f941a3a8000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f941a1b8000)
libunistring.so.2 => /home/evan/.guix-profile/lib/libunistring.so.2 (0x00007f941a030000)
libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f9419e88000)
libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007f9419e50000)
libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007f9419e18000)
libgmp.so.10 => /home/evan/.guix-profile/lib/libgmp.so.10 (0x00007f9419d80000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f9419d70000)
/lib64/ld-linux-x86-64.so.2 (0x00007f941abd8000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9419d68000)
libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f9419c88000)
libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f9419c50000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f9419c48000)
libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f9419c38000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f9419c18000)
libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f9419bf8000)
libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f9419bb0000)
libgcc_s.so.1 => /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/libgcc_s.so.1 (0x00007f9419b90000)
libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f9419a60000)
libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f9419848000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f9419840000)
libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007f9419830000)
libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007f94197a0000)
libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 (0x00007f94196f8000)
libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007f94196c0000)
libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 (0x00007f94196a0000)
libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f9419690000)
libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 (0x00007f9419660000)
libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007f9419648000)
libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 (0x00007f94195f8000)
libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f94194d8000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f9419498000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9419348000)
--8<---------------cut here---------------end--------------->8---

In order to load this library, I run the following Lisp code in the SBCL
REPL:

--8<---------------cut here---------------start------------->8---
CL-USER> (asdf:load-system :cffi)
T

;;; *foreign-library-directories* has the right directories in it
CL-USER> cffi:*foreign-library-directories*
("/usr/lib/x86_64-linux-gnu/"
 "/home/evan/.guix-extra-profiles/emacs/emacs/lib/"
 "/home/evan/.guix-profile/lib/")
 
CL-USER> (cffi:load-foreign-library "libcurl.so.4")
--8<---------------cut here---------------end--------------->8---

The last expression fails with the following error, which states that it
failed while trying to load libnghttp2.so.14, listed above as one of the
dependencies for libcurl:

--8<---------------cut here---------------start------------->8---
Unable to load foreign library (LIBCURL.SO.4-464).
  Error opening shared object "/usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0":
  libnghttp2.so.14: cannot open shared object file: No such file or directory.
   [Condition of type CFFI:LOAD-FOREIGN-LIBRARY-ERROR]

Restarts:
 0: [RETRY] Try loading the foreign library again.
 1: [USE-VALUE] Use another library instead.
 2: [RETRY] Retry SLIME REPL evaluation request.
 3: [*ABORT] Return to SLIME's top level.
 4: [ABORT] abort thread (#<THREAD "repl-thread" RUNNING {10046C9BC3}>)

Backtrace:
  0: (CFFI::FL-ERROR "Unable to load foreign library (~A).~%  ~A" #:LIBCURL.SO.4-464 "Error opening shared object \"/usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0\": ..)
  1: (CFFI::REPORT-SIMPLE-ERROR #:LIBCURL.SO.4-464 #<SIMPLE-ERROR "Error opening ~:[runtime~;shared object ~:*~S~]: ..)
  2: (CFFI::LOAD-FOREIGN-LIBRARY-PATH #:LIBCURL.SO.4-464 "libcurl.so.4" NIL)
  3: ((FLET CFFI::%DO-LOAD :IN CFFI::%DO-LOAD-FOREIGN-LIBRARY) #<CFFI:FOREIGN-LIBRARY LIBCURL.SO.4-464> #:LIBCURL.SO.4-464 "libcurl.so.4")
  4: (CFFI::%DO-LOAD-FOREIGN-LIBRARY "libcurl.so.4" NIL)
  5: (CFFI:LOAD-FOREIGN-LIBRARY "libcurl.so.4" :SEARCH-PATH NIL)
  6: (SB-INT:SIMPLE-EVAL-IN-LEXENV (CFFI:LOAD-FOREIGN-LIBRARY "libcurl.so.4") #<NULL-LEXENV>)
  7: (EVAL (CFFI:LOAD-FOREIGN-LIBRARY "libcurl.so.4"))
--8<---------------cut here---------------end--------------->8---



>> Ideally, I would just install the libraries I need from Guix, but I
>> specifically want to interface with a library from the system, in this
>> case PulseAudio (I've just been using libcurl as a test to make sure
>> it works).
>
> Does it work for the libraries installed with Guix?

To test this, I attempted to load `~/.guix-profile/lib/libguile-2.2.so`
since I don't have many libraries installed from Guix. This library
depends on the following libraries, as reported by `ldd`:

--8<---------------cut here---------------start------------->8---
linux-vdso.so.1 (0x00007ffe3a353000)
libgc.so.1 => /gnu/store/3xs3dnc28p9fi8in7hkfcdx20incrdvq-libgc-7.6.12/lib/libgc.so.1 (0x00007faeaee00000)
libpthread.so.0 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libpthread.so.0 (0x00007faeaedd8000)
libffi.so.6 => /gnu/store/ain96mrdwqd4s9shdd3s7m4syp5icdx5-libffi-3.2.1/lib/libffi.so.6 (0x00007faeaedc8000)
libunistring.so.2 => /gnu/store/nsikjxykcaqa0zjpfmkqd569bngbv5nl-libunistring-0.9.10/lib/libunistring.so.2 (0x00007faeaec40000)
libgmp.so.10 => /gnu/store/y7qk8raalgvdnxcglvxa320cfxrjk1x6-gmp-6.1.2/lib/libgmp.so.10 (0x00007faeaeba8000)
libltdl.so.7 => /gnu/store/zasz52va238yyaq68rjm8ljwl4ikij4p-libltdl-2.4.6/lib/libltdl.so.7 (0x00007faeaeb98000)
libdl.so.2 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libdl.so.2 (0x00007faeaeb90000)
libcrypt.so.1 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libcrypt.so.1 (0x00007faeaeb50000)
libm.so.6 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libm.so.6 (0x00007faeaea10000)
libgcc_s.so.1 => /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/libgcc_s.so.1 (0x00007faeae9f0000)
libc.so.6 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libc.so.6 (0x00007faeae830000)
/lib64/ld-linux-x86-64.so.2 (0x00007faeaefb0000)
--8<---------------cut here---------------end--------------->8---

Attempting the same steps in the SBCL REPL again succeeds with no
errors:

--8<---------------cut here---------------start------------->8---
CL-USER> (asdf:load-system :cffi)
T

CL-USER> (cffi:load-foreign-library "libguile-2.2.so")
#<CFFI:FOREIGN-LIBRARY LIBGUILE-2.2.SO-492 "libguile-2.2.so.1.4.1">
--8<---------------cut here---------------end--------------->8---



Hopefully this clarifies the problem somewhat.
-- Evan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]