Re: ccall mechanism in Julia

[Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>]

Andreas Enge writes:

> On Fri, Mar 06, 2015 at 04:48:59PM +0100, Ricardo Wurmus wrote:
>> Julia's bindings to these libraries use the ccall mechanism; ccall
>> builds a map from library names to paths by parsing the output of
>> "ldconfig -p" at runtime.  I worked around this problem by patching the
>> sources to include a static map of library names to store paths.  Then I
>> noticed, however, that my patch effectively cripples the FFI.  It is no
>> longer possible to call a function that is exported by a shared library
>> unless it happens to be in the static map that is created at build time.
>
> This looks like a good approach for guix, where we try to not use random
> libraries lying around in the user profile. Setting the rpath with our
> ld-wrapper behaves also like a static map. Otherwise, updating some other
> library in the user profile would modify the behaviour of julia, which
> would contradict our functional approach.

The problem I see is that a user won't be able to make foreign function
calls to installed libraries at runtime in an interactive Julia
session.

With Guile I'm not limited in this manner because I can arbitrarily
extend the GUILE_LOAD_PATH and use stuff that's not installed to the
store.  With C I can extend the paths as well and compile stuff that are
not in the store, in an exploratory fashion.

If we went with a static map of only the most rudamentary compile-time
dependencies users won't be able to use the flexible FFI in an ad-hoc
manner, severely limiting expression.

I would still want to hard-code the paths of input librarie providing
core features so that the behaviour of Julia would not change dependent
on what libraries are installed, but I think that we should avoid
disabling the FFI for anything but the core libraries.  I would like to
extend the ccall mechanism such that one could make foreign function
calls to installed libraries (or libraries available in some list of
paths).

From what I can tell Julia's FFI requires no boilerplate.  A user just
needs the library name (or rather the key in the sonameMap resolving to
a library path) and the function name to access.  I would very much like
to keep this feature.

How are Guile's or Python's FFI handled?

~~ Ricardo