Re: DSO-style FFI

[Ted Zlatanov <tzz@lifelogs.com>]

On Tue, 08 Oct 2013 14:43:39 -0600 Tom Tromey <tromey@redhat.com> wrote: 

Ted> The foreign function has to take Emacs Lisp_Objects (maybe just strings
Ted> and numbers) and package its return data in a Lisp_Object.  So how do we
Ted> handle that glue with libffi or anything else without promising some
Ted> minimal internal Emacs ABI?

Tom> The idea is to have the FFI code be part of Emacs.  So when Emacs
Tom> changes, it does too.  What doesn't change is the API presented to
Tom> elisp.

Tom> This is how FFI works in other languages -- Common Lisp, Python,
Tom> probably Guile (I didn't check), Java (it's part of reflection there),
Tom> etc.

It would really help if we discussed a small example, and forgive my
ignorance here.

Say I have a transforming function:

int gnutls_xform(char* datain, char* dataout);

I want to use a Lisp_Object, a string, for `datain'; and get back another
Lisp_Object, again a string, in `dataout'.

So, something like this would be done on a given function_pointer to
`gnutls_xform', with some given Lisp_Object `input_data':

       ffi_cif cif;
       ffi_type *args[2];
       void *values[2];
       char *in = SDATA (input_data);
       char *out = xzmalloc(100);
       int rc;
     
       args[0] = &ffi_type_pointer;
       values[0] = &in;

       args[1] = &ffi_type_pointer;
       values[1] = &out;
          
       /* Initialize the cif */
       if (ffi_prep_cif(&cif, FFI_DEFAULT_ABI, 1,
     		       &ffi_type_uint, args) == FFI_OK)
         {
           ffi_call(&cif, function_pointer, &rc, values);
           /* rc now holds the result of the call */
         }
       
       Lisp_Object ret = Qnil;
       // if rc shows success...
       if (rc != ERROR)
       {
          ret = make_string (out);
       }

       xfree(out);

This would be more generic of course, but keeping it simple for now
really helps.

As far as the dynamic function loading to give us function_pointer, I am
guessing it's done with libltdl, by calling

  module = lt_dlopen("emacs_gnutls_module");

and then

  lt_dlsym(module, "gnutls_xform");

as shown in
http://www.gnu.org/software/libtool/manual/html_node/Libltdl-interface.html

Is that more or less where we should be going?  Is this today's state of
the art?  Based on the other languages you listed, it seems so.

My immediate concerns are security (these modules can do almost
anything); concurrency between modules and Emacs; and between modules
loaded by concurrent Emacs threads in a hypothetical future where we
have such threads. But in terms of practical integration they seem
workable.

Thanks for your patience
Ted