Re: Binding generator, for Guile, to C libraries

[Danny Milosavljevic <dannym@scratchpost.org>]

Hi Ludo,

On Wed, 05 Jul 2017 23:53:36 +0200
ludo@gnu.org (Ludovic Courtès) wrote:

> Danny Milosavljevic <dannym@scratchpost.org> skribis:
> 
> > Okay, I've cleaned up what I have and put it on github, under:
> >
> > https://github.com/daym/guile-gcc-unit
> >
> > What this does is it allows you to read gcc output files which specify all the prototypes - in our case in order to extract the prototypes and generate Guile bindings (of parted, or whatever).
> >
> > The next steps:
> > - Provide the actual binding generator (I started a version of it - I'll commit a cleaned-up version in a few days I guess).

> > - Get it to work with non-toy examples (see file "tests/huge/input" for the parted non-toy example - the goal is to read this file, find all the record decls and all the function decls in it

This part is working now, also for parted.

> and generate the Guile pendants of them).  If you happen to know LALR parsing theory, I have a question :)
> > - Find out whether that can be integrated as a Guile "language".  Just being able to do something like ',load "foo.h"' and have it register all the functions from it would be way cool.  On the other hand, on non-Guix systems the header files aren't usually installed when the library is installed.  So maybe an offline version would be good as well.  
> 
> Matt Wette, the author of nyacc, was apparently in the process of
> writing a similar tool using nyacc’s C99 parser.  Might be worth looking
> at.

Nice!  I've actually thought about using nyacc and mes for that - but since we use gcc to compile everything, we can also use gcc to find out what it compiled - it's more probable that that actually matches up.  What I like is that gcc actually dumps the declarations even when you *don't* call the function.  So you can actually create a mostly-empty source file with #include in it and gcc will give you everything that is in scope.

But it's definitely worth taking a look at Matt Wette's actual Guile dynamic-* generator, maybe that can be shared between the projects.

> Besides and FWIW, I’m skeptical of binding generators.  Things like
> SWIG, for instance, generate interfaces that are often not quite what
> you’d like, so you end up writing a layer on top of the generated glue
> anyway, not to mention annotation in headers. 

Yeah, probably the bindings will suck a bit.  But one can always write a wrapper and not export all the original functions from there.  Rust for example has a "...-sys" convention where the package "foo-sys" contains the easily-generated bindings and the package "foo" contains the actually simple high-level bindings.

What I don't want is to manually maintain all the bindings.  It's a computer, it should automate it :P

Writing "override" declarations (or maybe even just literally overwriting the definition in Scheme) once every few months I'm fine with.  Adapting the bindings every time upstream adds a function?  No.

If I plan to use something like the "...-sys" convention in Guile, can I somehow re-export all the same names as the "-sys" package did, in a non-sys package?

For example:

(define-module (foo-sys)
  #:export (a b c))

...
------
(define-module (foo)
  #:re-export (same-names-as-in foo-sys, but use newer versions from below if applicable))

(define (b ...)
  fix it up)
-----
Hmm, I guess I can use (include-from-path "foo-sys") in the second module before I overwrite anything.

> (Bindings generated from
> high-level descriptions like GIR and XCB are a different story; those
> seem to work quite well.)

Yeah, I like glib's approach with the def files which specify also the object lifetimes etc.