Re: SWIG (and particularly Python) bindings

[Ben Gamari <bgamari@gmail.com>]

Excerpts from Adrian Perez de Castro's message of Wed Dec 30 05:52:23 -0500 2009:
> On Tue, 29 Dec 2009 04:16:43 -0500, Ben wrote:
> 
> > Regardless, I thought it might be nice to have access to the notmuch
> > backend from a language other than C (preferably my high-level language
> > of choice, python) [...]
> 
> Funny, I was just doing the same: a Python binding. Haha, so now we have
> two just-backed Python bindings. What should we do?

Heh, it's a small world, eh?

> 
> > [...] To this end, I took a few hours today acquainting
> > myself with SWIG and produced these bindings for Python. Unfortunately,
> > it doesn't appear that SWIG has particularly good support for
> > object-oriented C [...]
> 
> I already used SWIG sometimes in the past (and did not like it a lot), so
> my binding is using Cython [*] (which is exactly like Pyrex plus some extra
> features), so the binding is partly manual.
> 
I liked that SWIG would give us coverage for a multitude of languages
with little work. Unfortunately, getting an object oriented interface
requires wrapping the functions exposed by SWIG. Nevertheless, I think
being able to generically hide the ugliness of the binding glue itself
is quite nice. Moveover, there's effectively no SWIG interface code to
maintain.  It's literally just a [#%]include "notmuch.h". The rest is
all the Python wrapper. This seems like a nice binding model, short of
SWIG doing everything (the developers at some point might add support
for proper wrapping of object-oriented C code).

> > While the bindings are currently in the form of a patch to notmuch
> > (creating a top-level swig directory in the source tree), they could
> > certainly be moved out-of-tree if the powers that be don't feel it
> > appropriate to include them. [...]
> 
> Same here, see attached patch. It is currently unfinished, and I was just
> about to add support for iterating notmuch_threads_t and other similar
> structures. I can also publish a Git repo with the entire branch, just
> drop me a line if you want me to do that.
> 
In my approach, I just used the iterator protocol.

> > [...] Unfortunately, the build system is currently almost entirely
> > independent from the notmuch build system. If  these are to be
> > included in-tree, I would be curious to hear people have
> > to say about how we might integrate it into the sup build system.
>                                                   ^^^
> (Mmmh, I suppose you mean "notmuch build system" there :P)

Heh, yep.

> 
> Mine is a little more cooked, as I have added a distutils "setup.py"
> script. The bad news is that Python modules need to be compiled as
> relocatable object files (-fPIC to teh rescue!), and the linker will
> refuse to link the generated code with "notmuch.a" -- so I am instructing
> distutils to compile *all* sources again. Not nice.
> 
In my patch I simple I modified the LDFLAGS to include -fPIC. Not sure if
that's an acceptable option or not.

> BTW, I think that if more bindings start to appear, Notmuch might be built
> as a shared library, to avoid duplicating it everywhere. One option may be
> using *just* libtool but not the rest of auto-foo tools (for the record:
> I agree with Carl that they are slow and wicked).
> 
I think you're probably right. Libtool is probably the best option here
(Despite the fact that I, too, have a bitter hatred of autotools).

Cheers,
- Ben