SWIG (and particularly Python) bindings

SWIG (and particularly Python) bindings

[Ben Gamari]

Hey all,

I've been looking at switching away from sup recently to something with
a slightly little less everything-and-the-kitchen-sink philosophy.
Notmuch looks excellent, although it appears that its current front-end
for my editor of choice (vim) is a little lacking in some ways
(operations involving a call to notmuch invariably lock up vi for the
duration of the operation).

Regardless, I thought it might be nice to have access to the notmuch
backend from a language other than C (preferably my high-level language
of choice, python). To this end, I took a few hours today acquainting
myself with SWIG and produced these bindings for Python. Unfortunately,
it doesn't appear that SWIG has particularly good support for
object-oriented C, so the approach I took was using SWIG to generate the
low-level glue, on top of which I wrote a properly structured set of
Python bindings. While I theoretically have full interface coverage,
I unfortunately haven't had a chance to use these for anything useful,
so who knows whether much of it actually works. Regardless, I thought
folks might be interested.

While the bindings are currently in the form of a patch to notmuch
(creating a top-level swig directory in the source tree), they could
certainly be moved out-of-tree if the powers that be don't feel it
appropriate to include them. Unfortunately, the build system is
currently almost entirely independent from the notmuch build system. If
these are to be included in-tree, I would be curious to hear people have
to say about how we might integrate it into the sup build system.

Hope this helps someone. I'll hopefully have a chance to work with the
bindings soon and will send a new set of patches once I think I've found
all of the obvious issues.

- Ben


---
 swig/Makefile        |   18 ++++
 swig/notmuch.py      |  222 ++++++++++++++++++++++++++++++++++++++++++++++++++
 swig/notmuch_funcs.i |    9 ++
 3 files changed, 249 insertions(+), 0 deletions(-)
 create mode 100644 swig/Makefile
 create mode 100644 swig/notmuch.py
 create mode 100644 swig/notmuch_funcs.i

diff --git a/swig/Makefile b/swig/Makefile
new file mode 100644
index 0000000..c01c427
--- /dev/null
+++ b/swig/Makefile
@@ -0,0 +1,18 @@
+include ../Makefile.config
+
+INCLUDES=-I../lib -I/usr/include/python2.6
+CFLAGS=${INCLUDES}
+
+all : python
+
+python : _notmuch_funcs.so notmuch.py
+
+_notmuch_funcs.so : notmuch_funcs_wrap.o
+	g++ -shared ${XAPIAN_LDFLAGS} ${GMIME_LDFLAGS} ${TALLOC_LDFLAGS} -o $@ $< ../lib/notmuch.a
+
+%_wrap.o : %_wrap.c
+	gcc ${CFLAGS} -fPIC -c -o $@ $<
+
+%_wrap.c %.py : %.i 
+	swig -python ${INCLUDES} $<
+
diff --git a/swig/notmuch.py b/swig/notmuch.py
new file mode 100644
index 0000000..95b81ad
--- /dev/null
+++ b/swig/notmuch.py
@@ -0,0 +1,222 @@
+import notmuch_funcs as nm
+from datetime import datetime
+
+class Exception(Exception):
+        def get_message():
+                errors = {
+                        nm.NOTMUCH_STATUS_OUT_OF_MEMORY: 'out of memory',
+                        nm.NOTMUCH_STATUS_READONLY_DATABASE: 'database opened as read-only',
+                        nm.NOTMUCH_STATUS_XAPIAN_EXCEPTION: 'xapian error',
+                        nm.NOTMUCH_STATUS_FILE_ERROR: 'file error',
+                        nm.NOTMUCH_STATUS_FILE_NOT_EMAIL: 'file not email message',
+                        nm.NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID: 'duplicate message id',
+                        nm.NOTMUCH_STATUS_NULL_POINTER: 'null pointer',
+                        nm.NOTMUCH_STATUS_TAG_TOO_LONG: 'tag name too long',
+                        nm.NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW: 'unbalanced freeze/thaw',
+                }
+                return errors.get(self.status, 'unknown error')
+
+        def __init__(self, status):
+                self.status = status
+
+def _handle_status(status):
+        if (status != nm.NOTMUCH_STATUS_SUCCESS):
+                raise Exception(status)
+
+class Database(object):
+        MODE_READ_ONLY = nm.NOTMUCH_DATABASE_MODE_READ_ONLY
+        MODE_READ_WRITE = nm.NOTMUCH_DATABASE_MODE_READ_WRITE
+
+        def __init__(self, db):
+                if not db:
+                        raise "Failed to open database"
+                self.db = db
+
+        @staticmethod
+        def create(path):
+                return Database(nm.notmuch_database_create(path))
+
+        @staticmethod
+        def open(path, mode):
+                return Database(nm.notmuch_database_open(path, mode))
+
+        def close(self):
+                nm.notmuch_database_close(self.db)
+
+        def get_path(self):
+                return nm.notmuch_database_get_path(self.db)
+
+        def set_timestamp(self, key, timestamp):
+                _handle_status(nm.notmuch_database_set_timestamp(self.db, key, timestamp))
+
+        def get_timestamp(self, key):
+                return datetime.fromtimestamp(nm.notmuch_database_get_timestamp(self.db, key))
+
+        def add_message(self, filename, message):
+                _handle_status(nm.notmuch_database_add_message(self.db, filename, message.message))
+
+        def find_message(self, message_id):
+                return Message(nm.notmuch_database_find_message(self.db, message_id))
+
+        def get_all_tags(self):
+                return Tags(nm.notmuch_database_get_all_tags(self.db))
+
+        def __destroy__(self):
+                self.close()
+        
+class Query(object):
+        def __init__(self, db, query_string):
+                self.query = nm.notmuch_query_create(db, query_string)
+                if not self.query: # This might not work
+                        raise "Failed to create query"
+
+        def set_sort(self, sort):
+                nm.notmuch_query_set_sort(self.query, sort)
+                
+        def search_threads(self):
+                return Threads(nm.notmuch_query_search_threads(self.query))
+
+        def search_messages(self):
+                return Messages(nm.notmuch_query_search_messages(self.query))
+
+        def count_message(self):
+                return nm.notmuch_query_count_messages(self.query)
+
+        def __destroy__(self):
+                nm.notmuch_query_destroy(self.query)
+
+class Tags(object):
+        def __init__(self, tags):
+                self.tags = tags
+
+        def __iter__(self):
+                return self
+
+        def next(self):
+                if not nm.notmuch_tags_has_more(self.tags):
+                        raise StopIteration
+                else:
+                        h = nm.notmuch_tags_get(self.tags)
+                        nm.notmuch_tags_advance(self.tags)
+                        return h
+
+        def __destroy__(self):
+                nm.notmuch_messages_destroy(self.tags)
+
+class Thread(object):
+        def __init__(self, thread):
+                self.thread = thread
+
+        def get_thread_id(self):
+                return nm.notmuch_thread_get_thread_id(self.thread)
+
+        def get_total_messages(self):
+                return nm.notmuch_thread_total_messages(self.thread)
+        
+        def get_toplevel_messages(self):
+                return Messages(nm.notmuch_thread_get_toplevel_messages(self.thread))
+
+        def get_matched_messages(self):
+                return nm.notmuch_thread_get_matched_messages(self.thread)
+
+        def get_authors(self):
+                return nm.notmuch_thread_get_authors(self.thread)
+
+        def get_subject(self):
+                return nm.notmuch_thread_get_subject(self.thread)
+
+        def get_oldest_date(self):
+                return nm.notmuch_thread_get_oldest_date(self.thread)
+
+        def get_newest_date(self):
+                return nm.notmuch_thread_get_newest_date(self.thread)
+
+        def get_tags(self):
+                return Tags(nm.notmuch_thread_get_tags(self.thread))
+
+        def __destroy__(self):
+                nm.notmuch_thread_destroy(self.thread)
+
+class Threads(object):
+        def __init__(self, threads):
+                self.threads = threads
+
+        def __iter__(self):
+                return self
+
+        def next(self):
+                if not nm.notmuch_threads_has_more(self.threads):
+                        raise StopIteration
+                else:
+                        h = Thread(nm.notmuch_threads_get(self.threads))
+                        nm.notmuch_threads_advance(self.threads)
+                        return h
+
+        def __destroy__(self):
+                nm.notmuch_threads_destroy(self.threads)
+
+class Messages(object):
+        def __init__(self, messages):
+                self.messages = messages
+
+        def __iter__(self):
+                return self
+
+        def next(self):
+                if not nm.notmuch_messages_has_more(self.messages):
+                        raise StopIteration
+                else:
+                        h = Message(nm.notmuch_messages_get(self.messages))
+                        nm.notmuch_messages_advance(self.messages)
+                        return h
+
+        def __destroy__(self):
+                nm.notmuch_messages_destroy(self.messages)
+
+        def collect_tags(self):
+                return Tags(nm.notmuch_messages_collect_tags(self.messages))
+
+class Message(object):
+        def __init__(self, message):
+                self.message = message
+
+        def get_replies(self):
+                return Messages(nm.notmuch_message_get_replies(self.message))
+
+        def get_filename(self):
+                return nm.notmuch_message_get_filename(self.message)
+
+        def get_flag(self, flag):
+                return bool(nm.notmuch_message_get_flag(self.message, flag))
+
+        def set_flag(self, flag, value):
+                return nm.notmuch_message_set_flag(self.message, flag, value)
+
+        def get_date(self):
+                return datetime.fromtimestamp(nm.notmuch_message_get_date(self.message))
+
+        def get_header(self, header):
+                return nm.notmuch_message_get_header(self.message, header)
+
+        def get_tags(self):
+                return Tags(nm.notmuch_message_get_tags(self.message))
+
+        def add_tag(self, tag):
+                _handle_status(nm.notmuch_message_add_tag(self.message, tag))
+
+        def remove_tag(self, tag):
+                _handle_status(nm.notmuch_message_remove_tag(self.message, tag))
+
+        def remove_all_tags(self):
+                nm.notmuch_message_remove_all_tags(self.message)
+
+        def freeze(self):
+                nm.notmuch_message_freeze(self.message)
+
+        def thaw(self):
+                nm.notmuch_message_thaw(self.message)
+
+        def __destroy__(self):
+                nm.notmuch_message_destroy(self.message)
+
+
diff --git a/swig/notmuch_funcs.i b/swig/notmuch_funcs.i
new file mode 100644
index 0000000..cc1826e
--- /dev/null
+++ b/swig/notmuch_funcs.i
@@ -0,0 +1,9 @@
+%module notmuch_funcs
+
+%{
+#define SWIG_FILE_WITH_INIT
+#include "notmuch.h"
+%}
+
+%include "notmuch.h"
+
-- 
1.6.3.3

Re: SWIG (and particularly Python) bindings

[Adrian Perez de Castro]

[-- Attachment #1.1: Type: text/plain, Size: 2425 bytes --]

On Tue, 29 Dec 2009 04:16:43 -0500, Ben wrote:

> Regardless, I thought it might be nice to have access to the notmuch
> backend from a language other than C (preferably my high-level language
> of choice, python) [...]

Funny, I was just doing the same: a Python binding. Haha, so now we have
two just-backed Python bindings. What should we do?

> [...] To this end, I took a few hours today acquainting
> myself with SWIG and produced these bindings for Python. Unfortunately,
> it doesn't appear that SWIG has particularly good support for
> object-oriented C [...]

I already used SWIG sometimes in the past (and did not like it a lot), so
my binding is using Cython [*] (which is exactly like Pyrex plus some extra
features), so the binding is partly manual.

> While the bindings are currently in the form of a patch to notmuch
> (creating a top-level swig directory in the source tree), they could
> certainly be moved out-of-tree if the powers that be don't feel it
> appropriate to include them. [...]

Same here, see attached patch. It is currently unfinished, and I was just
about to add support for iterating notmuch_threads_t and other similar
structures. I can also publish a Git repo with the entire branch, just
drop me a line if you want me to do that.

> [...] Unfortunately, the build system is currently almost entirely
> independent from the notmuch build system. If  these are to be
> included in-tree, I would be curious to hear people have
> to say about how we might integrate it into the sup build system.
                                                  ^^^
(Mmmh, I suppose you mean "notmuch build system" there :P)

Mine is a little more cooked, as I have added a distutils "setup.py"
script. The bad news is that Python modules need to be compiled as
relocatable object files (-fPIC to teh rescue!), and the linker will
refuse to link the generated code with "notmuch.a" -- so I am instructing
distutils to compile *all* sources again. Not nice.

BTW, I think that if more bindings start to appear, Notmuch might be built
as a shared library, to avoid duplicating it everywhere. One option may be
using *just* libtool but not the rest of auto-foo tools (for the record:
I agree with Carl that they are slow and wicked).

Regards,


[*] http://www.cython.org/
-- 
Adrian Perez de Castro <aperez@igalia.com>
Igalia - Free Software Engineering

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: notmuch-python-wip.patch --]
[-- Type: text/x-patch, Size: 17456 bytes --]

 Makefile              |    1 +
 python/.gitignore     |    2 +
 python/Makefile       |    6 +
 python/Makefile.local |   15 ++
 python/notmuch.pyx    |  397 +++++++++++++++++++++++++++++++++++++++++++++++++
 python/pyutil.h       |   16 ++
 python/setup.py       |   89 +++++++++++
 7 files changed, 526 insertions(+), 0 deletions(-)

diff --git a/Makefile b/Makefile
index 021fdb8..081d670 100644
--- a/Makefile
+++ b/Makefile
@@ -37,6 +37,7 @@ include Makefile.config
 
 include lib/Makefile.local
 include compat/Makefile.local
+include python/Makefile.local
 include Makefile.local
 
 # The user has not set any verbosity, default to quiet mode and inform the
diff --git a/python/.gitignore b/python/.gitignore
new file mode 100644
index 0000000..7f0efa8
--- /dev/null
+++ b/python/.gitignore
@@ -0,0 +1,2 @@
+notmuch.c
+build/
diff --git a/python/Makefile b/python/Makefile
new file mode 100644
index 0000000..e1e5c43
--- /dev/null
+++ b/python/Makefile
@@ -0,0 +1,6 @@
+
+all: python
+
+%:
+	make -C .. $@
+
diff --git a/python/Makefile.local b/python/Makefile.local
new file mode 100644
index 0000000..140a701
--- /dev/null
+++ b/python/Makefile.local
@@ -0,0 +1,15 @@
+dir=python
+
+python: $(dir)/build/.stamp
+	(cd $(dir) && python setup.py build)
+	touch $@
+
+$(dir)/build/.stamp: lib/notmuch.a
+
+clean: clean-python
+
+clean-python:
+	$(RM) -r $(dir)/build
+
+.PHONY: clean-python python
+
diff --git a/python/notmuch.pyx b/python/notmuch.pyx
new file mode 100644
index 0000000..f38b719
--- /dev/null
+++ b/python/notmuch.pyx
@@ -0,0 +1,397 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+# vim: fenc=utf-8 ft=pyrex
+#
+# Copyright © 2009 Adrian Perez <aperez@igalia.com>
+#
+# Distributed under terms of the GPLv3 license.
+#
+
+cdef extern from "talloc.h":
+	void* talloc_init(char *fmt, ...)
+	int   talloc_free(void *ctx)
+
+
+cdef extern from "pyutil.h":
+	#
+	# Utility macros
+	#
+	char** pyutil_alloc_strv(void *ctx, unsigned nitems)
+
+
+cdef extern from "notmuch.h":
+	#
+	# Return status handling
+	#
+	ctypedef enum notmuch_status_t:
+		NOTMUCH_STATUS_SUCCESS
+		NOTMUCH_STATUS_OUT_OF_MEMORY
+		NOTMUCH_STATUS_READONLY_DATABASE
+		NOTMUCH_STATUS_XAPIAN_EXCEPTION
+		NOTMUCH_STATUS_FILE_ERROR
+		NOTMUCH_STATUS_FILE_NOT_EMAIL
+		NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID
+		NOTMUCH_STATUS_NULL_POINTER
+		NOTMUCH_STATUS_TAG_TOO_LONG
+		NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW
+
+	char* notmuch_status_to_string(notmuch_status_t status)
+
+	#
+	# notmuch_database_* -> notmuch.Database
+	#
+	ctypedef enum notmuch_database_mode_t:
+		NOTMUCH_DATABASE_MODE_READ_ONLY
+		NOTMUCH_DATABASE_MODE_READ_WRITE
+
+	ctypedef enum notmuch_sort_t:
+		NOTMUCH_SORT_OLDEST_FIRST
+		NOTMUCH_SORT_NEWEST_FIRST
+		NOTMUCH_SORT_MESSAGE_ID
+
+	ctypedef struct notmuch_database_t
+	ctypedef struct notmuch_messages_t
+	ctypedef struct notmuch_message_t
+	ctypedef struct notmuch_threads_t
+	ctypedef struct notmuch_thread_t
+	ctypedef struct notmuch_query_t
+	ctypedef struct notmuch_tags_t
+	ctypedef int time_t
+
+	int notmuch_threads_has_more(notmuch_threads_t *threads)
+	void notmuch_threads_advance(notmuch_threads_t *threads)
+	void notmuch_threads_destroy(notmuch_threads_t *threads)
+	notmuch_thread_t* notmuch_threads_get(notmuch_threads_t *threads)
+
+	void   notmuch_thread_destroy(notmuch_thread_t *thread)
+	char*  notmuch_thread_get_authors(notmuch_thread_t *thread)
+	char*  notmuch_thread_get_subject(notmuch_thread_t *thread)
+	char*  notmuch_thread_get_thread_id(notmuch_thread_t *thread)
+	time_t notmuch_thread_get_oldest_date(notmuch_thread_t *thread)
+	time_t notmuch_thread_get_newest_date(notmuch_thread_t *thread)
+	int    notmuch_thread_get_total_messages(notmuch_thread_t *thread)
+	int    notmuch_thread_get_matched_messages(notmuch_thread_t *thread)
+	notmuch_tags_t*     notmuch_thread_get_tags(notmuch_thread_t *thread)
+	notmuch_messages_t* notmuch_thread_get_toplevel_messages(notmuch_thread_t *thread)
+
+	notmuch_database_t* notmuch_database_create(char *path)
+	notmuch_database_t* notmuch_database_open(char *path, notmuch_database_mode_t mode)
+	char* notmuch_database_get_path(notmuch_database_t *db)
+	void  notmuch_database_close(notmuch_database_t *db)
+
+	time_t notmuch_database_get_timestamp(notmuch_database_t *db, char *key)
+	notmuch_status_t notmuch_database_set_timestamp(
+			notmuch_database_t *db, char *key, time_t timestamp)
+
+	notmuch_status_t notmuch_database_add_message(
+			notmuch_database_t *db, char *filename, notmuch_message_t **mesage)
+
+	notmuch_message_t* notmuch_database_find_message(
+			notmuch_database_t *db, char *message_id)
+
+	notmuch_tags_t* notmuch_database_get_all_tags(notmuch_database_t *db)
+
+	notmuch_query_t* notmuch_query_create(notmuch_database_t *db, char *qstring)
+
+	void notmuch_query_destroy(notmuch_query_t *query)
+	void notmuch_query_set_sort(notmuch_query_t *query, notmuch_sort_t sort)
+	unsigned notmuch_query_count_messages(notmuch_query_t *query)
+	notmuch_threads_t*  notmuch_query_search_threads(notmuch_query_t *query)
+	notmuch_messages_t* notmuch_query_search_messages(notmuch_query_t *query)
+
+	char* notmuch_message_get_message_id(notmuch_message_t *msg)
+	char* notmuch_message_get_thread_id(notmuch_message_t *msg)
+	char* notmuch_message_get_filename(notmuch_message_t *msg)
+	char* notmuch_message_get_header(notmuch_message_t *msg, char *name)
+	notmuch_status_t notmuch_message_add_tag(notmuch_message_t *msg, char *tag)
+	notmuch_status_t notmuch_message_remove_tag(notmuch_message_t *msg, char *tag)
+	void notmuch_message_remove_all_tags(notmuch_message_t *msg)
+	void notmuch_message_destroy(notmuch_message_t *msg)
+	void notmuch_message_freeze(notmuch_message_t *msg)
+	void notmuch_message_thaw(notmuch_message_t *msg)
+
+
+cdef extern from "notmuch-client.h":
+	#
+	# notmuch_config_* -> notmuch.Config
+	#
+	ctypedef struct notmuch_config_t
+
+	notmuch_config_t* notmuch_config_open(void *ctx, char *filename, int *is_new_ret)
+	void   notmuch_config_close(notmuch_config_t *cfg)
+	int    notmuch_config_save(notmuch_config_t *cfg)
+	char*  notmuch_config_get_database_path(notmuch_config_t *cfg)
+	void   notmuch_config_set_database_path(notmuch_config_t *cfg, char *path)
+	char*  notmuch_config_get_user_name(notmuch_config_t *cfg)
+	void   notmuch_config_set_user_name(notmuch_config_t *cfg, char *name)
+	char*  notmuch_config_get_user_primary_email(notmuch_config_t *cfg)
+	void   notmuch_config_set_user_primary_email(notmuch_config_t *cfg, char *email)
+	char** notmuch_config_get_user_other_email(notmuch_config_t *cfg, size_t *length)
+	void   notmuch_config_set_user_other_email(notmuch_config_t *cfg, char **other_email, size_t length)
+
+	#
+	# Miscellaneous
+	#
+	int debugger_is_active()
+
+
+#
+# Import needed Python built-in modules
+#
+from datetime import datetime, date
+
+#
+# Miscellaneous functions and information
+#
+debugger_active = bool(debugger_is_active())
+
+
+#
+# notmuch_database_* -> notmuch.Database
+#
+
+
+#
+# notmuch_config_* -> notmuch.Config
+#
+cdef class Config:
+	"""Handles the Notmuch configuration file.
+	"""
+	cdef notmuch_config_t * _cfg
+	cdef void * _ctx
+	cdef object _filename
+
+	def __init__(self, filename="~/.notmuch-config"):
+		"""Open a Notmuch configuration file.
+		"""
+		cdef int newret
+		self._ctx = talloc_init("notmuch.Config")
+		self._cfg = notmuch_config_open(self._ctx, filename, &newret)
+		self._filename = filename
+
+	def __dealloc__(self):
+		notmuch_config_close(self._cfg)
+		talloc_free(self._ctx)
+
+	property filename:
+		"""File name containing the configuration (string)"""
+		def __get__(self):
+			return self._filename
+
+	property database_path:
+		"""Path to the Notmuch database (string)"""
+		def __get__(self):
+			return notmuch_config_get_database_path(self._cfg)
+		def __set__(self, path):
+			notmuch_config_set_database_path(self._cfg, path)
+
+	property user_name:
+		"""User name (string)"""
+		def __get__(self):
+			return notmuch_config_get_user_name(self._cfg)
+		def __set__(self, name):
+			notmuch_config_set_user_name(self._cfg, name)
+
+	property user_primary_email:
+		"""Primary e-mail address of the user (string)"""
+		def __get__(self):
+			return notmuch_config_get_user_primary_email(self._cfg)
+		def __set__(self, email):
+			notmuch_config_set_user_primary_email(self._cfg, email)
+
+	property other_email:
+		"""List of other e-mail addresses of the user (tuple of strings)"""
+		def __get__(self):
+			cdef size_t length
+			cdef size_t i
+			cdef char **emails
+			emails = notmuch_config_get_user_other_email(self._cfg, &length)
+
+			result = []
+			for i from 0 <= i < length:
+				result.append(emails[i])
+
+			# XXX We do not want the result to be modifiable, because the property
+			#     must be assigned as a whole, and not just modified only in the
+			#     Python side of the world.
+			return tuple(result)
+
+		def __set__(self, emaillist):
+			cdef size_t length = len(emaillist)
+			cdef char **emails = pyutil_alloc_strv(self._ctx, length)
+			cdef size_t i
+
+			for i from 0 <= i < len(emaillist):
+				emails[i] = emaillist[i]
+			notmuch_config_set_user_other_email(self._cfg, emails, len(emaillist))
+
+
+	def save(self):
+		"""Save the Notmuch configuration"""
+		notmuch_config_save(self._cfg)
+
+
+cdef class Database
+
+
+cdef class Message:
+	cdef notmuch_message_t *_msg
+
+	def __cinit__(self, object messageptr):
+		# XXX Counterpart of bogus cast
+		self._msg = <notmuch_message_t*> messageptr
+
+	def __dealloc__(self):
+		notmuch_message_destroy(self._msg)
+
+	property message_id:
+		def __get__(self):
+			return notmuch_message_get_message_id(self._msg)
+
+	property thread_id:
+		def __get__(self):
+			return notmuch_message_get_thread_id(self._msg)
+
+	property filename:
+		def __get__(self):
+			return notmuch_message_get_filename(self._msg)
+
+	def get_header(self, name):
+		return notmuch_message_get_header(self._msg, name)
+
+	def add_tag(self, tag):
+		cdef notmuch_status_t ret = notmuch_message_add_tag(self._msg, tag)
+		if ret != NOTMUCH_STATUS_SUCCESS:
+			raise ValueError(notmuch_status_to_string(ret))
+
+	def remove_tag(self, tag):
+		cdef notmuch_status_t ret = notmuch_message_remove_tag(self._msg, tag)
+		if ret != NOTMUCH_STATUS_SUCCESS:
+			raise ValueError(notmuch_status_to_string(ret))
+
+	def remove_all_tags(self):
+		notmuch_message_remove_all_tags(self._msg)
+
+	def freeze(self):
+		notmuch_message_freeze(self._msg)
+
+	def thaw(self):
+		notmuch_message_thaw(self._msg)
+
+
+cdef class Thread:
+	cdef notmuch_thread_t *_thread
+
+	def __cinit__(self, object threadptr):
+		self._thread = <notmuch_thread_t*> threadptr
+
+	def __dealloc__(self):
+		notmuch_thread_destroy(self._thread)
+
+	property authors:
+		def __get__(self):
+			return notmuch_thread_get_authors(self._thread)
+
+	property subject:
+		def __get__(self):
+			return notmuch_thread_get_subject(self._thread)
+
+	property thread_id:
+		def __get__(self):
+			return notmuch_thread_get_thread_id(self._thread)
+
+	property oldest_date:
+		def __get__(self):
+			return datetime.fromtimestamp(notmuch_thread_get_oldest_date(self._thread))
+
+	property newest_date:
+		def __get__(self):
+			return datetime.fromtimestamp(notmuch_thread_get_newest_date(self._thread))
+
+	property total_messages:
+		def __get__(self):
+			return notmuch_thread_get_total_messages(self._thread)
+
+	property matched_messages:
+		def __get__(self):
+			return notmuch_thread_get_matched_messages(self._thread)
+
+	def __len__(self):
+		return self.matched_messages
+
+
+cdef class Query:
+	cdef notmuch_query_t *_query
+
+	def __cinit__(self, Database db not None, qs):
+		self._query = notmuch_query_create(db._db, qs)
+
+	def __dealloc__(self):
+		notmuch_query_destroy(self._query)
+
+	def __len__(self):
+		return notmuch_query_count_messages(self._query)
+
+	def sort(self, notmuch_sort_t ordering):
+		notmuch_query_set_sort(self._query, ordering)
+		return self
+
+
+
+cdef class Database:
+	cdef notmuch_database_t *_db
+
+	def __init__(self, path=None, readonly=True):
+		cdef notmuch_database_mode_t mode = NOTMUCH_DATABASE_MODE_READ_WRITE
+		if path is None:
+			path = Config().database_path
+		if readonly:
+			mode = NOTMUCH_DATABASE_MODE_READ_ONLY
+		self._db = notmuch_database_open(path, mode)
+
+	def __dealloc__(self):
+		notmuch_database_close(self._db)
+
+	def get_timestamp(self, key):
+		cdef time_t ts = notmuch_database_get_timestamp(self._db, key)
+		return datetime.fromtimestamp(ts)
+
+	def set_timestamp(self, key, timestamp):
+		cdef time_t ts
+		if isinstance(timestamp, date):
+			ts = int(timestamp.strftime("%s"))
+		elif isinstance(timestamp, float):
+			ts = <time_t> timestamp
+		elif isinstance(timestamp, int):
+			ts = timestamp
+		else:
+			raise ValueError("Numeric timestamp or datetime.date object expected")
+
+	def add_message(self, filename):
+		cdef notmuch_message_t *message = NULL
+		cdef notmuch_status_t   status
+		status = notmuch_database_add_message(self._db, filename, &message)
+		if status == NOTMUCH_STATUS_SUCCESS:
+			# XXX This cast seems bogus, it may work, though
+			return Message(<object> message)
+		else:
+			if message != NULL:
+				notmuch_message_destroy(message)
+			raise ValueError(notmuch_status_to_string(status))
+
+	def find_message(self, message_id):
+		cdef notmuch_message_t *message
+		message = notmuch_database_find_message(self._db, message_id)
+		if message == NULL:
+			raise KeyError(message_id)
+		return Message(<object> message)
+
+	property path:
+		"""Database path"""
+		def __get__(self):
+			return notmuch_database_get_path(self._db)
+
+	def query(self, qstring):
+		return Query(self, qstring)
+
+
diff --git a/python/pyutil.h b/python/pyutil.h
new file mode 100644
index 0000000..64d93bf
--- /dev/null
+++ b/python/pyutil.h
@@ -0,0 +1,16 @@
+/*
+ * pyutil.h
+ * Copyright (C) 2009 Adrian Perez <aperez@igalia.com>
+ *
+ * Distributed under terms of the GPLv3 license.
+ */
+
+#ifndef __pyutil_h__
+#define __pyutil_h__
+
+#include <talloc.h>
+
+#define pyutil_alloc_strv(_ctx, _n)  talloc_array ((_ctx), char*, (_n))
+
+#endif /* !__pyutil_h__ */
+
diff --git a/python/setup.py b/python/setup.py
new file mode 100644
index 0000000..ffd43b2
--- /dev/null
+++ b/python/setup.py
@@ -0,0 +1,89 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+# vim:fenc=utf-8
+#
+# Copyright © 2009 Adrian Perez <aperez@igalia.com>
+#
+# Distributed under terms of the GPLv3 license.
+
+from distutils.core import setup
+from distutils.extension import Extension
+from Cython.Distutils import build_ext
+import commands
+
+
+class FlagOMatic(dict):
+    _KEYS = ("extra_link_args", "include_dirs", "library_dirs", "libraries")
+
+    def __init__(self, *arg, **kw):
+        super(FlagOMatic, self).__init__(*arg, **kw)
+        for key in self._KEYS:
+            self[key] = set(self.get(key, []))
+
+    extra_link_args = property(lambda self: self["extra_link_args"])
+    include_dirs    = property(lambda self: self["include_dirs"])
+    library_dirs    = property(lambda self: self["library_dirs"])
+    libraries       = property(lambda self: self["libraries"])
+
+    _FLAG_MAP = {
+            "-I": "include_dirs",
+            "-L": "library_dirs",
+            "-l": "libraries",
+        }
+
+    def add_compiler_flags(self, text):
+        for token in text.split():
+            key = self._FLAG_MAP.get(token[:2], "extra_link_args")
+            if key == "extra_link_args":
+                self.extra_link_args.add(token)
+            else:
+                self[key].add(token[2:])
+
+    def add_pkgconfig_flags(self, *modules):
+        self.add_command_output_flags(
+                "pkg-config --libs --cflags %s" % " ".join(modules))
+
+    def add_command_output_flags(self, command):
+        self.add_compiler_flags(commands.getoutput(command))
+
+    @property
+    def kwargs(self):
+        return dict((k, list(v)) for k, v in self.iteritems())
+
+
+# Gather compiler flags
+#
+flags = FlagOMatic(
+       #extra_link_args = ("../lib/notmuch.a",),
+        include_dirs    = ("..", "../lib", "../compat"))
+
+flags.add_pkgconfig_flags("gmime-2.4", "talloc")
+flags.add_command_output_flags("xapian-config --cxxflags --libs")
+
+# We are building a extension module
+#
+import os
+
+srcs = ["notmuch.pyx"]
+srcs.extend(
+    map(lambda s: "../"+s,
+        filter(lambda s: s.endswith(".c"),
+            os.listdir(".."))))
+srcs.extend(
+    map(lambda s: "../lib/"+s,
+        filter(lambda s: s.endswith(".cc") or s.endswith(".c"),
+            os.listdir("../lib"))))
+
+ext_modules = [Extension("notmuch", srcs, **flags.kwargs)]
+
+
+# And now for the easy part :-)
+#
+setup(
+    name         = "notmuch",
+    author       = "Adrian Perez",
+    author_email = "aperez@igalia.com",
+    cmdclass     = {"build_ext": build_ext},
+    ext_modules  = ext_modules,
+)
+

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: SWIG (and particularly Python) bindings

[Scott Robinson]

Excerpts from Adrian Perez de Castro's message of Wed Dec 30 02:52:23 -0800 2009:
> BTW, I think that if more bindings start to appear, Notmuch might be built
> as a shared library, to avoid duplicating it everywhere. One option may be
> using *just* libtool but not the rest of auto-foo tools (for the record:
> I agree with Carl that they are slow and wicked).

http://github.com/quad/notmuch/tree/libtool
-- 
Scott Robinson | http://quadhome.com/

Q: Why are my replies five sentences or less?
A: http://five.sentenc.es/

Re: SWIG (and particularly Python) bindings

[Ben Gamari]

Excerpts from Adrian Perez de Castro's message of Wed Dec 30 05:52:23 -0500 2009:
> On Tue, 29 Dec 2009 04:16:43 -0500, Ben wrote:
> 
> > Regardless, I thought it might be nice to have access to the notmuch
> > backend from a language other than C (preferably my high-level language
> > of choice, python) [...]
> 
> Funny, I was just doing the same: a Python binding. Haha, so now we have
> two just-backed Python bindings. What should we do?

Heh, it's a small world, eh?

> 
> > [...] To this end, I took a few hours today acquainting
> > myself with SWIG and produced these bindings for Python. Unfortunately,
> > it doesn't appear that SWIG has particularly good support for
> > object-oriented C [...]
> 
> I already used SWIG sometimes in the past (and did not like it a lot), so
> my binding is using Cython [*] (which is exactly like Pyrex plus some extra
> features), so the binding is partly manual.
> 
I liked that SWIG would give us coverage for a multitude of languages
with little work. Unfortunately, getting an object oriented interface
requires wrapping the functions exposed by SWIG. Nevertheless, I think
being able to generically hide the ugliness of the binding glue itself
is quite nice. Moveover, there's effectively no SWIG interface code to
maintain.  It's literally just a [#%]include "notmuch.h". The rest is
all the Python wrapper. This seems like a nice binding model, short of
SWIG doing everything (the developers at some point might add support
for proper wrapping of object-oriented C code).

> > While the bindings are currently in the form of a patch to notmuch
> > (creating a top-level swig directory in the source tree), they could
> > certainly be moved out-of-tree if the powers that be don't feel it
> > appropriate to include them. [...]
> 
> Same here, see attached patch. It is currently unfinished, and I was just
> about to add support for iterating notmuch_threads_t and other similar
> structures. I can also publish a Git repo with the entire branch, just
> drop me a line if you want me to do that.
> 
In my approach, I just used the iterator protocol.

> > [...] Unfortunately, the build system is currently almost entirely
> > independent from the notmuch build system. If  these are to be
> > included in-tree, I would be curious to hear people have
> > to say about how we might integrate it into the sup build system.
>                                                   ^^^
> (Mmmh, I suppose you mean "notmuch build system" there :P)

Heh, yep.

> 
> Mine is a little more cooked, as I have added a distutils "setup.py"
> script. The bad news is that Python modules need to be compiled as
> relocatable object files (-fPIC to teh rescue!), and the linker will
> refuse to link the generated code with "notmuch.a" -- so I am instructing
> distutils to compile *all* sources again. Not nice.
> 
In my patch I simple I modified the LDFLAGS to include -fPIC. Not sure if
that's an acceptable option or not.

> BTW, I think that if more bindings start to appear, Notmuch might be built
> as a shared library, to avoid duplicating it everywhere. One option may be
> using *just* libtool but not the rest of auto-foo tools (for the record:
> I agree with Carl that they are slow and wicked).
> 
I think you're probably right. Libtool is probably the best option here
(Despite the fact that I, too, have a bitter hatred of autotools).

Cheers,
- Ben

Re: SWIG (and particularly Python) bindings

[Ben Gamari]

Excerpts from Scott Robinson's message of Wed Dec 30 06:34:15 -0500 2009:
> Excerpts from Adrian Perez de Castro's message of Wed Dec 30 02:52:23 -0800 2009:
> > BTW, I think that if more bindings start to appear, Notmuch might be built
> > as a shared library, to avoid duplicating it everywhere. One option may be
> > using *just* libtool but not the rest of auto-foo tools (for the record:
> > I agree with Carl that they are slow and wicked).
> 
> http://github.com/quad/notmuch/tree/libtool

Is this branch being considered for merge? The build output arguably
isn't as pretty as it was before, but it seems to work. Moreover, my
SWIG patches currently depend upon having a shared library involved.

- Ben

[PATCH] libtoolize notmuch.

[Ben Gamari]

Here is a patch based on  quad's libtool branch rebased on current master. It
has been tested and verified to work on my machine

---
 Makefile           |   27 ++++++++++++++++-----------
 Makefile.local     |   12 ++++++++----
 lib/Makefile.local |    8 ++++----
 3 files changed, 28 insertions(+), 19 deletions(-)

diff --git a/Makefile b/Makefile
index 021fdb8..9022a39 100644
--- a/Makefile
+++ b/Makefile
@@ -3,6 +3,10 @@ WARN_CFLAGS=$(WARN_CXXFLAGS) -Wmissing-declarations
 
 # Additional programs that are used during the compilation process.
 EMACS ?= emacs
+libtool = libtool
+# Needed so libtool replaces parameters in a modern fashion.
+CC = gcc
+CXX = g++
 # Lowercase to avoid clash with GZIP environment variable for passing
 # arguments to gzip.
 gzip = gzip
@@ -43,34 +47,35 @@ include Makefile.local
 # user how to enable verbose compiles.
 ifeq ($(V),)
 quiet_DOC := "Use \"$(MAKE) V=1\" to see the verbose compile lines.\n"
-quiet = @printf $(quiet_DOC)$(eval quiet_DOC:=)"  $1 $2	$@\n"; $($1)
+V = 0
 endif
 # The user has explicitly enabled quiet compilation.
 ifeq ($(V),0)
-quiet = @printf "  $1	$@\n"; $($1)
+quiet = @printf $(quiet_DOC)$(eval quiet_DOC:=)"  $1 $2	$@\n"; $($1)
+libtool += --silent
 endif
 # Otherwise, print the full command line.
 quiet ?= $($1)
 
-%.o: %.cc $(all_deps)
-	$(call quiet,CXX,$(CXXFLAGS)) -c $(FINAL_CXXFLAGS) $< -o $@
+%.lo: %.cc $(all_deps)
+	$(call quiet,libtool,$(CXXFLAGS)) --mode=compile $(CXX) -c $(FINAL_CXXFLAGS) $< -o $@
 
-%.o: %.c $(all_deps)
-	$(call quiet,CC,$(CFLAGS)) -c $(FINAL_CFLAGS) $< -o $@
+%.lo: %.c $(all_deps)
+	$(call quiet,libtool,$(CFLAGS)) --mode=compile $(CC) -c $(FINAL_CFLAGS) $< -o $@
 
 %.elc: %.el
 	$(call quiet,EMACS) -batch -f batch-byte-compile $<
 
 .deps/%.d: %.c $(all_deps)
 	@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
-	$(CC) -M $(CPPFLAGS) $(FINAL_CFLAGS) $< > $@.$$$$ 2>/dev/null ; \
-	sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@.$$$$ > $@; \
+	$(CC) -M $(CPPFLAGS) $(FINAL_CFLAGS) $< > $@.$$$$; \
+	sed 's,'$$(basename $*)'\.lo[ :]*,$*.lo $@ : ,g' < $@.$$$$ > $@; \
 	rm -f $@.$$$$
 
 .deps/%.d: %.cc $(all_deps)
 	@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
-	$(CXX) -M $(CPPFLAGS) $(FINAL_CXXFLAGS) $< > $@.$$$$ 2>/dev/null ; \
-	sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@.$$$$ > $@; \
+	$(CXX) -M $(CPPFLAGS) $(FINAL_CXXFLAGS) $< > $@.$$$$; \
+	sed 's,'$$(basename $*)'\.lo[ :]*,$*.lo $@ : ,g' < $@.$$$$ > $@; \
 	rm -f $@.$$$$
 
 DEPS := $(SRCS:%.c=.deps/%.d)
@@ -79,4 +84,4 @@ DEPS := $(DEPS:%.cc=.deps/%.d)
 
 .PHONY : clean
 clean:
-	rm -f $(CLEAN); rm -rf .deps
+	$(libtool) --mode=clean rm -f $(CLEAN); rm -rf .deps
diff --git a/Makefile.local b/Makefile.local
index 933ff4c..ccff76c 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -20,9 +20,9 @@ notmuch_client_srcs =		\
 	query-string.c		\
 	show-message.c
 
-notmuch_client_modules = $(notmuch_client_srcs:.c=.o)
-notmuch: $(notmuch_client_modules) lib/notmuch.a
-	$(call quiet,CXX,$(LDFLAGS)) $^ $(FINAL_LDFLAGS) -o $@
+notmuch_client_modules = $(notmuch_client_srcs:.c=.lo)
+notmuch: $(notmuch_client_modules) lib/libnotmuch.la
+	$(call quiet,libtool,$(LDFLAGS)) --mode=link $(CXX) $^ $(FINAL_LDFLAGS) -o $@
 
 notmuch.1.gz: notmuch.1
 	$(call quiet,gzip) --stdout $^ > $@
@@ -32,8 +32,12 @@ install: all notmuch.1.gz
 	do \
 		install -d $$d ; \
 	done ;
-	install notmuch $(DESTDIR)$(prefix)/bin/
+	$(libtool) --mode=install install -c lib/libnotmuch.la $(DESTDIR)$(prefix)/lib/
+	$(libtool) --mode=install install -c notmuch $(DESTDIR)$(prefix)/bin/
 	install -m0644 notmuch.1.gz $(DESTDIR)$(prefix)/share/man/man1/
+	install contrib/notmuch-completion.bash \
+		$(DESTDIR)$(bash_completion_dir)/notmuch
+	$(libtool) --mode=finish $(DESTDIR)$(prefix)/lib/
 
 install-emacs: install emacs
 	for d in $(DESTDIR)/$(emacs_lispdir) ; \
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 70489e1..b03dff6 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -17,9 +17,9 @@ libnotmuch_cxx_srcs =		\
 	$(dir)/query.cc		\
 	$(dir)/thread.cc
 
-libnotmuch_modules = $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
-$(dir)/notmuch.a: $(libnotmuch_modules)
-	$(call quiet,AR) rcs $@ $^
+libnotmuch_modules = $(libnotmuch_c_srcs:.c=.lo) $(libnotmuch_cxx_srcs:.cc=.lo)
+$(dir)/libnotmuch.la: $(libnotmuch_modules)
+	$(call quiet,libtool) --mode=link $(CXX) -rpath $(DESTDIR)$(prefix)/lib -o $@ $^
 
 SRCS  := $(SRCS) $(libnotmuch_c_srcs) $(libnotmuch_cxx_srcs)
-CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/notmuch.a
+CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/notmuch.la
-- 
1.6.3.3

Re: SWIG (and particularly Python) bindings

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1084 bytes --]

On Tue, 29 Dec 2009 04:16:43 -0500, Ben Gamari <bgamari@gmail.com> wrote:
> I've been looking at switching away from sup recently to something with
> a slightly little less everything-and-the-kitchen-sink philosophy.

Hi Ben,

Welcome to Notmuch!

> Notmuch looks excellent, although it appears that its current front-end
> for my editor of choice (vim) is a little lacking in some ways
> (operations involving a call to notmuch invariably lock up vi for the
> duration of the operation).

Yes. The emacs mode was similarly crippled initially and fairly painful
until we fixed that, (and quite nice afterwards).

> these are to be included in-tree, I would be curious to hear people have
> to say about how we might integrate it into the sup build system.

I can't say much about integrating with the sup build system...

;-)

> Hope this helps someone. I'll hopefully have a chance to work with the
> bindings soon and will send a new set of patches once I think I've found
> all of the obvious issues.

Thanks for sharing. I'll look forward to future contributions from you!

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: SWIG (and particularly Python) bindings

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 981 bytes --]

On Wed, 30 Dec 2009 11:52:23 +0100, Adrian Perez de Castro <aperez@igalia.com> wrote:
> BTW, I think that if more bindings start to appear, Notmuch might be built
> as a shared library, to avoid duplicating it everywhere.

Yes. As soon as we have users of the library we can install it as a
shared library.

>                                                          One option may be
> using *just* libtool but not the rest of auto-foo tools (for the record:
> I agree with Carl that they are slow and wicked).

Totally uninterested in libtool personally. It does so many things wrong
(.la files that *only* cause breakage on Linux, a tendency to shove
hard-coded rpath entries in the library where unwanted), and doesn't
seem to do much of anything that's really interesting. I'm happy to just
accept patches that tweak the command in the Makefile for "build a
shared library now" for each platform that's of practical interest. The
job here is just not that complicated.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] libtoolize notmuch.

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 472 bytes --]

On Fri, 15 Jan 2010 23:28:55 -0500, Ben Gamari <bgamari.foss@gmail.com> wrote:
> Here is a patch based on  quad's libtool branch rebased on current master. It
> has been tested and verified to work on my machine

How about something similar that simply creates a .so with ld?

I'd be interested in using automake before libtool, (but similarly,
don't actually see any benefit in using automake as opposed to just
using GNU make features as notmuch does currently).

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: SWIG (and particularly Python) bindings

[Ben Gamari]

Excerpts from Carl Worth's message of Wed Jan 20 03:45:34 -0500 2010:
> Welcome to Notmuch!
> 
Thanks!

> > Notmuch looks excellent, although it appears that its current front-end
> > for my editor of choice (vim) is a little lacking in some ways
> > (operations involving a call to notmuch invariably lock up vi for the
> > duration of the operation).
> 
> Yes. The emacs mode was similarly crippled initially and fairly painful
> until we fixed that, (and quite nice afterwards).
> 
I can easily believe that. I've tried using the vim mode and quickly
gave up. I think I'm going to have to stick with sup until I've added
the appropriate subprocess support into vim. I have some half-completed
code, but still have a ways to go.

> > these are to be included in-tree, I would be curious to hear people have
> > to say about how we might integrate it into the sup build system.
> 
> I can't say much about integrating with the sup build system...
> 
Heh. My bad.

- Ben