Using notmuch from Ruby, Python, et al.

Using notmuch from Ruby, Python, et al.

[Scott Robinson]

I'm hoping this thread will provide a guide for my efforts.

I'm writing a web frontend for notmuch in Ruby. But, the output from the
notmuch command-line is ambiguous. Therefore, I've started two branches
experimenting with different ways of providing hooks.

Which of these approaches are likely to be pulled upstream?

== Shared Library

I have modified the build to output a shared library. It's a small change
relying on libtool.

However, the archives indicate Carl dislikes libtool?

== JSON output

I added a --output=(text|json) option to "search" and "show." Then, emit json
results appropriately.

To accomplish this, I used the cJSON library sans parsing logic.

However, I saw discussion about providing a --format option?
-- 
Scott Robinson | http://quadhome.com/

Q: Why are my replies five sentences or less?
A: http://five.sentenc.es/

Re: Using notmuch from Ruby, Python, et al.

[Jeffrey Ollie]

On Wed, Nov 25, 2009 at 10:16 AM, Scott Robinson <scott@quadhome.com> wrote:
>
> == Shared Library
>
> I have modified the build to output a shared library. It's a small change
> relying on libtool.
>
> However, the archives indicate Carl dislikes libtool?

Ugh, please, let's not go down the autoconf/automake/libtool route.

> == JSON output
>
> I added a --output=(text|json) option to "search" and "show." Then, emit json
> results appropriately.
>
> To accomplish this, I used the cJSON library sans parsing logic.
>
> However, I saw discussion about providing a --format option?

Awesome, I was just looking at using json-glib but I haven't gotten
very far yet, mainly because I've never used it before (nor done much
of anything with JSON). I was going to call the option
'--output-format=(standard|json)' but I'm not going to worry about the
color of that bikeshed.

IMHO, I think both approaches would be valuable, that way people
writing front ends could choose the methods that worked best for them.
 Do you have patches that you are ready to share?

-- 
Jeff Ollie

Re: Using notmuch from Ruby, Python, et al.

[Scott Robinson]

[-- Attachment #1: Type: text/plain, Size: 1567 bytes --]

Excerpts from Jeffrey Ollie's message of Wed Nov 25 08:33:39 -0800 2009:
> On Wed, Nov 25, 2009 at 10:16 AM, Scott Robinson <scott@quadhome.com> wrote:
> >
> > == Shared Library
> >
> > I have modified the build to output a shared library. It's a small change
> > relying on libtool.
> >
> > However, the archives indicate Carl dislikes libtool?
> 
> Ugh, please, let's not go down the autoconf/automake/libtool route.

I didn't use autoconf and friends. Just put in libtool where $(CXX) and $(CC)
are. Going with the system defaults.

See the attached patch.

> > == JSON output
> >
> > I added a --output=(text|json) option to "search" and "show." Then, emit json
> > results appropriately.
> >
> > To accomplish this, I used the cJSON library sans parsing logic.
> >
> > However, I saw discussion about providing a --format option?
> 
> Awesome, I was just looking at using json-glib but I haven't gotten
> very far yet, mainly because I've never used it before (nor done much
> of anything with JSON). I was going to call the option
> '--output-format=(standard|json)' but I'm not going to worry about the
> color of that bikeshed.
> 
> IMHO, I think both approaches would be valuable, that way people
> writing front ends could choose the methods that worked best for them.
>  Do you have patches that you are ready to share?
> 

Yes!

As the json work is a bit more substantial, is it preferable to start a [PATCH]
thread or link to a git repo?
-- 
Scott Robinson | http://quadhome.com/

Q: Why are my replies five sentences or less?
A: http://five.sentenc.es/

[-- Attachment #2: libtoolize-notmuch.diff --]
[-- Type: application/octet-stream, Size: 3996 bytes --]

diff --git a/Makefile b/Makefile
index 2cd1b1b..b065006 100644
--- a/Makefile
+++ b/Makefile
@@ -4,6 +4,10 @@ CFLAGS=-O2
 
 # Additional programs that are used during the compilation process.
 EMACS ?= emacs
+libtool = libtool
+# Needed so libtool replaces parameters in a modern fashion.
+CC = gcc
+CXX = g++
 # Lowercase to avoid clash with GZIP environment variable for passing
 # arguments to gzip.
 gzip = gzip
@@ -41,20 +45,21 @@ include Makefile.config
 # user how to enable verbose compiles.
 ifeq ($(V),)
 quiet_DOC := "Use \"$(MAKE) V=1\" to see the verbose compile lines.\n"
-quiet = @echo $(quiet_DOC)$(eval quiet_DOC:=)"  $1	$@"; $($1)
+V = 0
 endif
 # The user has explicitly enabled quiet compilation.
 ifeq ($(V),0)
-quiet = @echo "  $1	$@"; $($1)
+quiet = @echo $(quiet_DOC)$(eval quiet_DOC:=)"  $1	$@"; $($1)
+libtool += --silent
 endif
 # Otherwise, print the full command line.
 quiet ?= $($1)
 
-%.o: %.cc $(all_deps)
-	$(call quiet,CXX) -c $(CXXFLAGS) $< -o $@
+%.lo: %.cc $(all_deps)
+	$(call quiet,libtool) --mode=compile $(CXX) -c $(CXXFLAGS) $< -o $@
 
-%.o: %.c $(all_deps)
-	$(call quiet,CC) -c $(CFLAGS) $< -o $@
+%.lo: %.c $(all_deps)
+	$(call quiet,libtool) --mode=compile $(CC) -c $(CFLAGS) $< -o $@
 
 %.elc: %.el
 	$(call quiet,EMACS) -batch -f batch-byte-compile $<
@@ -62,13 +67,13 @@ quiet ?= $($1)
 .deps/%.d: %.c $(all_deps)
 	@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
 	$(CC) -M $(CPPFLAGS) $(CFLAGS) $< > $@.$$$$; \
-	sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@.$$$$ > $@; \
+	sed 's,'$$(basename $*)'\.lo[ :]*,$*.lo $@ : ,g' < $@.$$$$ > $@; \
 	rm -f $@.$$$$
 
 .deps/%.d: %.cc $(all_deps)
 	@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
 	$(CXX) -M $(CPPFLAGS) $(CXXFLAGS) $< > $@.$$$$; \
-	sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@.$$$$ > $@; \
+	sed 's,'$$(basename $*)'\.lo[ :]*,$*.lo $@ : ,g' < $@.$$$$ > $@; \
 	rm -f $@.$$$$
 
 DEPS := $(SRCS:%.c=.deps/%.d)
@@ -77,4 +82,4 @@ DEPS := $(DEPS:%.cc=.deps/%.d)
 
 .PHONY : clean
 clean:
-	rm -f $(CLEAN); rm -rf .deps
+	$(libtool) --mode=clean rm -f $(CLEAN); rm -rf .deps
diff --git a/Makefile.local b/Makefile.local
index a3a19de..0a3a201 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -20,9 +20,9 @@ notmuch_client_srcs =		\
 	query-string.c		\
 	show-message.c
 
-notmuch_client_modules = $(notmuch_client_srcs:.c=.o)
-notmuch: $(notmuch_client_modules) lib/notmuch.a
-	$(call quiet,CXX) $^ $(LDFLAGS) -o $@
+notmuch_client_modules = $(notmuch_client_srcs:.c=.lo)
+notmuch: $(notmuch_client_modules) lib/libnotmuch.la
+	$(call quiet,libtool) --mode=link $(CXX) $^ $(LDFLAGS) -o $@
 
 notmuch.1.gz: notmuch.1
 	$(call quiet,gzip) --stdout $^ > $@
@@ -33,10 +33,12 @@ install: all notmuch.1.gz
 	do \
 		install -d $$d ; \
 	done ;
-	install notmuch $(DESTDIR)$(prefix)/bin/
+	$(libtool) --mode=install install -c lib/libnotmuch.la $(DESTDIR)$(prefix)/lib/
+	$(libtool) --mode=install install -c notmuch $(DESTDIR)$(prefix)/bin/
 	install -m0644 notmuch.1.gz $(DESTDIR)$(prefix)/share/man/man1/
 	install contrib/notmuch-completion.bash \
 		$(DESTDIR)$(bash_completion_dir)/notmuch
+	$(libtool) --mode=finish $(DESTDIR)$(prefix)/lib/
 
 install-emacs: install emacs
 	for d in $(DESTDIR)/$(emacs_lispdir) ; \
diff --git a/lib/Makefile.local b/lib/Makefile.local
index a7562c9..1bc0991 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -16,9 +16,9 @@ libnotmuch_cxx_srcs =		\
 	$(dir)/query.cc		\
 	$(dir)/thread.cc
 
-libnotmuch_modules = $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
-$(dir)/notmuch.a: $(libnotmuch_modules)
-	$(call quiet,AR) rcs $@ $^
+libnotmuch_modules = $(libnotmuch_c_srcs:.c=.lo) $(libnotmuch_cxx_srcs:.cc=.lo)
+$(dir)/libnotmuch.la: $(libnotmuch_modules)
+	$(call quiet,libtool) --mode=link $(CXX) -rpath $(DESTDIR)$(prefix)/lib -o $@ $^
 
 SRCS  := $(SRCS) $(libnotmuch_c_srcs) $(libnotmuch_cxx_srcs)
-CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/notmuch.a
+CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/libnotmuch.la

Re: Using notmuch from Ruby, Python, et al.

[Jeffrey Ollie]

On Wed, Nov 25, 2009 at 11:40 AM, Scott Robinson <scott@quadhome.com> wrote:
>
> As the json work is a bit more substantial, is it preferable to start a [PATCH]
> thread or link to a git repo?

Probably a new [PATCH] thread so that the changes can be discussed.

-- 
Jeff Ollie

Re: Using notmuch from Ruby, Python, et al.

[Jeffrey Ollie]

On Wed, Nov 25, 2009 at 10:16 AM, Scott Robinson <scott@quadhome.com> wrote:
>
> To accomplish this, I used the cJSON library sans parsing logic.

I wasn't familiar with cJSON so I went over to take a look.  I haven't
really examined the code, but I really don't like the fact that you
are expected to copy the cJSON code into your project.

-- 
Jeff Ollie

Re: Using notmuch from Ruby, Python, et al.

[Scott Robinson]

Excerpts from Jeffrey Ollie's message of Wed Nov 25 09:46:47 -0800 2009:
> On Wed, Nov 25, 2009 at 10:16 AM, Scott Robinson <scott@quadhome.com> wrote:
> >
> > To accomplish this, I used the cJSON library sans parsing logic.
> 
> I wasn't familiar with cJSON so I went over to take a look.  I haven't
> really examined the code, but I really don't like the fact that you
> are expected to copy the cJSON code into your project.

I selected cJSON because it was simple for integration. Given how libsha1.c is
included in lib/, I figured adding a depedency wouldn't be very popular.

Would it be better to link with a previously existing library?
-- 
Scott Robinson | http://quadhome.com/

Q: Why are my replies five sentences or less?
A: http://five.sentenc.es/

Re: Using notmuch from Ruby, Python, et al.

[Jeffrey Ollie]

On Wed, Nov 25, 2009 at 11:58 AM, Scott Robinson <scott@quadhome.com> wrote:
>
> I selected cJSON because it was simple for integration. Given how libsha1.c is
> included in lib/, I figured adding a depedency wouldn't be very popular.

Oh, I noticed libsha1.c as well, and it bugs the heck out of me.  I
just haven't gotten around to doing anything about it yet.

> Would it be better to link with a previously existing library?

IMHO yes.  I was looking at json-glib because it seems to be the only
C JSON API in Fedora, plus it's based upon glib.  From what I've read
it should be really easy to serialize a GMimeObject to a JSON
structure with very few calls using json-glib.

-- 
Jeff Ollie

Re: Using notmuch from Ruby, Python, et al.

[Scott Robinson]

Excerpts from Jeffrey Ollie's message of Wed Nov 25 10:03:05 -0800 2009:
> On Wed, Nov 25, 2009 at 11:58 AM, Scott Robinson <scott@quadhome.com> wrote:
> > Would it be better to link with a previously existing library?
> 
> IMHO yes.  I was looking at json-glib because it seems to be the only
> C JSON API in Fedora, plus it's based upon glib.  From what I've read
> it should be really easy to serialize a GMimeObject to a JSON
> structure with very few calls using json-glib.

I'm taking a look at json-glib now.

But, it doesn't look to support streaming output.
-- 
Scott Robinson | http://quadhome.com/

Q: Why are my replies five sentences or less?
A: http://five.sentenc.es/

Re: Using notmuch from Ruby, Python, et al.

[Carl Worth]

On Wed, 25 Nov 2009 09:58:00 -0800, Scott Robinson <scott@quadhome.com> wrote:
> Excerpts from Jeffrey Ollie's message of Wed Nov 25 09:46:47 -0800 2009:
> 
> I selected cJSON because it was simple for integration. Given how libsha1.c is
> included in lib/, I figured adding a depedency wouldn't be very
> popular.

I'm not dead-set against additional dependencies, but I think I do take
a conservative approach, (hence copying in the tiny libsha1.c rather
than linking to any enormous encryption library).

> Would it be better to link with a previously existing library?

For something like this, I don't see the need for any library at
all---generating any particular format is always *much* easier than
parsing it.

So I wouldn't mind seeing a couple of tiny functions in notmuch for this
(or other formats). And I think it would be better to start with code
that's already written and debugged (and license compatible!) rather
than starting from scratch.

> Q: Why are my replies five sentences or less?
> A: http://five.sentenc.es/

There---just five this time. :-)

-Carl

Re: Using notmuch from Ruby, Python, et al.

[Mikhail Gusarov]

[-- Attachment #1: Type: text/plain, Size: 344 bytes --]


Twas brillig at 18:55:28 27.11.2009 UTC-08 when cworth@cworth.org did gyre and gimble:

 >> Q: Why are my replies five sentences or less?
 >> A: http://five.sentenc.es/

 CW> There---just five this time. :-)

I imagine hook that counts the number of sentences and inserts proper
signature :)

-- 
  http://fossarchy.blogspot.com/

[-- Attachment #2: Type: application/pgp-signature, Size: 834 bytes --]

Re: Using notmuch from Ruby, Python, et al.

[Carl Worth]

On Wed, 25 Nov 2009 08:16:54 -0800, Scott Robinson <scott@quadhome.com> wrote:
> I'm hoping this thread will provide a guide for my efforts.
> 
> I'm writing a web frontend for notmuch in Ruby. But, the output from the
> notmuch command-line is ambiguous. Therefore, I've started two branches
> experimenting with different ways of providing hooks.

Hi Scott, welcome to notmuch!

A web frontend sounds very interesting, and I'll look forward to see
what you come up with here.

> Which of these approaches are likely to be pulled upstream?

Both, in fact!

> == Shared Library
> 
> I have modified the build to output a shared library. It's a small change
> relying on libtool.

We definitely want to have a shared library at some point. To some
extent, publishing a shared library commits us to maintaining API/ABI
for that library, and that's something I'm not willing to commit to
at this point, (since we're still seeing a fair amount of churn in the
interface).

But then again, we're also not even putting out a tar-file release yet
either, so I think it's easy to say that we're not "publishing" anything
yet and the API is all still subject to change.

> However, the archives indicate Carl dislikes libtool?

I've been maintaining a mainstream library (cairo) for several years
that uses libtool in its build system. So I know what that's like and I
definitely don't like it.

I haven't yet tried maintaining a library without using libtool, but I'm
optimistic that it will be much less painful, so that's what I'd like to
do for now.

> == JSON output
> 
> I added a --output=(text|json) option to "search" and "show." Then, emit json
> results appropriately.
> 
> To accomplish this, I used the cJSON library sans parsing logic.
> 
> However, I saw discussion about providing a --format option?

Obviously the exact name isn't that important and is easy to change. But
yes, this kind of thing looks very interesting.

> Q: Why are my replies five sentences or less?
> A: http://five.sentenc.es/

I guess I lose (or you do since you're the one receiving it). I count
this as sentence number 13 in my reply. :-)

-Carl