Multithreaded access

Multithreaded access

[Peter Wang]

Hi,

I am also toying around with a curses frontend for notmuch :-)
One thing that I would really like to have is asynchronous search:
I should be able to begin reading mail even while search results
are rolling in.

At the moment I'm using one read-only database_t* and creating multiple
queries off it.  There's a background thread[1] slurping up <n> search
results at a time, then passes them to the main thread.  The main thread
also handles the pager.  When the user goes to open a thread, it creates
a separate query_t so that it can look up the messages and their headers
and filenames.

It seems to work, but sometimes crashes.  I'm wondering if there is
something inherently wrong with this design, so that I should try a
different approach.

A future concern is how to handle tag updates.  Could I open the database
for writing in one thread, and read-only in another?

Peter

[1] Actually, I am playing around with Go at the same time so they are
really goroutines.  Any problems could very well be related to this.

Re: Multithreaded access

[Carl Worth]

On Mon, 23 Nov 2009 22:07:42 +1100, Peter Wang <novalazy@gmail.com> wrote:
> I am also toying around with a curses frontend for notmuch :-)
> One thing that I would really like to have is asynchronous search:
> I should be able to begin reading mail even while search results
> are rolling in.

Absolutely! I just coded this up on my plane ride home for notmuch.el
and it is *fantastic*. It takes notmuch from "OK, as long as your
searches don't have a lot of results" to "Blisteringly fast,
regardless of what you're doing."

And it's not like sup, where you page down and then wait for more
search results to page in. Things just keep filling up in the buffer
while you're reading and navigating, just as it should be.

I'll be cleaning this up and pushing it out soon. I'm very excited
about it.

> At the moment I'm using one read-only database_t* and creating multiple
> queries off it.  There's a background thread[1] slurping up <n> search
> results at a time, then passes them to the main thread.  The main thread
> also handles the pager.  When the user goes to open a thread, it creates
> a separate query_t so that it can look up the messages and their headers
> and filenames.

Hmmm... in my code with emacs, I'm just using separate processes,
(which seems to be working fine). I haven't looked into Xapian's
documentation with regard to threads yet, so I can't actually say how
well notmuch's library work with threads yet.

> A future concern is how to handle tag updates.  Could I open the database
> for writing in one thread, and read-only in another?

Again, I don't know about the threading yet. But you can definitely
open the database read-write in one process while also having it be
opened read-only in another.

-Carl