Search results now start appearing "instantly"

Search results now start appearing "instantly"

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 2308 bytes --]

Here's a nice feature for any of you who have been suffering with
"notmuch search" taking forever before it would complete, (such as when
trying to display "notmuch search tag:inbox" with every message in your
collection having the inbox tag).

I've fixed the "notmuch search" command-line program to not do any of
the "chunking" that it was doing before, but to instead stream results
out as quickly as possible. So it takes maybe a second or two before the
results start appearing, but then it's a steady stream from then on.

I've also fixed the emacs interface to "notmuch search" which previously
would wait until the command completed and would then do the processing
it needs to on the buffer. The new implementation instead fires off
"notmuch search" as an asynchronous process and filters the results as
they come in.

The net effect is that searches now appear instantly and you can just
watch the scrollbar shrink as the results keep coming in, (and you can
navigate and read messages as much as you'd like while results keep
coming).

It's working out fairly well, (but for one minor bug which is that you
lose your current position when you refresh). And I hope people are
happy with it. For some it might take notmuch from "interesting, but
too slow to be usable" to "blisteringly fast, and where have you been
all my life".

The one thing that might still be undesirable is that the "notmuch
search" process will still continue to burn CPU until all of the results
are complete. We could probably take some clues from the user's actions
to ameliorate this. It would be easy to suspend the search process when
the user obviously isn't needing more results and then resume it
later. Detecting what the user needs is a little tricky, but in some
cases will be obvious, (such as the user switching away from the scroll
buffer or the user viewing a tiny fraction of the search results and not
scrolling through them).

So please update to the latest code and let me know what you think. I'll
be interested to hear if this helps people, and also to know if the CPU
usage is a problem for anybody.

And for the authors of the other interfaces, let us know when you've got
similar support for streaming searches, (or if you didn't get this
automatically as soon as "notmuch search" was fixed).

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Search results now start appearing "instantly"

[Peter Wang]

On 2009-11-25, Carl Worth <cworth@cworth.org> wrote:
> 
> So please update to the latest code and let me know what you think. I'll
> be interested to hear if this helps people, and also to know if the CPU
> usage is a problem for anybody.

Brilliant, thanks!

Peter

Re: Search results now start appearing "instantly"

[Carl Worth]

On Thu, 26 Nov 2009 23:34:34 +1100, Peter Wang <novalazy@gmail.com> wrote:
> On 2009-11-25, Carl Worth <cworth@cworth.org> wrote:
> > 
> > So please update to the latest code and let me know what you think. I'll
> > be interested to hear if this helps people, and also to know if the CPU
> > usage is a problem for anybody.
> 
> Brilliant, thanks!

You're quite welcome. I only wish I would have done it right the first
time.

Anyway, enjoy.

-Carl

Re: Search results now start appearing "instantly"

[Ruben Pollan]

[-- Attachment #1: Type: text/plain, Size: 803 bytes --]

On 11:17, Wed 25 Nov 09, Carl Worth wrote:
> And for the authors of the other interfaces, let us know when you've got
> similar support for streaming searches, (or if you didn't get this
> automatically as soon as "notmuch search" was fixed).

Yes, actually is faster now.

I had to rewrite some of the code, because before I used the
notmuch_query_search_threads for pagging with the parameters for number of
threads and index. But I like it more now.

Thank you.

-- 
Rubén Pollán  | jabber:meskio@jabber.org
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Ésta es la historia de una sociedad que se hunde
y mientras cae se repite:
hasta ahora todo va bien, hasta ahora todo va bien ...
Pero lo importante no es la caida sino el aterrizaje.
                               el odio

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Search results now start appearing "instantly"

[Carl Worth]

On Thu, 26 Nov 2009 14:27:37 +0100, Ruben Pollan <meskio@sindominio.net> wrote:
> On 11:17, Wed 25 Nov 09, Carl Worth wrote:
> > And for the authors of the other interfaces, let us know when you've got
> > similar support for streaming searches, (or if you didn't get this
> > automatically as soon as "notmuch search" was fixed).
> 
> Yes, actually is faster now.
> 
> I had to rewrite some of the code, because before I used the
> notmuch_query_search_threads for pagging with the parameters for number of
> threads and index. But I like it more now.

Hi Ruben,

I'm glad it worked out. I probably should have sent the patches to the
list for review before just pushing them out. I'll have to adjust from
this being just my own thing where I do whatever I want, to instead work
within our community.

So please feel free to call me out on that,

-Carl