Re: About the json output and the number of results shown.

[Carl Worth <cworth@cworth.org>]

[-- Attachment #1: Type: text/plain, Size: 2515 bytes --]

On Wed, 12 Jan 2011 19:37:21 +0100, Christophe-Marie Duquesne <chm.duquesne@gmail.com> wrote:
> So I am wondering: what is the point of having a tool that is able to
> output json and ending in not using it? Is there a solution to make
> the json output more useable? One solution I've been thinking about
> would be to add an option: the range of results to show (something
> like --range=25:50). Is it doable easily?

This is fairly easy to do, yes. We even had functionality like this
once, and I'll probably even add it back soon, (since a client like the
vim interface isn't able to do the kind of asynchronous processing that
you would really want).

One problem with the ranged output (for "notmuch search" at least) is
that small ranges with large initial offsets will take longer than
expected. This is because in this case notmuch can't directly use
Xapian's range offset support. The user is asking for an offset as a
number of threads, but within Xapian we only have messages stored. So
notmuch will have to search for messages from the beginning, construct a
bunch of useless threads, and then throw those threads away after doing
no more than counting them.

This inefficiency in this API was one of the reasons I dropped this
functionality before. It's pretty ugly. But I don't see a really good
answer for that.

> feature request. In any case, do you have any proposal for making
> sense of this json output without modifications in the notmuch CLI?

We've run into basically the same issue with the emacs interface. We've
been avoiding using the json output precisely because the emacs JSON
parsing would need to see all the output before it could start
parsing. And that wouldn't give us the responsive user interface that we
want.

One idea I've had for this is to change the output (perhaps with a
command-line option) to avoid emitting the outer array. That is, the
results would instead be a series of independent JSON objects rather
than a single JSON object. That should let the application treat things
quickly by simply calling the JSON parser for each complete
object. (Though, here, the application would likely want a cheap way to
know when the input represented a complete object.)

If anyone wants to help improve our JSON output here, then that would be
great.

For any change to the structure of the JSON output, I'd also like to see
some documentation added to specify that structure clearly.

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]