JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...]

JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...]

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 1873 bytes --]

Hey, Carl.  I mentioned this is an email a while back
(id:87eil4ygar.fsf@servo.finestructure.net), but I want to bring it up
again now that you're looking at the JSON output stuff.  I would like to
make the case for JSON being the one and only output format.  Here's the
arguments:

* JSON is a very well spec'd API
* JSON is a *very* well supported standard (it's basically well
  supported by every language)
* the JSONformat is perfectly suited for notmuch's output
* emacs has a json.el library, so it would not be hard at all to modify
  the notmuch.el to parse json output
* using only one output format would considerably reduce the code base
  in notmuch, keeping things simpler and easier to maintain

I think these are pretty good reason to just move to JSON as *the*
output format.

While I'm on this topic, let me rehash another point I made previously:
If we move to JSON output, it would be *so* sweet to provide means to
filter the output JSON fields.  The "--output" flag could be used to
pick the desired output fields.  For instance, the current "search"
output could be something like:

search --output=thread_id,date,number,author,subject,tags

while the current "show" output could be something like:

search --output=message_id,tags,path,header,body,attachments

This would all make things *much* easier for clients and wrappers.  For
instance, here are just a couple of things that would be made much
easier for wrapper scripts:

* Proper maildir sync ("search --output=message_id,tags,path" ...)

* Purging "delete" tagged messages ("search --output=path" tags:delete)

* Moving/archiving messages based on search results ("search --output=path" ...)

I would personally love to see this.  I think it would make notmuch
cleaner, simpler, and more powerful all at the same time.

Interested in hearing what other's thoughts on this idea are.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...]

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 1908 bytes --]

On Tue, 23 Feb 2010 16:00:01 -0500, Jameson Rollins <jrollins@finestructure.net> wrote:
> Hey, Carl.  I mentioned this is an email a while back
> (id:87eil4ygar.fsf@servo.finestructure.net), but I want to bring it up
> again now that you're looking at the JSON output stuff.  I would like to
> make the case for JSON being the one and only output format.  Here's the
> arguments:

Oh, I'm definitely in favor of killing the %message{ format.

As soon as we get the emacs and vi frontends switched to the json
output, then that old thing can be eliminated.

> * using only one output format would considerably reduce the code base
>   in notmuch, keeping things simpler and easier to maintain

But I actually still want text-only output. It will be even less
"parseable" than the current output, (no delimiters at all), but also
have some niceties like thread indentation, etc.

The idea there is to support direct, human-readable output for quick
command-line use. (It also means that notmuch provides a much gentler
introduction to people just experimenting with it). And I'm willing to
keep the code base just a "little" bit complicated for that.

> If we move to JSON output, it would be *so* sweet to provide means to
> filter the output JSON fields.  The "--output" flag could be used to
> pick the desired output fields.  For instance, the current "search"
> output could be something like:

I definitely want to enable the user to select which fields are printed.

But I don't see how that depends on JSON output. In fact, for easy shell
scripts, etc. I want to be able to do search that give me filenames,
email addresses, etc. without any delimiters at all.

I definitely want to be able to pipe single-field lists coming from
notmuch to grep, xargs, shell for loops, etc. without having to decode
JSON.

So I'm still in favor of having a "plain text" output going forward,
(and by default).

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...]

[Sebastian Spaeth]

> I definitely want to be able to pipe single-field lists coming from
> notmuch to grep, xargs, shell for loops, etc. without having to decode
> JSON.

While I would love to see JSON (even by default), I agree. If I just
want to code up a notmuch-based address book with sth like:

notmuch show to:Diana --output=to --sort=relevance --limit=20

just getting back a plain list of mail addresses is the easiest to handle.

Re: JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...]

[racin]

> > I definitely want to be able to pipe single-field lists coming from
> > notmuch to grep, xargs, shell for loops, etc. without having to
> decode
> > JSON.
> 
> While I would love to see JSON (even by default), I agree. If I just
> want to code up a notmuch-based address book with sth like:
> 
> notmuch show to:Diana --output=to --sort=relevance --limit=20
> 
> just getting back a plain list of mail addresses is the easiest to
> handle.

This would also be useful for the Emacs/Vim interfaces. For instance, my smart completion patch
would really benefit from notmuch being capable of outputing various fields in all messages in plain text
separated by newlines (this is even easier to handle in emacs code than JSON). In fact, most of the C code I had
to write for this patch is better replaced by the --output option...

Re: JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...]

[Jameson Rollins]

[-- Attachment #1: Type: text/plain, Size: 1592 bytes --]

On Wed, 24 Feb 2010 15:27:18 +0100 (CET), racin@free.fr wrote:
> > > I definitely want to be able to pipe single-field lists coming from
> > > notmuch to grep, xargs, shell for loops, etc. without having to
> > decode
> > > JSON.
> > 
> > While I would love to see JSON (even by default), I agree. If I just
> > want to code up a notmuch-based address book with sth like:
> > 
> > notmuch show to:Diana --output=to --sort=relevance --limit=20
> > 
> > just getting back a plain list of mail addresses is the easiest to
> > handle.
> 
> This would also be useful for the Emacs/Vim interfaces. For instance, my smart completion patch
> would really benefit from notmuch being capable of outputing various fields in all messages in plain text
> separated by newlines (this is even easier to handle in emacs code than JSON). In fact, most of the C code I had
> to write for this patch is better replaced by the --output option...

Ok, I'm convinced.  I can see how they're both useful.  I had been
thinking more about the fact that text output isn't so useful for
multiline content (like message content), but I can see how it would be
useful for single line output.

I had also been thinking about the fact that the current "text" output,
specifically for the "show" command, *is* structured, but just not
according to any standard that I know of.  If the output is going to be
structured (ie. "show" output) then it should be in JSON format.  If
not, like the output of a single field that is a single line, then
having text output is definitely useful.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]