* Re: JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...] [not found] <1257161639.5014171267021461605.JavaMail.root@zimbra1-e1.priv.proxad.net> @ 2010-02-24 14:27 ` racin 2010-02-24 14:56 ` Jameson Rollins 0 siblings, 1 reply; 5+ messages in thread From: racin @ 2010-02-24 14:27 UTC (permalink / raw To: Sebastian Spaeth; +Cc: notmuch > > I definitely want to be able to pipe single-field lists coming from > > notmuch to grep, xargs, shell for loops, etc. without having to > decode > > JSON. > > While I would love to see JSON (even by default), I agree. If I just > want to code up a notmuch-based address book with sth like: > > notmuch show to:Diana --output=to --sort=relevance --limit=20 > > just getting back a plain list of mail addresses is the easiest to > handle. This would also be useful for the Emacs/Vim interfaces. For instance, my smart completion patch would really benefit from notmuch being capable of outputing various fields in all messages in plain text separated by newlines (this is even easier to handle in emacs code than JSON). In fact, most of the C code I had to write for this patch is better replaced by the --output option... ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...] 2010-02-24 14:27 ` JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...] racin @ 2010-02-24 14:56 ` Jameson Rollins 0 siblings, 0 replies; 5+ messages in thread From: Jameson Rollins @ 2010-02-24 14:56 UTC (permalink / raw To: racin, Sebastian Spaeth; +Cc: notmuch [-- Attachment #1: Type: text/plain, Size: 1592 bytes --] On Wed, 24 Feb 2010 15:27:18 +0100 (CET), racin@free.fr wrote: > > > I definitely want to be able to pipe single-field lists coming from > > > notmuch to grep, xargs, shell for loops, etc. without having to > > decode > > > JSON. > > > > While I would love to see JSON (even by default), I agree. If I just > > want to code up a notmuch-based address book with sth like: > > > > notmuch show to:Diana --output=to --sort=relevance --limit=20 > > > > just getting back a plain list of mail addresses is the easiest to > > handle. > > This would also be useful for the Emacs/Vim interfaces. For instance, my smart completion patch > would really benefit from notmuch being capable of outputing various fields in all messages in plain text > separated by newlines (this is even easier to handle in emacs code than JSON). In fact, most of the C code I had > to write for this patch is better replaced by the --output option... Ok, I'm convinced. I can see how they're both useful. I had been thinking more about the fact that text output isn't so useful for multiline content (like message content), but I can see how it would be useful for single line output. I had also been thinking about the fact that the current "text" output, specifically for the "show" command, *is* structured, but just not according to any standard that I know of. If the output is going to be structured (ie. "show" output) then it should be in JSON format. If not, like the output of a single field that is a single line, then having text output is definitely useful. jamie. [-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH] JSON output for notmuch-search and notmuch-show. @ 2009-12-18 5:33 Scott Robinson 2009-12-18 12:59 ` [PATCH] Add an "--output=(json|text|)" command-line option to both " david 0 siblings, 1 reply; 5+ messages in thread From: Scott Robinson @ 2009-12-18 5:33 UTC (permalink / raw To: notmuch Excerpts from David Bremner's message of Mon Dec 14 15:10:12 -0800 2009: > OK, I'll wait for Scott's patch, and see how he is using cJSON. Just > stealing a few functions might be the way to go. > > The cJSON code is less horrifying after running through indent. Now all > we need is "indent --carl" :) > I took an earlier suggestion and didn't use cJSON, instead writing custom code for emitting the new format. Added an "--output=(json|text|)" command-line option to both notmuch-search and notmuch-show. In the case of notmuch-show, "--output=json" also implies "--entire-thread" as the thread structure is implicit in the emitted document tree. As a coincidence to the implementation, multipart message ID numbers are now incremented with each part printed. This changes the previous semantics, which were unclear and not necessary related to the actual ordering of the message parts. --- Makefile.local | 3 +- json.c | 73 ++++++++++++++ notmuch-client.h | 3 + notmuch-search.c | 163 +++++++++++++++++++++++++++++--- notmuch-show.c | 275 ++++++++++++++++++++++++++++++++++++++++++++++-------- notmuch.c | 24 ++++-- show-message.c | 4 +- 7 files changed, 481 insertions(+), 64 deletions(-) create mode 100644 json.c diff --git a/Makefile.local b/Makefile.local index 933ff4c..53b474b 100644 --- a/Makefile.local +++ b/Makefile.local @@ -18,7 +18,8 @@ notmuch_client_srcs = \ notmuch-tag.c \ notmuch-time.c \ query-string.c \ - show-message.c + show-message.c \ + json.c notmuch_client_modules = $(notmuch_client_srcs:.c=.o) notmuch: $(notmuch_client_modules) lib/notmuch.a diff --git a/json.c b/json.c new file mode 100644 index 0000000..ee563d6 --- /dev/null +++ b/json.c @@ -0,0 +1,73 @@ +/* notmuch - Not much of an email program, (just index and search) + * + * Copyright © 2009 Carl Worth + * Copyright © 2009 Keith Packard + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see http://www.gnu.org/licenses/ . + * + * Authors: Carl Worth <cworth@cworth.org> + * Keith Packard <keithp@keithp.com> + */ + +#include "notmuch-client.h" + +/* + * json_quote_str derived from cJSON's print_string_ptr, + * Copyright (c) 2009 Dave Gamble + */ + +char * +json_quote_str(const void *ctx, const char *str) +{ + const char *ptr; + char *ptr2; + char *out; + int len = 0; + + if (!str) + return NULL; + + for (ptr = str; *ptr; len++, ptr++) { + if (*ptr < 32 || *ptr == '\"' || *ptr == '\\') + len++; + } + + out = talloc_array (ctx, char, len + 3); + + ptr = str; + ptr2 = out; + + *ptr2++ = '\"'; + while (*ptr) { + if (*ptr > 31 && *ptr != '\"' && *ptr != '\\') { + *ptr2++ = *ptr++; + } else { + *ptr2++ = '\\'; + switch (*ptr++) { + case '\"': *ptr2++ = '\"'; break; + case '\\': *ptr2++ = '\\'; break; + case '\b': *ptr2++ = 'b'; break; + case '\f': *ptr2++ = 'f'; break; + case '\n': *ptr2++ = 'n'; break; + case '\r': *ptr2++ = 'r'; break; + case '\t': *ptr2++ = 't'; break; + default: ptr2--; break; + } + } + } + *ptr2++ = '\"'; + *ptr2++ = '\0'; + + return out; +} diff --git a/notmuch-client.h b/notmuch-client.h index 50a30fe..7b844b9 100644 --- a/notmuch-client.h +++ b/notmuch-client.h @@ -143,6 +143,9 @@ notmuch_status_t show_message_body (const char *filename, void (*show_part) (GMimeObject *part, int *part_count)); +char * +json_quote_str (const void *ctx, const char *str); + /* notmuch-config.c */ typedef struct _notmuch_config notmuch_config_t; diff --git a/notmuch-search.c b/notmuch-search.c index dc44eb6..e243747 100644 --- a/notmuch-search.c +++ b/notmuch-search.c @@ -20,8 +20,120 @@ #include "notmuch-client.h" +typedef struct search_format { + const char *results_start; + const char *thread_start; + void (*thread) (const void *ctx, + const char *id, + const time_t date, + const int matched, + const int total, + const char *authors, + const char *subject); + const char *tag_start; + const char *tag; + const char *tag_sep; + const char *tag_end; + const char *thread_sep; + const char *thread_end; + const char *results_end; +} search_format_t; + +static void +format_thread_text (const void *ctx, + const char *id, + const time_t date, + const int matched, + const int total, + const char *authors, + const char *subject); +static const search_format_t format_text = { + "", + "", + format_thread_text, + " (", + "%s", " ", + ")", "", + "\n", + "", +}; + +static void +format_thread_json (const void *ctx, + const char *id, + const time_t date, + const int matched, + const int total, + const char *authors, + const char *subject); +static const search_format_t format_json = { + "[", + "{", + format_thread_json, + "\"tags\": [", + "\"%s\"", ", ", + "]", ",\n", + "}", + "]\n", +}; + +static void +format_thread_text (const void *ctx, + const char *id, + const time_t date, + const int matched, + const int total, + const char *authors, + const char *subject) +{ + printf ("thread:%s %12s [%d/%d] %s; %s", + id, + notmuch_time_relative_date (ctx, date), + matched, + total, + authors, + subject); +} + +static void +format_thread_json (const void *ctx, + const char *id, + const time_t date, + const int matched, + const int total, + const char *authors, + const char *subject) +{ + struct tm *tm; + char timestamp[40]; + void *ctx_quote = talloc_new (ctx); + + tm = gmtime (&date); + if (tm == NULL) + INTERNAL_ERROR ("gmtime failed on thread %s.", id); + + if (strftime (timestamp, sizeof (timestamp), "%s", tm) == 0) + INTERNAL_ERROR ("strftime failed on thread %s.", id); + + printf ("\"id\": %s,\n" + "\"timestamp\": %s,\n" + "\"matched\": %d,\n" + "\"total\": %d,\n" + "\"authors\": %s,\n" + "\"subject\": %s,\n", + json_quote_str (ctx_quote, id), + timestamp, + matched, + total, + json_quote_str (ctx_quote, authors), + json_quote_str (ctx_quote, subject)); + + talloc_free (ctx_quote); +} + static void do_search_threads (const void *ctx, + const search_format_t *format, notmuch_query_t *query, notmuch_sort_t sort) { @@ -29,7 +141,9 @@ do_search_threads (const void *ctx, notmuch_threads_t *threads; notmuch_tags_t *tags; time_t date; - const char *relative_date; + int first_thread = 1; + + fputs (format->results_start, stdout); for (threads = notmuch_query_search_threads (query); notmuch_threads_has_more (threads); @@ -37,6 +151,9 @@ do_search_threads (const void *ctx, { int first_tag = 1; + if (! first_thread) + fputs (format->thread_sep, stdout); + thread = notmuch_threads_get (threads); if (sort == NOTMUCH_SORT_OLDEST_FIRST) @@ -44,30 +161,37 @@ do_search_threads (const void *ctx, else date = notmuch_thread_get_newest_date (thread); - relative_date = notmuch_time_relative_date (ctx, date); + fputs (format->thread_start, stdout); + + format->thread (ctx, + notmuch_thread_get_thread_id (thread), + date, + notmuch_thread_get_matched_messages (thread), + notmuch_thread_get_total_messages (thread), + notmuch_thread_get_authors (thread), + notmuch_thread_get_subject (thread)); - printf ("thread:%s %12s [%d/%d] %s; %s", - notmuch_thread_get_thread_id (thread), - relative_date, - notmuch_thread_get_matched_messages (thread), - notmuch_thread_get_total_messages (thread), - notmuch_thread_get_authors (thread), - notmuch_thread_get_subject (thread)); + fputs (format->tag_start, stdout); - printf (" ("); for (tags = notmuch_thread_get_tags (thread); notmuch_tags_has_more (tags); notmuch_tags_advance (tags)) { if (! first_tag) - printf (" "); - printf ("%s", notmuch_tags_get (tags)); + fputs (format->tag_sep, stdout); + printf (format->tag, notmuch_tags_get (tags)); first_tag = 0; } - printf (")\n"); + + fputs (format->tag_end, stdout); + fputs (format->thread_end, stdout); + + first_thread = 0; notmuch_thread_destroy (thread); } + + fputs (format->results_end, stdout); } int @@ -79,6 +203,7 @@ notmuch_search_command (void *ctx, int argc, char *argv[]) char *query_str; char *opt; notmuch_sort_t sort = NOTMUCH_SORT_NEWEST_FIRST; + const search_format_t *format = &format_text; int i; for (i = 0; i < argc && argv[i][0] == '-'; i++) { @@ -96,6 +221,16 @@ notmuch_search_command (void *ctx, int argc, char *argv[]) fprintf (stderr, "Invalid value for --sort: %s\n", opt); return 1; } + } else if (STRNCMP_LITERAL (argv[i], "--output=") == 0) { + opt = argv[i] + sizeof ("--output=") - 1; + if (strcmp (opt, "text") == 0) { + format = &format_text; + } else if (strcmp (opt, "json") == 0) { + format = &format_json; + } else { + fprintf (stderr, "Invalid value for --output: %s\n", opt); + return 1; + } } else { fprintf (stderr, "Unrecognized option: %s\n", argv[i]); return 1; @@ -132,7 +267,7 @@ notmuch_search_command (void *ctx, int argc, char *argv[]) notmuch_query_set_sort (query, sort); - do_search_threads (ctx, query, sort); + do_search_threads (ctx, format, query, sort); notmuch_query_destroy (query); notmuch_database_close (notmuch); diff --git a/notmuch-show.c b/notmuch-show.c index 376aacd..70d2605 100644 --- a/notmuch-show.c +++ b/notmuch-show.c @@ -20,8 +20,65 @@ #include "notmuch-client.h" +typedef struct show_format { + const char *message_set_start; + const char *message_start; + void (*message) (const void *ctx, + notmuch_message_t *message, + int indent); + const char *header_start; + void (*header) (const void *ctx, + notmuch_message_t *message); + const char *header_end; + const char *body_start; + void (*part) (GMimeObject *part, + int *part_count); + const char *body_end; + const char *message_end; + const char *message_set_sep; + const char *message_set_end; +} show_format_t; + +static void +format_message_text (unused (const void *ctx), + notmuch_message_t *message, + int indent); +static void +format_headers_text (const void *ctx, + notmuch_message_t *message); +static void +format_part_text (GMimeObject *part, + int *part_count); +static const show_format_t format_text = { + "", + "\fmessage{ ", format_message_text, + "\fheader{\n", format_headers_text, "\fheader}\n", + "\fbody{\n", format_part_text, "\fbody}\n", + "\fmessage}\n", "", + "" +}; + +static void +format_message_json (const void *ctx, + notmuch_message_t *message, + unused (int indent)); +static void +format_headers_json (const void *ctx, + notmuch_message_t *message); +static void +format_part_json (GMimeObject *part, + int *part_count); +static const show_format_t format_json = { + "[", + "{", format_message_json, + ", \"headers\": {", format_headers_json, "}", + ", \"body\": [", format_part_json, "]", + "}", ", ", + "]" +}; + static const char * -_get_tags_as_string (void *ctx, notmuch_message_t *message) +_get_tags_as_string (const void *ctx, notmuch_message_t *message) { notmuch_tags_t *tags; int first = 1; @@ -48,7 +105,7 @@ _get_tags_as_string (void *ctx, notmuch_message_t *message) /* Get a nice, single-line summary of message. */ static const char * -_get_one_line_summary (void *ctx, notmuch_message_t *message) +_get_one_line_summary (const void *ctx, notmuch_message_t *message) { const char *from; time_t date; @@ -67,18 +124,87 @@ _get_one_line_summary (void *ctx, notmuch_message_t *message) } static void -show_part_content (GMimeObject *part) +format_message_text (unused (const void *ctx), notmuch_message_t *message, int indent) +{ + printf ("id:%s depth:%d match:%d filename:%s\n", + notmuch_message_get_message_id (message), + indent, + notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH), + notmuch_message_get_filename (message)); +} + +static void +format_message_json (const void *ctx, notmuch_message_t *message, unused (int indent)) +{ + void *ctx_quote = talloc_new (ctx); + + printf ("\"id\": %s, \"match\": %s, \"filename\": %s", + json_quote_str (ctx_quote, notmuch_message_get_message_id (message)), + notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH) ? "true" : "false", + json_quote_str (ctx_quote, notmuch_message_get_filename (message))); + + talloc_free (ctx_quote); +} + +static void +format_headers_text (const void *ctx, notmuch_message_t *message) +{ + const char *headers[] = { + "Subject", "From", "To", "Cc", "Bcc", "Date" + }; + const char *name, *value; + unsigned int i; + + printf ("%s\n", _get_one_line_summary (ctx, message)); + + for (i = 0; i < ARRAY_SIZE (headers); i++) { + name = headers[i]; + value = notmuch_message_get_header (message, name); + if (value) + printf ("%s: %s\n", name, value); + } +} + +static void +format_headers_json (const void *ctx, notmuch_message_t *message) +{ + const char *headers[] = { + "Subject", "From", "To", "Cc", "Bcc", "Date" + }; + const char *name, *value; + unsigned int i; + int first_header = 1; + void *ctx_quote = talloc_new (ctx); + + for (i = 0; i < ARRAY_SIZE (headers); i++) { + name = headers[i]; + value = notmuch_message_get_header (message, name); + if (value) + { + if (!first_header) + fputs (", ", stdout); + first_header = 0; + + printf ("%s: %s", + json_quote_str (ctx_quote, name), + json_quote_str (ctx_quote, value)); + } + } + + talloc_free (ctx_quote); +} + +static void +show_part_content (GMimeObject *part, GMimeStream *stream_out) { - GMimeStream *stream_stdout = g_mime_stream_file_new (stdout); GMimeStream *stream_filter = NULL; GMimeDataWrapper *wrapper; const char *charset; charset = g_mime_object_get_content_type_parameter (part, "charset"); - if (stream_stdout) { - g_mime_stream_file_set_owner (GMIME_STREAM_FILE (stream_stdout), FALSE); - stream_filter = g_mime_stream_filter_new(stream_stdout); + if (stream_out) { + stream_filter = g_mime_stream_filter_new(stream_out); g_mime_stream_filter_add(GMIME_STREAM_FILTER(stream_filter), g_mime_filter_crlf_new(FALSE, FALSE)); if (charset) { @@ -92,15 +218,16 @@ show_part_content (GMimeObject *part) g_mime_data_wrapper_write_to_stream (wrapper, stream_filter); if (stream_filter) g_object_unref(stream_filter); - if (stream_stdout) - g_object_unref(stream_stdout); } static void -show_part (GMimeObject *part, int *part_count) +format_part_text (GMimeObject *part, int *part_count) { GMimeContentDisposition *disposition; GMimeContentType *content_type; + GMimeStream *stream_stdout = g_mime_stream_file_new (stdout); + + g_mime_stream_file_set_owner (GMIME_STREAM_FILE (stream_stdout), FALSE); disposition = g_mime_object_get_content_disposition (part); if (disposition && @@ -118,11 +245,14 @@ show_part (GMimeObject *part, int *part_count) if (g_mime_content_type_is_type (content_type, "text", "*") && !g_mime_content_type_is_type (content_type, "text", "html")) { - show_part_content (part); + show_part_content (part, stream_stdout); } printf ("\fattachment}\n"); + if (stream_stdout) + g_object_unref(stream_stdout); + return; } @@ -135,7 +265,7 @@ show_part (GMimeObject *part, int *part_count) if (g_mime_content_type_is_type (content_type, "text", "*") && !g_mime_content_type_is_type (content_type, "text", "html")) { - show_part_content (part); + show_part_content (part, stream_stdout); } else { @@ -144,57 +274,93 @@ show_part (GMimeObject *part, int *part_count) } printf ("\fpart}\n"); + + if (stream_stdout) + g_object_unref(stream_stdout); } static void -show_message (void *ctx, notmuch_message_t *message, int indent) +format_part_json (GMimeObject *part, int *part_count) { - const char *headers[] = { - "Subject", "From", "To", "Cc", "Bcc", "Date" - }; - const char *name, *value; - unsigned int i; + GMimeContentType *content_type; + GMimeContentDisposition *disposition; + void *ctx = talloc_new (NULL); + GMimeStream *stream_memory = g_mime_stream_mem_new (); + GByteArray *part_content; - printf ("\fmessage{ id:%s depth:%d match:%d filename:%s\n", - notmuch_message_get_message_id (message), - indent, - notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH), - notmuch_message_get_filename (message)); + content_type = g_mime_object_get_content_type (GMIME_OBJECT (part)); - printf ("\fheader{\n"); + if (*part_count > 1) + fputs (", ", stdout); - printf ("%s\n", _get_one_line_summary (ctx, message)); + printf ("{\"id\": %d, \"content-type\": %s", + *part_count, + json_quote_str (ctx, g_mime_content_type_to_string (content_type))); - for (i = 0; i < ARRAY_SIZE (headers); i++) { - name = headers[i]; - value = notmuch_message_get_header (message, name); - if (value) - printf ("%s: %s\n", name, value); + disposition = g_mime_object_get_content_disposition (part); + if (disposition && + strcmp (disposition->disposition, GMIME_DISPOSITION_ATTACHMENT) == 0) + { + const char *filename = g_mime_part_get_filename (GMIME_PART (part)); + + printf (", \"filename\": %s", json_quote_str (ctx, filename)); } - printf ("\fheader}\n"); - printf ("\fbody{\n"); + if (g_mime_content_type_is_type (content_type, "text", "*") && + !g_mime_content_type_is_type (content_type, "text", "html")) + { + show_part_content (part, stream_memory); + part_content = g_mime_stream_mem_get_byte_array (GMIME_STREAM_MEM (stream_memory)); + + printf (", \"content\": %s", json_quote_str (ctx, (char *) part_content->data)); + } + + fputs ("}", stdout); + + talloc_free (ctx); + if (stream_memory) + g_object_unref (stream_memory); +} + +static void +show_message (void *ctx, const show_format_t *format, notmuch_message_t *message, int indent) +{ + fputs (format->message_start, stdout); + format->message(ctx, message, indent); - show_message_body (notmuch_message_get_filename (message), show_part); + fputs (format->header_start, stdout); + format->header(ctx, message); + fputs (format->header_end, stdout); - printf ("\fbody}\n"); + fputs (format->body_start, stdout); + show_message_body (notmuch_message_get_filename (message), format->part); + fputs (format->body_end, stdout); - printf ("\fmessage}\n"); + fputs (format->message_end, stdout); } static void -show_messages (void *ctx, notmuch_messages_t *messages, int indent, +show_messages (void *ctx, const show_format_t *format, notmuch_messages_t *messages, int indent, notmuch_bool_t entire_thread) { notmuch_message_t *message; notmuch_bool_t match; + int first_set = 1; int next_indent; + fputs (format->message_set_start, stdout); + for (; notmuch_messages_has_more (messages); notmuch_messages_advance (messages)) { + if (!first_set) + fputs (format->message_set_sep, stdout); + first_set = 0; + + fputs (format->message_set_start, stdout); + message = notmuch_messages_get (messages); match = notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH); @@ -202,15 +368,21 @@ show_messages (void *ctx, notmuch_messages_t *messages, int indent, next_indent = indent; if (match || entire_thread) { - show_message (ctx, message, indent); + show_message (ctx, format, message, indent); next_indent = indent + 1; + + fputs (format->message_set_sep, stdout); } - show_messages (ctx, notmuch_message_get_replies (message), + show_messages (ctx, format, notmuch_message_get_replies (message), next_indent, entire_thread); notmuch_message_destroy (message); + + fputs (format->message_set_end, stdout); } + + fputs (format->message_set_end, stdout); } int @@ -223,15 +395,29 @@ notmuch_show_command (void *ctx, unused (int argc), unused (char *argv[])) notmuch_thread_t *thread; notmuch_messages_t *messages; char *query_string; + char *opt; + const show_format_t *format = &format_text; int entire_thread = 0; int i; + int first_toplevel = 1; for (i = 0; i < argc && argv[i][0] == '-'; i++) { if (strcmp (argv[i], "--") == 0) { i++; break; } - if (strcmp(argv[i], "--entire-thread") == 0) { + if (STRNCMP_LITERAL (argv[i], "--output=") == 0) { + opt = argv[i] + sizeof ("--output=") - 1; + if (strcmp (opt, "text") == 0) { + format = &format_text; + } else if (strcmp (opt, "json") == 0) { + format = &format_json; + entire_thread = 1; + } else { + fprintf (stderr, "Invalid value for --output: %s\n", opt); + return 1; + } + } else if (STRNCMP_LITERAL (argv[i], "--entire-thread") == 0) { entire_thread = 1; } else { fprintf (stderr, "Unrecognized option: %s\n", argv[i]); @@ -268,6 +454,8 @@ notmuch_show_command (void *ctx, unused (int argc), unused (char *argv[])) return 1; } + fputs (format->message_set_start, stdout); + for (threads = notmuch_query_search_threads (query); notmuch_threads_has_more (threads); notmuch_threads_advance (threads)) @@ -280,11 +468,18 @@ notmuch_show_command (void *ctx, unused (int argc), unused (char *argv[])) INTERNAL_ERROR ("Thread %s has no toplevel messages.\n", notmuch_thread_get_thread_id (thread)); - show_messages (ctx, messages, 0, entire_thread); + if (!first_toplevel) + fputs (format->message_set_sep, stdout); + first_toplevel = 0; + + show_messages (ctx, format, messages, 0, entire_thread); notmuch_thread_destroy (thread); + } + fputs (format->message_set_end, stdout); + notmuch_query_destroy (query); notmuch_database_close (notmuch); diff --git a/notmuch.c b/notmuch.c index 2ac8a59..aa2fc12 100644 --- a/notmuch.c +++ b/notmuch.c @@ -162,6 +162,11 @@ command_t commands[] = { "\n" "\t\tSupported options for search include:\n" "\n" + "\t\t--output=(json|text)\n" + "\n" + "\t\t\tPresents the results in either JSON or plain-text\n" + "\t\t\tformat, which is the default.\n" + "\n" "\t\t--sort=(newest-first|oldest-first)\n" "\n" "\t\t\tPresent results in either chronological order\n" @@ -186,13 +191,18 @@ command_t commands[] = { "\t\t\tall messages in the same thread as any matched\n" "\t\t\tmessage will be displayed.\n" "\n" - "\t\tThe output format is plain-text, with all text-content\n" - "\t\tMIME parts decoded. Various components in the output,\n" - "\t\t('message', 'header', 'body', 'attachment', and MIME 'part')\n" - "\t\tare delimited by easily-parsed markers. Each marker consists\n" - "\t\tof a Control-L character (ASCII decimal 12), the name of\n" - "\t\tthe marker, and then either an opening or closing brace,\n" - "\t\t'{' or '}' to either open or close the component.\n" + "\t\t--output=(json|text)\n" + "\n" + "\t\t\tPresents the results in either JSON or plain-text\n" + "\t\t\tformat, which is the default.\n" + "\n" + "\t\tThe plain-text has all text-content MIME parts decoded.\n" + "\t\tVarious components in the output, ('message', 'header',\n" + "\t\t'body', 'attachment', and MIME 'part') are delimited by\n" + "\t\teasily-parsed markers. Each marker consists of a Control-L\n" + "\t\tcharacter (ASCII decimal 12), the name of the marker, and\n" + "\t\tthen either an opening or closing brace, '{' or '}' to\n" + "\t\teither open or close the component.\n" "\n" "\t\tA common use of \"notmuch show\" is to display a single\n" "\t\tthread of email messages. For this, use a search term of\n" diff --git a/show-message.c b/show-message.c index 784981b..05ced9c 100644 --- a/show-message.c +++ b/show-message.c @@ -26,8 +26,6 @@ static void show_message_part (GMimeObject *part, int *part_count, void (*show_part) (GMimeObject *part, int *part_count)) { - *part_count = *part_count + 1; - if (GMIME_IS_MULTIPART (part)) { GMimeMultipart *multipart = GMIME_MULTIPART (part); int i; @@ -56,6 +54,8 @@ show_message_part (GMimeObject *part, int *part_count, return; } + *part_count = *part_count + 1; + (*show_part) (part, part_count); } -- Scott Robinson | http://quadhome.com/ Q: Why are my replies five sentences or less? A: http://five.sentenc.es/ ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH] Add an "--output=(json|text|)" command-line option to both notmuch-search and notmuch-show. 2009-12-18 5:33 [PATCH] JSON output for notmuch-search and notmuch-show Scott Robinson @ 2009-12-18 12:59 ` david 2010-02-23 19:56 ` Carl Worth 0 siblings, 1 reply; 5+ messages in thread From: david @ 2009-12-18 12:59 UTC (permalink / raw To: notmuch [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain, Size: 25645 bytes --] From: Scott Robinson <scott@quadhome.com> In the case of notmuch-show, "--output=json" also implies "--entire-thread" as the thread structure is implicit in the emitted document tree. As a coincidence to the implementation, multipart message ID numbers are now incremented with each part printed. This changes the previous semantics, which were unclear and not necessary related to the actual ordering of the message parts. Edited-By: David Bremner <david@tethera.net> Reviewed-By: David Bremner <david@tethera.net> --- It took me a little work to apply Scott's patch, so rather than asking him to resend it from git-send-email, I am just sending. I hope no-one is offended (much). Other than manually extracting the patch from the output of notmuch show (for me the message arrived base64 encoded), I deleted trailing whitespace on line 465. It compiles, it doesn't seem to screw up the original output, and at least in a few tests, it generates parseable json. Yay!. I'm thinking that the patch I sent out last night to only dump message ids could be reworked to use the framework of this patch. I also think it would be reasonably simple to add an --output=mbox option, for archiving and so on. Makefile.local | 3 +- json.c | 73 ++++++++++++++ notmuch-client.h | 3 + notmuch-search.c | 163 +++++++++++++++++++++++++++++--- notmuch-show.c | 275 ++++++++++++++++++++++++++++++++++++++++++++++-------- notmuch.c | 24 ++++-- show-message.c | 4 +- 7 files changed, 481 insertions(+), 64 deletions(-) create mode 100644 json.c diff --git a/Makefile.local b/Makefile.local index 933ff4c..53b474b 100644 --- a/Makefile.local +++ b/Makefile.local @@ -18,7 +18,8 @@ notmuch_client_srcs = \ notmuch-tag.c \ notmuch-time.c \ query-string.c \ - show-message.c + show-message.c \ + json.c notmuch_client_modules = $(notmuch_client_srcs:.c=.o) notmuch: $(notmuch_client_modules) lib/notmuch.a diff --git a/json.c b/json.c new file mode 100644 index 0000000..ee563d6 --- /dev/null +++ b/json.c @@ -0,0 +1,73 @@ +/* notmuch - Not much of an email program, (just index and search) + * + * Copyright © 2009 Carl Worth + * Copyright © 2009 Keith Packard + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see http://www.gnu.org/licenses/ . + * + * Authors: Carl Worth <cworth@cworth.org> + * Keith Packard <keithp@keithp.com> + */ + +#include "notmuch-client.h" + +/* + * json_quote_str derived from cJSON's print_string_ptr, + * Copyright (c) 2009 Dave Gamble + */ + +char * +json_quote_str(const void *ctx, const char *str) +{ + const char *ptr; + char *ptr2; + char *out; + int len = 0; + + if (!str) + return NULL; + + for (ptr = str; *ptr; len++, ptr++) { + if (*ptr < 32 || *ptr == '\"' || *ptr == '\\') + len++; + } + + out = talloc_array (ctx, char, len + 3); + + ptr = str; + ptr2 = out; + + *ptr2++ = '\"'; + while (*ptr) { + if (*ptr > 31 && *ptr != '\"' && *ptr != '\\') { + *ptr2++ = *ptr++; + } else { + *ptr2++ = '\\'; + switch (*ptr++) { + case '\"': *ptr2++ = '\"'; break; + case '\\': *ptr2++ = '\\'; break; + case '\b': *ptr2++ = 'b'; break; + case '\f': *ptr2++ = 'f'; break; + case '\n': *ptr2++ = 'n'; break; + case '\r': *ptr2++ = 'r'; break; + case '\t': *ptr2++ = 't'; break; + default: ptr2--; break; + } + } + } + *ptr2++ = '\"'; + *ptr2++ = '\0'; + + return out; +} diff --git a/notmuch-client.h b/notmuch-client.h index 50a30fe..7b844b9 100644 --- a/notmuch-client.h +++ b/notmuch-client.h @@ -143,6 +143,9 @@ notmuch_status_t show_message_body (const char *filename, void (*show_part) (GMimeObject *part, int *part_count)); +char * +json_quote_str (const void *ctx, const char *str); + /* notmuch-config.c */ typedef struct _notmuch_config notmuch_config_t; diff --git a/notmuch-search.c b/notmuch-search.c index dc44eb6..e243747 100644 --- a/notmuch-search.c +++ b/notmuch-search.c @@ -20,8 +20,120 @@ #include "notmuch-client.h" +typedef struct search_format { + const char *results_start; + const char *thread_start; + void (*thread) (const void *ctx, + const char *id, + const time_t date, + const int matched, + const int total, + const char *authors, + const char *subject); + const char *tag_start; + const char *tag; + const char *tag_sep; + const char *tag_end; + const char *thread_sep; + const char *thread_end; + const char *results_end; +} search_format_t; + +static void +format_thread_text (const void *ctx, + const char *id, + const time_t date, + const int matched, + const int total, + const char *authors, + const char *subject); +static const search_format_t format_text = { + "", + "", + format_thread_text, + " (", + "%s", " ", + ")", "", + "\n", + "", +}; + +static void +format_thread_json (const void *ctx, + const char *id, + const time_t date, + const int matched, + const int total, + const char *authors, + const char *subject); +static const search_format_t format_json = { + "[", + "{", + format_thread_json, + "\"tags\": [", + "\"%s\"", ", ", + "]", ",\n", + "}", + "]\n", +}; + +static void +format_thread_text (const void *ctx, + const char *id, + const time_t date, + const int matched, + const int total, + const char *authors, + const char *subject) +{ + printf ("thread:%s %12s [%d/%d] %s; %s", + id, + notmuch_time_relative_date (ctx, date), + matched, + total, + authors, + subject); +} + +static void +format_thread_json (const void *ctx, + const char *id, + const time_t date, + const int matched, + const int total, + const char *authors, + const char *subject) +{ + struct tm *tm; + char timestamp[40]; + void *ctx_quote = talloc_new (ctx); + + tm = gmtime (&date); + if (tm == NULL) + INTERNAL_ERROR ("gmtime failed on thread %s.", id); + + if (strftime (timestamp, sizeof (timestamp), "%s", tm) == 0) + INTERNAL_ERROR ("strftime failed on thread %s.", id); + + printf ("\"id\": %s,\n" + "\"timestamp\": %s,\n" + "\"matched\": %d,\n" + "\"total\": %d,\n" + "\"authors\": %s,\n" + "\"subject\": %s,\n", + json_quote_str (ctx_quote, id), + timestamp, + matched, + total, + json_quote_str (ctx_quote, authors), + json_quote_str (ctx_quote, subject)); + + talloc_free (ctx_quote); +} + static void do_search_threads (const void *ctx, + const search_format_t *format, notmuch_query_t *query, notmuch_sort_t sort) { @@ -29,7 +141,9 @@ do_search_threads (const void *ctx, notmuch_threads_t *threads; notmuch_tags_t *tags; time_t date; - const char *relative_date; + int first_thread = 1; + + fputs (format->results_start, stdout); for (threads = notmuch_query_search_threads (query); notmuch_threads_has_more (threads); @@ -37,6 +151,9 @@ do_search_threads (const void *ctx, { int first_tag = 1; + if (! first_thread) + fputs (format->thread_sep, stdout); + thread = notmuch_threads_get (threads); if (sort == NOTMUCH_SORT_OLDEST_FIRST) @@ -44,30 +161,37 @@ do_search_threads (const void *ctx, else date = notmuch_thread_get_newest_date (thread); - relative_date = notmuch_time_relative_date (ctx, date); + fputs (format->thread_start, stdout); + + format->thread (ctx, + notmuch_thread_get_thread_id (thread), + date, + notmuch_thread_get_matched_messages (thread), + notmuch_thread_get_total_messages (thread), + notmuch_thread_get_authors (thread), + notmuch_thread_get_subject (thread)); - printf ("thread:%s %12s [%d/%d] %s; %s", - notmuch_thread_get_thread_id (thread), - relative_date, - notmuch_thread_get_matched_messages (thread), - notmuch_thread_get_total_messages (thread), - notmuch_thread_get_authors (thread), - notmuch_thread_get_subject (thread)); + fputs (format->tag_start, stdout); - printf (" ("); for (tags = notmuch_thread_get_tags (thread); notmuch_tags_has_more (tags); notmuch_tags_advance (tags)) { if (! first_tag) - printf (" "); - printf ("%s", notmuch_tags_get (tags)); + fputs (format->tag_sep, stdout); + printf (format->tag, notmuch_tags_get (tags)); first_tag = 0; } - printf (")\n"); + + fputs (format->tag_end, stdout); + fputs (format->thread_end, stdout); + + first_thread = 0; notmuch_thread_destroy (thread); } + + fputs (format->results_end, stdout); } int @@ -79,6 +203,7 @@ notmuch_search_command (void *ctx, int argc, char *argv[]) char *query_str; char *opt; notmuch_sort_t sort = NOTMUCH_SORT_NEWEST_FIRST; + const search_format_t *format = &format_text; int i; for (i = 0; i < argc && argv[i][0] == '-'; i++) { @@ -96,6 +221,16 @@ notmuch_search_command (void *ctx, int argc, char *argv[]) fprintf (stderr, "Invalid value for --sort: %s\n", opt); return 1; } + } else if (STRNCMP_LITERAL (argv[i], "--output=") == 0) { + opt = argv[i] + sizeof ("--output=") - 1; + if (strcmp (opt, "text") == 0) { + format = &format_text; + } else if (strcmp (opt, "json") == 0) { + format = &format_json; + } else { + fprintf (stderr, "Invalid value for --output: %s\n", opt); + return 1; + } } else { fprintf (stderr, "Unrecognized option: %s\n", argv[i]); return 1; @@ -132,7 +267,7 @@ notmuch_search_command (void *ctx, int argc, char *argv[]) notmuch_query_set_sort (query, sort); - do_search_threads (ctx, query, sort); + do_search_threads (ctx, format, query, sort); notmuch_query_destroy (query); notmuch_database_close (notmuch); diff --git a/notmuch-show.c b/notmuch-show.c index 376aacd..b5b3eba 100644 --- a/notmuch-show.c +++ b/notmuch-show.c @@ -20,8 +20,65 @@ #include "notmuch-client.h" +typedef struct show_format { + const char *message_set_start; + const char *message_start; + void (*message) (const void *ctx, + notmuch_message_t *message, + int indent); + const char *header_start; + void (*header) (const void *ctx, + notmuch_message_t *message); + const char *header_end; + const char *body_start; + void (*part) (GMimeObject *part, + int *part_count); + const char *body_end; + const char *message_end; + const char *message_set_sep; + const char *message_set_end; +} show_format_t; + +static void +format_message_text (unused (const void *ctx), + notmuch_message_t *message, + int indent); +static void +format_headers_text (const void *ctx, + notmuch_message_t *message); +static void +format_part_text (GMimeObject *part, + int *part_count); +static const show_format_t format_text = { + "", + "\fmessage{ ", format_message_text, + "\fheader{\n", format_headers_text, "\fheader}\n", + "\fbody{\n", format_part_text, "\fbody}\n", + "\fmessage}\n", "", + "" +}; + +static void +format_message_json (const void *ctx, + notmuch_message_t *message, + unused (int indent)); +static void +format_headers_json (const void *ctx, + notmuch_message_t *message); +static void +format_part_json (GMimeObject *part, + int *part_count); +static const show_format_t format_json = { + "[", + "{", format_message_json, + ", \"headers\": {", format_headers_json, "}", + ", \"body\": [", format_part_json, "]", + "}", ", ", + "]" +}; + static const char * -_get_tags_as_string (void *ctx, notmuch_message_t *message) +_get_tags_as_string (const void *ctx, notmuch_message_t *message) { notmuch_tags_t *tags; int first = 1; @@ -48,7 +105,7 @@ _get_tags_as_string (void *ctx, notmuch_message_t *message) /* Get a nice, single-line summary of message. */ static const char * -_get_one_line_summary (void *ctx, notmuch_message_t *message) +_get_one_line_summary (const void *ctx, notmuch_message_t *message) { const char *from; time_t date; @@ -67,18 +124,87 @@ _get_one_line_summary (void *ctx, notmuch_message_t *message) } static void -show_part_content (GMimeObject *part) +format_message_text (unused (const void *ctx), notmuch_message_t *message, int indent) +{ + printf ("id:%s depth:%d match:%d filename:%s\n", + notmuch_message_get_message_id (message), + indent, + notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH), + notmuch_message_get_filename (message)); +} + +static void +format_message_json (const void *ctx, notmuch_message_t *message, unused (int indent)) +{ + void *ctx_quote = talloc_new (ctx); + + printf ("\"id\": %s, \"match\": %s, \"filename\": %s", + json_quote_str (ctx_quote, notmuch_message_get_message_id (message)), + notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH) ? "true" : "false", + json_quote_str (ctx_quote, notmuch_message_get_filename (message))); + + talloc_free (ctx_quote); +} + +static void +format_headers_text (const void *ctx, notmuch_message_t *message) +{ + const char *headers[] = { + "Subject", "From", "To", "Cc", "Bcc", "Date" + }; + const char *name, *value; + unsigned int i; + + printf ("%s\n", _get_one_line_summary (ctx, message)); + + for (i = 0; i < ARRAY_SIZE (headers); i++) { + name = headers[i]; + value = notmuch_message_get_header (message, name); + if (value) + printf ("%s: %s\n", name, value); + } +} + +static void +format_headers_json (const void *ctx, notmuch_message_t *message) +{ + const char *headers[] = { + "Subject", "From", "To", "Cc", "Bcc", "Date" + }; + const char *name, *value; + unsigned int i; + int first_header = 1; + void *ctx_quote = talloc_new (ctx); + + for (i = 0; i < ARRAY_SIZE (headers); i++) { + name = headers[i]; + value = notmuch_message_get_header (message, name); + if (value) + { + if (!first_header) + fputs (", ", stdout); + first_header = 0; + + printf ("%s: %s", + json_quote_str (ctx_quote, name), + json_quote_str (ctx_quote, value)); + } + } + + talloc_free (ctx_quote); +} + +static void +show_part_content (GMimeObject *part, GMimeStream *stream_out) { - GMimeStream *stream_stdout = g_mime_stream_file_new (stdout); GMimeStream *stream_filter = NULL; GMimeDataWrapper *wrapper; const char *charset; charset = g_mime_object_get_content_type_parameter (part, "charset"); - if (stream_stdout) { - g_mime_stream_file_set_owner (GMIME_STREAM_FILE (stream_stdout), FALSE); - stream_filter = g_mime_stream_filter_new(stream_stdout); + if (stream_out) { + stream_filter = g_mime_stream_filter_new(stream_out); g_mime_stream_filter_add(GMIME_STREAM_FILTER(stream_filter), g_mime_filter_crlf_new(FALSE, FALSE)); if (charset) { @@ -92,15 +218,16 @@ show_part_content (GMimeObject *part) g_mime_data_wrapper_write_to_stream (wrapper, stream_filter); if (stream_filter) g_object_unref(stream_filter); - if (stream_stdout) - g_object_unref(stream_stdout); } static void -show_part (GMimeObject *part, int *part_count) +format_part_text (GMimeObject *part, int *part_count) { GMimeContentDisposition *disposition; GMimeContentType *content_type; + GMimeStream *stream_stdout = g_mime_stream_file_new (stdout); + + g_mime_stream_file_set_owner (GMIME_STREAM_FILE (stream_stdout), FALSE); disposition = g_mime_object_get_content_disposition (part); if (disposition && @@ -118,11 +245,14 @@ show_part (GMimeObject *part, int *part_count) if (g_mime_content_type_is_type (content_type, "text", "*") && !g_mime_content_type_is_type (content_type, "text", "html")) { - show_part_content (part); + show_part_content (part, stream_stdout); } printf ("\fattachment}\n"); + if (stream_stdout) + g_object_unref(stream_stdout); + return; } @@ -135,7 +265,7 @@ show_part (GMimeObject *part, int *part_count) if (g_mime_content_type_is_type (content_type, "text", "*") && !g_mime_content_type_is_type (content_type, "text", "html")) { - show_part_content (part); + show_part_content (part, stream_stdout); } else { @@ -144,57 +274,93 @@ show_part (GMimeObject *part, int *part_count) } printf ("\fpart}\n"); + + if (stream_stdout) + g_object_unref(stream_stdout); } static void -show_message (void *ctx, notmuch_message_t *message, int indent) +format_part_json (GMimeObject *part, int *part_count) { - const char *headers[] = { - "Subject", "From", "To", "Cc", "Bcc", "Date" - }; - const char *name, *value; - unsigned int i; + GMimeContentType *content_type; + GMimeContentDisposition *disposition; + void *ctx = talloc_new (NULL); + GMimeStream *stream_memory = g_mime_stream_mem_new (); + GByteArray *part_content; - printf ("\fmessage{ id:%s depth:%d match:%d filename:%s\n", - notmuch_message_get_message_id (message), - indent, - notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH), - notmuch_message_get_filename (message)); + content_type = g_mime_object_get_content_type (GMIME_OBJECT (part)); - printf ("\fheader{\n"); + if (*part_count > 1) + fputs (", ", stdout); - printf ("%s\n", _get_one_line_summary (ctx, message)); + printf ("{\"id\": %d, \"content-type\": %s", + *part_count, + json_quote_str (ctx, g_mime_content_type_to_string (content_type))); - for (i = 0; i < ARRAY_SIZE (headers); i++) { - name = headers[i]; - value = notmuch_message_get_header (message, name); - if (value) - printf ("%s: %s\n", name, value); + disposition = g_mime_object_get_content_disposition (part); + if (disposition && + strcmp (disposition->disposition, GMIME_DISPOSITION_ATTACHMENT) == 0) + { + const char *filename = g_mime_part_get_filename (GMIME_PART (part)); + + printf (", \"filename\": %s", json_quote_str (ctx, filename)); } - printf ("\fheader}\n"); - printf ("\fbody{\n"); + if (g_mime_content_type_is_type (content_type, "text", "*") && + !g_mime_content_type_is_type (content_type, "text", "html")) + { + show_part_content (part, stream_memory); + part_content = g_mime_stream_mem_get_byte_array (GMIME_STREAM_MEM (stream_memory)); + + printf (", \"content\": %s", json_quote_str (ctx, (char *) part_content->data)); + } + + fputs ("}", stdout); + + talloc_free (ctx); + if (stream_memory) + g_object_unref (stream_memory); +} + +static void +show_message (void *ctx, const show_format_t *format, notmuch_message_t *message, int indent) +{ + fputs (format->message_start, stdout); + format->message(ctx, message, indent); - show_message_body (notmuch_message_get_filename (message), show_part); + fputs (format->header_start, stdout); + format->header(ctx, message); + fputs (format->header_end, stdout); - printf ("\fbody}\n"); + fputs (format->body_start, stdout); + show_message_body (notmuch_message_get_filename (message), format->part); + fputs (format->body_end, stdout); - printf ("\fmessage}\n"); + fputs (format->message_end, stdout); } static void -show_messages (void *ctx, notmuch_messages_t *messages, int indent, +show_messages (void *ctx, const show_format_t *format, notmuch_messages_t *messages, int indent, notmuch_bool_t entire_thread) { notmuch_message_t *message; notmuch_bool_t match; + int first_set = 1; int next_indent; + fputs (format->message_set_start, stdout); + for (; notmuch_messages_has_more (messages); notmuch_messages_advance (messages)) { + if (!first_set) + fputs (format->message_set_sep, stdout); + first_set = 0; + + fputs (format->message_set_start, stdout); + message = notmuch_messages_get (messages); match = notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH); @@ -202,15 +368,21 @@ show_messages (void *ctx, notmuch_messages_t *messages, int indent, next_indent = indent; if (match || entire_thread) { - show_message (ctx, message, indent); + show_message (ctx, format, message, indent); next_indent = indent + 1; + + fputs (format->message_set_sep, stdout); } - show_messages (ctx, notmuch_message_get_replies (message), + show_messages (ctx, format, notmuch_message_get_replies (message), next_indent, entire_thread); notmuch_message_destroy (message); + + fputs (format->message_set_end, stdout); } + + fputs (format->message_set_end, stdout); } int @@ -223,15 +395,29 @@ notmuch_show_command (void *ctx, unused (int argc), unused (char *argv[])) notmuch_thread_t *thread; notmuch_messages_t *messages; char *query_string; + char *opt; + const show_format_t *format = &format_text; int entire_thread = 0; int i; + int first_toplevel = 1; for (i = 0; i < argc && argv[i][0] == '-'; i++) { if (strcmp (argv[i], "--") == 0) { i++; break; } - if (strcmp(argv[i], "--entire-thread") == 0) { + if (STRNCMP_LITERAL (argv[i], "--output=") == 0) { + opt = argv[i] + sizeof ("--output=") - 1; + if (strcmp (opt, "text") == 0) { + format = &format_text; + } else if (strcmp (opt, "json") == 0) { + format = &format_json; + entire_thread = 1; + } else { + fprintf (stderr, "Invalid value for --output: %s\n", opt); + return 1; + } + } else if (STRNCMP_LITERAL (argv[i], "--entire-thread") == 0) { entire_thread = 1; } else { fprintf (stderr, "Unrecognized option: %s\n", argv[i]); @@ -268,6 +454,8 @@ notmuch_show_command (void *ctx, unused (int argc), unused (char *argv[])) return 1; } + fputs (format->message_set_start, stdout); + for (threads = notmuch_query_search_threads (query); notmuch_threads_has_more (threads); notmuch_threads_advance (threads)) @@ -280,11 +468,18 @@ notmuch_show_command (void *ctx, unused (int argc), unused (char *argv[])) INTERNAL_ERROR ("Thread %s has no toplevel messages.\n", notmuch_thread_get_thread_id (thread)); - show_messages (ctx, messages, 0, entire_thread); + if (!first_toplevel) + fputs (format->message_set_sep, stdout); + first_toplevel = 0; + + show_messages (ctx, format, messages, 0, entire_thread); notmuch_thread_destroy (thread); + } + fputs (format->message_set_end, stdout); + notmuch_query_destroy (query); notmuch_database_close (notmuch); diff --git a/notmuch.c b/notmuch.c index 2ac8a59..aa2fc12 100644 --- a/notmuch.c +++ b/notmuch.c @@ -162,6 +162,11 @@ command_t commands[] = { "\n" "\t\tSupported options for search include:\n" "\n" + "\t\t--output=(json|text)\n" + "\n" + "\t\t\tPresents the results in either JSON or plain-text\n" + "\t\t\tformat, which is the default.\n" + "\n" "\t\t--sort=(newest-first|oldest-first)\n" "\n" "\t\t\tPresent results in either chronological order\n" @@ -186,13 +191,18 @@ command_t commands[] = { "\t\t\tall messages in the same thread as any matched\n" "\t\t\tmessage will be displayed.\n" "\n" - "\t\tThe output format is plain-text, with all text-content\n" - "\t\tMIME parts decoded. Various components in the output,\n" - "\t\t('message', 'header', 'body', 'attachment', and MIME 'part')\n" - "\t\tare delimited by easily-parsed markers. Each marker consists\n" - "\t\tof a Control-L character (ASCII decimal 12), the name of\n" - "\t\tthe marker, and then either an opening or closing brace,\n" - "\t\t'{' or '}' to either open or close the component.\n" + "\t\t--output=(json|text)\n" + "\n" + "\t\t\tPresents the results in either JSON or plain-text\n" + "\t\t\tformat, which is the default.\n" + "\n" + "\t\tThe plain-text has all text-content MIME parts decoded.\n" + "\t\tVarious components in the output, ('message', 'header',\n" + "\t\t'body', 'attachment', and MIME 'part') are delimited by\n" + "\t\teasily-parsed markers. Each marker consists of a Control-L\n" + "\t\tcharacter (ASCII decimal 12), the name of the marker, and\n" + "\t\tthen either an opening or closing brace, '{' or '}' to\n" + "\t\teither open or close the component.\n" "\n" "\t\tA common use of \"notmuch show\" is to display a single\n" "\t\tthread of email messages. For this, use a search term of\n" diff --git a/show-message.c b/show-message.c index 784981b..05ced9c 100644 --- a/show-message.c +++ b/show-message.c @@ -26,8 +26,6 @@ static void show_message_part (GMimeObject *part, int *part_count, void (*show_part) (GMimeObject *part, int *part_count)) { - *part_count = *part_count + 1; - if (GMIME_IS_MULTIPART (part)) { GMimeMultipart *multipart = GMIME_MULTIPART (part); int i; @@ -56,6 +54,8 @@ show_message_part (GMimeObject *part, int *part_count, return; } + *part_count = *part_count + 1; + (*show_part) (part, part_count); } -- 1.6.5.3 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] Add an "--output=(json|text|)" command-line option to both notmuch-search and notmuch-show. 2009-12-18 12:59 ` [PATCH] Add an "--output=(json|text|)" command-line option to both " david @ 2010-02-23 19:56 ` Carl Worth 2010-02-23 21:00 ` JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...] Jameson Rollins 0 siblings, 1 reply; 5+ messages in thread From: Carl Worth @ 2010-02-23 19:56 UTC (permalink / raw To: david, notmuch [-- Attachment #1: Type: text/plain, Size: 3222 bytes --] On Fri, 18 Dec 2009 08:59:55 -0400, david@tethera.net wrote: > From: Scott Robinson <scott@quadhome.com> Hi Scott (and David), I'm finally getting around to looking closely at the JSON patches (hurrah!). Here are some comments: > In the case of notmuch-show, "--output=json" also implies > "--entire-thread" as the thread structure is implicit in the emitted > document tree. I think we've all agreed to use --output for selecting what to print and to instead use --format for how to format it. I also think David's got a patch to change that, but if we have to change the current patch anyway, we might change that at the same time. > --- /dev/null > +++ b/json.c > @@ -0,0 +1,73 @@ > +/* notmuch - Not much of an email program, (just index and search) > + * > + * Copyright © 2009 Carl Worth > + * Copyright © 2009 Keith Packard I know I didn't contribute any code to this file, so my name shouldn't be here. Scott, I imagine you have made some non-trivial contributions so your name (or suitable copyright-claiming entity) should be here. > + * Authors: Carl Worth <cworth@cworth.org> > + * Keith Packard <keithp@keithp.com> Same thing here. > +/* > + * json_quote_str derived from cJSON's print_string_ptr, > + * Copyright (c) 2009 Dave Gamble > + */ Thanks for attributing the source here, but let's please keep all the copyright statements up at the top of the file. It would still be reasonable to have a comment at this point that this particular code came from Dave Gamble and cJSON. But it should also mention the license under which the code is being integrated. I suggest leaving the notmuch-standar GPLv3+ blurb at the top of the file, but then quoting the license itself of the external code, (it's a short MIT-style license, right?). > +char * > +json_quote_str(const void *ctx, const char *str) It would be nice to have a little comment here describing what the function does, (what characters get quoted and how, that the return value is talloced with 'ctx' as a parent, and perhaps a pointer to the appropriate JSON reference/specification). But I definitely like this nice little function as opposed to some JSON library. Thanks! > +typedef struct search_format { > + const char *results_start; > + const char *thread_start; > + void (*thread) (const void *ctx, > + const char *id, > + const time_t date, > + const int matched, > + const int total, > + const char *authors, > + const char *subject); ... Definitely missing at least a quick comment for internal documentation here as well. But I do like the way this works. > --- a/notmuch.c > +++ b/notmuch.c > @@ -162,6 +162,11 @@ command_t commands[] = { > "\n" > "\t\tSupported options for search include:\n" > "\n" > + "\t\t--output=(json|text)\n" > + "\n" > + "\t\t\tPresents the results in either JSON or plain-text\n" > + "\t\t\tformat, which is the default.\n" > + "\n" Thanks for adding the documentation here. But please add to notmuch.1 as well. This all looks really great. And I can't wait to apply it and play with it more. -Carl [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...] 2010-02-23 19:56 ` Carl Worth @ 2010-02-23 21:00 ` Jameson Rollins 2010-02-23 23:35 ` Carl Worth 0 siblings, 1 reply; 5+ messages in thread From: Jameson Rollins @ 2010-02-23 21:00 UTC (permalink / raw To: Carl Worth, david, notmuch [-- Attachment #1: Type: text/plain, Size: 1873 bytes --] Hey, Carl. I mentioned this is an email a while back (id:87eil4ygar.fsf@servo.finestructure.net), but I want to bring it up again now that you're looking at the JSON output stuff. I would like to make the case for JSON being the one and only output format. Here's the arguments: * JSON is a very well spec'd API * JSON is a *very* well supported standard (it's basically well supported by every language) * the JSONformat is perfectly suited for notmuch's output * emacs has a json.el library, so it would not be hard at all to modify the notmuch.el to parse json output * using only one output format would considerably reduce the code base in notmuch, keeping things simpler and easier to maintain I think these are pretty good reason to just move to JSON as *the* output format. While I'm on this topic, let me rehash another point I made previously: If we move to JSON output, it would be *so* sweet to provide means to filter the output JSON fields. The "--output" flag could be used to pick the desired output fields. For instance, the current "search" output could be something like: search --output=thread_id,date,number,author,subject,tags while the current "show" output could be something like: search --output=message_id,tags,path,header,body,attachments This would all make things *much* easier for clients and wrappers. For instance, here are just a couple of things that would be made much easier for wrapper scripts: * Proper maildir sync ("search --output=message_id,tags,path" ...) * Purging "delete" tagged messages ("search --output=path" tags:delete) * Moving/archiving messages based on search results ("search --output=path" ...) I would personally love to see this. I think it would make notmuch cleaner, simpler, and more powerful all at the same time. Interested in hearing what other's thoughts on this idea are. jamie. [-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...] 2010-02-23 21:00 ` JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...] Jameson Rollins @ 2010-02-23 23:35 ` Carl Worth 2010-02-24 13:54 ` Sebastian Spaeth 0 siblings, 1 reply; 5+ messages in thread From: Carl Worth @ 2010-02-23 23:35 UTC (permalink / raw To: Jameson Rollins, david, notmuch [-- Attachment #1: Type: text/plain, Size: 1908 bytes --] On Tue, 23 Feb 2010 16:00:01 -0500, Jameson Rollins <jrollins@finestructure.net> wrote: > Hey, Carl. I mentioned this is an email a while back > (id:87eil4ygar.fsf@servo.finestructure.net), but I want to bring it up > again now that you're looking at the JSON output stuff. I would like to > make the case for JSON being the one and only output format. Here's the > arguments: Oh, I'm definitely in favor of killing the %message{ format. As soon as we get the emacs and vi frontends switched to the json output, then that old thing can be eliminated. > * using only one output format would considerably reduce the code base > in notmuch, keeping things simpler and easier to maintain But I actually still want text-only output. It will be even less "parseable" than the current output, (no delimiters at all), but also have some niceties like thread indentation, etc. The idea there is to support direct, human-readable output for quick command-line use. (It also means that notmuch provides a much gentler introduction to people just experimenting with it). And I'm willing to keep the code base just a "little" bit complicated for that. > If we move to JSON output, it would be *so* sweet to provide means to > filter the output JSON fields. The "--output" flag could be used to > pick the desired output fields. For instance, the current "search" > output could be something like: I definitely want to enable the user to select which fields are printed. But I don't see how that depends on JSON output. In fact, for easy shell scripts, etc. I want to be able to do search that give me filenames, email addresses, etc. without any delimiters at all. I definitely want to be able to pipe single-field lists coming from notmuch to grep, xargs, shell for loops, etc. without having to decode JSON. So I'm still in favor of having a "plain text" output going forward, (and by default). -Carl [-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...] 2010-02-23 23:35 ` Carl Worth @ 2010-02-24 13:54 ` Sebastian Spaeth 0 siblings, 0 replies; 5+ messages in thread From: Sebastian Spaeth @ 2010-02-24 13:54 UTC (permalink / raw To: notmuch > I definitely want to be able to pipe single-field lists coming from > notmuch to grep, xargs, shell for loops, etc. without having to decode > JSON. While I would love to see JSON (even by default), I agree. If I just want to code up a notmuch-based address book with sth like: notmuch show to:Diana --output=to --sort=relevance --limit=20 just getting back a plain list of mail addresses is the easiest to handle. ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2010-02-24 14:56 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- [not found] <1257161639.5014171267021461605.JavaMail.root@zimbra1-e1.priv.proxad.net> 2010-02-24 14:27 ` JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...] racin 2010-02-24 14:56 ` Jameson Rollins 2009-12-18 5:33 [PATCH] JSON output for notmuch-search and notmuch-show Scott Robinson 2009-12-18 12:59 ` [PATCH] Add an "--output=(json|text|)" command-line option to both " david 2010-02-23 19:56 ` Carl Worth 2010-02-23 21:00 ` JSON output as default [was: Re: [PATCH] Add an "--output=(json|text|)" command-line option...] Jameson Rollins 2010-02-23 23:35 ` Carl Worth 2010-02-24 13:54 ` Sebastian Spaeth
Code repositories for project(s) associated with this public inbox https://yhetil.org/notmuch.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).