[PATCH 0/3] Outline fix for emacs tagging race

[PATCH 0/3] Outline fix for emacs tagging race

[Mark Walters]

For a long time [1] there have been two related races in tagging from
the search buffer.

The first is that when tagging (including archiving) a thread message
which arrived after the buffer was created may also be tagged. This is
because the tagging is done based on the thread-id not on the
individual messages.

The second is when using the '*' command to tag all messages. This is
not quite the same as this command only tags messages matching the
query not all messages in all threads that contain a message matching
the query. Thus if more messages now match than when the buffer was
created (eg some external tagging script has run) then this command
can unexpectedly tag these messages too.

One solution that was discussed in [2] was for the search output of
notmuch to include the message-ids of both matching and non-matching
messages. At that time that was difficult to implement as it was
unclear how to escape the message ids when using the text
format. Since emacs now uses JSON for search mode this problem is
solved.

This patch series implements the above mentioned solution and seems to
work except for one problem.

Since emacs now tags each message in a thread in the search buffer it
is easy to ask it to tag a lot of messages. This could be done
individually which would be ridiculously slow so instead they are all
done in one batch. But now it is relatively easy to take notmuch over
the threshold for ARG_MAX.

In [3] Tomi did some experiments and found on a standard Debian system
with getconf ARG_MAX =131072 that command lines with 10000 200 byte
arguments worked. I am a little puzzled by that as I get the same
results and I getconf ARG_MAX gives 2097152 for me.

More importantly though, when trying to execute a command from emacs I
am finding that 131072 is the limit on the command length in bytes and
we can hit this with something around 1500 messages (see end for a
very hacky emacs test script). This is probably more than we can
expect in a single thread so tagging from show is probably safe but it
does cause a problem when tagging from search.

I can think of several possible solutions (e.g., batch it in my new
stuff, put some batching in notmuch-tag, all notmuch tag to read a
query from stdin). But before any larger more intrusive change do
people like the general approach? Does anyone have a good way to get
round the command line size problem?

Best wishes 

Mark


[1] id:87ocmtg9ni.fsf@yoom.home.cworth.org
[2] id:CAH-f9WticM4EN8F1_ik_-mcBcBtrXwSpO+Drbtp7=UN7McECrg@mail.gmail.com
[3] id:m2liody7av.fsf@guru.guru-group.fi

Mark Walters (3):
  test: test for race when tagging from emacs search
  cli: all search mode to include msg-ids with JSON output
  emacs: make emacs use message-ids for tagging

 emacs/notmuch.el |   22 ++++++++++++++++++++--
 notmuch-search.c |   40 ++++++++++++++++++++++++++++++++++++++--
 test/emacs       |   21 +++++++++++++++++++++
 3 files changed, 79 insertions(+), 4 deletions(-)


TEST SCRIPT

(progn
  (setq bigstring "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
  (setq n 1310)
  (setq big nil)
  (while (> n 0)
    (setq n (1- n))
    (setq big (concat big (format "%s" n) " " bigstring)))
  (call-process "echo" nil t nil big))



-- 
1.7.9.1

[PATCH 1/3] test: test for race when tagging from emacs search

[Mark Walters]

When tagging from search view in emacs there is a race condition: it
tags all messages in the thread even ones which arrived after the
search was made. This can cause dataloss (if, for example, a thread is
archived it could archive messages the user has never seen).

Mark this test known broken.
---
 test/emacs |   23 +++++++++++++++++++++++
 1 files changed, 23 insertions(+), 0 deletions(-)

diff --git a/test/emacs b/test/emacs
index 44f641e..6246177 100755
--- a/test/emacs
+++ b/test/emacs
@@ -122,6 +122,29 @@ test_emacs "(notmuch-search \"$os_x_darwin_thread\")
 output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)"
 
+test_begin_subtest "Tag all matching messages from search view"
+test_subtest_known_broken
+notmuch tag +test-tag-race from:cworth
+test_emacs "(notmuch-search \"tag:test-tag-race\")
+	    (notmuch-test-wait)"
+notmuch tag +test-tag-race "id:1258471718-6781-2-git-send-email-dottedmag@dottedmag.net"
+test_emacs "(execute-kbd-macro \"*+test-tag-race-2\")"
+output=$(notmuch count tag:test-tag-race-2)
+test_expect_equal "$output" "12"
+notmuch tag -test-tag-race '*'
+notmuch tag -test-tag-race-2 '*'
+
+test_begin_subtest "Change tags from search view: another message arriving after thread lookup"
+test_subtest_known_broken
+typsos_id="878we4qdqf.fsf@yoom.home.cworth.org"
+typsos_thread=$(notmuch search --output=threads id:$typsos_id)
+test_emacs "(notmuch-search \"$typsos_thread\")
+	    (notmuch-test-wait)"
+add_message "[subject]=\"new-thread-message\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"new-thread-message\"" "[in-reply-to]=\"<$typsos_id>\""
+test_emacs "(execute-kbd-macro \"+tag-from-search-view -unread\")"
+output=$(notmuch search tag:tag-from-search-view | notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2009-11-18 [2/3] Ingmar Vanhassel, Carl Worth| Notmuch Test Suite; [notmuch] [PATCH] Typsos (inbox tag-from-search-view unread)"
+
 test_begin_subtest "Add tag from notmuch-show view"
 test_emacs "(notmuch-show \"$os_x_darwin_thread\")
 	    (execute-kbd-macro \"+tag-from-show-view\")"
-- 
1.7.9.1

[PATCH 2/3] cli: all search mode to include msg-ids with JSON output

[Mark Walters]

This adds a --output=with-ids option which gives similar output to the
normal search summary output but with a list of message ids
too. Currently this is not implemented for text format.
---
 notmuch-search.c |   40 ++++++++++++++++++++++++++++++++++++++--
 1 files changed, 38 insertions(+), 2 deletions(-)

diff --git a/notmuch-search.c b/notmuch-search.c
index 830c4e4..82e168c 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -26,7 +26,8 @@ typedef enum {
     OUTPUT_THREADS,
     OUTPUT_MESSAGES,
     OUTPUT_FILES,
-    OUTPUT_TAGS
+    OUTPUT_TAGS,
+    OUTPUT_SUMMARY_WITH_IDS
 } output_t;
 
 static char *
@@ -46,6 +47,23 @@ sanitize_string (const void *ctx, const char *str)
     return out;
 }
 
+static void
+output_msg_ids (sprinter_t *format,
+		notmuch_bool_t matching,
+		notmuch_messages_t *messages)
+{
+    notmuch_message_t *message;
+    for (;
+	 notmuch_messages_valid (messages);
+	 notmuch_messages_move_to_next (messages))
+    {
+	message = notmuch_messages_get (messages);
+	if (notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH) == matching)
+	    format->string (format, notmuch_message_get_message_id (message));
+	output_msg_ids (format, matching, notmuch_message_get_replies (message));
+    }
+}
+
 static int
 do_search_threads (sprinter_t *format,
 		   notmuch_query_t *query,
@@ -88,7 +106,7 @@ do_search_threads (sprinter_t *format,
 	    format->string (format,
 			    notmuch_thread_get_thread_id (thread));
 	    format->separator (format);
-	} else { /* output == OUTPUT_SUMMARY */
+	} else { /* output == OUTPUT_SUMMARY or OUTPUT_SUMMARY_WITH_IDS */
 	    void *ctx_quote = talloc_new (thread);
 	    const char *authors = notmuch_thread_get_authors (thread);
 	    const char *subject = notmuch_thread_get_subject (thread);
@@ -160,6 +178,18 @@ do_search_threads (sprinter_t *format,
 		printf (")");
 
 	    format->end (format);
+
+	if (output == OUTPUT_SUMMARY_WITH_IDS) {
+	    format->map_key (format, "matching_msg_ids");
+	    format->begin_list (format);
+	    output_msg_ids (format, TRUE, notmuch_thread_get_toplevel_messages (thread));
+	    format->end (format);
+	    format->map_key (format, "nonmatching_msg_ids");
+	    format->begin_list (format);
+	    output_msg_ids (format, FALSE, notmuch_thread_get_toplevel_messages (thread));
+	    format->end (format);
+	}
+
 	    format->end (format);
 	    format->separator (format);
 	}
@@ -323,6 +353,7 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
 				  { "messages", OUTPUT_MESSAGES },
 				  { "files", OUTPUT_FILES },
 				  { "tags", OUTPUT_TAGS },
+				  { "with-ids", OUTPUT_SUMMARY_WITH_IDS },
 				  { 0, 0 } } },
         { NOTMUCH_OPT_KEYWORD, &exclude, "exclude", 'x',
           (notmuch_keyword_t []){ { "true", EXCLUDE_TRUE },
@@ -398,6 +429,11 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
 	    notmuch_query_set_omit_excluded (query, FALSE);
     }
 
+    if (output == OUTPUT_SUMMARY_WITH_IDS && format_sel == NOTMUCH_FORMAT_TEXT) {
+	fprintf (stderr, "Warning: --output=with-ids not implemented for text format.\n");
+	output = OUTPUT_SUMMARY;
+    }
+
     switch (output) {
     default:
     case OUTPUT_SUMMARY:
-- 
1.7.9.1

[PATCH 3/3] emacs: make emacs use message-ids for tagging

[Mark Walters]

This makes emacs use the new --output=with-ids in search mode and use
this for tagging.  This fixes the race condition in tagging from
search mode so mark the tests fixed.
---
 emacs/notmuch.el |   22 ++++++++++++++++++++--
 test/emacs       |    2 --
 2 files changed, 20 insertions(+), 4 deletions(-)

diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index f9454d8..49a57b2 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -492,6 +492,21 @@ If BARE is set then do not prefix with \"thread:\""
   "Return a search string for threads for the current region"
   (mapconcat 'identity (notmuch-search-find-thread-id-region beg end) " or "))
 
+(defun notmuch-search-find-msg-ids-region (beg end &optional only-matching)
+  "Return a list of msg-ids for the current region"
+  (let ((msg-ids))
+    (mapc (lambda (msg-id) (setq msg-ids (append msg-id msg-ids)))
+	    (append (notmuch-search-properties-in-region :matching_msg_ids beg end)
+		    (unless only-matching
+		      (notmuch-search-properties-in-region :nonmatching_msg_ids beg end))))
+    msg-ids))
+
+(defun notmuch-search-find-msg-ids-region-search (beg end &optional only-matching)
+  "Return a search string for msg-ids in threads for the current region"
+  (mapconcat (lambda (id) (notmuch-id-to-query id))
+	     (notmuch-search-find-msg-ids-region beg end only-matching)
+	     " or "))
+
 (defun notmuch-search-find-authors ()
   "Return the authors for the current thread"
   (plist-get (notmuch-search-get-result) :authors))
@@ -566,7 +581,7 @@ and will also appear in a buffer named \"*Notmuch errors*\"."
 
 (defun notmuch-search-tag-region (beg end &optional tag-changes)
   "Change tags for threads in the given region."
-  (let ((search-string (notmuch-search-find-thread-id-region-search beg end)))
+  (let ((search-string (notmuch-search-find-msg-ids-region-search beg end)))
     (setq tag-changes (funcall 'notmuch-tag search-string tag-changes))
     (notmuch-search-foreach-result beg end
       (lambda (pos)
@@ -842,7 +857,9 @@ non-authors is found, assume that all of the authors match."
 
 See `notmuch-tag' for information on the format of TAG-CHANGES."
   (interactive)
-  (apply 'notmuch-tag notmuch-search-query-string tag-changes))
+  (apply 'notmuch-tag (notmuch-search-find-msg-ids-region-search
+		       (point-min) (point-max) t)
+	 tag-changes))
 
 (defun notmuch-search-buffer-title (query)
   "Returns the title for a buffer with notmuch search results."
@@ -939,6 +956,7 @@ Other optional parameters are used as follows:
 		     "notmuch-search" buffer
 		     notmuch-command "search"
 		     "--format=json"
+		     "--output=with-ids"
 		     (if oldest-first
 			 "--sort=oldest-first"
 		       "--sort=newest-first")
diff --git a/test/emacs b/test/emacs
index 6246177..4102198 100755
--- a/test/emacs
+++ b/test/emacs
@@ -123,7 +123,6 @@ output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)"
 
 test_begin_subtest "Tag all matching messages from search view"
-test_subtest_known_broken
 notmuch tag +test-tag-race from:cworth
 test_emacs "(notmuch-search \"tag:test-tag-race\")
 	    (notmuch-test-wait)"
@@ -135,7 +134,6 @@ notmuch tag -test-tag-race '*'
 notmuch tag -test-tag-race-2 '*'
 
 test_begin_subtest "Change tags from search view: another message arriving after thread lookup"
-test_subtest_known_broken
 typsos_id="878we4qdqf.fsf@yoom.home.cworth.org"
 typsos_thread=$(notmuch search --output=threads id:$typsos_id)
 test_emacs "(notmuch-search \"$typsos_thread\")
-- 
1.7.9.1

Re: [PATCH 0/3] Outline fix for emacs tagging race

[Tomi Ollila]

On Fri, Nov 09 2012, Mark Walters <markwalters1009@gmail.com> wrote:

>
> This patch series implements the above mentioned solution and seems to
> work except for one problem.
>
> Since emacs now tags each message in a thread in the search buffer it
> is easy to ask it to tag a lot of messages. This could be done
> individually which would be ridiculously slow so instead they are all
> done in one batch. But now it is relatively easy to take notmuch over
> the threshold for ARG_MAX.
>
> In [3] Tomi did some experiments and found on a standard Debian system
> with getconf ARG_MAX =131072 that command lines with 10000 200 byte
> arguments worked. I am a little puzzled by that as I get the same
> results and I getconf ARG_MAX gives 2097152 for me.
>
> More importantly though, when trying to execute a command from emacs I
> am finding that 131072 is the limit on the command length in bytes and
> we can hit this with something around 1500 messages (see end for a
> very hacky emacs test script). This is probably more than we can
> expect in a single thread so tagging from show is probably safe but it
> does cause a problem when tagging from search.

That's because your test script below constructs one very loooong
string (and used just one arg). Like you said, [3] used 10000 200-char args.

This test script is more like what the perl(1) script did in [3]

(progn
  (setq arglist '("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"))
  (setq n 10000)
  (while (> n 0)
    (setq n (1- n))
    (setq arglist (cons "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" arglist)))

  (apply 'call-process "/bin/echo" nil t nil arglist))

(except that the args here are just 80 chars but you should get the point :)


I did not check your patches yet.. buf if you concatenate there you 
could try to provide those as separate args...

... then we can experiment how notmuch (and xapian) copes with thosew
IIRC notmuch will concatenate one long string out of those and do
xapian stuff using that (but then we're not limited by command line
arguments restrictions).

>
> Best wishes 
>
> Mark

Tomi

>
> [3] id:m2liody7av.fsf@guru.guru-group.fi

> TEST SCRIPT

> (progn
>   (setq bigstring
> "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
>   (setq n 1310)
>   (setq big nil)
>   (while (> n 0)
>    (setq n (1- n))
>     (setq big (concat big (format "%s" n) " " bigstring)))
>   (call-process "echo" nil t nil big))

Re: [PATCH 0/3] Outline fix for emacs tagging race

[David Bremner]

Mark Walters <markwalters1009@gmail.com> writes:

> I can think of several possible solutions (e.g., batch it in my new
> stuff, put some batching in notmuch-tag, all notmuch tag to read a
> query from stdin). But before any larger more intrusive change do
> people like the general approach? Does anyone have a good way to get
> round the command line size problem?

Jani posted some patches to do batch tagging from stdin. I guess the
last full series posted was at id:cover.1334404979.git.jani@nikula.org

I believe Jani has slightly updated (or at least rebased) the patches in
git. I reworked and re-posted some of the foundational parts of the
series at id:1345382314-5330-1-git-send-email-david@tethera.net

I want to have some of this infrastucture for a new style of
backup/restore, and it's possible that batch tagging would also speed up
nmbug.  

d

Re: [PATCH 0/3] Outline fix for emacs tagging race

[Austin Clements]

Quoth Mark Walters on Nov 09 at  6:58 pm:
> For a long time [1] there have been two related races in tagging from
> the search buffer.
> 
> The first is that when tagging (including archiving) a thread message
> which arrived after the buffer was created may also be tagged. This is
> because the tagging is done based on the thread-id not on the
> individual messages.
> 
> The second is when using the '*' command to tag all messages. This is
> not quite the same as this command only tags messages matching the
> query not all messages in all threads that contain a message matching
> the query. Thus if more messages now match than when the buffer was
> created (eg some external tagging script has run) then this command
> can unexpectedly tag these messages too.
> 
> One solution that was discussed in [2] was for the search output of
> notmuch to include the message-ids of both matching and non-matching
> messages. At that time that was difficult to implement as it was
> unclear how to escape the message ids when using the text
> format. Since emacs now uses JSON for search mode this problem is
> solved.
> 
> This patch series implements the above mentioned solution and seems to
> work except for one problem.
> 
> Since emacs now tags each message in a thread in the search buffer it
> is easy to ask it to tag a lot of messages. This could be done
> individually which would be ridiculously slow so instead they are all
> done in one batch. But now it is relatively easy to take notmuch over
> the threshold for ARG_MAX.
> 
> In [3] Tomi did some experiments and found on a standard Debian system
> with getconf ARG_MAX =131072 that command lines with 10000 200 byte
> arguments worked. I am a little puzzled by that as I get the same
> results and I getconf ARG_MAX gives 2097152 for me.
> 
> More importantly though, when trying to execute a command from emacs I
> am finding that 131072 is the limit on the command length in bytes and
> we can hit this with something around 1500 messages (see end for a
> very hacky emacs test script). This is probably more than we can
> expect in a single thread so tagging from show is probably safe but it
> does cause a problem when tagging from search.
> 
> I can think of several possible solutions (e.g., batch it in my new
> stuff, put some batching in notmuch-tag, all notmuch tag to read a
> query from stdin). But before any larger more intrusive change do
> people like the general approach? Does anyone have a good way to get
> round the command line size problem?
> 
> Best wishes 
> 
> Mark
> 
> 
> [1] id:87ocmtg9ni.fsf@yoom.home.cworth.org
> [2] id:CAH-f9WticM4EN8F1_ik_-mcBcBtrXwSpO+Drbtp7=UN7McECrg@mail.gmail.com
> [3] id:m2liody7av.fsf@guru.guru-group.fi

I'm glad to see someone picking up this bug.  Besides somehow dealing
with long command-lines, when we were last exploring this race, I had
found that it was 3-4x more efficient to use Xapian document IDs
directly rather than message IDs [1].  It's probably best *not* to do
this initially for the sake of simplicity, but I think a simple tweak
to your approach would let us seamlessly transition to this in the
future.  Rather than extending the JSON output with what are
explicitly message IDs, instead extend the output with opaque queries
that are guaranteed to match the matching/non-matching messages in the
thread and are guaranteed to be combinable, but aren't guaranteed to
be of any particular form.  For now, the CLI can simply output id:
queries for these, but in the future we could easily add a special
query syntax for docid queries or, if we ever move to a custom query
parser, support docids in any query.

For the long command line problem, one easy solution is to support,
say, '-' as a query syntax that means "read the query from stdin."
This would be a simple addition to query_string_from_args and would
work across the CLI.

[1] id:CAH-f9WsPj=1Eu=g3sOePJgCTBFs6HrLdLq18xMEnJ8aZ00yCEg@mail.gmail.com

Re: [PATCH 2/3] cli: all search mode to include msg-ids with JSON output

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 977 bytes --]

On Fri, Nov 09 2012, Mark Walters <markwalters1009@gmail.com> wrote:
> This adds a --output=with-ids option which gives similar output to the
> normal search summary output but with a list of message ids
> too. Currently this is not implemented for text format.

Hey, Mark.  Very nit-picky comment here, but I'm not sure "with-ids" is
an appropriate name for an output type.  "with-ids" sounds like a
modifier, as opposed to a output type unto itself.

But I wonder if this separate output type is really necessary.  Can the
emacs interface just make two separate search calls to the binary when
constructing the buffer, one with --output=summary and one with
--output=messages?  Wouldn't that provide all the needed info?  I guess
there would still be a race condition, especially for really long search
results, but I wonder if the calls could actually be made in parallel at
the same time.  Maybe that would require more work.  Sorry, just
thinking out loud here...

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 2/3] cli: all search mode to include msg-ids with JSON output

[Michal Nazarewicz]

[-- Attachment #1: Type: text/plain, Size: 640 bytes --]

On Thu, Nov 22 2012, Jameson Graef Rollins wrote:
> I guess there would still be a race condition, especially for really
> long search results, but I wonder if the calls could actually be made
> in parallel at the same time.

If we are nitpicking, strictly speaking, running things in parallel does
not solve race conditions (at best it may make them less likely).

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michał “mina86” Nazarewicz    (o o)
ooo +----<email/xmpp: mpn@google.com>--------------ooO--(_)--Ooo--

[-- Attachment #2.1: Type: text/plain, Size: 0 bytes --]



[-- Attachment #2.2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 2/3] cli: all search mode to include msg-ids with JSON output

[Mark Walters]

Hi

On Thu, 22 Nov 2012, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Fri, Nov 09 2012, Mark Walters <markwalters1009@gmail.com> wrote:
>> This adds a --output=with-ids option which gives similar output to the
>> normal search summary output but with a list of message ids
>> too. Currently this is not implemented for text format.
>
> Hey, Mark.  Very nit-picky comment here, but I'm not sure "with-ids" is
> an appropriate name for an output type.  "with-ids" sounds like a
> modifier, as opposed to a output type unto itself.

Yes that is true: I am very happy for any suggestions. Perhaps
summary-with-ids?

> But I wonder if this separate output type is really necessary.  Can the
> emacs interface just make two separate search calls to the binary when
> constructing the buffer, one with --output=summary and one with
> --output=messages?  Wouldn't that provide all the needed info?  I guess
> there would still be a race condition, especially for really long search
> results, but I wonder if the calls could actually be made in parallel at
> the same time.  Maybe that would require more work.  Sorry, just
> thinking out loud here...

As you say this doesn't fully solve the race. But more importantly there
are two races: one for * (apply tag change to all matching messages) and
one for tagging singles threads. I don't see how this would help with
the latter. In my use this is the worrying race: I archive a thread to
say I have dealt with it but I may archive a reply which arrived after I
populated the search buffer.

Best wishes

Mark

Re: [PATCH 2/3] cli: all search mode to include msg-ids with JSON output

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 878 bytes --]

On Thu, Nov 22 2012, Mark Walters <markwalters1009@gmail.com> wrote:
> On Thu, 22 Nov 2012, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
>> On Fri, Nov 09 2012, Mark Walters <markwalters1009@gmail.com> wrote:
>>> This adds a --output=with-ids option which gives similar output to the
>>> normal search summary output but with a list of message ids
>>> too. Currently this is not implemented for text format.
>>
>> Hey, Mark.  Very nit-picky comment here, but I'm not sure "with-ids" is
>> an appropriate name for an output type.  "with-ids" sounds like a
>> modifier, as opposed to a output type unto itself.
>
> Yes that is true: I am very happy for any suggestions. Perhaps
> summary-with-ids?

"summary-with-ids" seems fine with me.  This is something that will
probably only be called by other programs, so having the name be longer
seems fine with me.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 2/3] cli: all search mode to include msg-ids with JSON output

[Tomi Ollila]

On Fri, Nov 23 2012, Jameson Graef Rollins <jrollins@finestructure.net> wrote:

> On Thu, Nov 22 2012, Mark Walters <markwalters1009@gmail.com> wrote:
>> On Thu, 22 Nov 2012, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
>>> On Fri, Nov 09 2012, Mark Walters <markwalters1009@gmail.com> wrote:
>>>> This adds a --output=with-ids option which gives similar output to the
>>>> normal search summary output but with a list of message ids
>>>> too. Currently this is not implemented for text format.
>>>
>>> Hey, Mark.  Very nit-picky comment here, but I'm not sure "with-ids" is
>>> an appropriate name for an output type.  "with-ids" sounds like a
>>> modifier, as opposed to a output type unto itself.
>>
>> Yes that is true: I am very happy for any suggestions. Perhaps
>> summary-with-ids?
>
> "summary-with-ids" seems fine with me.  This is something that will
> probably only be called by other programs, so having the name be longer
> seems fine with me.

Or “summary-with-queries” -- to take Austin's suggestion into account ?

> jamie.

Tomi

Re: [PATCH 2/3] cli: all search mode to include msg-ids with JSON output

[Austin Clements]

Quoth Jameson Graef Rollins on Nov 22 at  8:15 am:
> On Fri, Nov 09 2012, Mark Walters <markwalters1009@gmail.com> wrote:
> > This adds a --output=with-ids option which gives similar output to the
> > normal search summary output but with a list of message ids
> > too. Currently this is not implemented for text format.
> 
> Hey, Mark.  Very nit-picky comment here, but I'm not sure "with-ids" is
> an appropriate name for an output type.  "with-ids" sounds like a
> modifier, as opposed to a output type unto itself.

In fact, it sounds so much like a modifier that I wonder if it should
be implemented as a modifier.  This isn't the first time that we've
wanted to include something in the JSON output but make it optional
for performance reasons.  For example, show has a boolean --body
argument for this reason.  What if this wasn't a new type of output,
but simply a new --ids=true (or --queries=true) argument?