search summary and exclusions

search summary and exclusions

[Peter Wang]

Hi,

Exclusions don't work the way I expected with the search command and
--output=summary.  I would like messages with excluded tags to be
treated as if they don't exist at all, but currently:

  * excluded messages are counted towards the "total"
  * excluded tags are included in the "tags" set

Are these deliberate?  Especially the second point seems to conflict
with search --output=tags, which doesn't show excluded tags.

Peter

Re: search summary and exclusions

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 987 bytes --]

On Mon, May 28 2012, Peter Wang <novalazy@gmail.com> wrote:
> Exclusions don't work the way I expected with the search command and
> --output=summary.  I would like messages with excluded tags to be
> treated as if they don't exist at all, but currently:
>
>   * excluded messages are counted towards the "total"
>   * excluded tags are included in the "tags" set
>
> Are these deliberate?  Especially the second point seems to conflict
> with search --output=tags, which doesn't show excluded tags.

Hi, Peter.  Can you explain more what you mean?  I'm not sure what your
two bullets have to do with --output=summary.  Messages are counted with
the count subcommand, and tags are shown with search --output=tags,
neither of which have anything to do with --output=summary, and both of
which accept the exclude flag:

notmuch count --exclude=true '*'
notmuch search --output=tags --exclude=true '*'

Maybe you can give a clearer explanation of what your issue is.
Examples help.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: search summary and exclusions

[Peter Wang]

On Mon, 28 May 2012 10:03:35 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Mon, May 28 2012, Peter Wang <novalazy@gmail.com> wrote:
> > Exclusions don't work the way I expected with the search command and
> > --output=summary.  I would like messages with excluded tags to be
> > treated as if they don't exist at all, but currently:
> >
> >   * excluded messages are counted towards the "total"
> >   * excluded tags are included in the "tags" set
> >
> > Are these deliberate?  Especially the second point seems to conflict
> > with search --output=tags, which doesn't show excluded tags.
> 
> Hi, Peter.  Can you explain more what you mean?  I'm not sure what your
> two bullets have to do with --output=summary.  Messages are counted with
> the count subcommand, and tags are shown with search --output=tags,
> neither of which have anything to do with --output=summary, and both of
> which accept the exclude flag:
> 
> notmuch count --exclude=true '*'
> notmuch search --output=tags --exclude=true '*'
> 
> Maybe you can give a clearer explanation of what your issue is.
> Examples help.

Sure.  I keep draft messages in the mail store, and tag them with
'draft', and later 'deleted'.  I would like the exclusions to treat
draft and deleted messages as if they didn't exist in the store.

% ./notmuch search --format=json --exclude=true -- thread:0000000000009598 tag:unread
[{"thread": "0000000000009598",
"timestamp": 1338231998,
"date_relative": "Today 05:06",
"matched": 1,
"total": 15,
"authors": "Mark Walters| Peter Wang",
"subject": "[PATCH v6 3/6] cli: make --entire-thread=false work for format=json.",
"tags": ["deleted", "draft", "replied", "sent", "unread"]}]

Here is a thread I participated in.  From this, my MUA displays "1/15",
suggesting that there is 1 unread message out of a total of 15.  But
upon opening the thread, there are only 11 messages visible: 4 were
drafts (possibly deleted) which have been excluded.  To the user, it
looks like some messages went missing.

Therefore I would like search --output=summary --exclude=true
to report the total number of non-excluded messages.  It doesn't need to
be via the "total" field; a new field would be fine.

Peter

Re: search summary and exclusions

[Mark Walters]

Hi
On Tue, 29 May 2012, Peter Wang <novalazy@gmail.com> wrote:
> On Mon, 28 May 2012 10:03:35 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
>> On Mon, May 28 2012, Peter Wang <novalazy@gmail.com> wrote:
>> > Exclusions don't work the way I expected with the search command and
>> > --output=summary.  I would like messages with excluded tags to be
>> > treated as if they don't exist at all, but currently:
>> >
>> >   * excluded messages are counted towards the "total"
>> >   * excluded tags are included in the "tags" set
>> >
>> > Are these deliberate?  Especially the second point seems to conflict
>> > with search --output=tags, which doesn't show excluded tags.
>> 
>> Hi, Peter.  Can you explain more what you mean?  I'm not sure what your
>> two bullets have to do with --output=summary.  Messages are counted with
>> the count subcommand, and tags are shown with search --output=tags,
>> neither of which have anything to do with --output=summary, and both of
>> which accept the exclude flag:
>> 
>> notmuch count --exclude=true '*'
>> notmuch search --output=tags --exclude=true '*'
>> 
>> Maybe you can give a clearer explanation of what your issue is.
>> Examples help.
>
> Sure.  I keep draft messages in the mail store, and tag them with
> 'draft', and later 'deleted'.  I would like the exclusions to treat
> draft and deleted messages as if they didn't exist in the store.
>
> % ./notmuch search --format=json --exclude=true -- thread:0000000000009598 tag:unread
> [{"thread": "0000000000009598",
> "timestamp": 1338231998,
> "date_relative": "Today 05:06",
> "matched": 1,
> "total": 15,
> "authors": "Mark Walters| Peter Wang",
> "subject": "[PATCH v6 3/6] cli: make --entire-thread=false work for format=json.",
> "tags": ["deleted", "draft", "replied", "sent", "unread"]}]
>
> Here is a thread I participated in.  From this, my MUA displays "1/15",
> suggesting that there is 1 unread message out of a total of 15.  But
> upon opening the thread, there are only 11 messages visible: 4 were
> drafts (possibly deleted) which have been excluded.  To the user, it
> looks like some messages went missing.
>
> Therefore I would like search --output=summary --exclude=true
> to report the total number of non-excluded messages.  It doesn't need to
> be via the "total" field; a new field would be fine.

It is easy to add this to the JSON format but adding it to the text
format would probably break things, and I would prefer not to have the
formats diverge.

Does anyone have any suggestions? Would people happy with a change in the
text summary output?

Best wishes

Mark

Re: search summary and exclusions

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]

On Mon, May 28 2012, Peter Wang <novalazy@gmail.com> wrote:
> % ./notmuch search --format=json --exclude=true -- thread:0000000000009598 tag:unread
> [{"thread": "0000000000009598",
> "timestamp": 1338231998,
> "date_relative": "Today 05:06",
> "matched": 1,
> "total": 15,
> "authors": "Mark Walters| Peter Wang",
> "subject": "[PATCH v6 3/6] cli: make --entire-thread=false work for format=json.",
> "tags": ["deleted", "draft", "replied", "sent", "unread"]}]
>
> Here is a thread I participated in.  From this, my MUA displays "1/15",
> suggesting that there is 1 unread message out of a total of 15.  But
> upon opening the thread, there are only 11 messages visible: 4 were
> drafts (possibly deleted) which have been excluded.  To the user, it
> looks like some messages went missing.
>
> Therefore I would like search --output=summary --exclude=true
> to report the total number of non-excluded messages.  It doesn't need to
> be via the "total" field; a new field would be fine.

What you have pasted above is --output=json, not --output=summary.  The
formats are quite different.  What are you asking for a change in?

But regardless, I don't think I would like to see the changes you
suggest.  I would like for the thread total to list the total number of
messages in the thread, regardless whether they're excluded or not.
Same for the tags.  I think I want to continue to see if excluded
messages are in a returned thread.  The desire to hide the excluded
messages in the output is why they're marked as hidden/not visible.

Think about the excludes as acting on the search itself, and less on the
output.  We exclude messages from search, but if they show up in a
returned thread we at least acknowledge that they're there.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: search summary and exclusions

[Peter Wang]

On Tue, 29 May 2012 08:00:00 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Mon, May 28 2012, Peter Wang <novalazy@gmail.com> wrote:
> > % ./notmuch search --format=json --exclude=true -- thread:0000000000009598 tag:unread
> > [{"thread": "0000000000009598",
> > "timestamp": 1338231998,
> > "date_relative": "Today 05:06",
> > "matched": 1,
> > "total": 15,
> > "authors": "Mark Walters| Peter Wang",
> > "subject": "[PATCH v6 3/6] cli: make --entire-thread=false work for format=json.",
> > "tags": ["deleted", "draft", "replied", "sent", "unread"]}]
> >
> > Here is a thread I participated in.  From this, my MUA displays "1/15",
> > suggesting that there is 1 unread message out of a total of 15.  But
> > upon opening the thread, there are only 11 messages visible: 4 were
> > drafts (possibly deleted) which have been excluded.  To the user, it
> > looks like some messages went missing.
> >
> > Therefore I would like search --output=summary --exclude=true
> > to report the total number of non-excluded messages.  It doesn't need to
> > be via the "total" field; a new field would be fine.
> 
> What you have pasted above is --output=json, not --output=summary.  The
> formats are quite different.  What are you asking for a change in?

--output=summary is the default.  --format=json only changes the
surface syntax.

% ./notmuch search --output=summary --exclude=true -- thread:0000000000009598 tag:unread
thread:0000000000009598  Yest. 05:06 [1/15] Mark Walters| Peter Wang; [PATCH v6 3/6] cli: make --entire-thread=false work for format=json. (deleted draft replied sent unread)

> But regardless, I don't think I would like to see the changes you
> suggest.  I would like for the thread total to list the total number of
> messages in the thread, regardless whether they're excluded or not.
> Same for the tags.  I think I want to continue to see if excluded
> messages are in a returned thread.  The desire to hide the excluded
> messages in the output is why they're marked as hidden/not visible.
> 
> Think about the excludes as acting on the search itself, and less on the
> output.  We exclude messages from search, but if they show up in a
> returned thread we at least acknowledge that they're there.

Understood; it's a two-phase process, and I wasn't making the
distinction.  If it is the expected and useful behaviour, so be it.

Maybe there is room for another keyword under --exclude?

Peter

Re: search summary and exclusions

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 508 bytes --]

On Tue, May 29 2012, Peter Wang <novalazy@gmail.com> wrote:
> --output=summary is the default.  --format=json only changes the
> surface syntax.

I'm sorry.  For some reason this whole time I was thinking --format=text
when you were saying --output=summary.  My bad.

> Understood; it's a two-phase process, and I wasn't making the
> distinction.  If it is the expected and useful behaviour, so be it.
>
> Maybe there is room for another keyword under --exclude?

Maybe.  As always, patches welcome!

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: search summary and exclusions

[Mark Walters]

On Wed, 30 May 2012, Peter Wang <novalazy@gmail.com> wrote:
> On Tue, 29 May 2012 08:00:00 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
>> On Mon, May 28 2012, Peter Wang <novalazy@gmail.com> wrote:
>> > % ./notmuch search --format=json --exclude=true -- thread:0000000000009598 tag:unread
>> > [{"thread": "0000000000009598",
>> > "timestamp": 1338231998,
>> > "date_relative": "Today 05:06",
>> > "matched": 1,
>> > "total": 15,
>> > "authors": "Mark Walters| Peter Wang",
>> > "subject": "[PATCH v6 3/6] cli: make --entire-thread=false work for format=json.",
>> > "tags": ["deleted", "draft", "replied", "sent", "unread"]}]
>> >
>> > Here is a thread I participated in.  From this, my MUA displays "1/15",
>> > suggesting that there is 1 unread message out of a total of 15.  But
>> > upon opening the thread, there are only 11 messages visible: 4 were
>> > drafts (possibly deleted) which have been excluded.  To the user, it
>> > looks like some messages went missing.
>> >
>> > Therefore I would like search --output=summary --exclude=true
>> > to report the total number of non-excluded messages.  It doesn't need to
>> > be via the "total" field; a new field would be fine.
>> 
>> What you have pasted above is --output=json, not --output=summary.  The
>> formats are quite different.  What are you asking for a change in?
>
> --output=summary is the default.  --format=json only changes the
> surface syntax.
>
> % ./notmuch search --output=summary --exclude=true -- thread:0000000000009598 tag:unread
> thread:0000000000009598  Yest. 05:06 [1/15] Mark Walters| Peter Wang; [PATCH v6 3/6] cli: make --entire-thread=false work for format=json. (deleted draft replied sent unread)
>
>> But regardless, I don't think I would like to see the changes you
>> suggest.  I would like for the thread total to list the total number of
>> messages in the thread, regardless whether they're excluded or not.
>> Same for the tags.  I think I want to continue to see if excluded
>> messages are in a returned thread.  The desire to hide the excluded
>> messages in the output is why they're marked as hidden/not visible.
>> 
>> Think about the excludes as acting on the search itself, and less on the
>> output.  We exclude messages from search, but if they show up in a
>> returned thread we at least acknowledge that they're there.
>
> Understood; it's a two-phase process, and I wasn't making the
> distinction.  If it is the expected and useful behaviour, so be it.
>
> Maybe there is room for another keyword under --exclude?

Yes that might be the best way to go. Something like --exclude=all and
then excluded messages never appear anywhere?

I think it should be easy: I will try to send a patch tonight.

Best wishes

Mark

Re: search summary and exclusions

[Peter Wang]

On Wed, 30 May 2012 08:49:31 +0100, Mark Walters <markwalters1009@gmail.com> wrote:
> 
> On Wed, 30 May 2012, Peter Wang <novalazy@gmail.com> wrote:
> >
> > Maybe there is room for another keyword under --exclude?
> 
> Yes that might be the best way to go. Something like --exclude=all and
> then excluded messages never appear anywhere?
> 
> I think it should be easy: I will try to send a patch tonight.

Hi Mark,

Did you get a chance to try this?  Otherwise I can have a look.

Peter

Re: search summary and exclusions

[Mark Walters]

On Sun, 17 Jun 2012, Peter Wang <novalazy@gmail.com> wrote:
> On Wed, 30 May 2012 08:49:31 +0100, Mark Walters <markwalters1009@gmail.com> wrote:
>> 
>> On Wed, 30 May 2012, Peter Wang <novalazy@gmail.com> wrote:
>> >
>> > Maybe there is room for another keyword under --exclude?
>> 
>> Yes that might be the best way to go. Something like --exclude=all and
>> then excluded messages never appear anywhere?
>> 
>> I think it should be easy: I will try to send a patch tonight.
>
> Hi Mark,
>
> Did you get a chance to try this?  Otherwise I can have a look.

Hi sorry I did try this but got distracted. Doing it neatly turned out
to be a bit bigger than I expected. I will post what I had as a reply to
this message, but I don't think I will be able to work on it any time
soon so feel free to take it over.

I think it all works but I have not tested it very much.

Best wishes

Mark

[WIP 1/2] lib: add --exclude=all option

[Mark Walters]

Adds a exclude all option to the lib which means that excluded
messages are completely ignored (as if they had actually been
deleted).

---
 lib/notmuch-private.h |    3 ++-
 lib/notmuch.h         |   10 +++++++++-
 lib/query.cc          |   11 ++++++-----
 lib/thread.cc         |   41 +++++++++++++++++++++++++++++------------
 4 files changed, 46 insertions(+), 19 deletions(-)

diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index bfb4111..3b92ac2 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -232,7 +232,8 @@ _notmuch_thread_create (void *ctx,
 			unsigned int seed_doc_id,
 			notmuch_doc_id_set_t *match_set,
 			notmuch_string_list_t *excluded_terms,
-			notmuch_sort_t sort);
+			notmuch_sort_t sort,
+			notmuch_exclude_t omit_exclude);
 
 /* message.cc */
 
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 3633bed..57cc700 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -500,6 +500,14 @@ typedef enum {
 const char *
 notmuch_query_get_query_string (notmuch_query_t *query);
 
+/* Exclude values for notmuch_query_set_omit_excluded */
+typedef enum {
+    NOTMUCH_EXCLUDE_FALSE,
+    NOTMUCH_EXCLUDE_TRUE,
+    NOTMUCH_EXCLUDE_ALL
+} notmuch_exclude_t;
+
+
 /* Specify whether to omit excluded results or simply flag them.  By
  * default, this is set to TRUE.
  *
@@ -518,7 +526,7 @@ notmuch_query_get_query_string (notmuch_query_t *query);
  */
 
 void
-notmuch_query_set_omit_excluded (notmuch_query_t *query, notmuch_bool_t omit_excluded);
+notmuch_query_set_omit_excluded (notmuch_query_t *query, notmuch_exclude_t omit_excluded);
 
 /* Specify the sorting desired for this query. */
 void
diff --git a/lib/query.cc b/lib/query.cc
index e9c1a2d..fea76f8 100644
--- a/lib/query.cc
+++ b/lib/query.cc
@@ -28,7 +28,7 @@ struct _notmuch_query {
     const char *query_string;
     notmuch_sort_t sort;
     notmuch_string_list_t *exclude_terms;
-    notmuch_bool_t omit_excluded;
+    notmuch_exclude_t omit_excluded;
 };
 
 typedef struct _notmuch_mset_messages {
@@ -92,7 +92,7 @@ notmuch_query_create (notmuch_database_t *notmuch,
 
     query->exclude_terms = _notmuch_string_list_create (query);
 
-    query->omit_excluded = TRUE;
+    query->omit_excluded = NOTMUCH_EXCLUDE_TRUE;
 
     return query;
 }
@@ -104,7 +104,7 @@ notmuch_query_get_query_string (notmuch_query_t *query)
 }
 
 void
-notmuch_query_set_omit_excluded (notmuch_query_t *query, notmuch_bool_t omit_excluded)
+notmuch_query_set_omit_excluded (notmuch_query_t *query, notmuch_exclude_t omit_excluded)
 {
     query->omit_excluded = omit_excluded;
 }
@@ -220,7 +220,7 @@ notmuch_query_search_messages (notmuch_query_t *query)
 	if (query->exclude_terms) {
 	    exclude_query = _notmuch_exclude_tags (query, final_query);
 
-	    if (query->omit_excluded)
+	    if (query->omit_excluded != NOTMUCH_EXCLUDE_FALSE)
 		final_query = Xapian::Query (Xapian::Query::OP_AND_NOT,
 					     final_query, exclude_query);
 	    else {
@@ -486,7 +486,8 @@ notmuch_threads_get (notmuch_threads_t *threads)
 				   doc_id,
 				   &threads->match_set,
 				   threads->query->exclude_terms,
-				   threads->query->sort);
+				   threads->query->sort,
+				   threads->query->omit_excluded);
 }
 
 void
diff --git a/lib/thread.cc b/lib/thread.cc
index e976d64..6b52792 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -215,7 +215,8 @@ _thread_cleanup_author (notmuch_thread_t *thread,
 static void
 _thread_add_message (notmuch_thread_t *thread,
 		     notmuch_message_t *message,
-		     notmuch_string_list_t *exclude_terms)
+		     notmuch_string_list_t *exclude_terms,
+		     notmuch_exclude_t omit_exclude)
 {
     notmuch_tags_t *tags;
     const char *tag;
@@ -223,6 +224,26 @@ _thread_add_message (notmuch_thread_t *thread,
     InternetAddress *address;
     const char *from, *author;
     char *clean_author;
+    notmuch_bool_t message_excluded = FALSE;
+
+    for (tags = notmuch_message_get_tags (message);
+	 notmuch_tags_valid (tags);
+	 notmuch_tags_move_to_next (tags))
+    {
+	tag = notmuch_tags_get (tags);
+	/* Is message excluded. */
+	for (notmuch_string_node_t *term = exclude_terms->head; term;
+	     term = term->next) {
+	    /* We ignore initial 'K'. */
+	    if (strcmp(tag, (term->string + 1)) == 0) {
+		message_excluded = TRUE;
+		break;
+	    }
+	}
+    }
+
+    if (message_excluded && omit_exclude == NOTMUCH_EXCLUDE_ALL)
+	return;
 
     _notmuch_message_list_add_message (thread->message_list,
 				       talloc_steal (thread, message));
@@ -263,17 +284,12 @@ _thread_add_message (notmuch_thread_t *thread,
 	 notmuch_tags_move_to_next (tags))
     {
 	tag = notmuch_tags_get (tags);
-	/* Mark excluded messages. */
-	for (notmuch_string_node_t *term = exclude_terms->head; term;
-	     term = term->next) {
-	    /* We ignore initial 'K'. */
-	    if (strcmp(tag, (term->string + 1)) == 0) {
-		notmuch_message_set_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED, TRUE);
-		break;
-	    }
-	}
 	g_hash_table_insert (thread->tags, xstrdup (tag), NULL);
     }
+
+    /* Mark excluded messages. */
+    if (message_excluded)
+	notmuch_message_set_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED, TRUE);
 }
 
 static void
@@ -404,7 +420,8 @@ _notmuch_thread_create (void *ctx,
 			unsigned int seed_doc_id,
 			notmuch_doc_id_set_t *match_set,
 			notmuch_string_list_t *exclude_terms,
-			notmuch_sort_t sort)
+			notmuch_sort_t sort,
+			notmuch_exclude_t omit_excluded)
 {
     notmuch_thread_t *thread;
     notmuch_message_t *seed_message;
@@ -479,7 +496,7 @@ _notmuch_thread_create (void *ctx,
 	if (doc_id == seed_doc_id)
 	    message = seed_message;
 
-	_thread_add_message (thread, message, exclude_terms);
+	_thread_add_message (thread, message, exclude_terms, omit_excluded);
 
 	if ( _notmuch_doc_id_set_contains (match_set, doc_id)) {
 	    _notmuch_doc_id_set_remove (match_set, doc_id);
-- 
1.7.9.1

[WIP 2/2] cli: add --exclude=all option to notmuch-search.c

[Mark Walters]

Add a --exclude=all option to notmuch search.

---
 notmuch-search.c |    8 ++++++--
 test/excludes    |    5 +++++
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/notmuch-search.c b/notmuch-search.c
index 3be296d..89b5bf9 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -420,6 +420,7 @@ enum {
     EXCLUDE_TRUE,
     EXCLUDE_FALSE,
     EXCLUDE_FLAG,
+    EXCLUDE_ALL
 };
 
 int
@@ -461,6 +462,7 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
           (notmuch_keyword_t []){ { "true", EXCLUDE_TRUE },
                                   { "false", EXCLUDE_FALSE },
                                   { "flag", EXCLUDE_FLAG },
+                                  { "all", EXCLUDE_ALL },
                                   { 0, 0 } } },
 	{ NOTMUCH_OPT_INT, &offset, "offset", 'O', 0 },
 	{ NOTMUCH_OPT_INT, &limit, "limit", 'L', 0  },
@@ -516,7 +518,7 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
 	exclude = EXCLUDE_FALSE;
     }
 
-    if (exclude == EXCLUDE_TRUE || exclude == EXCLUDE_FLAG) {
+    if (exclude != EXCLUDE_FALSE) {
 	const char **search_exclude_tags;
 	size_t search_exclude_tags_length;
 
@@ -525,7 +527,9 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
 	for (i = 0; i < search_exclude_tags_length; i++)
 	    notmuch_query_add_tag_exclude (query, search_exclude_tags[i]);
 	if (exclude == EXCLUDE_FLAG)
-	    notmuch_query_set_omit_excluded (query, FALSE);
+	    notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_FALSE);
+	if (exclude == EXCLUDE_ALL)
+	    notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_ALL);
     }
 
     switch (output) {
diff --git a/test/excludes b/test/excludes
index 24d653e..7be8d79 100755
--- a/test/excludes
+++ b/test/excludes
@@ -166,6 +166,11 @@ ${matching_message_ids[3]}
 ${matching_message_ids[4]}
 ${matching_message_ids[5]}"
 
+test_begin_subtest "Search, exclude=all (thread summary)"
+output=$(notmuch search --exclude=all tag:test | notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2001-01-05 [1/5] Notmuch Test Suite; Some messages excluded: single non-excluded match: reply 4 (inbox test unread)
+thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; No messages excluded: single match: reply 3 (inbox test unread)"
+
 test_begin_subtest "Search, default exclusion: tag in query (thread summary)"
 output=$(notmuch search tag:test and tag:deleted | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2001-01-05 [1/6] Notmuch Test Suite; All messages excluded: single match: reply 2 (deleted inbox test unread)
-- 
1.7.9.1

Re: search summary and exclusions

[Mark Walters]

Hi

On Mon, 28 May 2012, Peter Wang <novalazy@gmail.com> wrote:
> Hi,
>
> Exclusions don't work the way I expected with the search command and
> --output=summary.  I would like messages with excluded tags to be
> treated as if they don't exist at all, but currently:
>
>   * excluded messages are counted towards the "total"
>   * excluded tags are included in the "tags" set
>
> Are these deliberate?  Especially the second point seems to conflict
> with search --output=tags, which doesn't show excluded tags.

The general idea is that commands that deciding whether something
matches the search checks the excludes but constructing a thread does
not. The difference in tags when outputting tags and summary is that the
former outputs tags of *messages* matching the query (so respects the
excludes) whereas summary outputs the tags of all messages in threads
matching the query (so does not respect the excludes) but see below for
more precise explanations.

notmuch search --output=tags <search>

The tags from messages which match the search and are not excluded are
returned.

notmuch search --output=threads <search>

returns all threads containing a message which matches the search and is
not excluded

notmuch search --output=summary <search>

returns the summary output of all these threads. The summary output
includes all tags of all messages (whether excluded or not) of the
thread and the count is the number of messages (including excluded
messages) of the thread.

This seems consistent with

notmuch show --entire-thread <search>

which returns all messages (whether excluded or not) of all these
threads.

Best wishes

Mark