[PATCH] show: add In-reply-to, References fields to structured formats

[PATCH] show: add In-reply-to, References fields to structured formats

[Peter Wang]

This is useful when 'show' is used to retrieve a draft message
which is in reply to another message.
---
 devel/schemata      |  9 ++++++++-
 notmuch-show.c      | 16 ++++++++++++----
 test/thread-replies |  7 +++++++
 3 files changed, 27 insertions(+), 5 deletions(-)

diff --git a/devel/schemata b/devel/schemata
index 41dc4a6..dd41217 100644
--- a/devel/schemata
+++ b/devel/schemata
@@ -14,7 +14,7 @@ are interleaved. Keys are printed as keywords (symbols preceded by a
 colon), e.g. (:id "123" :time 54321 :from "foobar"). Null is printed as
 nil, true as t and false as nil.
 
-This is version 2 of the structured output format.
+This is version 3 of the structured output format.
 
 Version history
 ---------------
@@ -26,6 +26,9 @@ v1
 v2
 - Added the thread_summary.query field.
 
+v3
+- Added headers.in-reply-to and headers.references fields.
+
 Common non-terminals
 --------------------
 
@@ -105,6 +108,10 @@ headers = {
     Cc?:            string,
     Bcc?:           string,
     Reply-To?:      string,
+    # Added in schema version 3.
+    In-reply-to?:   string,
+    # Added in schema version 3.
+    References?:    string,
     Date:           string
 }
 
diff --git a/notmuch-show.c b/notmuch-show.c
index c07f887..774ba44 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -222,6 +222,8 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
     InternetAddressList *recipients;
     const char *recipients_string;
     const char *reply_to_string;
+    const char *in_reply_to_string;
+    const char *references_string;
 
     sp->begin_map (sp);
 
@@ -258,13 +260,19 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
 	sp->string (sp, reply_to_string);
     }
 
-    if (reply) {
+    in_reply_to_string = g_mime_object_get_header (GMIME_OBJECT (message), "In-reply-to");
+    if (in_reply_to_string || reply) {
 	sp->map_key (sp, "In-reply-to");
-	sp->string (sp, g_mime_object_get_header (GMIME_OBJECT (message), "In-reply-to"));
+	sp->string (sp, in_reply_to_string);
+    }
 
+    references_string = g_mime_object_get_header (GMIME_OBJECT (message), "References");
+    if (references_string || reply) {
 	sp->map_key (sp, "References");
-	sp->string (sp, g_mime_object_get_header (GMIME_OBJECT (message), "References"));
-    } else {
+	sp->string (sp, references_string);
+    }
+
+    if (! reply) {
 	sp->map_key (sp, "Date");
 	sp->string (sp, g_mime_message_get_date_as_string (message));
     }
diff --git a/test/thread-replies b/test/thread-replies
index eeb70d0..9d4b379 100755
--- a/test/thread-replies
+++ b/test/thread-replies
@@ -39,6 +39,8 @@ expected='[[[{"id": "foo@one.com",
  "tags": ["inbox", "unread"], "headers": {"Subject": "Re: one",
  "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
  "To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
+ "In-reply-to": "mumble",
+ "References": "<foo@one.com>",
  "Date": "Fri, 05 Jan 2001 15:43:57 +0000"},
  "body": [{"id": 1, "content-type": "text/plain",
  "content": "This is just a test message (#2)\n"}]}, []]]]]]'
@@ -68,6 +70,8 @@ expected='[[[{"id": "foo@two.com",
  "headers": {"Subject": "Re: two",
  "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
  "To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
+ "In-reply-to": "<bar@baz.com>",
+ "References": "<foo@two.com>",
  "Date": "Fri, 05 Jan 2001 15:43:57 +0000"},
  "body": [{"id": 1,
  "content-type": "text/plain", "content": "This is just a test message (#4)\n"}]},
@@ -95,6 +99,7 @@ expected='[[[{"id": "foo@three.com", "match": true, "excluded": false,
  "headers": {"Subject": "Re: three",
  "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
  "To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
+ "In-reply-to": "<foo@three.com>",
  "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [{"id": 1,
  "content-type": "text/plain", "content": "This is just a test message (#6)\n"}]},
  []]]]]]'
@@ -124,6 +129,8 @@ expected='[[[{"id": "foo@four.com", "match": true, "excluded": false,
  "headers": {"Subject": "neither",
  "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
  "To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
+ "In-reply-to": "<baz@four.com>",
+ "References": "<baz@four.com> <foo@four.com>",
  "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [{"id": 1,
  "content-type": "text/plain", "content": "This is just a test message (#9)\n"}]},
  []]]]], [[{"id": "bar@four.com", "match": true, "excluded": false,
-- 
1.8.4

Re: [PATCH] show: add In-reply-to, References fields to structured formats

[Jani Nikula]

On Sun, 12 Jan 2014, Peter Wang <novalazy@gmail.com> wrote:
> This is useful when 'show' is used to retrieve a draft message
> which is in reply to another message.

I'd like to know more about *how* this is useful. Indeed the whole big
picture about supporting draft or postponed messages is foggy. I would
like to have some clarity about that first.

Apparently the idea is to index draft messages. How do you save them?
What guarantees are there that they look enough like real messages that
they get indexed? Does this patch mean that the idea is to resume draft
messages using the structured formats instead of opening the raw file?
Why?  What do you plan to do with the saved draft? And so on...

BR,
Jani.

> ---
>  devel/schemata      |  9 ++++++++-
>  notmuch-show.c      | 16 ++++++++++++----
>  test/thread-replies |  7 +++++++
>  3 files changed, 27 insertions(+), 5 deletions(-)
>
> diff --git a/devel/schemata b/devel/schemata
> index 41dc4a6..dd41217 100644
> --- a/devel/schemata
> +++ b/devel/schemata
> @@ -14,7 +14,7 @@ are interleaved. Keys are printed as keywords (symbols preceded by a
>  colon), e.g. (:id "123" :time 54321 :from "foobar"). Null is printed as
>  nil, true as t and false as nil.
>  
> -This is version 2 of the structured output format.
> +This is version 3 of the structured output format.
>  
>  Version history
>  ---------------
> @@ -26,6 +26,9 @@ v1
>  v2
>  - Added the thread_summary.query field.
>  
> +v3
> +- Added headers.in-reply-to and headers.references fields.
> +
>  Common non-terminals
>  --------------------
>  
> @@ -105,6 +108,10 @@ headers = {
>      Cc?:            string,
>      Bcc?:           string,
>      Reply-To?:      string,
> +    # Added in schema version 3.
> +    In-reply-to?:   string,
> +    # Added in schema version 3.
> +    References?:    string,
>      Date:           string
>  }
>  
> diff --git a/notmuch-show.c b/notmuch-show.c
> index c07f887..774ba44 100644
> --- a/notmuch-show.c
> +++ b/notmuch-show.c
> @@ -222,6 +222,8 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
>      InternetAddressList *recipients;
>      const char *recipients_string;
>      const char *reply_to_string;
> +    const char *in_reply_to_string;
> +    const char *references_string;
>  
>      sp->begin_map (sp);
>  
> @@ -258,13 +260,19 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage *message,
>  	sp->string (sp, reply_to_string);
>      }
>  
> -    if (reply) {
> +    in_reply_to_string = g_mime_object_get_header (GMIME_OBJECT (message), "In-reply-to");
> +    if (in_reply_to_string || reply) {
>  	sp->map_key (sp, "In-reply-to");
> -	sp->string (sp, g_mime_object_get_header (GMIME_OBJECT (message), "In-reply-to"));
> +	sp->string (sp, in_reply_to_string);
> +    }
>  
> +    references_string = g_mime_object_get_header (GMIME_OBJECT (message), "References");
> +    if (references_string || reply) {
>  	sp->map_key (sp, "References");
> -	sp->string (sp, g_mime_object_get_header (GMIME_OBJECT (message), "References"));
> -    } else {
> +	sp->string (sp, references_string);
> +    }
> +
> +    if (! reply) {
>  	sp->map_key (sp, "Date");
>  	sp->string (sp, g_mime_message_get_date_as_string (message));
>      }
> diff --git a/test/thread-replies b/test/thread-replies
> index eeb70d0..9d4b379 100755
> --- a/test/thread-replies
> +++ b/test/thread-replies
> @@ -39,6 +39,8 @@ expected='[[[{"id": "foo@one.com",
>   "tags": ["inbox", "unread"], "headers": {"Subject": "Re: one",
>   "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
>   "To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
> + "In-reply-to": "mumble",
> + "References": "<foo@one.com>",
>   "Date": "Fri, 05 Jan 2001 15:43:57 +0000"},
>   "body": [{"id": 1, "content-type": "text/plain",
>   "content": "This is just a test message (#2)\n"}]}, []]]]]]'
> @@ -68,6 +70,8 @@ expected='[[[{"id": "foo@two.com",
>   "headers": {"Subject": "Re: two",
>   "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
>   "To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
> + "In-reply-to": "<bar@baz.com>",
> + "References": "<foo@two.com>",
>   "Date": "Fri, 05 Jan 2001 15:43:57 +0000"},
>   "body": [{"id": 1,
>   "content-type": "text/plain", "content": "This is just a test message (#4)\n"}]},
> @@ -95,6 +99,7 @@ expected='[[[{"id": "foo@three.com", "match": true, "excluded": false,
>   "headers": {"Subject": "Re: three",
>   "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
>   "To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
> + "In-reply-to": "<foo@three.com>",
>   "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [{"id": 1,
>   "content-type": "text/plain", "content": "This is just a test message (#6)\n"}]},
>   []]]]]]'
> @@ -124,6 +129,8 @@ expected='[[[{"id": "foo@four.com", "match": true, "excluded": false,
>   "headers": {"Subject": "neither",
>   "From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
>   "To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
> + "In-reply-to": "<baz@four.com>",
> + "References": "<baz@four.com> <foo@four.com>",
>   "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [{"id": 1,
>   "content-type": "text/plain", "content": "This is just a test message (#9)\n"}]},
>   []]]]], [[{"id": "bar@four.com", "match": true, "excluded": false,
> -- 
> 1.8.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] show: add In-reply-to, References fields to structured formats

[Peter Wang]

On Sun, 12 Jan 2014 17:31:32 +0200, Jani Nikula <jani@nikula.org> wrote:
> On Sun, 12 Jan 2014, Peter Wang <novalazy@gmail.com> wrote:
> > This is useful when 'show' is used to retrieve a draft message
> > which is in reply to another message.
> 
> I'd like to know more about *how* this is useful. Indeed the whole big
> picture about supporting draft or postponed messages is foggy. I would
> like to have some clarity about that first.
> 
> Apparently the idea is to index draft messages. How do you save them?
> What guarantees are there that they look enough like real messages that
> they get indexed? Does this patch mean that the idea is to resume draft
> messages using the structured formats instead of opening the raw file?
> Why?  What do you plan to do with the saved draft? And so on...

I didn't realise storing drafts in your Maildir was unusual.

A draft message (including its attachments) may be added to a Maildir
folder with notmuch insert or notmuch-deliver, and then tagged.
The message must look enough like a real message for indexing but it is
not hard for an email client to arrange -- the message is ostensibly to
be sent anyway.  Unlike saving in a local file, keeping the draft in a
central mail store means it will be accessible anywhere that you can
access the rest of your mail.

Draft messages may be retrieved for previewing or resumption with
notmuch show, like other messages.  The raw file may be on another
machine so it is not always possible to read it directly.  The advantage
of using a structured output format instead of the raw output format is
the same as for any other message -- notmuch has already parsed it for
you.  The disadvantage is that notmuch show's structured output only
presents a subset of headers, so other headers will be lost.
(This suggests an alternative change if notmuch maintainers are
receptive.)

Indexed draft messages will, by default, show up in normal display
so they will need to be hidden with notmuch search exclusions.
Drafts may tagged with 'delete' for eventual removal.

Peter