[PATCH 1/3] test: add basic show, search --format=text tests

[PATCH 1/3] test: add basic show, search --format=text tests

[Peter Wang]

There didn't seem to be these basic tests for --format=text,
as there are for --format=json.  These are just the tests from
the `json' script, with adjusted expected outputs.
---
 test/notmuch-test |    1 +
 test/text         |   55 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 56 insertions(+), 0 deletions(-)
 create mode 100755 test/text

diff --git a/test/notmuch-test b/test/notmuch-test
index bfad5d3..ea39dfc 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -31,6 +31,7 @@ TESTS="
   excludes
   tagging
   json
+  text
   multipart
   thread-naming
   raw
diff --git a/test/text b/test/text
new file mode 100755
index 0000000..428c89b
--- /dev/null
+++ b/test/text
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+test_description="--format=text output"
+. ./test-lib.sh
+
+test_begin_subtest "Show message: text"
+add_message "[subject]=\"text-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"text-show-message\""
+output=$(notmuch show --format=text "text-show-message" | notmuch_show_sanitize_all)
+test_expect_equal "$output" "\
+\fmessage{ id:XXXXX depth:0 match:1 excluded:0 filename:XXXXX
+\fheader{
+Notmuch Test Suite <test_suite@notmuchmail.org> (2000-01-01) (inbox unread)
+Subject: text-show-subject
+From: Notmuch Test Suite <test_suite@notmuchmail.org>
+To: Notmuch Test Suite <test_suite@notmuchmail.org>
+Date: Sat, 01 Jan 2000 12:00:00 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 1, Content-type: text/plain
+text-show-message
+\fpart}
+\fbody}
+\fmessage}"
+
+test_begin_subtest "Search message: text"
+add_message "[subject]=\"text-search-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"text-search-message\""
+output=$(notmuch search --format=text "text-search-message" | notmuch_search_sanitize)
+test_expect_equal "$output" \
+"thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; text-search-subject (inbox unread)"
+
+test_begin_subtest "Show message: text, utf-8"
+add_message "[subject]=\"text-show-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"tëxt-show-méssage\""
+output=$(notmuch show --format=text "tëxt-show-méssage" | notmuch_show_sanitize_all)
+test_expect_equal "$output" "\
+\fmessage{ id:XXXXX depth:0 match:1 excluded:0 filename:XXXXX
+\fheader{
+Notmuch Test Suite <test_suite@notmuchmail.org> (2000-01-01) (inbox unread)
+Subject: text-show-utf8-body-sübjéct
+From: Notmuch Test Suite <test_suite@notmuchmail.org>
+To: Notmuch Test Suite <test_suite@notmuchmail.org>
+Date: Sat, 01 Jan 2000 12:00:00 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 1, Content-type: text/plain
+tëxt-show-méssage
+\fpart}
+\fbody}
+\fmessage}"
+
+test_begin_subtest "Search message: text, utf-8"
+add_message "[subject]=\"text-search-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"tëxt-search-méssage\""
+output=$(notmuch search --format=text "tëxt-search-méssage" | notmuch_search_sanitize)
+test_expect_equal "$output" \
+"thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; text-search-utf8-body-sübjéct (inbox unread)"
+
+test_done
-- 
1.7.4.4

[PATCH 2/3] show: output Reply-To headers

[Peter Wang]

Output Reply-To headers when present in a message.
These were missing in text and json output formats.
---
 notmuch-show.c |   10 ++++++++++
 1 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 8247f1d..de63f06 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -198,6 +198,7 @@ format_headers_json (const void *ctx, GMimeMessage *message, notmuch_bool_t repl
     void *local = talloc_new (ctx);
     InternetAddressList *recipients;
     const char *recipients_string;
+    const char *reply_to_string;
 
     printf ("{%s: %s",
 	    json_quote_str (local, "Subject"),
@@ -217,6 +218,11 @@ format_headers_json (const void *ctx, GMimeMessage *message, notmuch_bool_t repl
 	printf (", %s: %s",
 		json_quote_str (local, "Cc"),
 		json_quote_str (local, recipients_string));
+    reply_to_string = g_mime_message_get_reply_to (message);
+    if (reply_to_string)
+	printf (", %s: %s",
+		json_quote_str (local, "Reply-To"),
+		json_quote_str (local, reply_to_string));
 
     if (reply) {
 	printf (", %s: %s",
@@ -512,6 +518,7 @@ format_part_text (const void *ctx, mime_node_t *node,
 	GMimeMessage *message = GMIME_MESSAGE (node->part);
 	InternetAddressList *recipients;
 	const char *recipients_string;
+	const char *reply_to_string;
 
 	printf ("\fheader{\n");
 	if (node->envelope_file)
@@ -526,6 +533,9 @@ format_part_text (const void *ctx, mime_node_t *node,
 	recipients_string = internet_address_list_to_string (recipients, 0);
 	if (recipients_string)
 	    printf ("Cc: %s\n", recipients_string);
+	reply_to_string = g_mime_message_get_reply_to (message);
+	if (reply_to_string)
+	    printf ("Reply-To: %s\n", reply_to_string);
 	printf ("Date: %s\n", g_mime_message_get_date_as_string (message));
 	printf ("\fheader}\n");
 
-- 
1.7.4.4

[PATCH 3/3] test: add test for showing Reply-To headers

[Peter Wang]

Test that show --format=text and --format=json now output
Reply-To headers when present.
---
 test/json |    4 ++--
 test/text |    3 ++-
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/test/json b/test/json
index 6439788..0a692ac 100755
--- a/test/json
+++ b/test/json
@@ -3,9 +3,9 @@ test_description="--format=json output"
 . ./test-lib.sh
 
 test_begin_subtest "Show message: json"
-add_message "[subject]=\"json-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"json-show-message\""
+add_message "[subject]=\"json-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"json-show-message\"" "[reply-to]=\"json-show-reply-to\""
 output=$(notmuch show --format=json "json-show-message")
-test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, \"excluded\": false, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"json-show-message\n\"}]}, []]]]"
+test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, \"excluded\": false, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"Reply-To\": \"json-show-reply-to\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"json-show-message\n\"}]}, []]]]"
 
 test_begin_subtest "Search message: json"
 add_message "[subject]=\"json-search-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"json-search-message\""
diff --git a/test/text b/test/text
index 428c89b..bf1dfd1 100755
--- a/test/text
+++ b/test/text
@@ -3,7 +3,7 @@ test_description="--format=text output"
 . ./test-lib.sh
 
 test_begin_subtest "Show message: text"
-add_message "[subject]=\"text-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"text-show-message\""
+add_message "[subject]=\"text-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"text-show-message\"" "[reply-to]=\"text-show-reply-to\""
 output=$(notmuch show --format=text "text-show-message" | notmuch_show_sanitize_all)
 test_expect_equal "$output" "\
 \fmessage{ id:XXXXX depth:0 match:1 excluded:0 filename:XXXXX
@@ -12,6 +12,7 @@ Notmuch Test Suite <test_suite@notmuchmail.org> (2000-01-01) (inbox unread)
 Subject: text-show-subject
 From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: Notmuch Test Suite <test_suite@notmuchmail.org>
+Reply-To: text-show-reply-to
 Date: Sat, 01 Jan 2000 12:00:00 +0000
 \fheader}
 \fbody{
-- 
1.7.4.4

Re: [PATCH 1/3] test: add basic show, search --format=text tests

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 453 bytes --]

On Sat, Jun 23 2012, Peter Wang <novalazy@gmail.com> wrote:
> There didn't seem to be these basic tests for --format=text,
> as there are for --format=json.  These are just the tests from
> the `json' script, with adjusted expected outputs.

As long as we're going to support this silly output format we should
probably have some basic tests for it, so we should definitely push this
patch, regardless if we want the rest of this series or not.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 2/3] show: output Reply-To headers

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 394 bytes --]

On Sat, Jun 23 2012, Peter Wang <novalazy@gmail.com> wrote:
> Output Reply-To headers when present in a message.
> These were missing in text and json output formats.

I don't see why the Reply-To header is needed, given that we are
constructing reply templates, but I guess if someone thinks it's needed
I don't see much harm in including it.

I didn't test it, but it looks ok to me.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 2/3] show: output Reply-To headers

[Peter Wang]

On Tue, 03 Jul 2012 18:27:23 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Sat, Jun 23 2012, Peter Wang <novalazy@gmail.com> wrote:
> > Output Reply-To headers when present in a message.
> > These were missing in text and json output formats.
> 
> I don't see why the Reply-To header is needed, given that we are
> constructing reply templates, but I guess if someone thinks it's needed
> I don't see much harm in including it.

I want to see what the sender intended, before hitting reply.

Peter

Re: [PATCH 1/3] test: add basic show, search --format=text tests

[Austin Clements]

Series LGTM, though I'm curious why you want the reply-to header,
given that nothing I'm aware of consumes it.

I second Jameson's opinion that we should push this first patch even
if we wind up not pushing the others.

Quoth Peter Wang on Jun 24 at  1:27 pm:
> There didn't seem to be these basic tests for --format=text,
> as there are for --format=json.  These are just the tests from
> the `json' script, with adjusted expected outputs.
> ---
>  test/notmuch-test |    1 +
>  test/text         |   55 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 56 insertions(+), 0 deletions(-)
>  create mode 100755 test/text
> 
> diff --git a/test/notmuch-test b/test/notmuch-test
> index bfad5d3..ea39dfc 100755
> --- a/test/notmuch-test
> +++ b/test/notmuch-test
> @@ -31,6 +31,7 @@ TESTS="
>    excludes
>    tagging
>    json
> +  text
>    multipart
>    thread-naming
>    raw
> diff --git a/test/text b/test/text
> new file mode 100755
> index 0000000..428c89b
> --- /dev/null
> +++ b/test/text
> @@ -0,0 +1,55 @@
> +#!/usr/bin/env bash
> +test_description="--format=text output"
> +. ./test-lib.sh
> +
> +test_begin_subtest "Show message: text"
> +add_message "[subject]=\"text-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"text-show-message\""
> +output=$(notmuch show --format=text "text-show-message" | notmuch_show_sanitize_all)
> +test_expect_equal "$output" "\
> +\fmessage{ id:XXXXX depth:0 match:1 excluded:0 filename:XXXXX
> +\fheader{
> +Notmuch Test Suite <test_suite@notmuchmail.org> (2000-01-01) (inbox unread)
> +Subject: text-show-subject
> +From: Notmuch Test Suite <test_suite@notmuchmail.org>
> +To: Notmuch Test Suite <test_suite@notmuchmail.org>
> +Date: Sat, 01 Jan 2000 12:00:00 +0000
> +\fheader}
> +\fbody{
> +\fpart{ ID: 1, Content-type: text/plain
> +text-show-message
> +\fpart}
> +\fbody}
> +\fmessage}"
> +
> +test_begin_subtest "Search message: text"
> +add_message "[subject]=\"text-search-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"text-search-message\""
> +output=$(notmuch search --format=text "text-search-message" | notmuch_search_sanitize)
> +test_expect_equal "$output" \
> +"thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; text-search-subject (inbox unread)"
> +
> +test_begin_subtest "Show message: text, utf-8"
> +add_message "[subject]=\"text-show-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"tëxt-show-méssage\""
> +output=$(notmuch show --format=text "tëxt-show-méssage" | notmuch_show_sanitize_all)
> +test_expect_equal "$output" "\
> +\fmessage{ id:XXXXX depth:0 match:1 excluded:0 filename:XXXXX
> +\fheader{
> +Notmuch Test Suite <test_suite@notmuchmail.org> (2000-01-01) (inbox unread)
> +Subject: text-show-utf8-body-sübjéct
> +From: Notmuch Test Suite <test_suite@notmuchmail.org>
> +To: Notmuch Test Suite <test_suite@notmuchmail.org>
> +Date: Sat, 01 Jan 2000 12:00:00 +0000
> +\fheader}
> +\fbody{
> +\fpart{ ID: 1, Content-type: text/plain
> +tëxt-show-méssage
> +\fpart}
> +\fbody}
> +\fmessage}"
> +
> +test_begin_subtest "Search message: text, utf-8"
> +add_message "[subject]=\"text-search-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"tëxt-search-méssage\""
> +output=$(notmuch search --format=text "tëxt-search-méssage" | notmuch_search_sanitize)
> +test_expect_equal "$output" \
> +"thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; text-search-utf8-body-sübjéct (inbox unread)"
> +
> +test_done

Re: [PATCH 2/3] show: output Reply-To headers

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 403 bytes --]

On Tue, Jul 03 2012, Peter Wang <novalazy@gmail.com> wrote:
> I want to see what the sender intended, before hitting reply.

Given that there have been requests to see a lot of other headers as
well, we probably need to have a discussion about which ones are worth
of emitting, and how we give the user some more general control to see
the ones they want.  Either that or we just emit them all?

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: [PATCH 2/3] show: output Reply-To headers

[Peter Wang]

On Tue, 03 Jul 2012 19:22:18 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Tue, Jul 03 2012, Peter Wang <novalazy@gmail.com> wrote:
> > I want to see what the sender intended, before hitting reply.
> 
> Given that there have been requests to see a lot of other headers as
> well, we probably need to have a discussion about which ones are worth
> of emitting, and how we give the user some more general control to see
> the ones they want.  Either that or we just emit them all?

If we start with the obvious:

  notmuch show --output-headers=date,from,subject,to,cc,reply-to ...

with the default being the current set.

Emitting everything would be easier but seems wasteful.  I just looked
at a random message: in RFC822 syntax the header is 4073 bytes, and the
body is 1116 bytes.  Keeping only the fields that notmuch emits reduces
the header to 295 bytes.  Reply-To is 92 bytes, but not every message
has that.

Peter

Re: [PATCH 2/3] show: output Reply-To headers

[Mark Walters]

On Wed, 04 Jul 2012, Peter Wang <novalazy@gmail.com> wrote:
> On Tue, 03 Jul 2012 19:22:18 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
>> On Tue, Jul 03 2012, Peter Wang <novalazy@gmail.com> wrote:
>> > I want to see what the sender intended, before hitting reply.
>> 
>> Given that there have been requests to see a lot of other headers as
>> well, we probably need to have a discussion about which ones are worth
>> of emitting, and how we give the user some more general control to see
>> the ones they want.  Either that or we just emit them all?
>
> If we start with the obvious:
>
>   notmuch show --output-headers=date,from,subject,to,cc,reply-to ...
>
> with the default being the current set.
>
> Emitting everything would be easier but seems wasteful.  I just looked
> at a random message: in RFC822 syntax the header is 4073 bytes, and the
> body is 1116 bytes.  Keeping only the fields that notmuch emits reduces
> the header to 295 bytes.  Reply-To is 92 bytes, but not every message
> has that.

I wonder if it would make sense for this option to be combined with
something like
id:"1341041595-5858-1-git-send-email-markwalters1009@gmail.com" which
chooses whether to output the body of the message or not.

Maybe something like --output=short|medium|full
with short being just the brief headers, medium being the current
default of brief headers and text bodies, and full being message with
all headers.

I am not sure I like it (as someone will want full headers and no
bodies!) but we don't want the command line to get too cluttered.

Another possibility for this particular choice: could a list of wanted
headers be included in the config file? Since I think you want it for
"user wants to see it" reasons rather than "program needs it to do
something" reasons that might make sense.

Best wishes

Mark

Re: [PATCH 2/3] show: output Reply-To headers

[Austin Clements]

Quoth Mark Walters on Jul 05 at  9:47 am:
> On Wed, 04 Jul 2012, Peter Wang <novalazy@gmail.com> wrote:
> > On Tue, 03 Jul 2012 19:22:18 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> >> On Tue, Jul 03 2012, Peter Wang <novalazy@gmail.com> wrote:
> >> > I want to see what the sender intended, before hitting reply.
> >> 
> >> Given that there have been requests to see a lot of other headers as
> >> well, we probably need to have a discussion about which ones are worth
> >> of emitting, and how we give the user some more general control to see
> >> the ones they want.  Either that or we just emit them all?
> >
> > If we start with the obvious:
> >
> >   notmuch show --output-headers=date,from,subject,to,cc,reply-to ...
> >
> > with the default being the current set.
> >
> > Emitting everything would be easier but seems wasteful.  I just looked
> > at a random message: in RFC822 syntax the header is 4073 bytes, and the
> > body is 1116 bytes.  Keeping only the fields that notmuch emits reduces
> > the header to 295 bytes.  Reply-To is 92 bytes, but not every message
> > has that.
> 
> I wonder if it would make sense for this option to be combined with
> something like
> id:"1341041595-5858-1-git-send-email-markwalters1009@gmail.com" which
> chooses whether to output the body of the message or not.
> 
> Maybe something like --output=short|medium|full
> with short being just the brief headers, medium being the current
> default of brief headers and text bodies, and full being message with
> all headers.
> 
> I am not sure I like it (as someone will want full headers and no
> bodies!) but we don't want the command line to get too cluttered.

This option seems like the type of thing that would be used primarily
by frontends, so it matters much more that they have control over the
output than that the command line is terse.  We should provide
reasonable defaults for the occasional flesh and blood call to the
CLI, but I don't think we need to cater to that use.

> Another possibility for this particular choice: could a list of wanted
> headers be included in the config file? Since I think you want it for
> "user wants to see it" reasons rather than "program needs it to do
> something" reasons that might make sense.

This would make a lot of sense in the frontend configuration (so much
so that we already have it!  But it's currently limited to removing
headers).  Also, if it's in the notmuch configuration, then when
programs do need the headers to do something, it becomes very
difficult to request what they need.

Re: [PATCH 1/3] test: add basic show, search --format=text tests

[David Bremner]

On Sun, 24 Jun 2012 13:27:48 +1000, Peter Wang <novalazy@gmail.com> wrote:
> There didn't seem to be these basic tests for --format=text,
> as there are for --format=json.  These are just the tests from
> the `json' script, with adjusted expected outputs.

I have pushed the first patch in the series.

d