problems with message/rfc822 parts

problems with message/rfc822 parts

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 2513 bytes --]

Hey, folks.  I've been noticing a couple of issues with message/rfc822
part handling in recent builds of notmuch/master.  They are rooted in
the new part handling rework that was done recently.  I just want to
mention them here, to make people aware of them, until we get a chance
to address them.

The first issue is that show --format=raw --part on a message/rfc822
part is throwing a gmime exception, even though notmuch is still
producing output and returning successfully.  I just sent a patch to the
list addressing the issues.  However, the test suite was apparently
unable to catch that something unexpected was being output to stderr, so
we still need to address that.

Another issue is that we're no longer outputting any of the headers of
message/rfc822 parts.  This shouldn't be too hard to hack back in, so
hopefully I can get to that this weekend.

There was also a problem with the emacs display of message/rfc822 parts,
and I just sent a patch to the list about that as well.

However, there is a larger issue related to the ones above which is that
message parts in general are not being handled consistently, including
the top level message.  amdragon and I were actually discussing this on
irc the other day.  A possible solution is to make a general
message-part-handling function, that handles output of headers and the
message body, and call that function both on the top level message and
on any message sub parts.  I think that would make things a lot cleaner,
and would make the output more consistent and intuitive.

A somewhat related issue is that the "raw" format for parts is not
really doing the expected thing in certain cases, particularly with
regards to message parts.  dkg and I sort of realized this when we were
reworking the part handling a couple weeks ago.  One expects, rightfully
so, that show --format=raw --part on a message part would output the
full, raw, well-formatted rfc822 message enclosed in that part.
Currently that is not the case, since, for instance, the headers are not
output and the part handling is recursing over all the message
sub-parts.

I don't know necessarily know the right solution for either of these
issues, although I have some ideas (assuming I don't run into gmime
limitations).  Feedback and suggestions would be wonderful.
Unfortunately I don't have time to work on any of these issues at the
moment (hopefully this weekend), but I just wanted to throw them out
there, in case anyone wants to tackle them before I get to them.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

improving message/rfc822 part handling

[Jameson Graef Rollins]

So the following patch series is my attempt to improve handling of
message/rfc822 parts.  The first couple of patches fix/overhaul the
multipart test, and the last two improve the message/rfc822 part
output and emacs handling, respectively.

The fix outputs the rfc822 message in a format similar to that of the
outer message, including "headers" and "body" fields.

What this doesn't yet address is the raw output of message/rfc822
parts.  The raw output of message/rfc822 parts /should/ be the full
raw message.  However, I have not yet figured out how to do that yet,
so the multipart test for raw message/rfc822 parts is not yet fixed.

The problem is really gmime's handling of message parts.  There
doesn't seem to be a gmime function that will return the message part
as a GMimeObject.  This means that we can't access all the message
headers, or the messages sub parts as mime objects.  There may be a
way to hack around gmime's limitations, but I haven't figured it out
yet.  If someone else wants to take a crack at it, that would be
great.  In the mean time, I'm opening a bug report with gmime to
request a function to return the rfc822 part as a GMimeObject.

jamie.

[PATCH 1/5] test: some small fixes to multipart test

[Jameson Graef Rollins]

There were two "--format=text --part=0" tests.  One of them was
supposed to be a test for "--format=text --part=1".

There were also two errant "test_expect_equal_file OUTPUT EXPECTED"
lines, that are removed here.
---
 test/multipart |   17 ++---------------
 1 files changed, 2 insertions(+), 15 deletions(-)

diff --git a/test/multipart b/test/multipart
index 0879696..9d6a9d1 100755
--- a/test/multipart
+++ b/test/multipart
@@ -129,18 +129,9 @@ Non-text part: application/pgp-signature
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=text --part=0, full message"
-notmuch show --format=text --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=text --part=1, message body"
+notmuch show --format=text --part=1 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-\fmessage{ id:87liy5ap00.fsf@yoom.home.cworth.org depth:0 match:1 filename:${MAIL_DIR}/multipart
-\fheader{
-Carl Worth <cworth@cworth.org> (2001-01-05) (attachment inbox signed unread)
-Subject: Multipart message
-From: Carl Worth <cworth@cworth.org>
-To: cworth@cworth.org
-Date: Tue, 05 Jan 2001 15:43:57 -0000
-\fheader}
-\fbody{
 \fpart{ ID: 1, Content-type: multipart/signed
 \fpart{ ID: 2, Content-type: multipart/mixed
 \fpart{ ID: 3, Content-type: message/rfc822
@@ -162,8 +153,6 @@ And this message is signed.
 Non-text part: application/pgp-signature
 \fpart}
 \fpart}
-\fbody}
-\fmessage}
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
@@ -257,13 +246,11 @@ test_begin_subtest "--format=json --part=2, multipart/mixed"
 output=$(notmuch show --format=json --part=2 'id:87liy5ap00.fsf@yoom.home.cworth.org')
 test_expect_equal "$output" \
 '{"id": 2, "content-type": "multipart/mixed", "content": [{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}, {"id": 5, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, {"id": 6, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}'
-test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "--format=json --part=3, rfc822 multipart"
 output=$(notmuch show --format=json --part=3 'id:87liy5ap00.fsf@yoom.home.cworth.org')
 test_expect_equal "$output" \
 '{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}'
-test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "--format=json --part=4, html part"
 output=$(notmuch show --format=json --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-- 
1.7.4.4

[PATCH 2/5] test: Fix date in test message in multipart test.

[Jameson Graef Rollins]

The test message date, "Tue, 05 Jan 2001 15:43:57 -0000", is not
actually a real date.  05 Jan 2001 was in fact a Friday, not a
Tuesday.  Date parsers (such as "date" in coreutils) will return "Fri"
as the day for this string, even if "Tue" is specified.

Also, the time zone "-0000" is actually always returned as "+0000", so
we change that here was well.

This will be relevant for later patches when we begin parsing rfc822
part headers, where gmime returns a parsed date string.

If we do want to test date parsing, we should do that in a separate
test.
---
 test/multipart |   12 ++++++------
 1 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/test/multipart b/test/multipart
index 9d6a9d1..67e04d9 100755
--- a/test/multipart
+++ b/test/multipart
@@ -6,7 +6,7 @@ cat <<EOF > ${MAIL_DIR}/multipart
 From: Carl Worth <cworth@cworth.org>
 To: cworth@cworth.org
 Subject: Multipart message
-Date: Tue, 05 Jan 2001 15:43:57 -0000
+Date: Fri, 05 Jan 2001 15:43:57 +0000
 User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 (i486-pc-linux-gnu)
 Message-ID: <87liy5ap00.fsf@yoom.home.cworth.org>
 MIME-Version: 1.0
@@ -23,7 +23,7 @@ Content-Disposition: inline
 From: Carl Worth <cworth@cworth.org>
 To: cworth@cworth.org
 Subject: html message
-Date: Tue, 05 Jan 2001 15:42:57 -0000
+Date: Fri, 05 Jan 2001 15:42:57 +0000
 User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 (i486-pc-linux-gnu)
 Message-ID: <87liy5ap01.fsf@yoom.home.cworth.org>
 MIME-Version: 1.0
@@ -61,7 +61,7 @@ cat <<EOF > ${MAIL_DIR}/base64-part-with-crlf
 From: Carl Worth <cworth@cworth.org>
 To: cworth@cworth.org
 Subject: Test message with a BASE64 encoded binary containing CRLF pair
-Date: Tue, 05 Jan 2001 15:43:57 -0000
+Date: Fri, 05 Jan 2001 15:43:57 +0000
 User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 (i486-pc-linux-gnu)
 Message-ID: <base64-part-with-crlf>
 MIME-Version: 1.0
@@ -100,7 +100,7 @@ Carl Worth <cworth@cworth.org> (2001-01-05) (attachment inbox signed unread)
 Subject: Multipart message
 From: Carl Worth <cworth@cworth.org>
 To: cworth@cworth.org
-Date: Tue, 05 Jan 2001 15:43:57 -0000
+Date: Fri, 05 Jan 2001 15:43:57 +0000
 \fheader}
 \fbody{
 \fpart{ ID: 1, Content-type: multipart/signed
@@ -235,7 +235,7 @@ test_expect_success \
 test_begin_subtest "--format=json --part=0, full message"
 output=$(notmuch show --format=json --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org')
 test_expect_equal "$output" \
-'{"id": "87liy5ap00.fsf@yoom.home.cworth.org", "match": true, "filename": "'"${MAIL_DIR}/multipart"'", "timestamp": 978709437, "date_relative": "2001-01-05", "tags": ["attachment","inbox","signed","unread"], "headers": {"Subject": "Multipart message", "From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Cc": "", "Bcc": "", "Date": "Tue, 05 Jan 2001 15:43:57 -0000"}, "body": [{"id": 1, "content-type": "multipart/signed", "content": [{"id": 2, "content-type": "multipart/mixed", "content": [{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}, {"id": 5, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, {"id": 6, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}, {"id": 7, "content-type": "application/pgp-signature"}]}]}'
+'{"id": "87liy5ap00.fsf@yoom.home.cworth.org", "match": true, "filename": "'"${MAIL_DIR}/multipart"'", "timestamp": 978709437, "date_relative": "2001-01-05", "tags": ["attachment","inbox","signed","unread"], "headers": {"Subject": "Multipart message", "From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Cc": "", "Bcc": "", "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [{"id": 1, "content-type": "multipart/signed", "content": [{"id": 2, "content-type": "multipart/mixed", "content": [{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}, {"id": 5, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, {"id": 6, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}, {"id": 7, "content-type": "application/pgp-signature"}]}]}'
 
 test_begin_subtest "--format=json --part=1, message body"
 output=$(notmuch show --format=json --part=1 'id:87liy5ap00.fsf@yoom.home.cworth.org')
@@ -385,7 +385,7 @@ To: Carl Worth <cworth@cworth.org>, cworth@cworth.org
 In-Reply-To: <87liy5ap00.fsf@yoom.home.cworth.org>
 References: <87liy5ap00.fsf@yoom.home.cworth.org>
 
-On Tue, 05 Jan 2001 15:43:57 -0000, Carl Worth <cworth@cworth.org> wrote:
+On Fri, 05 Jan 2001 15:43:57 +0000, Carl Worth <cworth@cworth.org> wrote:
 Non-text part: multipart/signed
 Non-text part: multipart/mixed
 Non-text part: message/rfc822
-- 
1.7.4.4

[PATCH 3/5] test: overhaul multipart test to test for improved message/rfc822 handling

[Jameson Graef Rollins]

The main goal of this overhaul is to define how message/rfc822 parts
should be handled.  message/rfc822 parts should be output in a similar
fashion to the outer message, including some subset of the rfc822
headers.  The following decisions about formatting of message/rfc822
parts were made:

The format and content of message/rfc822 parts shall be as similar as
possible to that of full messages.  In particular, for formatted
outputs, the "content" of rfc822 part output should include "headers"
and "body" fields).

The "body" field shall include the body of the message.

The "headers" field shall include the following headers, since these
are the ones available from the GMimeMessage:

  "From"
  "To"
  "Cc"
  "Subject"
  "Date"

However, for the case of --format=raw the raw rfc822 should be output,
including all headers.

A subset of relevant headers shall be output in reply.

The test embedded rfc822 message is also modified to be itself
multipart, so we can more fully test how all sub parts of the message
part are output.
---
 test/multipart |  368 +++++++++++++++++++++++++++++++++++++++++++-------------
 1 files changed, 283 insertions(+), 85 deletions(-)

diff --git a/test/multipart b/test/multipart
index 67e04d9..95577dc 100755
--- a/test/multipart
+++ b/test/multipart
@@ -2,6 +2,29 @@
 test_description="output of multipart message"
 . ./test-lib.sh
 
+cat <<EOF > embedded_message
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 (i486-pc-linux-gnu)
+Message-ID: <87liy5ap01.fsf@yoom.home.cworth.org>
+MIME-Version: 1.0
+Content-Type: multipart/alternative; boundary="==-=-=="
+
+--==-=-==
+Content-Type: text/html
+
+<p>This is an embedded message, with a multipart/alternative part.</p>
+
+--==-=-==
+Content-Type: text/plain
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==--
+EOF
+
 cat <<EOF > ${MAIL_DIR}/multipart
 From: Carl Worth <cworth@cworth.org>
 To: cworth@cworth.org
@@ -20,17 +43,9 @@ Content-Type: multipart/mixed; boundary="=-=-="
 Content-Type: message/rfc822
 Content-Disposition: inline
 
-From: Carl Worth <cworth@cworth.org>
-To: cworth@cworth.org
-Subject: html message
-Date: Fri, 05 Jan 2001 15:42:57 +0000
-User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 (i486-pc-linux-gnu)
-Message-ID: <87liy5ap01.fsf@yoom.home.cworth.org>
-MIME-Version: 1.0
-Content-Type: text/html
-
-<p>This is an embedded message, with a single html part.</p>
-
+EOF
+cat embedded_message >> ${MAIL_DIR}/multipart
+cat <<EOF >> ${MAIL_DIR}/multipart
 --=-=-=
 Content-Disposition: attachment; filename=attachment
 
@@ -106,21 +121,34 @@ Date: Fri, 05 Jan 2001 15:43:57 +0000
 \fpart{ ID: 1, Content-type: multipart/signed
 \fpart{ ID: 2, Content-type: multipart/mixed
 \fpart{ ID: 3, Content-type: message/rfc822
-\fpart{ ID: 4, Content-type: text/html
+\fheader{
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 4, Content-type: multipart/alternative
+\fpart{ ID: 5, Content-type: text/html
 Non-text part: text/html
 \fpart}
+\fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
+\fpart}
+\fbody}
 \fpart}
-\fattachment{ ID: 5, Content-type: text/plain
+\fattachment{ ID: 7, Content-type: text/plain
 Attachment: attachment (text/plain)
 This is a text attachment.
 \fattachment}
-\fpart{ ID: 6, Content-type: text/plain
+\fpart{ ID: 8, Content-type: text/plain
 And this message is signed.
 
 -Carl
 \fpart}
 \fpart}
-\fpart{ ID: 7, Content-type: application/pgp-signature
+\fpart{ ID: 9, Content-type: application/pgp-signature
 Non-text part: application/pgp-signature
 \fpart}
 \fpart}
@@ -135,21 +163,34 @@ cat <<EOF >EXPECTED
 \fpart{ ID: 1, Content-type: multipart/signed
 \fpart{ ID: 2, Content-type: multipart/mixed
 \fpart{ ID: 3, Content-type: message/rfc822
-\fpart{ ID: 4, Content-type: text/html
+\fheader{
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 4, Content-type: multipart/alternative
+\fpart{ ID: 5, Content-type: text/html
 Non-text part: text/html
 \fpart}
+\fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
 \fpart}
-\fattachment{ ID: 5, Content-type: text/plain
+\fbody}
+\fpart}
+\fattachment{ ID: 7, Content-type: text/plain
 Attachment: attachment (text/plain)
 This is a text attachment.
 \fattachment}
-\fpart{ ID: 6, Content-type: text/plain
+\fpart{ ID: 8, Content-type: text/plain
 And this message is signed.
 
 -Carl
 \fpart}
 \fpart}
-\fpart{ ID: 7, Content-type: application/pgp-signature
+\fpart{ ID: 9, Content-type: application/pgp-signature
 Non-text part: application/pgp-signature
 \fpart}
 \fpart}
@@ -161,15 +202,28 @@ notmuch show --format=text --part=2 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OU
 cat <<EOF >EXPECTED
 \fpart{ ID: 2, Content-type: multipart/mixed
 \fpart{ ID: 3, Content-type: message/rfc822
-\fpart{ ID: 4, Content-type: text/html
+\fheader{
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 4, Content-type: multipart/alternative
+\fpart{ ID: 5, Content-type: text/html
 Non-text part: text/html
 \fpart}
+\fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
 \fpart}
-\fattachment{ ID: 5, Content-type: text/plain
+\fbody}
+\fpart}
+\fattachment{ ID: 7, Content-type: text/plain
 Attachment: attachment (text/plain)
 This is a text attachment.
 \fattachment}
-\fpart{ ID: 6, Content-type: text/plain
+\fpart{ ID: 8, Content-type: text/plain
 And this message is signed.
 
 -Carl
@@ -178,40 +232,76 @@ And this message is signed.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=text --part=3, rfc822 multipart"
+test_begin_subtest "--format=text --part=3, rfc822 part"
 notmuch show --format=text --part=3 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
 \fpart{ ID: 3, Content-type: message/rfc822
-\fpart{ ID: 4, Content-type: text/html
+\fheader{
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 4, Content-type: multipart/alternative
+\fpart{ ID: 5, Content-type: text/html
 Non-text part: text/html
 \fpart}
+\fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
+\fpart}
+\fbody}
 \fpart}
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=text --part=4, html part"
+test_begin_subtest "--format=text --part=4, rfc822's multipart"
 notmuch show --format=text --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-\fpart{ ID: 4, Content-type: text/html
+\fpart{ ID: 4, Content-type: multipart/alternative
+\fpart{ ID: 5, Content-type: text/html
 Non-text part: text/html
 \fpart}
+\fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
+\fpart}
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=text --part=5, inline attachement"
+test_begin_subtest "--format=text --part=5, rfc822's html part"
 notmuch show --format=text --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-\fattachment{ ID: 5, Content-type: text/plain
+\fpart{ ID: 5, Content-type: text/html
+Non-text part: text/html
+\fpart}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=text --part=6, rfc822's text part"
+notmuch show --format=text --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+cat <<EOF >EXPECTED
+\fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=text --part=7, inline attachement"
+notmuch show --format=text --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+cat <<EOF >EXPECTED
+\fattachment{ ID: 7, Content-type: text/plain
 Attachment: attachment (text/plain)
 This is a text attachment.
 \fattachment}
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=text --part=6, plain text part"
-notmuch show --format=text --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=text --part=8, plain text part"
+notmuch show --format=text --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-\fpart{ ID: 6, Content-type: text/plain
+\fpart{ ID: 8, Content-type: text/plain
 And this message is signed.
 
 -Carl
@@ -219,10 +309,10 @@ And this message is signed.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=text --part=7, pgp signature (unverified)"
-notmuch show --format=text --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=text --part=9, pgp signature (unverified)"
+notmuch show --format=text --part=9 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-\fpart{ ID: 7, Content-type: application/pgp-signature
+\fpart{ ID: 9, Content-type: application/pgp-signature
 Non-text part: application/pgp-signature
 \fpart}
 EOF
@@ -233,48 +323,126 @@ test_expect_success \
     "notmuch show --format=text --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
 
 test_begin_subtest "--format=json --part=0, full message"
-output=$(notmuch show --format=json --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": "87liy5ap00.fsf@yoom.home.cworth.org", "match": true, "filename": "'"${MAIL_DIR}/multipart"'", "timestamp": 978709437, "date_relative": "2001-01-05", "tags": ["attachment","inbox","signed","unread"], "headers": {"Subject": "Multipart message", "From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Cc": "", "Bcc": "", "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [{"id": 1, "content-type": "multipart/signed", "content": [{"id": 2, "content-type": "multipart/mixed", "content": [{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}, {"id": 5, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, {"id": 6, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}, {"id": 7, "content-type": "application/pgp-signature"}]}]}'
+notmuch show --format=json --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": "87liy5ap00.fsf@yoom.home.cworth.org", "match": true, "filename": "${MAIL_DIR}/multipart", "timestamp": 978709437, "date_relative": "2001-01-05", "tags": ["attachment","inbox","signed","unread"], "headers": {"Subject": "Multipart message", "From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Cc": "", "Bcc": "", "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [
+{"id": 1, "content-type": "multipart/signed", "content": [
+{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
+{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 5, "content-type": "text/html"}, 
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]}, 
+{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, 
+{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}, 
+{"id": 9, "content-type": "application/pgp-signature"}]}]}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "--format=json --part=1, message body"
-output=$(notmuch show --format=json --part=1 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 1, "content-type": "multipart/signed", "content": [{"id": 2, "content-type": "multipart/mixed", "content": [{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}, {"id": 5, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, {"id": 6, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}, {"id": 7, "content-type": "application/pgp-signature"}]}'
+notmuch show --format=json --part=1 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 1, "content-type": "multipart/signed", "content": [
+{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
+{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 5, "content-type": "text/html"}, 
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]}, 
+{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, 
+{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}, 
+{"id": 9, "content-type": "application/pgp-signature"}]}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "--format=json --part=2, multipart/mixed"
-output=$(notmuch show --format=json --part=2 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 2, "content-type": "multipart/mixed", "content": [{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}, {"id": 5, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, {"id": 6, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}'
-
-test_begin_subtest "--format=json --part=3, rfc822 multipart"
-output=$(notmuch show --format=json --part=3 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}'
-
-test_begin_subtest "--format=json --part=4, html part"
-output=$(notmuch show --format=json --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 4, "content-type": "text/html"}'
-
-test_begin_subtest "--format=json --part=5, inline attachment"
-output=$(notmuch show --format=json --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 5, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}'
-
-test_begin_subtest "--format=json --part=6, plain text part"
-output=$(notmuch show --format=json --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 6, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}'
-
-test_begin_subtest "--format=json --part=7, pgp signature (unverified)"
-output=$(notmuch show --format=json --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 7, "content-type": "application/pgp-signature"}'
+notmuch show --format=json --part=2 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
+{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 5, "content-type": "text/html"}, 
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]}, 
+{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, 
+{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=3, rfc822 part"
+notmuch show --format=json --part=3 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
+{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 5, "content-type": "text/html"}, 
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=4, rfc822's multipart/alternative"
+notmuch show --format=json --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 5, "content-type": "text/html"}, 
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=5, rfc822's html part"
+notmuch show --format=json --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 5, "content-type": "text/html"}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=6, rfc822's text part"
+notmuch show --format=json --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=7, inline attachment"
+notmuch show --format=json --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=8, plain text part"
+notmuch show --format=json --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=9, pgp signature (unverified)"
+notmuch show --format=json --part=9 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 9, "content-type": "application/pgp-signature"}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
 
 test_expect_success \
-    "--format=json --part=8, no part, expect error" \
-    "notmuch show --format=json --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
+    "--format=json --part=10, no part, expect error" \
+    "notmuch show --format=json --part=10 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
 
 test_begin_subtest "--format=raw"
 notmuch show --format=raw 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
@@ -289,7 +457,13 @@ notmuch show --format=raw --part=1 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUT
 # output should *not* include newline
 echo >>OUTPUT
 cat <<EOF >EXPECTED
-<p>This is an embedded message, with a single html part.</p>
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+
+<p>This is an embedded message, with a multipart/alternative part.</p>
+This is an embedded message, with a multipart/alternative part.
 This is a text attachment.
 And this message is signed.
 
@@ -307,7 +481,13 @@ test_expect_equal_file OUTPUT EXPECTED
 test_begin_subtest "--format=raw --part=2, multipart/mixed"
 notmuch show --format=raw --part=2 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-<p>This is an embedded message, with a single html part.</p>
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+
+<p>This is an embedded message, with a multipart/alternative part.</p>
+This is an embedded message, with a multipart/alternative part.
 This is a text attachment.
 And this message is signed.
 
@@ -315,29 +495,41 @@ And this message is signed.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=raw --part=3, rfc822 multipart"
+test_begin_subtest "--format=raw --part=3, rfc822 part"
 notmuch show --format=raw --part=3 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_expect_equal_file OUTPUT embedded_message
+
+test_begin_subtest "--format=raw --part=4, rfc822's html part"
+notmuch show --format=raw --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-<p>This is an embedded message, with a single html part.</p>
+<p>This is an embedded message, with a multipart/alternative part.</p>
+This is an embedded message, with a multipart/alternative part.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=raw --part=4, html part"
-notmuch show --format=raw --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=raw --part=5, rfc822's html part"
+notmuch show --format=raw --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-<p>This is an embedded message, with a single html part.</p>
+<p>This is an embedded message, with a multipart/alternative part.</p>
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=raw --part=5, inline attachment"
-notmuch show --format=raw --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=raw --part=6, rfc822's text part"
+notmuch show --format=raw --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+cat <<EOF >EXPECTED
+This is an embedded message, with a multipart/alternative part.
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=raw --part=7, inline attachment"
+notmuch show --format=raw --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
 This is a text attachment.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=raw --part=6, plain text part"
-notmuch show --format=raw --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=raw --part=8, plain text part"
+notmuch show --format=raw --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
 And this message is signed.
 
@@ -345,8 +537,8 @@ And this message is signed.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=raw --part=7, pgp signature (unverified)"
-notmuch show --format=raw --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=raw --part=9, pgp signature (unverified)"
+notmuch show --format=raw --part=9 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 # output should *not* include newline
 echo >>OUTPUT
 cat <<EOF >EXPECTED
@@ -361,7 +553,7 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_expect_success \
-    "--format=raw --part=8, no part, expect error" \
+    "--format=raw --part=10, no part, expect error" \
     "notmuch show --format=raw --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
 
 test_begin_subtest "--format=mbox"
@@ -388,8 +580,14 @@ References: <87liy5ap00.fsf@yoom.home.cworth.org>
 On Fri, 05 Jan 2001 15:43:57 +0000, Carl Worth <cworth@cworth.org> wrote:
 Non-text part: multipart/signed
 Non-text part: multipart/mixed
-Non-text part: message/rfc822
+> From: Carl Worth <cworth@cworth.org>
+> To: cworth@cworth.org
+> Subject: html message
+> Date: Fri, 05 Jan 2001 15:42:57 +0000
+>
+Non-text part: multipart/alternative
 Non-text part: text/html
+> This is an embedded message, with a multipart/alternative part.
 > This is a text attachment.
 > And this message is signed.
 > 
-- 
1.7.4.4

[PATCH 4/5] Improve handling of message/rfc822 parts by adding a new header_message_part function to the formating structure.

[Jameson Graef Rollins]

This new function takes a GMimeMessage as input, and outputs the
formatted headers.  This allows for message/rfc822 parts to be
formatted on output in a similar way to full messages (see previous
patch that overhauls the multipart test for more info).
---
 notmuch-client.h |    1 +
 notmuch-reply.c  |   31 ++++++++++++++++++-
 notmuch-show.c   |   88 ++++++++++++++++++++++++++++++++++++++++++++++-------
 show-message.c   |   12 +++++++
 4 files changed, 119 insertions(+), 13 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 63be337..b50cb38 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -63,6 +63,7 @@ typedef struct notmuch_show_format {
     const char *header_start;
     void (*header) (const void *ctx,
 		    notmuch_message_t *message);
+    void (*header_message_part) (GMimeMessage *message);
     const char *header_end;
     const char *body_start;
     void (*part_start) (GMimeObject *part,
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 514bbc6..a19eb19 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -25,12 +25,15 @@
 #include "gmime-filter-headers.h"
 
 static void
+reply_headers_message_part (GMimeMessage *message);
+
+static void
 reply_part_content (GMimeObject *part);
 
 static const notmuch_show_format_t format_reply = {
     "",
 	"", NULL,
-	    "", NULL, "",
+	    "", NULL, reply_headers_message_part, ">\n",
 	    "",
 	        NULL,
 	        NULL,
@@ -63,6 +66,28 @@ show_reply_headers (GMimeMessage *message)
 }
 
 static void
+reply_headers_message_part (GMimeMessage *message)
+{
+    InternetAddressList *recipients;
+    const char *recipients_string;
+
+    printf ("> From: %s\n", g_mime_message_get_sender (message));
+    recipients = g_mime_message_get_recipients (message, GMIME_RECIPIENT_TYPE_TO);
+    recipients_string = internet_address_list_to_string (recipients, 0);
+    if (recipients_string)
+	printf ("> To: %s\n",
+		recipients_string);
+    recipients = g_mime_message_get_recipients (message, GMIME_RECIPIENT_TYPE_CC);
+    recipients_string = internet_address_list_to_string (recipients, 0);
+    if (recipients_string)
+	printf ("> Cc: %s\n",
+		recipients_string);
+    printf ("> Subject: %s\n", g_mime_message_get_subject (message));
+    printf ("> Date: %s\n", g_mime_message_get_date_as_string (message));
+}
+
+
+static void
 reply_part_content (GMimeObject *part)
 {
     GMimeContentType *content_type = g_mime_object_get_content_type (GMIME_OBJECT (part));
@@ -95,6 +120,10 @@ reply_part_content (GMimeObject *part)
 	if (stream_stdout)
 	    g_object_unref(stream_stdout);
     }
+    else if (g_mime_content_type_is_type (content_type, "message", "rfc822"))
+    {
+	/* Output nothing, since rfc822 subparts will be handled individually. */
+    }
     else
     {
 	if (disposition &&
diff --git a/notmuch-show.c b/notmuch-show.c
index 0e9d00d..6e2e560 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -29,6 +29,9 @@ format_headers_text (const void *ctx,
 		     notmuch_message_t *message);
 
 static void
+format_headers_message_part_text (GMimeMessage *message);
+
+static void
 format_part_start_text (GMimeObject *part,
 			int *part_count);
 
@@ -41,7 +44,7 @@ format_part_end_text (GMimeObject *part);
 static const notmuch_show_format_t format_text = {
     "",
 	"\fmessage{ ", format_message_text,
-	    "\fheader{\n", format_headers_text, "\fheader}\n",
+	    "\fheader{\n", format_headers_text, format_headers_message_part_text, "\fheader}\n",
 	    "\fbody{\n",
 	        format_part_start_text,
 	        NULL,
@@ -63,6 +66,9 @@ format_headers_json (const void *ctx,
 		     notmuch_message_t *message);
 
 static void
+format_headers_message_part_json (GMimeMessage *message);
+
+static void
 format_part_start_json (unused (GMimeObject *part),
 			int *part_count);
 
@@ -81,7 +87,7 @@ format_part_end_json (GMimeObject *part);
 static const notmuch_show_format_t format_json = {
     "[",
 	"{", format_message_json,
-	    ", \"headers\": {", format_headers_json, "}",
+	    "\"headers\": {", format_headers_json, format_headers_message_part_json, "}",
 	    ", \"body\": [",
 	        format_part_start_json,
 	        format_part_encstatus_json,
@@ -102,7 +108,7 @@ format_message_mbox (const void *ctx,
 static const notmuch_show_format_t format_mbox = {
     "",
         "", format_message_mbox,
-            "", NULL, "",
+            "", NULL, NULL, "",
             "",
                 NULL,
                 NULL,
@@ -121,7 +127,7 @@ format_part_content_raw (GMimeObject *part);
 static const notmuch_show_format_t format_raw = {
     "",
 	"", NULL,
-	    "", NULL, "",
+	    "", NULL, format_headers_message_part_text, "\n",
             "",
                 NULL,
                 NULL,
@@ -216,7 +222,7 @@ format_message_json (const void *ctx, notmuch_message_t *message, unused (int in
                json_quote_str (ctx_quote, notmuch_tags_get (tags)));
          first = 0;
     }
-    printf("]");
+    printf("], ");
     talloc_free (ctx_quote);
 }
 
@@ -326,6 +332,7 @@ format_message_mbox (const void *ctx,
     fclose (file);
 }
 
+
 static void
 format_headers_text (const void *ctx, notmuch_message_t *message)
 {
@@ -346,6 +353,27 @@ format_headers_text (const void *ctx, notmuch_message_t *message)
 }
 
 static void
+format_headers_message_part_text (GMimeMessage *message)
+{
+    InternetAddressList *recipients;
+    const char *recipients_string;
+
+    printf ("From: %s\n", g_mime_message_get_sender (message));
+    recipients = g_mime_message_get_recipients (message, GMIME_RECIPIENT_TYPE_TO);
+    recipients_string = internet_address_list_to_string (recipients, 0);
+    if (recipients_string)
+	printf ("To: %s\n",
+		recipients_string);
+    recipients = g_mime_message_get_recipients (message, GMIME_RECIPIENT_TYPE_CC);
+    recipients_string = internet_address_list_to_string (recipients, 0);
+    if (recipients_string)
+	printf ("Cc: %s\n",
+		recipients_string);
+    printf ("Subject: %s\n", g_mime_message_get_subject (message));
+    printf ("Date: %s\n", g_mime_message_get_date_as_string (message));
+}
+
+static void
 format_headers_json (const void *ctx, notmuch_message_t *message)
 {
     const char *headers[] = {
@@ -374,6 +402,40 @@ format_headers_json (const void *ctx, notmuch_message_t *message)
     talloc_free (ctx_quote);
 }
 
+static void
+format_headers_message_part_json (GMimeMessage *message)
+{
+    void *ctx = talloc_new (NULL);
+    void *ctx_quote = talloc_new (ctx);
+    InternetAddressList *recipients;
+    const char *recipients_string;
+
+    printf ("%s: %s",
+	    json_quote_str (ctx_quote, "From"),
+	    json_quote_str (ctx_quote, g_mime_message_get_sender (message)));
+    recipients = g_mime_message_get_recipients (message, GMIME_RECIPIENT_TYPE_TO);
+    recipients_string = internet_address_list_to_string (recipients, 0);
+    if (recipients_string)
+	printf (", %s: %s",
+		json_quote_str (ctx_quote, "To"),
+		json_quote_str (ctx_quote, recipients_string));
+    recipients = g_mime_message_get_recipients (message, GMIME_RECIPIENT_TYPE_CC);
+    recipients_string = internet_address_list_to_string (recipients, 0);
+    if (recipients_string)
+	printf (", %s: %s",
+		json_quote_str (ctx_quote, "Cc"),
+		json_quote_str (ctx_quote, recipients_string));
+    printf (", %s: %s",
+	    json_quote_str (ctx_quote, "Subject"),
+	    json_quote_str (ctx_quote, g_mime_message_get_subject (message)));
+    printf (", %s: %s",
+	    json_quote_str (ctx_quote, "Date"),
+	    json_quote_str (ctx_quote, g_mime_message_get_date_as_string (message)));
+
+    talloc_free (ctx_quote);
+    talloc_free (ctx);
+}
+
 /* Write a MIME text part out to the given stream.
  *
  * Both line-ending conversion (CRLF->LF) and charset conversion ( ->
@@ -616,11 +678,14 @@ format_part_content_json (GMimeObject *part)
 
 	printf (", \"content\": %s", json_quote_chararray (ctx, (char *) part_content->data, part_content->len));
     }
-    else if (g_mime_content_type_is_type (content_type, "multipart", "*") ||
-	     g_mime_content_type_is_type (content_type, "message", "rfc822"))
+    else if (g_mime_content_type_is_type (content_type, "multipart", "*"))
     {
 	printf (", \"content\": [");
     }
+    else if (g_mime_content_type_is_type (content_type, "message", "rfc822"))
+    {
+	printf (", \"content\": [{");
+    }
 
     talloc_free (ctx);
     if (stream_memory)
@@ -630,13 +695,12 @@ format_part_content_json (GMimeObject *part)
 static void
 format_part_end_json (GMimeObject *part)
 {
-    GMimeContentType *content_type;
-
-    content_type = g_mime_object_get_content_type (GMIME_OBJECT (part));
+    GMimeContentType *content_type = g_mime_object_get_content_type (GMIME_OBJECT (part));
 
-    if (g_mime_content_type_is_type (content_type, "multipart", "*") ||
-	g_mime_content_type_is_type (content_type, "message", "rfc822"))
+    if (g_mime_content_type_is_type (content_type, "multipart", "*"))
 	printf ("]");
+    else if (g_mime_content_type_is_type (content_type, "message", "rfc822"))
+	printf ("}]");
 
     printf ("}");
 }
diff --git a/show-message.c b/show-message.c
index 7a4bbc2..d83f04e 100644
--- a/show-message.c
+++ b/show-message.c
@@ -149,9 +149,21 @@ show_message_part (GMimeObject *part,
 	if (selected)
 	    state->in_zone = 1;
 
+	if (selected || (!selected && state->in_zone)) {
+	    fputs (format->header_start, stdout);
+	    if (format->header_message_part)
+		format->header_message_part (mime_message);
+	    fputs (format->header_end, stdout);
+
+	    fputs (format->body_start, stdout);
+	}
+
 	show_message_part (g_mime_message_get_mime_part (mime_message),
 			   state, format, params, TRUE);
 
+	if (selected || (!selected && state->in_zone))
+	    fputs (format->body_end, stdout);
+
 	if (selected)
 	    state->in_zone = 0;
     }
-- 
1.7.4.4

[PATCH 5/5] emacs: Improve support for message/rfc822 parts.

[Jameson Graef Rollins]

The insert-part-message/rfc822 function is overhauled to properly
processes the new formatting of message/rfc822 parts.  The json output
for message parts now includes "headers" and "body" fields, which are
now parsed and output appropriately.
---
 emacs/notmuch-show.el |   21 +++++++++++++--------
 1 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 7d848e3..0f1fe69 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -519,21 +519,26 @@ current buffer, if possible."
   t)
 
 (defun notmuch-show-insert-part-message/rfc822 (msg part content-type nth depth declared-type)
-  (let* ((message-part (plist-get part :content))
-	 (inner-parts (plist-get message-part :content)))
-    (notmuch-show-insert-part-header nth declared-type content-type nil)
+  (notmuch-show-insert-part-header nth declared-type content-type nil)
+  (let* ((message (car (plist-get part :content)))
+	 (headers (plist-get message :headers))
+	 (body (car (plist-get message :body)))
+	 (start (point)))
+
     ;; Override `notmuch-message-headers' to force `From' to be
     ;; displayed.
     (let ((notmuch-message-headers '("From" "Subject" "To" "Cc" "Date")))
-      (notmuch-show-insert-headers (plist-get part :headers)))
+      (notmuch-show-insert-headers (plist-get message :headers)))
+
     ;; Blank line after headers to be compatible with the normal
     ;; message display.
     (insert "\n")
 
-    ;; Show all of the parts.
-    (mapc (lambda (inner-part)
-	    (notmuch-show-insert-bodypart msg inner-part depth))
-	  inner-parts))
+    ;; Show the body
+    (notmuch-show-insert-bodypart msg body depth)
+
+    (when notmuch-show-indent-multipart
+      (indent-rigidly start (point) 1)))
   t)
 
 (defun notmuch-show-insert-part-text/plain (msg part content-type nth depth declared-type)
-- 
1.7.4.4

Re: improving message/rfc822 part handling

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 221 bytes --]

I forgot to mention that this patch series (well the last patch in
particular) supersedes the previous emacs rfc822 part handling patch I
sent in:

id:"1307034386-6107-1-git-send-email-jrollins@finestructure.net"

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: improving message/rfc822 part handling

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 129 bytes --]

Here's the gmime bug about returning rfc822 messages as GMimeObjects:

https://bugzilla.gnome.org/show_bug.cgi?id=651964

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

tweaks for handling parts in reply

[Jameson Graef Rollins]

This is a small set of tweaks to remove unneccessary "Non-text part:"
lines in reply, for parts that really don't need to be mentioned.

This patch set is not really related to the series that it is being
sent in reply to, but it has been worked on top of the message/rfc822
rework of the multipart test, so it's being sent in reply to that
message.

jamie.

[PATCH 1/4] test/multipart: test for absence of "Non-text part:" lines in reply for multipart/* and message/rfc822 parts

[Jameson Graef Rollins]

There's no reason to output "Non-text part:" lines for parts that are
not leaf nodes, eg. multipart/* and message/rfc822.  We fix the text
here to test for their absence.  The next patch will fix reply
accordingly.
---
 test/multipart |    3 ---
 1 files changed, 0 insertions(+), 3 deletions(-)

diff --git a/test/multipart b/test/multipart
index 95577dc..c7431cc 100755
--- a/test/multipart
+++ b/test/multipart
@@ -578,14 +578,11 @@ In-Reply-To: <87liy5ap00.fsf@yoom.home.cworth.org>
 References: <87liy5ap00.fsf@yoom.home.cworth.org>
 
 On Fri, 05 Jan 2001 15:43:57 +0000, Carl Worth <cworth@cworth.org> wrote:
-Non-text part: multipart/signed
-Non-text part: multipart/mixed
 > From: Carl Worth <cworth@cworth.org>
 > To: cworth@cworth.org
 > Subject: html message
 > Date: Fri, 05 Jan 2001 15:42:57 +0000
 >
-Non-text part: multipart/alternative
 Non-text part: text/html
 > This is an embedded message, with a multipart/alternative part.
 > This is a text attachment.
-- 
1.7.5.3

[PATCH 2/4] Fix notmuch-reply to not output "Non-text part:" lines for non-leafnode parts.

[Jameson Graef Rollins]

These lines are just cruft in this case, and can be removed.
---
 notmuch-reply.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/notmuch-reply.c b/notmuch-reply.c
index a19eb19..7a76ba3 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -93,7 +93,12 @@ reply_part_content (GMimeObject *part)
     GMimeContentType *content_type = g_mime_object_get_content_type (GMIME_OBJECT (part));
     GMimeContentDisposition *disposition = g_mime_object_get_content_disposition (part);
 
-    if (g_mime_content_type_is_type (content_type, "text", "*") &&
+    if (g_mime_content_type_is_type (content_type, "multipart", "*") ||
+	g_mime_content_type_is_type (content_type, "message", "rfc822"))
+    {
+	/* Output nothing, since multipart subparts will be handled individually. */
+    }
+    else if (g_mime_content_type_is_type (content_type, "text", "*") &&
 	!g_mime_content_type_is_type (content_type, "text", "html"))
     {
 	GMimeStream *stream_stdout = NULL, *stream_filter = NULL;
@@ -120,10 +125,6 @@ reply_part_content (GMimeObject *part)
 	if (stream_stdout)
 	    g_object_unref(stream_stdout);
     }
-    else if (g_mime_content_type_is_type (content_type, "message", "rfc822"))
-    {
-	/* Output nothing, since rfc822 subparts will be handled individually. */
-    }
     else
     {
 	if (disposition &&
-- 
1.7.5.3

[PATCH 3/4] test: test for absence of "Non-text part: application/pgp-*" lines in reply

[Jameson Graef Rollins]

In reply, the quoted text does not need to mention that the original
message had "application/pgp-signed" or "application/pgp-encrypted"
parts.
---
 test/crypto    |    2 --
 test/multipart |    1 -
 2 files changed, 0 insertions(+), 3 deletions(-)

diff --git a/test/crypto b/test/crypto
index 8e92016..cf096c0 100755
--- a/test/crypto
+++ b/test/crypto
@@ -310,8 +310,6 @@ expected='From: Notmuch Test Suite <test_suite@notmuchmail.org>
 Subject: Re: test encrypted message 002
 
 On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote:
-Non-text part: multipart/encrypted
-Non-text part: application/pgp-encrypted
 > This is another test encrypted message.'
 test_expect_equal \
     "$output" \
diff --git a/test/multipart b/test/multipart
index c7431cc..e333a67 100755
--- a/test/multipart
+++ b/test/multipart
@@ -589,7 +589,6 @@ Non-text part: text/html
 > And this message is signed.
 > 
 > -Carl
-Non-text part: application/pgp-signature
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-- 
1.7.5.3

[PATCH 4/4] Ignore "application/pgp-*" parts in reply.

[Jameson Graef Rollins]

The quoted text doesn't need to mention that the message being replied
to had these crufty parts.
---
 notmuch-reply.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/notmuch-reply.c b/notmuch-reply.c
index 7a76ba3..0eb295e 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -98,6 +98,11 @@ reply_part_content (GMimeObject *part)
     {
 	/* Output nothing, since multipart subparts will be handled individually. */
     }
+    else if (g_mime_content_type_is_type (content_type, "application", "pgp-encrypted") ||
+	     g_mime_content_type_is_type (content_type, "application", "pgp-signature"))
+    {
+	/* Ignore PGP/MIME cruft parts */
+    }
     else if (g_mime_content_type_is_type (content_type, "text", "*") &&
 	!g_mime_content_type_is_type (content_type, "text", "html"))
     {
-- 
1.7.5.3

Re: tweaks for handling parts in reply

[Austin Clements]

It's too bad this requires lots of special-casing.  I dug into mutt,
hoping to find some simpler way to choose what to reply to, but all I
found was a far more complicated collection of heuristics (see
mutt_body_handler if you're interested).

Should reply *ever* include a "Non-text part" message?  It certainly
doesn't help the receiver of the reply and presumably the sender will
either notice that a part is missing if they care about it, or
probably miss the Non-text part message anyway if they don't.

On Wed, Jun 8, 2011 at 3:30 PM, Jameson Graef Rollins
<jrollins@finestructure.net> wrote:
> This is a small set of tweaks to remove unneccessary "Non-text part:"
> lines in reply, for parts that really don't need to be mentioned.
>
> This patch set is not really related to the series that it is being
> sent in reply to, but it has been worked on top of the message/rfc822
> rework of the multipart test, so it's being sent in reply to that
> message.
>
> jamie.
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
>

Re: [PATCH 2/5] test: Fix date in test message in multipart test.

[David Bremner]

On Sun,  5 Jun 2011 17:29:26 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> The test message date, "Tue, 05 Jan 2001 15:43:57 -0000", is not
> actually a real date.  05 Jan 2001 was in fact a Friday, not a
> Tuesday.  Date parsers (such as "date" in coreutils) will return "Fri"
> as the day for this string, even if "Tue" is specified.
> 

I pushed parts 1 and 2 of this series. As discussed on IRC, part 3 has
conflicts currently, but I think the first two patches stand by
themselves anyway.

d

[PATCH] test: overhaul multipart test to test for improved message/rfc822 handling

[Jameson Graef Rollins]

The main goal of this overhaul is to define how message/rfc822 parts
should be handled.  message/rfc822 parts should be output in a similar
fashion to the outer message, including some subset of the rfc822
headers.  The following decisions about formatting of message/rfc822
parts were made:

The format and content of message/rfc822 parts shall be as similar as
possible to that of full messages.  In particular, for formatted
outputs, the "content" of rfc822 part output should include "headers"
and "body" fields).

The "body" field shall include the body of the message.

The "headers" field shall include the following headers, since these
are the ones available from the GMimeMessage:

  "From"
  "To"
  "Cc"
  "Subject"
  "Date"

However, for the case of --format=raw the raw rfc822 should be output,
including all headers.

A subset of relevant headers shall be output in reply.

The test embedded rfc822 message is also modified to be itself
multipart, so we can more fully test how all sub parts of the message
part are output.
---
This updated patch fixes some conflicts with the current master head.

 test/multipart |  368 +++++++++++++++++++++++++++++++++++++++++++-------------
 1 files changed, 283 insertions(+), 85 deletions(-)

diff --git a/test/multipart b/test/multipart
index ef859d1..0fd1c7e 100755
--- a/test/multipart
+++ b/test/multipart
@@ -2,6 +2,29 @@
 test_description="output of multipart message"
 . ./test-lib.sh
 
+cat <<EOF > embedded_message
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 (i486-pc-linux-gnu)
+Message-ID: <87liy5ap01.fsf@yoom.home.cworth.org>
+MIME-Version: 1.0
+Content-Type: multipart/alternative; boundary="==-=-=="
+
+--==-=-==
+Content-Type: text/html
+
+<p>This is an embedded message, with a multipart/alternative part.</p>
+
+--==-=-==
+Content-Type: text/plain
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==--
+EOF
+
 cat <<EOF > ${MAIL_DIR}/multipart
 From: Carl Worth <cworth@cworth.org>
 To: cworth@cworth.org
@@ -20,17 +43,9 @@ Content-Type: multipart/mixed; boundary="=-=-="
 Content-Type: message/rfc822
 Content-Disposition: inline
 
-From: Carl Worth <cworth@cworth.org>
-To: cworth@cworth.org
-Subject: html message
-Date: Fri, 05 Jan 2001 15:42:57 +0000
-User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 (i486-pc-linux-gnu)
-Message-ID: <87liy5ap01.fsf@yoom.home.cworth.org>
-MIME-Version: 1.0
-Content-Type: text/html
-
-<p>This is an embedded message, with a single html part.</p>
-
+EOF
+cat embedded_message >> ${MAIL_DIR}/multipart
+cat <<EOF >> ${MAIL_DIR}/multipart
 --=-=-=
 Content-Disposition: attachment; filename=attachment
 
@@ -105,20 +120,33 @@ Date: Fri, 05 Jan 2001 15:43:57 +0000
 \fpart{ ID: 1, Content-type: multipart/signed
 \fpart{ ID: 2, Content-type: multipart/mixed
 \fpart{ ID: 3, Content-type: message/rfc822
-\fpart{ ID: 4, Content-type: text/html
+\fheader{
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 4, Content-type: multipart/alternative
+\fpart{ ID: 5, Content-type: text/html
 Non-text part: text/html
 \fpart}
+\fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
+\fpart}
+\fbody}
 \fpart}
-\fattachment{ ID: 5, Filename: attachment, Content-type: text/plain
+\fattachment{ ID: 7, Filename: attachment, Content-type: text/plain
 This is a text attachment.
 \fattachment}
-\fpart{ ID: 6, Content-type: text/plain
+\fpart{ ID: 8, Content-type: text/plain
 And this message is signed.
 
 -Carl
 \fpart}
 \fpart}
-\fpart{ ID: 7, Content-type: application/pgp-signature
+\fpart{ ID: 9, Content-type: application/pgp-signature
 Non-text part: application/pgp-signature
 \fpart}
 \fpart}
@@ -133,20 +161,33 @@ cat <<EOF >EXPECTED
 \fpart{ ID: 1, Content-type: multipart/signed
 \fpart{ ID: 2, Content-type: multipart/mixed
 \fpart{ ID: 3, Content-type: message/rfc822
-\fpart{ ID: 4, Content-type: text/html
+\fheader{
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 4, Content-type: multipart/alternative
+\fpart{ ID: 5, Content-type: text/html
 Non-text part: text/html
 \fpart}
+\fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
 \fpart}
-\fattachment{ ID: 5, Filename: attachment, Content-type: text/plain
+\fbody}
+\fpart}
+\fattachment{ ID: 7, Filename: attachment, Content-type: text/plain
 This is a text attachment.
 \fattachment}
-\fpart{ ID: 6, Content-type: text/plain
+\fpart{ ID: 8, Content-type: text/plain
 And this message is signed.
 
 -Carl
 \fpart}
 \fpart}
-\fpart{ ID: 7, Content-type: application/pgp-signature
+\fpart{ ID: 9, Content-type: application/pgp-signature
 Non-text part: application/pgp-signature
 \fpart}
 \fpart}
@@ -158,14 +199,27 @@ notmuch show --format=text --part=2 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OU
 cat <<EOF >EXPECTED
 \fpart{ ID: 2, Content-type: multipart/mixed
 \fpart{ ID: 3, Content-type: message/rfc822
-\fpart{ ID: 4, Content-type: text/html
+\fheader{
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 4, Content-type: multipart/alternative
+\fpart{ ID: 5, Content-type: text/html
 Non-text part: text/html
 \fpart}
+\fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
 \fpart}
-\fattachment{ ID: 5, Filename: attachment, Content-type: text/plain
+\fbody}
+\fpart}
+\fattachment{ ID: 7, Filename: attachment, Content-type: text/plain
 This is a text attachment.
 \fattachment}
-\fpart{ ID: 6, Content-type: text/plain
+\fpart{ ID: 8, Content-type: text/plain
 And this message is signed.
 
 -Carl
@@ -174,39 +228,75 @@ And this message is signed.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=text --part=3, rfc822 multipart"
+test_begin_subtest "--format=text --part=3, rfc822 part"
 notmuch show --format=text --part=3 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
 \fpart{ ID: 3, Content-type: message/rfc822
-\fpart{ ID: 4, Content-type: text/html
+\fheader{
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+\fheader}
+\fbody{
+\fpart{ ID: 4, Content-type: multipart/alternative
+\fpart{ ID: 5, Content-type: text/html
 Non-text part: text/html
 \fpart}
+\fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
+\fpart}
+\fbody}
 \fpart}
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=text --part=4, html part"
+test_begin_subtest "--format=text --part=4, rfc822's multipart"
 notmuch show --format=text --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-\fpart{ ID: 4, Content-type: text/html
+\fpart{ ID: 4, Content-type: multipart/alternative
+\fpart{ ID: 5, Content-type: text/html
 Non-text part: text/html
 \fpart}
+\fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
+\fpart}
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=text --part=5, inline attachment"
+test_begin_subtest "--format=text --part=5, rfc822's html part"
 notmuch show --format=text --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-\fattachment{ ID: 5, Filename: attachment, Content-type: text/plain
-This is a text attachment.
-\fattachment}
+\fpart{ ID: 5, Content-type: text/html
+Non-text part: text/html
+\fpart}
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=text --part=6, plain text part"
+test_begin_subtest "--format=text --part=6, rfc822's text part"
 notmuch show --format=text --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
 \fpart{ ID: 6, Content-type: text/plain
+This is an embedded message, with a multipart/alternative part.
+\fpart}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=text --part=7, inline attachement"
+notmuch show --format=text --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+cat <<EOF >EXPECTED
+\fattachment{ ID: 7, Filename: attachment, Content-type: text/plain
+This is a text attachment.
+\fattachment}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=text --part=8, plain text part"
+notmuch show --format=text --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+cat <<EOF >EXPECTED
+\fpart{ ID: 8, Content-type: text/plain
 And this message is signed.
 
 -Carl
@@ -214,10 +304,10 @@ And this message is signed.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=text --part=7, pgp signature (unverified)"
-notmuch show --format=text --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=text --part=9, pgp signature (unverified)"
+notmuch show --format=text --part=9 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-\fpart{ ID: 7, Content-type: application/pgp-signature
+\fpart{ ID: 9, Content-type: application/pgp-signature
 Non-text part: application/pgp-signature
 \fpart}
 EOF
@@ -228,48 +318,126 @@ test_expect_success \
     "notmuch show --format=text --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
 
 test_begin_subtest "--format=json --part=0, full message"
-output=$(notmuch show --format=json --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": "87liy5ap00.fsf@yoom.home.cworth.org", "match": true, "filename": "'"${MAIL_DIR}/multipart"'", "timestamp": 978709437, "date_relative": "2001-01-05", "tags": ["attachment","inbox","signed","unread"], "headers": {"Subject": "Multipart message", "From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Cc": "", "Bcc": "", "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [{"id": 1, "content-type": "multipart/signed", "content": [{"id": 2, "content-type": "multipart/mixed", "content": [{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}, {"id": 5, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, {"id": 6, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}, {"id": 7, "content-type": "application/pgp-signature"}]}]}'
+notmuch show --format=json --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": "87liy5ap00.fsf@yoom.home.cworth.org", "match": true, "filename": "${MAIL_DIR}/multipart", "timestamp": 978709437, "date_relative": "2001-01-05", "tags": ["attachment","inbox","signed","unread"], "headers": {"Subject": "Multipart message", "From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Cc": "", "Bcc": "", "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [
+{"id": 1, "content-type": "multipart/signed", "content": [
+{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
+{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 5, "content-type": "text/html"}, 
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]}, 
+{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, 
+{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}, 
+{"id": 9, "content-type": "application/pgp-signature"}]}]}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "--format=json --part=1, message body"
-output=$(notmuch show --format=json --part=1 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 1, "content-type": "multipart/signed", "content": [{"id": 2, "content-type": "multipart/mixed", "content": [{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}, {"id": 5, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, {"id": 6, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}, {"id": 7, "content-type": "application/pgp-signature"}]}'
+notmuch show --format=json --part=1 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 1, "content-type": "multipart/signed", "content": [
+{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
+{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 5, "content-type": "text/html"}, 
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]}, 
+{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, 
+{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}, 
+{"id": 9, "content-type": "application/pgp-signature"}]}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "--format=json --part=2, multipart/mixed"
-output=$(notmuch show --format=json --part=2 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 2, "content-type": "multipart/mixed", "content": [{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}, {"id": 5, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, {"id": 6, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}'
-
-test_begin_subtest "--format=json --part=3, rfc822 multipart"
-output=$(notmuch show --format=json --part=3 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 3, "content-type": "message/rfc822", "content": [{"id": 4, "content-type": "text/html"}]}'
-
-test_begin_subtest "--format=json --part=4, html part"
-output=$(notmuch show --format=json --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 4, "content-type": "text/html"}'
-
-test_begin_subtest "--format=json --part=5, inline attachment"
-output=$(notmuch show --format=json --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 5, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}'
-
-test_begin_subtest "--format=json --part=6, plain text part"
-output=$(notmuch show --format=json --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 6, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}'
-
-test_begin_subtest "--format=json --part=7, pgp signature (unverified)"
-output=$(notmuch show --format=json --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org')
-test_expect_equal "$output" \
-'{"id": 7, "content-type": "application/pgp-signature"}'
+notmuch show --format=json --part=2 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
+{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 5, "content-type": "text/html"}, 
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]}, 
+{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}, 
+{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=3, rfc822 part"
+notmuch show --format=json --part=3 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
+{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 5, "content-type": "text/html"}, 
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=4, rfc822's multipart/alternative"
+notmuch show --format=json --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 5, "content-type": "text/html"}, 
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=5, rfc822's html part"
+notmuch show --format=json --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 5, "content-type": "text/html"}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=6, rfc822's text part"
+notmuch show --format=json --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=7, inline attachment"
+notmuch show --format=json --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=8, plain text part"
+notmuch show --format=json --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=json --part=9, pgp signature (unverified)"
+notmuch show --format=json --part=9 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
+echo >>OUTPUT # expect *no* newline at end of output
+cat <<EOF >EXPECTED
+
+{"id": 9, "content-type": "application/pgp-signature"}
+EOF
+test_expect_equal_file OUTPUT EXPECTED
 
 test_expect_success \
-    "--format=json --part=8, no part, expect error" \
-    "notmuch show --format=json --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
+    "--format=json --part=10, no part, expect error" \
+    "notmuch show --format=json --part=10 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
 
 test_begin_subtest "--format=raw"
 notmuch show --format=raw 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
@@ -284,7 +452,13 @@ notmuch show --format=raw --part=1 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUT
 # output should *not* include newline
 echo >>OUTPUT
 cat <<EOF >EXPECTED
-<p>This is an embedded message, with a single html part.</p>
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+
+<p>This is an embedded message, with a multipart/alternative part.</p>
+This is an embedded message, with a multipart/alternative part.
 This is a text attachment.
 And this message is signed.
 
@@ -302,7 +476,13 @@ test_expect_equal_file OUTPUT EXPECTED
 test_begin_subtest "--format=raw --part=2, multipart/mixed"
 notmuch show --format=raw --part=2 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-<p>This is an embedded message, with a single html part.</p>
+From: Carl Worth <cworth@cworth.org>
+To: cworth@cworth.org
+Subject: html message
+Date: Fri, 05 Jan 2001 15:42:57 +0000
+
+<p>This is an embedded message, with a multipart/alternative part.</p>
+This is an embedded message, with a multipart/alternative part.
 This is a text attachment.
 And this message is signed.
 
@@ -310,29 +490,41 @@ And this message is signed.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=raw --part=3, rfc822 multipart"
+test_begin_subtest "--format=raw --part=3, rfc822 part"
 notmuch show --format=raw --part=3 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_expect_equal_file OUTPUT embedded_message
+
+test_begin_subtest "--format=raw --part=4, rfc822's html part"
+notmuch show --format=raw --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-<p>This is an embedded message, with a single html part.</p>
+<p>This is an embedded message, with a multipart/alternative part.</p>
+This is an embedded message, with a multipart/alternative part.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=raw --part=4, html part"
-notmuch show --format=raw --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=raw --part=5, rfc822's html part"
+notmuch show --format=raw --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
-<p>This is an embedded message, with a single html part.</p>
+<p>This is an embedded message, with a multipart/alternative part.</p>
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=raw --part=5, inline attachment"
-notmuch show --format=raw --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=raw --part=6, rfc822's text part"
+notmuch show --format=raw --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+cat <<EOF >EXPECTED
+This is an embedded message, with a multipart/alternative part.
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "--format=raw --part=7, inline attachment"
+notmuch show --format=raw --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
 This is a text attachment.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=raw --part=6, plain text part"
-notmuch show --format=raw --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=raw --part=8, plain text part"
+notmuch show --format=raw --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 cat <<EOF >EXPECTED
 And this message is signed.
 
@@ -340,8 +532,8 @@ And this message is signed.
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-test_begin_subtest "--format=raw --part=7, pgp signature (unverified)"
-notmuch show --format=raw --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
+test_begin_subtest "--format=raw --part=9, pgp signature (unverified)"
+notmuch show --format=raw --part=9 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
 # output should *not* include newline
 echo >>OUTPUT
 cat <<EOF >EXPECTED
@@ -356,7 +548,7 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_expect_success \
-    "--format=raw --part=8, no part, expect error" \
+    "--format=raw --part=10, no part, expect error" \
     "notmuch show --format=raw --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
 
 test_begin_subtest "--format=mbox"
@@ -383,8 +575,14 @@ References: <87liy5ap00.fsf@yoom.home.cworth.org>
 On Fri, 05 Jan 2001 15:43:57 +0000, Carl Worth <cworth@cworth.org> wrote:
 Non-text part: multipart/signed
 Non-text part: multipart/mixed
-Non-text part: message/rfc822
+> From: Carl Worth <cworth@cworth.org>
+> To: cworth@cworth.org
+> Subject: html message
+> Date: Fri, 05 Jan 2001 15:42:57 +0000
+>
+Non-text part: multipart/alternative
 Non-text part: text/html
+> This is an embedded message, with a multipart/alternative part.
 > This is a text attachment.
 > And this message is signed.
 > 
-- 
1.7.5.4

Re: [PATCH] test: overhaul multipart test to test for improved message/rfc822 handling

[David Bremner]

On Mon,  5 Sep 2011 12:01:46 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> The main goal of this overhaul is to define how message/rfc822 parts
> should be handled.  message/rfc822 parts should be output in a similar
> fashion to the outer message, including some subset of the rfc822
> headers.  The following decisions about formatting of message/rfc822
> parts were made:

I have pushed the remaining 3 patches in the series. Namely this one,
and the last 2 from the previous series. I added a brief note to the
commit message about the one test which fails.  I think on the whole the
approach Jamie took of writing the test as the specification is the
correct one. 

d

Re: tweaks for handling parts in reply

[David Bremner]

On Wed,  8 Jun 2011 12:30:05 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:

> This is a small set of tweaks to remove unneccessary "Non-text part:"
> lines in reply, for parts that really don't need to be mentioned.

I have applied this series on top of master, and will push a bit later today.

d

Re: tweaks for handling parts in reply

[Jameson Graef Rollins]

On Thu, 06 Oct 2011 10:29:52 -0300, David Bremner <david@tethera.net> wrote:
> I have applied this series on top of master, and will push a bit later today.

Great.  Thanks so much, David.

jamie.