On Sat, Jan 17 2015, David Bremner <david@tethera.net> wrote:
> Generating the certs was very much trial and error.  The net of
> a thousand lies may have led me astray a bit in that it may be
> possible to do this all with gpgsm and avoid the dependency on
> openssl. On the other hand, some tests is better than no tests.

Hey, David.  Thanks so much for covering our butts and finally putting
together these tests.

They look good to me.  Unfortunately, one of the tests is failing for
me, but I'm completely perplexed as to why:

T355-smime: Testing S/MIME signature verification and decryption
 PASS   Generate CA Cert
 PASS   Generate User Cert
 PASS   emacs delivery of S/MIME signed message
 FAIL   Signature verification (openssl)
	--- T355-smime.4.OUTPUT	2015-01-17 19:06:46.806054727 +0000
	+++ T355-smime.4.EXPECTED	2015-01-17 19:06:46.806054727 +0000
	@@ -1,4 +1,4 @@
	 Verification successful
	-Content-Type: text/plain
	-
	-This is a test signed message.
	+Content-Type: text/plain
	+
	+This is a test signed message.
 PASS   signature verification (notmuch CLI)

??  There's visually no difference between the supposedly diff'd text.
A hd of the output files being compared shows that openssl is using a
carriage return '0d' followed by line feed '0a' for every newline,
in place of a simple line feed '0a' in the original message file:

servo:~/src/notmuch/git [master*] 0$ hd test/tmp.T355-smime/T355-smime.4.EXPECTED 
00000000  43 6f 6e 74 65 6e 74 2d  54 79 70 65 3a 20 74 65  |Content-Type: te|
00000010  78 74 2f 70 6c 61 69 6e  0a 0a 54 68 69 73 20 69  |xt/plain..This i|
00000020  73 20 61 20 74 65 73 74  20 73 69 67 6e 65 64 20  |s a test signed |
00000030  6d 65 73 73 61 67 65 2e  0a 56 65 72 69 66 69 63  |message..Verific|
00000040  61 74 69 6f 6e 20 73 75  63 63 65 73 73 66 75 6c  |ation successful|
00000050  0a                                                |.|
00000051
servo:~/src/notmuch/git [master*] 0$ hd test/tmp.T355-smime/T355-smime.4.OUTPUT 
00000000  43 6f 6e 74 65 6e 74 2d  54 79 70 65 3a 20 74 65  |Content-Type: te|
00000010  78 74 2f 70 6c 61 69 6e  0d 0a 0d 0a 54 68 69 73  |xt/plain....This|
00000020  20 69 73 20 61 20 74 65  73 74 20 73 69 67 6e 65  | is a test signe|
00000030  64 20 6d 65 73 73 61 67  65 2e 0d 0a 56 65 72 69  |d message...Veri|
00000040  66 69 63 61 74 69 6f 6e  20 73 75 63 63 65 73 73  |fication success|
00000050  66 75 6c 0a                                       |ful.|
00000054
servo:~/src/notmuch/git [master*] 0$ 

Bad openssl.  (Daniel off stage screaming: "why aren't you using
certtool!")

I also noticed that the "Verification successful" string is not reliably
being printed to stderr before the message output.

Two possible patches to fix the problems are attached below.  The second
is maybe slightly preferred, since it eliminates any reliance on broken
openssl message output whatsoever.

Thanks again for working on this, David.

jamie.


diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 0e5fd4a..5e3ec72 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -43,7 +43,9 @@ test_expect_success 'emacs delivery of S/MIME signed mes
 
 test_begin_subtest "Signature verification (openssl)"
 notmuch show --format=raw subject:"test signed message 001" |\
-    openssl smime -verify -CAfile ca.crt >& OUTPUT
+    openssl smime -verify -CAfile ca.crt 2> OUTPUT
+notmuch show --format=raw subject:"test signed message 001" |\
+    openssl smime -verify -CAfile ca.crt | tr -d '\015' >> OUTPUT
 cat <<EOF > EXPECTED
 Verification successful
 Content-Type: text/plain


diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 0e5fd4a..cba23e0 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -43,12 +43,9 @@ test_expect_success 'emacs delivery of S/MIME signed me
 
 test_begin_subtest "Signature verification (openssl)"
 notmuch show --format=raw subject:"test signed message 001" |\
-    openssl smime -verify -CAfile ca.crt >& OUTPUT
+    openssl smime -verify -CAfile ca.crt 2> OUTPUT
 cat <<EOF > EXPECTED
 Verification successful
-Content-Type: text/plain
-
-This is a test signed message.
 EOF
 test_expect_equal_file OUTPUT EXPECTED