[PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[Alexander Adolf]

[-- Attachment #1: Type: text/plain, Size: 891 bytes --]

Hello,

kindly find below a patch which I have been using in my local setup for
about a year without seeing any hiccups. So I thought it would be worth
proposing to the developers.

The issue I saw with S/MIME encrypted messages was, that the processing
of the MIME tree stopped after decryption. It turned out there was no
handler for application/(x-)?pkcs-mime type entities in notmuch-show.el,
and the "handler of last resort" notmuch-show-insert-part-*/* was
called. Thus, this patch just adds a notmuch-show handler for
application/(x-)?pkcs-mime type entities.

RFC 8551 defines that there is a single child element only ("A body can
consist of a single MIME entity or a tree of MIME entities (rooted with
a multipart).", clause 3.1), hence no list processing of the children is
needed (there's only one).


Hoping to have helped, and looking forward to your thoughts,

  --alexander


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-emacs-smime-render-decrypted-MIME-entities-in-notmuc.patch --]
[-- Type: text/x-patch, Size: 2116 bytes --]

From d10f4964173ae644077de91b56a98d48f7253ce8 Mon Sep 17 00:00:00 2001
From: Alexander Adolf <alexander.adolf@condition-alpha.com>
Date: Fri, 25 Mar 2022 14:13:28 +0100
Subject: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

When processing encrypted S/MIME messages, after decryption the "last
resort" handler notmuch-show-insert-part-*/* was called, because there
was no application/pkcs7-mime handler, resulting in the decrypted
contents not being displayed.

This commit adds a new function
notmuch-show-insert-part-application/pkcs7-mime (and an alias
notmuch-show-insert-part-application/x-pkcs7-mime for the legacy MIME
type) to render the S/MIME protected part after decryption.
---
 emacs/notmuch-show.el | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 7c1f02c9..b7edfc98 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -712,6 +712,23 @@ will return nil if the CID is unknown or cannot be retrieved."
 (defun notmuch-show-insert-part-application/pgp-encrypted (_msg _part _content-type _nth _depth _button)
   t)
 
+(defun notmuch-show-insert-part-application/pkcs7-mime (msg part _content-type _nth depth _button)
+  "Render S/MIME protected content after decryption.
+
+An alias for this function is also defined to handle entities
+using the legacy application/x-pkcs7-mime MIME type."
+  (let* ((encstatus (car (plist-get part :encstatus)))
+	 (inner-part (car (plist-get part :content))))
+    ;; Insert a button detailing the encryption status.
+    (notmuch-crypto-insert-encstatus-button encstatus)
+    (if (not (string= (plist-get encstatus :status) "bad"))
+        ;; Show all decrypted parts.
+        (notmuch-show-insert-bodypart msg inner-part depth))))
+
+;; Support for the legacy "x-" type.
+(fset 'notmuch-show-insert-part-application/x-pkcs7-mime
+  'notmuch-show-insert-part-application/pkcs7-mime)
+
 (defun notmuch-show-insert-part-multipart/* (msg part _content-type _nth depth _button)
   (let ((inner-parts (plist-get part :content))
 	(start (point)))
-- 
2.35.1


[-- Attachment #3: Type: text/plain, Size: 0 bytes --]

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[David Bremner]

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

> The issue I saw with S/MIME encrypted messages was, that the processing
> of the MIME tree stopped after decryption. It turned out there was no
> handler for application/(x-)?pkcs-mime type entities in notmuch-show.el,
> and the "handler of last resort" notmuch-show-insert-part-*/* was
> called. Thus, this patch just adds a notmuch-show handler for
> application/(x-)?pkcs-mime type entities.

Hi Alexander;

Thanks for sending that. It's a bit tricky to review SMIME stuff as I
believe none of the devs is really using SMIME regularly. One thing that
would help is adding one or more tests. Basically the test suite already
has a SMIME certificate, so if you can encrypt a message to that cert
(or use one of the existing ones), you can test that the message
displays OK in Emacs. The corresponding test set would probably be
T450-emacs-show, but it might be easier to add to T355-smime. Eventually
we'll also need a short update to doc/notmuch-emacs.rst.

I hope that helps move review forward,

David

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[Alexander Adolf]

Hello David,

David Bremner <david@tethera.net> writes:

> [...]
> Thanks for sending that. It's a bit tricky to review SMIME stuff as I
> believe none of the devs is really using SMIME regularly. One thing that
> would help is adding one or more tests. Basically the test suite already
> has a SMIME certificate, so if you can encrypt a message to that cert
> (or use one of the existing ones), you can test that the message
> displays OK in Emacs. The corresponding test set would probably be
> T450-emacs-show, but it might be easier to add to T355-smime. Eventually
> we'll also need a short update to doc/notmuch-emacs.rst.
> [...]

Apologies for the delay in getting back to you, and thanks for the
pointers.

I have had a look to both files, and I _think_ T450-emacs-show would
seem more appropriate. It seems that T355-smime is more aimed at the
pure mechanics of en-/decryption, and the handling of that in
libnotmuch. Neither of these have been a problem for me.

Further looking at T450-emacs-show, I find that it uses a somewhat
extensive framework to implement its test cases. Any chances of giving
me a fast start with this, or will it be down to "use the source, Luke"?


Many thanks in advance and looking forward to your thoughts,

  --alexander

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[David Bremner]

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

>
> I have had a look to both files, and I _think_ T450-emacs-show would
> seem more appropriate. It seems that T355-smime is more aimed at the
> pure mechanics of en-/decryption, and the handling of that in
> libnotmuch. Neither of these have been a problem for me.
>
> Further looking at T450-emacs-show, I find that it uses a somewhat
> extensive framework to implement its test cases. Any chances of giving
> me a fast start with this, or will it be down to "use the source, Luke"?
>

I think something like the following test (but with an smime encrypted
message instead of the existing PGP/MIME encrypted message.

    test_begin_subtest "process cryptographic MIME parts"
    ;; additional variables can be dynamically bound as needed, but this one
    ;; in particular is needed for decryption
    test_emacs '(let ((notmuch-crypto-process-mime t))
    ;; simulate the user action
            (notmuch-show
            "id:20091117203301.GV3165@dottiness.seas.harvard.edu")
    ;; this is the test framework, write the terminal display to a file
    ;; called OUTPUT        
            (test-visible-output))'
    test_expect_equal_file $EXPECTED/notmuch-show-process-crypto-mime-parts-on OUTPUT

you would also need to create a file with a "screen display" in
$EXPECTED, and add that as part of your patch.

I hope that helps,

David

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[Alexander Adolf]

David Bremner <david@tethera.net> writes:

> [...]
> I think something like the following test (but with an smime encrypted
> message instead of the existing PGP/MIME encrypted message.
>
>     test_begin_subtest "process cryptographic MIME parts"
>     ;; additional variables can be dynamically bound as needed, but this one
>     ;; in particular is needed for decryption
>     test_emacs '(let ((notmuch-crypto-process-mime t))
>     ;; simulate the user action
>             (notmuch-show
>             "id:20091117203301.GV3165@dottiness.seas.harvard.edu")
>     ;; this is the test framework, write the terminal display to a file
>     ;; called OUTPUT        
>             (test-visible-output))'
>     test_expect_equal_file $EXPECTED/notmuch-show-process-crypto-mime-parts-on OUTPUT
>
> you would also need to create a file with a "screen display" in
> $EXPECTED, and add that as part of your patch.
>
> I hope that helps,
> [...]

To make this work, I will need to generate two S/MIME test messages. One
encrypted only, and one encrypted and signed.

Where can I find the S/MIME test key? test/smime directory?

Is there a better way for generating the test messages than importing
the test key into my "production" keychain? Look to T355-smime for
inspiration?

Where do I put the generated test messages to be able to use them in the
test?


Many thanks in advance and cheers,

  --alexander

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[David Bremner]

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

>
> Where can I find the S/MIME test key? test/smime directory?

test/smime/0xE0972A47.p12

> Is there a better way for generating the test messages than importing
> the test key into my "production" keychain? Look to T355-smime for
> inspiration?

The former is probably the simplest. You could also run T355-smime with
--debug and inspect tmp.T355-smime/mail afterwards, there should be some
signed and encrypted mails.

> Where do I put the generated test messages to be able to use them in the
> test?

It should be fine to add to corpora/crypto, then call

add_email_corpus crypto

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[Alexander Adolf]

David Bremner <david@tethera.net> writes:

> [...]
> You could also run T355-smime with --debug and inspect
> tmp.T355-smime/mail afterwards, there should be some signed and
> encrypted mails.
> [...]

$ ./T355-smime.sh --debug
Run tests in a subdir of built notmuch tree.

Any way around the effort of building from source? I'm interested in the
elisp part only, and I have notmuch installed in /usr/local.


Cheers,

  --alexander

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[David Bremner]

[-- Attachment #1: Type: text/plain, Size: 889 bytes --]

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

> David Bremner <david@tethera.net> writes:
>
>> [...]
>> You could also run T355-smime with --debug and inspect
>> tmp.T355-smime/mail afterwards, there should be some signed and
>> encrypted mails.
>> [...]
>
> $ ./T355-smime.sh --debug
> Run tests in a subdir of built notmuch tree.
>
> Any way around the effort of building from source? I'm interested in the
> elisp part only, and I have notmuch installed in /usr/local.
>

I'm not sure it's less effort, but I started making the tests work
without building from source at

  https://nmbug.notmuchmail.org/nmweb/show/20211025145753.3031094-2-david@tethera.net

It turns out to be actually slightly tricky (the messages are erased
part way through that file), do I attach the corresponding message.

Hopefully you can figure out how to run your test once it is
written. 


[-- Attachment #2: smime-enc.eml:2,S --]
[-- Type: application/octet-stream, Size: 3855 bytes --]

From: Notmuch Test Suite <test_suite@notmuchmail.org>
To: test_suite@notmuchmail.org
Subject: test encrypted message 001
Date: 01 Jan 2000 12:00:00 -0000
Message-ID: <87ilrfryci.fsf@example.com>
MIME-Version: 1.0
Content-Type: application/pkcs7-mime;
 smime-type=enveloped-data;
 name=smime.p7m
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=smime.p7m

MIAGCSqGSIb3DQEHA6CAMIACAQAxggFFMIIBQQIBADApMB0xGzAZBgNVBAMTEk5vdG11Y2ggVGVz
dCBTdWl0ZQIIb3SMlL0MZ6kwDQYJKoZIhvcNAQEBBQAEggEAtjC5r28KtqMzu/vwXXR5B1kta6Hn
dtW5dZcFOBVDCUq9Y8xOhl/dCjnxP0p8gLPHYqVaTb6YTlFI0YYHZtXU3XY8/NJnI3ZdFrD4zAEC
QRlvJptFFM67lDQZCfPnn63DiydjUy00WPcZnMF9ZMhfatPJv1sgPrF62re8LJbm+By6x21GkiEu
3+XYFnFMge8rEd8FZSAxfBUY5gvjD+++BaTZK9mgNvOks+3OPbtDvqR2d4ocgAkJfolxAL1yYqYO
BdS/EUYVMYRWruPEnemY5emu5z4Wa5EC7VZKSP7H+GDRkYNBU/w9H3VRf86tXJRjryDGGJG3iXr9
LMcJ48zPgzCABgkqhkiG9w0BBwEwHQYJYIZIAWUDBAECBBCi4eShF0rUOIjUCi5XvAkQoIAEgghg
WXo1Gwa9E2oXRiu/11KnkfO7x6gwNm5LMYSMdGpTJuxw71US4ts1zzXVKqP1chrFfCHsRrK6aYjt
uzL1gD9FD5ZoG6q0urHEG6xrZgt86DmVVhySiUiTrhEZMRv4d0nrEI3OxyYzpvOHWWw0Jg0ZRwGT
Irvz5CXEwG1aZ37ab7V3pRN91TNrzX7saRlEaDvv2Rl1vpV5qZCwtl16G8KBAojAQCjRrMNBxg/A
P3s5I9qpLKnV4bQ4MUTks8fX+uiFQmfOPfsSU76U8M9xtZFobE2Qk+JjgsEUNU9fxQY5+aik0jSq
Yc1MMzQOkifKPzm2o3Uyka6CWgUzesiWF/Z1zLw16K4xlaWG7tWotK1WdAA4RyiE3kaXzbQa/DHa
NomLZJ73/nB7yOECrBbV904PX/70I+oizsE0ccTZrUPMSBoos/fweXzR5Fv/IlxXCQYeGp71FxWJ
KMU/0jMGc9sRW6z8gzDUzTvAu+SlkPU+1iq4mF29urfePA9HjvFbN9go4/aDr14OnnTc16Q016qF
7EOkQP2fT1BhLU4nPo9iLd8EfV5GAq43l+YPM5DsYRcqXf2shNKx40Byuuk3MEtG4720Yt1qJkMv
KfvXfzliU6dHnn6eSYtocKP0igCySEza/VVijfatJ5sD3uZDMCokEGN1k0y4Vq+V0Bw+MZDFxKd6
nEM7mdQBBeWfwEFperVns6GLRaQ0i4q6PWCNXpIFZ46LEb7rKoyWA3DB2Sff7fklnGUNaIR5cp6y
uyqnPlM3z3iaiOjOAK7ZRLdMssfXjXv7JJ6DCPa6p3HTfa46bZaspDHOYDQftOu0mfFWz2ZI9B3D
zWVbdXn/f25W519qsuKxIdeh7/8oL0WfdTDR2Ixzv1Su05tyiWGbuEZm1DMlQzXlq1NHKbLlhsQh
RhjqL+ZICCupXJzJj8SdWbxcz9fcCvWI+etEZ2LWGtaqQVUcbCowTFByVeHDI/CEtnwDajhVvQLP
qMIzabHHYyLQ9+baLzkznkJim/7dZxGkCdu4zSCyeHwCicIJbX32YmkFdjUz6U/G5H2T4uKQCVon
kHTd3x0s4F7RG0E0zIr9RbRHbZQkTFx2F34VAwiCIv6rJbf4UAOLnVGJBwP73aGfSjdzwBEceAak
Jaz2a9rrLFrZ2+g2pjvoV5Fpk+0jrtXzw+tQSXWO+RsHsqOn6XjcjqCPlgIXYZYPFxFeoaHUPii8
Eih+SnNOJahDWNMBWDoB4rpqdFJMlrclA0neEwPf0le3rm08FQPhL9xvBu/wYJt2ToSML/dU6Fti
MCBpEL642STNQznKGyiD7V5BMcA4wLi1Lzfd/H3OpV1OQuaZjLWh5HlSH2+82cgNxKHB8CbJCl8i
vT/uhpFvCQ0BfFtMboAzrAqhX/v6wEyz9G2SjBefVU0sk8SSSaZcUTi1jQdUmhdcTO6g5/zwxbkt
4YkQ27+jMnWXjNhr6W55vLfZe6MQUKraynIaWEH7MN9shVD7Taq9/FHkoFH356/mojPR8W8M1+3T
QCvbeqZc61KpyW3Jr029BpXUvOCZX6XZfk1yR/mQ/Fl/XGT1siISTnCaFAtB9MYQQr4DYqcNqFRv
wEnf9mNIOsVr4q7uMR3rfmkU2A697gve1CKkeYQQbQEXJEOzytLIxUXAU4Fr7VzUYM//nrmBwkct
viZmpGxPKBNH8DxBk2g4Ttz52HKlyQIGdcv0Swgx6+XwtDnV9GnyHM2NzR0ryp5c9C7ZUk1hs34z
dQW5L6MWkPS/tFaXA38ZnNFK/XrFxgL1ouf+fL0NLVFrgESanKz/RXR1LxnBrBBr4LTfhVwWg9fv
AQMC2qRNYKxXMgOkPdzm1BteRZVBLEOaCqpmqyX0NUTg6x9RmVWLArN5xUggrz8Ut0/TXqkmnimN
G0UCAyyaG3V/H9l9IyXNLjvldAZ32JjReTZkVpcZBzyOdDovmJwycGsgeo8r8PVOBJEW+Xc9nUcK
PnUdjkt/vFkBF0Z545L7zxRad04gqeAx1yL7b8kJeGgypXkbTSvXWeV74ouowSqUuw9UAY01s7iM
nEd955MajtUvbtFzNu2azPAAmHSt8MtYWjZRhee4CYvjpCGNrdCfcMnkhWKJxKgMmMgXdTU1bUpe
hr5lT3AiIJqnFSiR6pFKJabAwB5+zyKKJTxkNvcWyfvlFl7d1C1Y+kcgv8rnwqVgCo7GvaUIyG00
tvuObtB7WMNdLE7LCh64SbR+5S/L3pKl0/tnJvCn/Dx4Oz4IDSGgykLZ4Qph37bBO+57KgLdpIAW
G6T+P7uXgfhvnRpQGqeregJ3VGViYerhZV2kWbI6VUBWf+rTQ/O2HyxmwVKP8HgZK47h1pOsXR3w
IjKw+LO512INyVfH1sfsjQTzt963Jowxe409sLFdmjso4ebFwOQJS7AH6RP2tFgJmTzz3FN7I3ie
rlHTeBeOZgs9w9QmtzIQdHevYApVy/VQP9++h2sthnZINlxHxcWAJyq3oJ3Cx4Bqf15fdQcYMU8i
ihLM7X5IEmHSrxvMZyFyLduo1LZwtQBJZrl+0bclXM4eiYyX9BNSdnwbpB2nMG0jPVLMr5a9aebn
Iglakpv56GgehxoAkqa1LQtcApdjmE31h3evnw/VMHH8IHW2ipa7b4UZIVNqRz5a5gWCkQoUkQWW
nG3MQo/J3Dn5ARni7lzF9qlBWrZsxK3Oz89aX/bpTKWhL2GyK/E9/U6zwwPxzx+4ESRhBpPwDyFT
kDsuuUKroTeAlrUevZUmiFMSEjVpiBTkmLRSMy+wgcdH1FRXy/vnwU80OqT1VbdPQfbHMA2WIsQg
TEaNytqO7pVYZcewTRBZXf3Ak6Tu2YxDhsDnWOlMzPoJ5n4EEKOLfhCpWoYFUo2PX1pP8SwAAAAA
AAAAAAAA

[-- Attachment #3: Type: text/plain, Size: 0 bytes --]

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[Alexander Adolf]

[-- Attachment #1: Type: text/plain, Size: 268 bytes --]

Hello David,

David Bremner <david@tethera.net> writes:

> [...]
> I'm not sure it's less effort, 
> [...]

Neither am I... ;-))

It might perhaps seem easier to run the tests I have added to the
attached, updated patch in your complete environment?

  --alexander




[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-emacs-smime-render-decrypted-MIME-entities-in-notmuc.patch --]
[-- Type: text/x-patch, Size: 8180 bytes --]

From ba6f00bbd1803f5cccfafdb262f17b79b6c95252 Mon Sep 17 00:00:00 2001
From: Alexander Adolf <alexander.adolf@condition-alpha.com>
Date: Fri, 25 Mar 2022 14:13:28 +0100
Subject: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

When processing encrypted S/MIME messages, after decryption the "last
resort" handler notmuch-show-insert-part-*/* was called, because there
was no application/pkcs7-mime handler, resulting in the decrypted
contents not being displayed.

This commit adds a new function
notmuch-show-insert-part-application/pkcs7-mime (and an alias
notmuch-show-insert-part-application/x-pkcs7-mime for the legacy MIME
type) to render the S/MIME protected part after decryption.
---
 emacs/notmuch-show.el                         | 17 +++++++
 test/T450-emacs-show.sh                       | 11 +++-
 .../crypto/smime-encrypted-signed-multipart   | 51 +++++++++++++++++++
 ...much-show-smime-encrypted-signed-multipart | 15 ++++++
 4 files changed, 93 insertions(+), 1 deletion(-)
 create mode 100644 test/corpora/crypto/smime-encrypted-signed-multipart
 create mode 100644 test/emacs-show.expected-output/notmuch-show-smime-encrypted-signed-multipart

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 7c1f02c9..b7edfc98 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -712,6 +712,23 @@ will return nil if the CID is unknown or cannot be retrieved."
 (defun notmuch-show-insert-part-application/pgp-encrypted (_msg _part _content-type _nth _depth _button)
   t)
 
+(defun notmuch-show-insert-part-application/pkcs7-mime (msg part _content-type _nth depth _button)
+  "Render S/MIME protected content after decryption.
+
+An alias for this function is also defined to handle entities
+using the legacy application/x-pkcs7-mime MIME type."
+  (let* ((encstatus (car (plist-get part :encstatus)))
+	 (inner-part (car (plist-get part :content))))
+    ;; Insert a button detailing the encryption status.
+    (notmuch-crypto-insert-encstatus-button encstatus)
+    (if (not (string= (plist-get encstatus :status) "bad"))
+        ;; Show all decrypted parts.
+        (notmuch-show-insert-bodypart msg inner-part depth))))
+
+;; Support for the legacy "x-" type.
+(fset 'notmuch-show-insert-part-application/x-pkcs7-mime
+  'notmuch-show-insert-part-application/pkcs7-mime)
+
 (defun notmuch-show-insert-part-multipart/* (msg part _content-type _nth depth _button)
   (let ((inner-parts (plist-get part :content))
 	(start (point)))
diff --git a/test/T450-emacs-show.sh b/test/T450-emacs-show.sh
index 057ad37e..5c4bdd7e 100755
--- a/test/T450-emacs-show.sh
+++ b/test/T450-emacs-show.sh
@@ -49,7 +49,7 @@ test_emacs '(let ((notmuch-crypto-process-mime nil))
 	(test-visible-output))'
 test_expect_equal_file $EXPECTED/notmuch-show-process-crypto-mime-parts-off OUTPUT
 
-test_begin_subtest "process cryptographic MIME parts"
+test_begin_subtest "process cryptographic MIME parts (PGP)"
 test_emacs '(let ((notmuch-crypto-process-mime t))
 	(notmuch-show "id:20091117203301.GV3165@dottiness.seas.harvard.edu")
 	(test-visible-output))'
@@ -245,4 +245,13 @@ test_emacs "(test-log-error
 	        (notmuch-show \"$tid\")))"
 test_expect_equal "$(cat MESSAGES)" "COMPLETE"
 
+# more crypto tests, using S/MIME from here on:
+add_gpgsm_home
+
+test_begin_subtest "process cryptographic MIME parts (S/MIME)"
+test_emacs '(let ((notmuch-crypto-process-mime t))
+	(notmuch-show "id:575ddaaf0b234fd85e077cfb4d44d467@notmuchmail.org")
+	(test-visible-output))'
+test_expect_equal_file $EXPECTED/notmuch-show-smime-encrypted-signed-multipart OUTPUT
+
 test_done
diff --git a/test/corpora/crypto/smime-encrypted-signed-multipart b/test/corpora/crypto/smime-encrypted-signed-multipart
new file mode 100644
index 00000000..f0bd6c51
--- /dev/null
+++ b/test/corpora/crypto/smime-encrypted-signed-multipart
@@ -0,0 +1,51 @@
+Message-Id: <575ddaaf0b234fd85e077cfb4d44d467@notmuchmail.org>
+From: test_suite@notmuchmail.org
+To: test_suite@notmuchmail.org
+Subject: notmuch-show S/MIME test
+Date: Tue, 12 Apr 2022 16:57:30 +0200
+MIME-Version: 1.0
+Content-Type: application/pkcs7-mime;
+ smime-type=enveloped-data;
+ name=smime.p7m
+Content-Transfer-Encoding: base64
+Content-Disposition: attachment; filename=smime.p7m
+
+MIAGCSqGSIb3DQEHA6CAMIACAQAxggFFMIIBQQIBADApMB0xGzAZBgNVBAMTEk5vdG11Y2ggVGVz
+dCBTdWl0ZQIIb3SMlL0MZ6kwDQYJKoZIhvcNAQEBBQAEggEAeCEItxJpxL5frDmEeuMRpi8TcCCw
+WHraQ//IDMkA6fcbDIA8hfJpRNpiL5AvqQBipPELtb95rHKxOes7fUPMbT5FVA7dm72hur5N9VRj
+kN8Jgs6BjpXRKBA4S5eEzu9J2DJYnTWueZUItKlUzXUXg9AWwyEOKtlXfpOEGZ8FSTNQaE4thipO
+hmElscsz1tGmw2+8E1dFeXZyHArruqMAxzqOtiM6G3Y5dj1i8V+s6BSRLzep0JQZ0T/Jq5LE5T+E
+rYpgjopj1IT7IOwOP6B+YuhkalXjX7ursH9CNsDg+YjvWPn8RblAH5BRKLHNo1jMm9JLPFH9/0qJ
+IJAD0U2q4jCABgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBBbgb0RK4lPMIDflJo1BTd3oIAEggcA
+StUcSp6hZo/hgKSybFBEwoSpB5/SIcsNM5ej5t3afI6IV7Zg9PKG4jOZwihjJFtjYBEOkbHP44rc
+7bK4C15SHsNS4JvM0i0acKqti4kBO7+ly1R4e5KExGm8L5ZZbD6ejlufUAxzg7Yw1jzNTnNr211D
+fwYD7PkMKfotQtFReVrwBL+Ud+7Twkz0DyS0G00Qr+/WcN9v0FmTMbzqak9Aftd9nPSrWzRWPF9y
+AFIo2myYsJeWLFThE1FMMx/OjuxfSWRBzCv+pCsL4my+NIIeidyTcmn0Zs9EmREQF4yPfVX+k1XJ
+NwEbYCUcfF0Lo5fy6JxPxIto+zVgQxmQKf1kcLPSGbrS/AyaugztXwVONM4pQBKtk1RHudvSlWWN
+9J8e3fEHHqcdDdHOLxCursak+qKktwt+0QNiLAkbl/bBtN2NIb41ad6k+APUCG3Lcm4pA7p7KoYI
+UG6ZhXwid3rnRF8Va2vNADpt4SQaTtQxsK4dAMia9M6SuH96eaUcOQLq0JsHow4xWvotnb3XRVPV
+4s+NjqIBbnTSASq66GwcnKw3IAngXoJ7S35CPOix4RRiy2fUjPOh6TARWcXzTvEHoAnhVAM6pKRv
+3jztM1uhv4b6uO6SrfzdaLeMDAK+Dod14gP/rPR7iw5HLI6YkPVMLIM8Z5Af1C5Yzn5J8hRYmHdd
+d3sZsChTQ2KaPPhERWwaQutoBRk6toXHB6q7fpfJ7LcKXxd5wy5/8O6xTXpjcqk0U3BqCHDYb86B
+g5emFkj6Gt1At9e9/aIAGW5h3h56nditPITcnYC1Q+2U0jlc/RlEvEgjftzQC/iRXOOM0HP5sAH2
+e7vsM0FHU4lEq4IjCR/vm7mVPslUU7TG7fDG75tB+THPtIryAibPaEbDFzmag9fjMZpufcWIloUB
+KPXLgBP3ozW8kUbOfGHTNNA6Dy4/jKpo11Td99uoNtAbCUzIX/EGLYVn7UGaqrBY4ed9cZZo9ySs
+rYp9bCYhRe0Usgw1ia0ONzIavZH0uBpD5VgdjhawyMMCnMxFkRqio6ZP+txcL/G9rc9dbg7JT2zg
+SbDMBWIS4NRiFLfTVHgxX8wqjGsiB4sS8VwKDHC0h66MIqSVNlBey2tVSVIKrboMiW2gzRWpDTiB
+odXlppZ7SgAkDNUeDIvoSCz+2H6CNiisU8nfiO+bDo/Ua/b5leB9Oxt4uNCJIa8BSxetjfd8Ctrv
+yMuHHH/ce8GYffW6ciMPURpwK5O3WYxD2KmZSSvTNdbvv3QVda1eAKmxy/O15AfIYj8cB/yzsrry
+yVww1i2sJAHTFsCMA4hwxFesHtOBqBV9jG3/5kIon7tIQqM6gGJhEphZQNp6JZYvHRRaWOboM+Pe
+WRWsgy9py+bVHdh2PXTw69c6I+N7C2iEkYBJtnkGkSYFUCsnyogmEZCYd2piJLlBOEi1HHIVSVq+
+XyJVTknhz2sWeHD2kCAuvrRJcJ4ld8+58ZcxDyjjGp0oOaMX/+YMuDRtRHDMoHv3KaEnyXShlVI/
+/xUbVBkPGDtbQbfpxJ7ZtG1UOYIf4EjDXXiWWpedItQTVYzEfgLAdp5RWUfez6SU5eFmSpp0d64v
+GgW4EatLcV+PPsfroaR1wXiaAS8gJp1agjRldghbkH5FFPn9SnR/XagFRKBEX/m1/Lyf0YDW1JLf
+XmwfEcbkN+oxdbGfuDU4dlQvja5VNEXVqZ2+p43HvOByvGtfOVUnVJ8Ag0g8GfcG8IBgVa7sQHsj
+xHI/e1BVTAK4j0GVKYLRURRJgFpBkMBaAzFm211t/YiOInwQZe+n80EWHioTDtmAC6dbj2iR/w9I
+mkinAQBgNWFXrXJ6wNnixgw5kAPmR81TGv7CHMxKOHXGy9y5++7AvkUcMB8mY14y5iYOWh+BiBWv
+4iDGZiFt73Ur9ur7rz+t+EDXWuHXKuexOu5qH/izf0/hwY5c3jrl7Sg2ZFuRI58oua5Wn3JAtkQS
+75frVr67OG9EmUAu9Whv2oqjtyEwujwtddBM2IRbY49dtdenA6u6C+XsZn4v/fgIR46Dl+nC9oyZ
+3JrwLmu+l4e79glr+r4B/uB6fbC5gcMk8zlZDZeTVN3ZgJCpWSywpABE2GIPSH4kgiiBH8RHD4hV
+7kPAQqc9AXrvvXXN9Gzouy2FmWTOvnb3d63OjG01ZDIgVwaE6dfCZRgKN3P3Jx+/mQboJzhiQJbw
+M/cFlSwo/GuUjt1fhTq7Vtr8PLXnAORvTrXYTbpB2ozPxdVDHLfKYpta98s+61vVuoRO4yaLk+oP
+IuSI/j9wgPJkUOuN6oGjZ7fr2y/alw2F8JbmG/0eZxRE3JuvnUfwTRJeijMvaKS1Vk8BUoLAADys
+JLSbQhXdzvZGk5avYwv7IAkXASMRwODE3AQQzThdv7ONbleBxDPXMpAvRQAAAAAAAAAAAAA=
\ No newline at end of file
diff --git a/test/emacs-show.expected-output/notmuch-show-smime-encrypted-signed-multipart b/test/emacs-show.expected-output/notmuch-show-smime-encrypted-signed-multipart
new file mode 100644
index 00000000..556bd937
--- /dev/null
+++ b/test/emacs-show.expected-output/notmuch-show-smime-encrypted-signed-multipart
@@ -0,0 +1,15 @@
+test_suite@notmuchmail.org (0 mins. ago) (encrypted inbox)
+Subject: notmuch-show S/MIME test
+To: test_suite@notmuchmail.org
+Date: Tue, 12 Apr 2022 16:57:30 +0200
+
+[ smime.p7m: application/pkcs7-mime ]
+[ Decryption successful ]
+[ multipart/signed ]
+[ Good signature by: <test_suite@notmuchmail.org> ]
+[ multipart/mixed ]
+[ multipart/mixed ]
+[ text/plain ]
+The password is "12345678". But don't tell anyone!
+[ test.dtd: application/octet-stream ]
+[ smime.p7s: application/pkcs7-signature ]
-- 
2.35.1


[-- Attachment #3: Type: text/plain, Size: 0 bytes --]

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[David Bremner]

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

> Hello David,
>
> David Bremner <david@tethera.net> writes:
>
>> [...]
>> I'm not sure it's less effort, 
>> [...]
>
> Neither am I... ;-))
>
> It might perhaps seem easier to run the tests I have added to the
> attached, updated patch in your complete environment?
>
>   --alexander


It seems that it is mostly working, but there are a few issues to iron out.

The first is easy, I think. Since we added a message to the crypto
corpus, we need to adjust the tests. I think it is fine to just add
the new message to the failing test output, as in the diff below.

T357-index-decryption: Testing indexing decrypted mail
 FAIL   indexing message fails when secret key not available
	--- T357-index-decryption.31.expected	2022-04-13 23:15:02.258922959 +0000
	+++ T357-index-decryption.31.output	2022-04-13 23:15:02.258922959 +0000
	@@ -1,5 +1,6 @@
	 #= simple-encrypted@crypto.notmuchmail.org index.decryption=failure
	 #notmuch-dump batch-tag:3 config,properties,tags
	++encrypted +inbox +unread -- id:575ddaaf0b234fd85e077cfb4d44d467@notmuchmail.org
	 +encrypted +inbox +unread -- id:basic-encrypted@crypto.notmuchmail.org
	 +encrypted +inbox +unread -- id:encrypted-rfc822-attachment@crypto.notmuchmail.org
	 +encrypted +inbox +unread -- id:encrypted-signed@crypto.notmuchmail.org


T450-emacs-show: Testing emacs notmuch-show view
 BROKEN show encrypted rfc822 message
!!! Bodypart handler `notmuch-show-insert-part-*/*' threw an error:
!!! Symbol’s value as variable is void: gnus-newsgroup-charset
 FAIL   process cryptographic MIME parts (S/MIME)
	--- T450-emacs-show.21.notmuch-show-smime-encrypted-signed-multipart	2022-04-13 23:15:11.267167711 +0000
	+++ T450-emacs-show.21.OUTPUT	2022-04-13 23:15:11.267167711 +0000
	@@ -1,4 +1,4 @@
	-test_suite@notmuchmail.org (0 mins. ago) (encrypted inbox)
	+test_suite@notmuchmail.org (Yest. 14:57) (encrypted inbox)

If you set notmuch-show-relative-dates to to nil, something like, with
maybe better indentation:

-test_emacs '(let ((notmuch-crypto-process-mime t))
+test_emacs '(let ((notmuch-crypto-process-mime t)
+          (notmuch-show-relative-dates nil))

Then you will get an actual date that you can hard code.

	 Subject: notmuch-show S/MIME test
	 To: test_suite@notmuchmail.org
	 Date: Tue, 12 Apr 2022 16:57:30 +0200
	@@ -6,10 +6,10 @@
	 [ smime.p7m: application/pkcs7-mime ]
	 [ Decryption successful ]
	 [ multipart/signed ]
	-[ Good signature by: <test_suite@notmuchmail.org> ]
	+[ Good signature by: test_suite@notmuchmail.org ]

This is exactly the problem we just dealt with for T355-smime. I think
the same solution can be applied, but you will need to inline the output
so that you can do variable substitution.

	 [ multipart/mixed ]
	 [ multipart/mixed ]
	 [ text/plain ]
	 The password is "12345678". But don't tell anyone!
	-[ test.dtd: application/octet-stream ]
	+[ test.dtd: application/octet-stream (as application/xml-dtd) ]

This seems related to the function
#'notmuch-show-get-mime-type-of-application/octet-stream, which is
calling (mailcap-extension-to-mime "dtd"). Probably this is hard to make
reproducible, so just seding away an "(as ...)" string is a reasonable
alternative.

	 [ smime.p7s: application/pkcs7-signature ]


\r

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[Alexander Adolf]

David Bremner <david@tethera.net> writes:

> [...]
> It seems that it is mostly working, but there are a few issues to iron out.
> [...]

Hello David,

Apologies for the delay in getting back to you. I was busy with other
stuff, and will tend to this shortly.


Cheers,

  --alexander

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[Alexander Adolf]

Hello David,

admittedly being unfamiliar with the test suite, and not being able to
run the tests at my end, this is a bit of a head scratcher for me. Thus
up-front apologies, and thanks for bearing with me!

David Bremner <david@tethera.net> writes:

> [...]
> It seems that it is mostly working, but there are a few issues to iron out.
>
> The first is easy, I think. Since we added a message to the crypto
> corpus, we need to adjust the tests. I think it is fine to just add
> the new message to the failing test output, as in the diff below.
>
> T357-index-decryption: Testing indexing decrypted mail
>  FAIL   indexing message fails when secret key not available
> 	--- T357-index-decryption.31.expected	2022-04-13 23:15:02.258922959 +0000
> 	+++ T357-index-decryption.31.output	2022-04-13 23:15:02.258922959 +0000
> 	@@ -1,5 +1,6 @@
> 	 #= simple-encrypted@crypto.notmuchmail.org index.decryption=failure
> 	 #notmuch-dump batch-tag:3 config,properties,tags
> 	++encrypted +inbox +unread -- id:575ddaaf0b234fd85e077cfb4d44d467@notmuchmail.org
> 	 +encrypted +inbox +unread -- id:basic-encrypted@crypto.notmuchmail.org
> 	 +encrypted +inbox +unread -- id:encrypted-rfc822-attachment@crypto.notmuchmail.org
> 	 +encrypted +inbox +unread -- id:encrypted-signed@crypto.notmuchmail.org

Ok, I have added the line as you suggest. I was wondering though, as in
principle the secret key of test_suite@notmuchmail.org could be
available (it's in the key-ring at least)?

> T450-emacs-show: Testing emacs notmuch-show view
>  BROKEN show encrypted rfc822 message
> !!! Bodypart handler `notmuch-show-insert-part-*/*' threw an error:
> !!! Symbol’s value as variable is void: gnus-newsgroup-charset
>  FAIL   process cryptographic MIME parts (S/MIME)
> 	--- T450-emacs-show.21.notmuch-show-smime-encrypted-signed-multipart	2022-04-13 23:15:11.267167711 +0000
> 	+++ T450-emacs-show.21.OUTPUT	2022-04-13 23:15:11.267167711 +0000
> 	@@ -1,4 +1,4 @@
> 	-test_suite@notmuchmail.org (0 mins. ago) (encrypted inbox)
> 	+test_suite@notmuchmail.org (Yest. 14:57) (encrypted inbox)
>
> If you set notmuch-show-relative-dates to to nil, something like, with
> maybe better indentation:
>
> -test_emacs '(let ((notmuch-crypto-process-mime t))
> +test_emacs '(let ((notmuch-crypto-process-mime t)
> +          (notmuch-show-relative-dates nil))
>
> Then you will get an actual date that you can hard code.

It seems this refers to the very first line of the expected output? I
have updated the expected file to use an absolute date, and the test
script to set notmuch-show-relative-dates to nil as you suggest.

> 	 Subject: notmuch-show S/MIME test
> 	 To: test_suite@notmuchmail.org
> 	 Date: Tue, 12 Apr 2022 16:57:30 +0200
> 	@@ -6,10 +6,10 @@
> 	 [ smime.p7m: application/pkcs7-mime ]
> 	 [ Decryption successful ]
> 	 [ multipart/signed ]
> 	-[ Good signature by: <test_suite@notmuchmail.org> ]
> 	+[ Good signature by: test_suite@notmuchmail.org ]
>
> This is exactly the problem we just dealt with for T355-smime.

T357?

> I think the same solution can be applied, but you will need to inline
> the output so that you can do variable substitution.

I'm lost on what you're suggesting/expecting for this one. Remove the
angle brackets from the "Good signature" line of the expected file?

> 	 [ multipart/mixed ]
> 	 [ multipart/mixed ]
> 	 [ text/plain ]
> 	 The password is "12345678". But don't tell anyone!
> 	-[ test.dtd: application/octet-stream ]
> 	+[ test.dtd: application/octet-stream (as application/xml-dtd) ]
>
> This seems related to the function
> #'notmuch-show-get-mime-type-of-application/octet-stream, which is
> calling (mailcap-extension-to-mime "dtd"). Probably this is hard to make
> reproducible, so just seding away an "(as ...)" string is a reasonable
> alternative.
> [...]

I see; unfortunate choice of MIME part. As it seems I will need to
generate a new test message anyway, what would be a more "portable"
part? PNG?\r

Re: [PATCH] emacs/smime: render decrypted MIME entities in notmuch-show

[David Bremner]

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:


>
> Ok, I have added the line as you suggest. I was wondering though, as in
> principle the secret key of test_suite@notmuchmail.org could be
> available (it's in the key-ring at least)?

Yes, it's added to the test suite's temporary keyring. One of the
messages in test/corpora/crypto is not encrypted to that key though.

$ for file in *; do 
echo $file      
gpg < $file |& grep encrypted; echo
done

basic-encrypted.eml
gpg: encrypted with 1024-bit RSA key, ID C44D36DEAD54AB16, created 2011-02-05

encrypted-rfc822-attachment
gpg: encrypted with 4096-bit RSA key, ID 70E3C0DE87068451, created 2019-06-08
gpg: encrypted with 1024-bit RSA key, ID C44D36DEAD54AB16, created 2011-02-05

encrypted-signed.eml
gpg: encrypted with 1024-bit RSA key, ID C44D36DEAD54AB16, created 2011-02-05

simple-encrypted
gpg: encrypted with 4096-bit RSA key, ID 3B7AA7F014E69B5A, created 2016-12-21



> It seems this refers to the very first line of the expected output? I
> have updated the expected file to use an absolute date, and the test
> script to set notmuch-show-relative-dates to nil as you suggest.
>

Yes, it's referring to the first line of output.

>> 	 Subject: notmuch-show S/MIME test
>> 	 To: test_suite@notmuchmail.org
>> 	 Date: Tue, 12 Apr 2022 16:57:30 +0200
>> 	@@ -6,10 +6,10 @@
>> 	 [ smime.p7m: application/pkcs7-mime ]
>> 	 [ Decryption successful ]
>> 	 [ multipart/signed ]
>> 	-[ Good signature by: <test_suite@notmuchmail.org> ]
>> 	+[ Good signature by: test_suite@notmuchmail.org ]
>>
>> This is exactly the problem we just dealt with for T355-smime.
>
> T357?
>
>> I think the same solution can be applied, but you will need to inline
>> the output so that you can do variable substitution.
>
> I'm lost on what you're suggesting/expecting for this one. Remove the
> angle brackets from the "Good signature" line of the expected file?

Have a look at commit 8723e707c15f7b435f07f5d5ea693496bb9769bb, in
particular the lines

if [ $NOTMUCH_GMIME_EMITS_ANGLE_BRACKETS == 1 ]; then
   EXPECTED_EMAIL_ADDR='<test_suite@notmuchmail.org>'
else
   EXPECTED_EMAIL_ADDR='test_suite@notmuchmail.org'
fi

Then you can use $EXPECTED_EMAIL_ADDR in the expected output. This means
you'll have to generate the expected output rather than hard coding it
in a file. To pick some examples at random, you can see how to do
something similar on lines 157 and 222 of T035-read-config.sh

>
>> 	 [ multipart/mixed ]
>> 	 [ multipart/mixed ]
>> 	 [ text/plain ]
>> 	 The password is "12345678". But don't tell anyone!
>> 	-[ test.dtd: application/octet-stream ]
>> 	+[ test.dtd: application/octet-stream (as application/xml-dtd) ]
>>
>> This seems related to the function
>> #'notmuch-show-get-mime-type-of-application/octet-stream, which is
>> calling (mailcap-extension-to-mime "dtd"). Probably this is hard to make
>> reproducible, so just seding away an "(as ...)" string is a reasonable
>> alternative.
>> [...]
>
> I see; unfortunate choice of MIME part. As it seems I will need to
> generate a new test message anyway, what would be a more "portable"
> part? PNG?

PNG should not be worse, but the output might still need postprocessing
to avoid variation.