BUG: emacs: fcc duplicates messages with variant bodies due to mml security

BUG: emacs: fcc duplicates messages with variant bodies due to mml security

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 2077 bytes --]

I'm concerned that notmuch-emacs is causing variant messages of
encrypted/signed messages to be generated.

In particular, if notmuch-fcc-dirs is not nil, two variants of the
message bodies will be created: one will be sent through "notmuch
insert" and the other will be sent to sendmail.

the mml security modes (e.g. "#secure method=pgpmime mode=sign" (in
angle brackets)) take place after this bifurcation.

The result for a signed message is:

  * the message sent to "notmuch insert" has one cryptographic signature

  * the message sent to sendmail has a different cryptographic
    signature, in particular if it is made more than 1 second later
    (some signature algorithms will produce identical signatures if made
    in the same second)

You can try this out by:

 0) put ignore-cache-for-signing in ~/.gnupg/gpg-agent.conf

 1) run "gpgconf --kill gpg-agent" to ensure the config is re-read

 2) send yourself a signed message with message-id $mid.
 
 3) Pull it back in however you receive mail, and then:

     diff -u $(notmuch search --output=files "id:$mid")

If the messages were more than 1 second apart, you'll see the signature
part differs.

The result for an encrypted message is:

 * the message sent to "notmuch insert" is encrypted with one session
   key

 * the message sent to sendmail is encrypted with a different session key

So the entire body of the message will be different.

I'm currently working around this by setting notmuch-fcc-dirs to nil,
and setting message-send-mail-hook to (dkg-notmuch-fcc), which is defined
as:

    (defun dkg-notmuch-fcc ()
      (shell-command-on-region (point-min) (point-max) "notmuch insert"))

this takes advantage of message-send-mail-hook's late-send.  as the
documentation for that var says:

    This hook is run very late -- just before the message is sent as
    mail.

But this isn't something normal users should need to do manually.

Any emacs gurus want to propose a way to improve the situation?  i'm at
a bit of a loss.

  --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: BUG: emacs: fcc duplicates messages with variant bodies due to mml security

[David Bremner]

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

>
> I'm currently working around this by setting notmuch-fcc-dirs to nil,
> and setting message-send-mail-hook to (dkg-notmuch-fcc), which is defined
> as:
>
>     (defun dkg-notmuch-fcc ()
>       (shell-command-on-region (point-min) (point-max) "notmuch insert"))
>
> this takes advantage of message-send-mail-hook's late-send.  as the
> documentation for that var says:
>
>     This hook is run very late -- just before the message is sent as
>     mail.
>
> But this isn't something normal users should need to do manually.
>
> Any emacs gurus want to propose a way to improve the situation?  i'm at
> a bit of a loss.

Not an "emacs guru". However... 

Currently our fcc processing is run (in a slightly weird way by
rebinding message-do-fcc) just before message-sent-hook. This has the
potentially important semantics that it only runs for messages that are
successfully sent.

There is a variable message-encoded-mail-cache, which holds the raw sent
message. We should probably use this instead of doing our own
re-encoding (mml-generate-mime does the actual mime stuff, and is called
from message-encode-message-body. Larsi added this variable to gnus
after we copied and modified message-do-fcc. The change _looks_
relatively simple, guarding the call to message-encode-message-body as
follows:

	;; Avoid re-doing things like GPG-encoding secret parts.
	(if (not encoded-cache)
	    (message-encode-message-body)
	  (erase-buffer)
	  (insert encoded-cache))

[PATCH] test/emacs: known broken test for matching fcc and sent message

[David Bremner]

Based on the method outlined by Daniel Kahn Gilmour in
id:87k1zm225v.fsf@fifthhorseman.net.  With a delay of 0.2 seconds the
test becomes flaky on my machine. With a 1 second delay it fails
consistently for more than 1600 iterations.
---
 test/T350-crypto.sh | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/test/T350-crypto.sh b/test/T350-crypto.sh
index 8dbf8935..1283c52c 100755
--- a/test/T350-crypto.sh
+++ b/test/T350-crypto.sh
@@ -13,13 +13,23 @@ test_description='PGP/MIME signature verification and decryption'
 test_require_emacs
 add_gnupg_home
 
-test_begin_subtest "emacs delivery of signed message"
+test_begin_subtest "emacs delivery of signed message via fcc"
 test_expect_success \
 'emacs_fcc_message \
     "test signed message 001" \
     "This is a test signed message." \
     "(mml-secure-message-sign)"'
 
+test_begin_subtest "emacs delivery of signed message via fcc and smtp"
+test_subtest_known_broken
+emacs_deliver_message \
+    'signed message sent via SMTP' \
+    'This is a test that messages are sent via SMTP' \
+    "(add-hook 'message-send-mail-hook (lambda () (sleep-for 1)))
+     (mml-secure-message-sign)"
+msg_file=$(notmuch search --output=files subject:signed-message-sent-via-SMTP)
+test_expect_equal_file sent_message "$msg_file"
+
 test_begin_subtest "signed part content-type indexing"
 output=$(notmuch search mimetype:multipart/signed and mimetype:application/pgp-signature | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; test signed message 001 (inbox signed)"
-- 
2.34.1

Re: [PATCH] test/emacs: known broken test for matching fcc and sent message

[David Bremner]

David Bremner <david@tethera.net> writes:

> Based on the method outlined by Daniel Kahn Gilmour in
> id:87k1zm225v.fsf@fifthhorseman.net.  With a delay of 0.2 seconds the
> test becomes flaky on my machine. With a 1 second delay it fails
> consistently for more than 1600 iterations.

And yet I didn't test the next test in the file, which I broke by adding
another signed message. This is fixed in git.

d

Re: [PATCH] test/emacs: known broken test for matching fcc and sent message

[Tomi Ollila]

On Wed, Jan 19 2022, David Bremner wrote:

> David Bremner <david@tethera.net> writes:
>
>> Based on the method outlined by Daniel Kahn Gilmour in
>> id:87k1zm225v.fsf@fifthhorseman.net.  With a delay of 0.2 seconds the
>> test becomes flaky on my machine. With a 1 second delay it fails
>> consistently for more than 1600 iterations.
>
> And yet I didn't test the next test in the file, which I broke by adding
> another signed message. This is fixed in git.

I don't see commit im my pull (if that meant it, so probably only in your
own git) -- so you have chance to fix typo in the commit message :D

Tomi

>
> d

Re: [PATCH] test/emacs: known broken test for matching fcc and sent message

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

> On Wed, Jan 19 2022, David Bremner wrote:
>
>> David Bremner <david@tethera.net> writes:
>>
>>> Based on the method outlined by Daniel Kahn Gilmour in
>>> id:87k1zm225v.fsf@fifthhorseman.net.  With a delay of 0.2 seconds the
>>> test becomes flaky on my machine. With a 1 second delay it fails
>>> consistently for more than 1600 iterations.
>>
>> And yet I didn't test the next test in the file, which I broke by adding
>> another signed message. This is fixed in git.
>
> I don't see commit im my pull (if that meant it, so probably only in your
> own git) -- so you have chance to fix typo in the commit message :D
>
> Tomi
>
>>
>> d

The updated version is at
id:20220120123354.3999579-4-david@tethera.net. I feel like there must be
something more like a miscommunication than a typo here. What are you
looking at?

d

Re: [PATCH] test/emacs: known broken test for matching fcc and sent message

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

> On Wed, Jan 19 2022, David Bremner wrote:
>
>> David Bremner <david@tethera.net> writes:
>>
>>> Based on the method outlined by Daniel Kahn Gilmour in
>>> id:87k1zm225v.fsf@fifthhorseman.net.  With a delay of 0.2 seconds the
>>> test becomes flaky on my machine. With a 1 second delay it fails
>>> consistently for more than 1600 iterations.
>>
>> And yet I didn't test the next test in the file, which I broke by adding
>> another signed message. This is fixed in git.
>
> I don't see commit im my pull (if that meant it, so probably only in your
> own git) -- so you have chance to fix typo in the commit message :D
>

Ah, I see the typo now. Once more proving, never type what you can
copy-pasta.

d

Re: BUG: emacs: fcc duplicates messages with variant bodies due to mml security

[David Bremner]

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> I'm concerned that notmuch-emacs is causing variant messages of
> encrypted/signed messages to be generated.
>
> In particular, if notmuch-fcc-dirs is not nil, two variants of the
> message bodies will be created: one will be sent through "notmuch
> insert" and the other will be sent to sendmail.
>

This should be fixed as of da302e1cbaaab89b2bbb32c0f59e1aa6ee708455

d