bug#39884: 27.0.50; Emacs may destroy outgoing email messages during sending

["Rainer Gemulla" <rgemulla@gmx.de>]

[-- Attachment #1.1: Type: text/plain, Size: 4120 bytes --]

I am composing emails using Emacs message mode. Shortly after switching 
to Emacs 27, some people complained to me that they could not read some 
of emails I sent them. I checked in my local email archive: even my 
archived emails were not readable with Emacs any more. This is a major 
bug, and for this reason I switched back to Emacs 26. It is my current 
understanding that the bug occurs when (1) forwarding messages and (2) 
signing the forwarded messages using GPG. So not everyone seems to be 
affected. Nevertheless, I experienced this bug on multiple machines (in 
fact, on all machines that I tried).

The rest of this email explains where the bug arises and how to 
reproduce it.

Before sending an email, Emacs converts the email text to MIME. This 
conversion process is performed by a function called mml-to-mime. I 
found that this function (or it's interaction with signing) is 
responsible for Emacs 27 destroying emails: the produced MIME can be 
invalid. This happens mostly when there are multiple mml parts in the 
original email, which is the case when forwarding an email.

After some playing around, I found a way to reproduce the bug. It's 
somewhat painful, but the error consistently arises. I first list the 
steps, then the original email, then its correct MIME encoding, and the 
incorrect MIME encoding that Emacs may produce. Note that step 3+4 
encode the message for the first time (all fine), steps 5-7 add signing, 
steps 8-9 encode the orginal message again (without signing, now 
broken).

1. Run: emacs -Q

2. M-x message-mode
3. Clear scratch buffer, paste original message
4. M-: (mml-to-mime) --> gives CORRECT result

5. Clear scratch buffer, paste original message
6. Insert a new line "<#secure method=pgpmime mode=sign>" at start of 
message (after line "--text follows this line--"). This makes Emacs try 
to sign the mail.
7. M-: (mml-to-mime) --> throws (expected) signer name error

8. Clear scratch buffer, paste original message
9. M-: (mml-to-mime)
   --> broken result (first Content-Type after "text follows..." is 
wrong)

I tried to track down the bug further and found this piece of code in 
mml-parse-1, line 284:

(setq tag (list 'part '(type . "text/plain"))
       no-markup-p t
       warn t)

During step 9, this statement is executed, but afterwards, the tag 
variable is not set to the list mentioned in the statement (seriously!). 
In my case, it had value 'Content-Type: multipart/alternative; 
boundary="====-=-="' right afterwards (as in the incorrect result 
below). At this point I gave up, looks like a deeper problem.


ORIGINAL MESSAGE
From: a@b.ce
To: c@d.de
Subject: Test
--text follows this line--
Test

<#mml type=message/rfc822 disposition=inline>
<#multipart type=alternative>
<#part type=text/plain charset="UTF-8" disposition=inline nofile=yes>
Some text.
<#part type=text/html charset="UTF-8" nofile=yes>
Some more text.
<#/multipart>
<#/mml>


CORRECT RESULT
From: a <at> b.ce
To: c <at> d.de
Subject: Test
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
--text follows this line--
--=-=-=
Content-Type: text/plain

Test


--=-=-=
Content-Type: message/rfc822
Content-Disposition: inline

--====-=-=
Content-Disposition: inline
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="====-=-="

Some text.

--====-=-=
Content-Type: text/html; charset=utf-8

Some more text.

--====-=-=--

--=-=-=--


INCORRECT RESULT AFTER STEP 9
From: a <at> b.ce
To: c <at> d.de
Subject: Test
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
--text follows this line--
--=-=-=
Content-Type: multipart/alternative; boundary="====-=-="
Content-Transfer-Encoding: base64

VGV4dAoK
--=-=-=
Content-Type: message/rfc822
Content-Disposition: inline

--====-=-=
Content-Disposition: inline
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="====-=-="

Some text.

--====-=-=
Content-Type: text/html; charset=utf-8

Some more text.

--====-=-=--

--=-=-=--

[-- Attachment #1.2: Type: text/html, Size: 9420 bytes --]

[-- Attachment #2: Type: application/pgp-signature, Size: 876 bytes --]