bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Mark Lillibridge]

[-- Attachment #1: Type: text/plain, Size: 6105 bytes --]


[|'s added to quote From lines]

    If you forward a Rmail message containing a MIME message with "From "
lines, the mbox escaping of From lines is not properly done.


    Start emacs -q then evaluate the 9521 patch:

(require 'rmailmm)

(defun rmail-insert-mime-forwarded-message (forward-buffer)
  "Insert the message in FORWARD-BUFFER as a forwarded message.
This is the usual value of `rmail-insert-mime-forwarded-message-function'."
  (let ((message-buffer
	 (with-current-buffer forward-buffer
	   (if rmail-buffer-swapped
	       rmail-view-buffer
	     forward-buffer))))
    (save-restriction
      (narrow-to-region (point) (point))
      (message-forward-make-body-mime message-buffer))))


    Then run Rmail on the following mbox file (remove |'s at line start
first):

|From mdl  Wed Oct 26 10:18:58 2011
|X-RMAIL-ATTRIBUTES: --EF----
|X-RMAIL-KEYWORDS: *
|X-Spam-Status: No, score=-4.7 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED,
|	RP_MATCHES_RCVD autolearn=unavailable version=3.3.1
|From: Mark Lillibridge <mark.lillibridge <at> hp.com>
|To: <bug-gnu-emacs <at> gnu.org>
|Subject: 23.3; rmail-edit-current-message adds extra '>'s to Froms...
|Date: Sat, 22 Oct 2011 12:05:57 -0700
|Reply-To: <mark.lillibridge <at> hp.com>
|
|
|Start with a Rmail message (this one will do) not containing MIME that
|contains lines like the following:
|
|>From level 1
|>>From level 2
|>>>From level 3
|
|From mdl  Tue Jan  1 12:27:17 2013
|Delivered-To: lillibridge <at> gmail.com
|Received: from gmail-pop.l.google.com [74.125.25.108]
|	by foil.strangled.net with POP3 (fetchmail-6.3.18)
|	for <mdl <at> localhost> (single-drop); Tue, 01 Jan 2013 12:27:17 -0800 (PST)
|Received: by 10.49.49.9 with SMTP id q9csp680843qen;
|        Tue, 1 Jan 2013 12:27:08 -0800 (PST)
|X-Received: by 10.52.69.201 with SMTP id g9mr59443696vdu.98.1357072028236;
|        Tue, 01 Jan 2013 12:27:08 -0800 (PST)
|Return-Path: <mark.lillibridge <at> hp.com>
|Received: from alum-mailsec-relay-12.mit.edu (ALUM-MAILSEC-RELAY-12.MIT.EDU. [18.7.68.32])
|        by mx.google.com with ESMTPS id o8si46143160vdv.77.2013.01.01.12.27.08
|        (version=TLSv1/SSLv3 cipher=OTHER);
|        Tue, 01 Jan 2013 12:27:08 -0800 (PST)
|Received: from alum-mailsec-scanner-7.mit.edu (ALUM-MAILSEC-SCANNER-7.MIT.EDU [18.7.68.19])
|	by alum-mailsec-relay-12.mit.edu (8.13.8/8.12.8) with ESMTP id r01KR7kv027035
|	for <mdl <at> alum.mit.edu>; Tue, 1 Jan 2013 15:27:07 -0500
|Received: from madara.hpl.hp.com (madara.hpl.hp.com [192.6.19.124])
|	by alum-mailsec-scanner-7.mit.edu (Symantec Messaging Gateway) with SMTP id A5.53.02235.B9643E05; Tue,  1 Jan 2013 15:27:07 -0500 (EST)
|Received: from postal.hpl.hp.com (postal.hpl.hp.com [15.25.115.25])
|	by madara.hpl.hp.com (8.14.4/8.14.4/HPL-PA Relay) with ESMTP id r01KR5Ae021394
|	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT)
|	for <mdl <at> alum.mit.edu>; Tue, 1 Jan 2013 12:27:06 -0800
|Received: from build-debian-1.u.hpl.hp.com (build-debian-1.u.hpl.hp.com [15.8.240.3])
|	by postal.hpl.hp.com (8.14.3/8.14.3/HPL-PA Hub) with ESMTP id r01KR4Df022085;
|	Tue, 1 Jan 2013 12:27:04 -0800
|From: Mark Lillibridge <mark.lillibridge <at> hp.com>
|To: mdl <at> alum.mit.edu
|Subject: test message containing From's in an attachment
|Reply-to: mark.lillibridge <at> hp.com
|Date: Tue, 01 Jan 2013 12:27:04 -0800
|Message-ID: <qmhip7gmyxz.fsf <at> build-debian-1.u.hpl.hp.com>
|MIME-Version: 1.0
|Content-Type: text/plain
|Content-Disposition: attachment; filename=froms.txt
|X-RMAIL-ATTRIBUTES: ------U-
|
|>From level 1
|>>From level 2
|>>>From level 3
|
|From mdl  Tue Jan  1 13:04:30 2013
|Delivered-To: lillibridge <at> gmail.com
|Received: from gmail-pop.l.google.com [74.125.25.108]
|	by foil.strangled.net with POP3 (fetchmail-6.3.18)
|	for <mdl <at> localhost> (single-drop); Tue, 01 Jan 2013 13:04:30 -0800 (PST)
|Received: by 10.49.49.9 with SMTP id q9csp681694qen;
|        Tue, 1 Jan 2013 13:04:13 -0800 (PST)
|X-Received: by 10.52.75.100 with SMTP id b4mr59369212vdw.52.1357074253041;
|        Tue, 01 Jan 2013 13:04:13 -0800 (PST)
|Return-Path: <mark.lillibridge <at> hp.com>
|Received: from alum-mailsec-relay-12.mit.edu (ALUM-MAILSEC-RELAY-12.MIT.EDU. [18.7.68.32])
|        by mx.google.com with ESMTPS id u1si24918746vdh.137.2013.01.01.13.04.12
|        (version=TLSv1/SSLv3 cipher=OTHER);
|        Tue, 01 Jan 2013 13:04:13 -0800 (PST)
|Received: from alum-mailsec-scanner-3.mit.edu (ALUM-MAILSEC-SCANNER-3.MIT.EDU [18.7.68.14])
|	by alum-mailsec-relay-12.mit.edu (8.13.8/8.12.8) with ESMTP id r01L47Pu030367
|	for <mdl <at> alum.mit.edu>; Tue, 1 Jan 2013 16:04:12 -0500
|Authentication-Results: symauth.service.identifier
|Received: from madara.hpl.hp.com (madara.hpl.hp.com [192.6.19.124])
|	by alum-mailsec-scanner-3.mit.edu (Symantec Messaging Gateway) with SMTP id E2.ED.02222.B4F43E05; Tue,  1 Jan 2013 16:04:12 -0500 (EST)
|Received: from masterns.hpl.hp.com (masterns.hpl.hp.com [15.0.48.4])
|	by madara.hpl.hp.com (8.14.4/8.14.4/HPL-PA Relay) with ESMTP id r01L4Auq022776
|	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT)
|	for <mdl <at> alum.mit.edu>; Tue, 1 Jan 2013 13:04:10 -0800
|Received: from build-debian-1.u.hpl.hp.com (build-debian-1.u.hpl.hp.com [15.8.240.3])
|	by masterns.hpl.hp.com (8.14.3/8.14.3/HPL-PA Hub) with ESMTP id r01L47j3018514;
|	Tue, 1 Jan 2013 13:04:08 -0800
|From: Mark Lillibridge <mark.lillibridge <at> hp.com>
|To: mdl <at> alum.mit.edu
|Subject: test message containing From's in an attachment
|Reply-to: mark.lillibridge <at> hp.com
|Date: Tue, 01 Jan 2013 13:04:07 -0800
|Message-ID: <qmhhan0mx88.fsf <at> build-debian-1.u.hpl.hp.com>
|MIME-Version: 1.0
|Content-Type: multipart/mixed; boundary="=-=-="
|X-RMAIL-ATTRIBUTES: --------
|
|--=-=-=
|Content-Type: text/plain
|
|
|body part
|
|
|--=-=-=
|Content-Type: text/plain
|Content-Disposition: attachment; filename=froms.txt
|
|>From level 1
|>>From level 2
|>>>From level 3
|
|--=-=-=--
|

    The result should be 3 messages.  If you forward any of the
messages, you'll see that the resulting message has the form:

|To: 
|...
|
|

[-- Attachment #2: Type: message/rfc822, Size: 136 bytes --]


|>From level 1
|>>From level 2
|>>>From level 3
|...
|

[-- Attachment #3: Type: text/plain, Size: 194 bytes --]


Note that the From lines should be under mboxrd:

|From level 1
|>From level 2
|>>From level 3


    See #13328 for related discussion of properly decoding From lines in
MIME messages.

- Mark

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Lars Ingebrigtsen]

Mark Lillibridge <mdl@alum.mit.edu> writes:

>     If you forward a Rmail message containing a MIME message with "From "
> lines, the mbox escaping of From lines is not properly done.

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

I don't use rmail much myself, but I vaguely recall that there was a
similar bug report about this, and that this had been fixed at some
point.

Perhaps Eli remembers; added to the CCs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>>     If you forward a Rmail message containing a MIME message with "From "
>> lines, the mbox escaping of From lines is not properly done.
>
> (I'm going through old bug reports that unfortunately weren't resolved
> at the time.)
>
> I don't use rmail much myself, but I vaguely recall that there was a
> similar bug report about this, and that this had been fixed at some
> point.
>
> Perhaps Eli remembers; added to the CCs.

This was a month ago -- Eli, do you remember?

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  13329@debbugs.gnu.org
> Date: Thu, 09 Jun 2022 19:18:55 +0200
> 
> Lars Ingebrigtsen <larsi@gnus.org> writes:
> 
> >>     If you forward a Rmail message containing a MIME message with "From "
> >> lines, the mbox escaping of From lines is not properly done.
> >
> > (I'm going through old bug reports that unfortunately weren't resolved
> > at the time.)
> >
> > I don't use rmail much myself, but I vaguely recall that there was a
> > similar bug report about this, and that this had been fixed at some
> > point.
> >
> > Perhaps Eli remembers; added to the CCs.
> 
> This was a month ago -- Eli, do you remember?

I don't remember, sorry.  I also didn't see anything pertinent in the
Git logs.

Moreover, upon re-reading the original report several times, I don't
think I understand what exactly is the bug: the original report is a
multipart message which is hard to parse.  It also shows what I think
is a single message, but says "The result should be 3 messages", which
left me utterly confused.

I'd prefer to have a description with 2 attachments: the original
message, and the forwarded message.  And the description should tell
in full what was expected to be seen, and why, preferably without
using ellipsis and hard-to-understand terminology such as "under
mboxrd".

Is it possible to get such a detailed report?  Then I could try the
messages here and see if there's still a problem left in our current
sources.

At this point I can only say that I'm forwarding email with MIME
attachments quite a lot, and didn't spot any problems, although I
admit there's not a lot of ">>...>From" in them.

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> At this point I can only say that I'm forwarding email with MIME
> attachments quite a lot, and didn't spot any problems, although I
> admit there's not a lot of ">>...>From" in them.

Here's a message with a From line:

From thing lala yes

If you can forward this as MIME in rmail, then I think the bug has been
fixed?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Eli Zaretskii]

[-- Attachment #1: Type: text/plain, Size: 139 bytes --]

Here is your message, forwarded as MIME by Rmail (I just edited the
Subject, to avoid creating a new bug report).  Did you receive it
OK?


[-- Attachment #2: Type: message/rfc822, Size: 799 bytes --]

From: Lars Ingebrigtsen <larsi@gnus.org>
To: Eli Zaretskii <eliz@gnu.org>
Cc: mdl@alum.mit.edu,  13329@debbugs.gnu.org
Subject: Re: bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines
Date: Fri, 10 Jun 2022 11:24:27 +0200

Eli Zaretskii <eliz@gnu.org> writes:

> At this point I can only say that I'm forwarding email with MIME
> attachments quite a lot, and didn't spot any problems, although I
> admit there's not a lot of ">>...>From" in them.

Here's a message with a From line:

>From thing lala yes

If you can forward this as MIME in rmail, then I think the bug has been
fixed?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Here is your message, forwarded as MIME by Rmail (I just edited the
> Subject, to avoid creating a new bug report).  Did you receive it
> OK?

Yup; looks fine.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: mdl@alum.mit.edu,  13329@debbugs.gnu.org
> Date: Fri, 10 Jun 2022 13:02:22 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Here is your message, forwarded as MIME by Rmail (I just edited the
> > Subject, to avoid creating a new bug report).  Did you receive it
> > OK?
> 
> Yup; looks fine.

Then I guess this problem no longer exists, and we can close this bug;
done.

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Mark Lillibridge]

[-- Attachment #1: Type: text/plain, Size: 1264 bytes --]


Eli Zaretskii <eliz@gnu.org> writes:

>  I'm completely confused by the data in this thread.
>  
>  Could you please send a problematic message as a binary attachment?
>  By "problematic message" I mean a message that, when forwarded by the
>  Rmail's 'f' command, produces a problem when the forwarded message is
>  received and read by Rmail.
>  
>  Armed with that information, I can investigate the problem on my end.

    This bug concerns trying to forward an existing message in a Rmail
mbox.  Thus, the easiest way is to start with a known Rmail file.   I've
attached a gziped mbox file; save it and gunzip it.  

    Now, start emacs 28.1 -q, find the mbox file, then do M-x
rmail-mode.  You should have three messages.  Try forwarding any of them
with 'f'; you will see that there are extra >'s in the proposed message
to send; this is the bug.  (There should be no >s before From level 1, 1
'>' before From level 2, etc.)  It is no longer necessary to worry about
the patch for 9521.

    Pictorially, the text we are trying to forward should be:
|From level 1
|>From level 2
|>>From level 3
where the leading |s are to prevent quoting.

    Note that this bug is likely related to #13328
(https://debbugs.gnu.org/cgi/bugreport.cgi?bug=13328).

- Mark

[-- Attachment #2: mbox_13328.gz --]
[-- Type: application/gzip, Size: 1585 bytes --]

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Mark Lillibridge]

    If you don't want to start with an existing mbox, send a message to
yourself containing a line like "From nothing" in the body.  You will
see that a > incorrectly shows up in the Rmail display due to bug
#13328.  If you forward it, the > remains, which is this bug.

    If you look closely, you can actually see the bug in the your
previous test -- a '>' got added in front of the From in the message you
test forwarded ('>From thing lala yes').

- Mark

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Eli Zaretskii]

> From: Mark Lillibridge <mdl@alum.mit.edu>
> Cc: eliz@gnu.org, larsi@gnus.org, 13329@debbugs.gnu.org
> Date: Mon, 10 Oct 2022 11:25:55 -0700
> 
> 
>     If you don't want to start with an existing mbox, send a message to
> yourself containing a line like "From nothing" in the body.  You will
> see that a > incorrectly shows up in the Rmail display due to bug
> #13328.

I just did, and I don't see the bug (in Emacs 28.2).  The mbox file I
get does include a line that begins with ">From", but in the message
buffer presented by Rmail to me I see just "From", with the ">"
removed.

I see the same in Emacs 28.1, so I'm unsure why you see the problem
there.  Maybe the reproduction recipe above is too simple, and
something else is missing to reproduce the problem?

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Eli Zaretskii]

> Cc: larsi@gnus.org, 13329@debbugs.gnu.org
> Date: Mon, 10 Oct 2022 22:03:51 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > From: Mark Lillibridge <mdl@alum.mit.edu>
> > Cc: eliz@gnu.org, larsi@gnus.org, 13329@debbugs.gnu.org
> > Date: Mon, 10 Oct 2022 11:25:55 -0700
> > 
> > 
> >     If you don't want to start with an existing mbox, send a message to
> > yourself containing a line like "From nothing" in the body.  You will
> > see that a > incorrectly shows up in the Rmail display due to bug
> > #13328.
> 
> I just did, and I don't see the bug (in Emacs 28.2).  The mbox file I
> get does include a line that begins with ">From", but in the message
> buffer presented by Rmail to me I see just "From", with the ">"
> removed.
> 
> I see the same in Emacs 28.1, so I'm unsure why you see the problem
> there.  Maybe the reproduction recipe above is too simple, and
> something else is missing to reproduce the problem?

And now I've also sent to myself a message with a MIME attachment,
where the attachment was a text that included a line that began with a
"From".  In the message I received, if I tell Rmail to show the
attachment, I see that line without any leading ">" before "From".  So
again, I'm unable to reproduce the problem you are describing.

Are you sure you are using rmail.el which came with Emacs 28?

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Mark Lillibridge]

Eli Zaretskii <eliz@gnu.org> writes:

>  > Cc: larsi@gnus.org, 13329@debbugs.gnu.org
>  > Date: Mon, 10 Oct 2022 22:03:51 +0300
>  > From: Eli Zaretskii <eliz@gnu.org>
>  > 
>  > > From: Mark Lillibridge <mdl@alum.mit.edu>
>  > > Cc: eliz@gnu.org, larsi@gnus.org, 13329@debbugs.gnu.org
>  > > Date: Mon, 10 Oct 2022 11:25:55 -0700
>  > > 
>  > > 
>  > >     If you don't want to start with an existing mbox, send a message to
>  > > yourself containing a line like "From nothing" in the body.  You will
>  > > see that a > incorrectly shows up in the Rmail display due to bug
>  > > #13328.
>  > 
>  > I just did, and I don't see the bug (in Emacs 28.2).  The mbox file I
>  > get does include a line that begins with ">From", but in the message
>  > buffer presented by Rmail to me I see just "From", with the ">"
>  > removed.
>  > 
>  > I see the same in Emacs 28.1, so I'm unsure why you see the problem
>  > there.  Maybe the reproduction recipe above is too simple, and
>  > something else is missing to reproduce the problem?
>  
>  And now I've also sent to myself a message with a MIME attachment,
>  where the attachment was a text that included a line that began with a
>  "From".  In the message I received, if I tell Rmail to show the
>  attachment, I see that line without any leading ">" before "From".  So
>  again, I'm unable to reproduce the problem you are describing.

    How did your mailer encode the attachment?  If it used base64, then
I don't expect you'll see the bug.


>  Are you sure you are using rmail.el which came with Emacs 28?

    For the test with the existing mbox, yes.  I downloaded and built
from source Emacs 28.1 and invoked it with -q, which should mean it
looks at none of my elisp.

- Mark

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Eli Zaretskii]

> From: Mark Lillibridge <mdl@alum.mit.edu>
> Cc: larsi@gnus.org, 13329@debbugs.gnu.org
> Date: Mon, 10 Oct 2022 13:37:03 -0700
> 
> >  And now I've also sent to myself a message with a MIME attachment,
> >  where the attachment was a text that included a line that began with a
> >  "From".  In the message I received, if I tell Rmail to show the
> >  attachment, I see that line without any leading ">" before "From".  So
> >  again, I'm unable to reproduce the problem you are describing.
> 
>     How did your mailer encode the attachment?  If it used base64, then
> I don't expect you'll see the bug.

It was base64, but that's the default used by Rmail, because I didn''t
change anything, just used the mail-add-attachment command, and the
file I attached was plain-ASCII file.

> >  Are you sure you are using rmail.el which came with Emacs 28?
> 
>     For the test with the existing mbox, yes.  I downloaded and built
> from source Emacs 28.1 and invoked it with -q, which should mean it
> looks at none of my elisp.

Then I have no idea what is going on, but from my testing the problem
doesn't exist.  Perhaps your configuration of email happens to trigger
some rare situation, but I'd need a clear recipe starting from "emacs -Q"
to reproduce it.  Or maybe your MTA has something to do with that?

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Mark Lillibridge]

Eli Zaretskii <eliz@gnu.org> writes:

>  > From: Mark Lillibridge <mdl@alum.mit.edu>
>  > Cc: eliz@gnu.org, larsi@gnus.org, 13329@debbugs.gnu.org
>  > Date: Mon, 10 Oct 2022 11:25:55 -0700
>  > 
>  > 
>  >     If you don't want to start with an existing mbox, send a message to
>  > yourself containing a line like "From nothing" in the body.  You will
>  > see that a > incorrectly shows up in the Rmail display due to bug
>  > #13328.
>  
>  I just did, and I don't see the bug (in Emacs 28.2).  The mbox file I
>  get does include a line that begins with ">From", but in the message
>  buffer presented by Rmail to me I see just "From", with the ">"
>  removed.
>  
>  I see the same in Emacs 28.1, so I'm unsure why you see the problem
>  there.  Maybe the reproduction recipe above is too simple, and
>  something else is missing to reproduce the problem?

    Hmm.  What mailer did you used to send the message?  Maybe it wasn't
sending a MIME message?  We know that #13328 doesn't show the extra >
for non-MIME messages.

    I was using Emacs Rmail, but not the pure emacs 28.1 -q because I
need to change settings to send email.  Let me see if I can try a purer
reenactment.

    Can you check the mbox I sent to see if you see extra >s for the
last 2 messages there?  Can you send me the pieces of your mbox with the
2 test messages you sent (use binary attachment to avoid corruption)?

    Clearly, one of our two set ups is different.  :-(

- Mark

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Mark Lillibridge]

[-- Attachment #1: Type: text/plain, Size: 1103 bytes --]


Mark Lillibridge <mdl@alum.mit.edu> writes:

>      I was using Emacs Rmail, but not the pure emacs 28.1 -q because I
>  need to change settings to send email.  Let me see if I can try a purer
>  reenactment.

    Ok, I sent myself a message with last line "From nothing" (pure text
body); fetchmail produced the attached incoming_sample.gz file, which
Rmail 'g' added to an empty Rmail file to produce the attached
RMAIL.sample.gz.  I did this using as pure an Rmail set up as I can with
emacs 28.1.  My sending/receiving goes through MIT email forwarding and
uses Gmail as a receiving service.

   Both of these files have the proper mbox quoting, with a > before the
From in the body.  If I switch to emacs 28.1 -q and open that Rmail
file, I see the extra > and pressing 'f' includes it in the forwarded
message.  This shows both bugs (13328 and 13329).

   See if you can open RMAIL.sample and reproduce the bugs on your end.
If so, something must differ between our email sending/receiving before
rmail processes.  If you can't reproduce the bugs given the Rmail file,
I'm very confused.

- Mark

[-- Attachment #2: incoming_sample.gz --]
[-- Type: application/gzip, Size: 4569 bytes --]

[-- Attachment #3: RMAIL.sample.gz --]
[-- Type: application/gzip, Size: 4589 bytes --]

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Eli Zaretskii]

> From: Mark Lillibridge <mdl@alum.mit.edu>
> Cc: eliz@gnu.org, larsi@gnus.org, 13329@debbugs.gnu.org
> Date: Mon, 10 Oct 2022 14:04:58 -0700
> 
> >      I was using Emacs Rmail, but not the pure emacs 28.1 -q because I
> >  need to change settings to send email.  Let me see if I can try a purer
> >  reenactment.
> 
>     Ok, I sent myself a message with last line "From nothing" (pure text
> body); fetchmail produced the attached incoming_sample.gz file, which
> Rmail 'g' added to an empty Rmail file to produce the attached
> RMAIL.sample.gz.  I did this using as pure an Rmail set up as I can with
> emacs 28.1.  My sending/receiving goes through MIT email forwarding and
> uses Gmail as a receiving service.
> 
>    Both of these files have the proper mbox quoting, with a > before the
> >From in the body.  If I switch to emacs 28.1 -q and open that Rmail
> file, I see the extra > and pressing 'f' includes it in the forwarded
> message.  This shows both bugs (13328 and 13329).
> 
>    See if you can open RMAIL.sample and reproduce the bugs on your end.
> If so, something must differ between our email sending/receiving before
> rmail processes.  If you can't reproduce the bugs given the Rmail file,
> I'm very confused.

So, to summarize:

  . the problem happens only in emails that have MIME-Version: header
  . the problem happens only in emails that have just the body, no
    attachments
  . the problem happens because rmail-show-mime doesn't remove the ">"
    from ">From" when it shows the plan-text MIME content

Is that correct?

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Mark Lillibridge <mdl@alum.mit.edu>
>> Cc: eliz@gnu.org, larsi@gnus.org, 13329@debbugs.gnu.org
>> Date: Mon, 10 Oct 2022 14:04:58 -0700
>>
>> >      I was using Emacs Rmail, but not the pure emacs 28.1 -q because I
>> >  need to change settings to send email.  Let me see if I can try a purer
>> >  reenactment.
>>
>>     Ok, I sent myself a message with last line "From nothing" (pure text
>> body); fetchmail produced the attached incoming_sample.gz file, which
>> Rmail 'g' added to an empty Rmail file to produce the attached
>> RMAIL.sample.gz.  I did this using as pure an Rmail set up as I can with
>> emacs 28.1.  My sending/receiving goes through MIT email forwarding and
>> uses Gmail as a receiving service.
>>
>>    Both of these files have the proper mbox quoting, with a > before the
>> >From in the body.  If I switch to emacs 28.1 -q and open that Rmail
>> file, I see the extra > and pressing 'f' includes it in the forwarded
>> message.  This shows both bugs (13328 and 13329).
>>
>>    See if you can open RMAIL.sample and reproduce the bugs on your end.
>> If so, something must differ between our email sending/receiving before
>> rmail processes.  If you can't reproduce the bugs given the Rmail file,
>> I'm very confused.
>
> So, to summarize:
>
>   . the problem happens only in emails that have MIME-Version: header
>   . the problem happens only in emails that have just the body, no
>     attachments
>   . the problem happens because rmail-show-mime doesn't remove the ">"
>     from ">From" when it shows the plan-text MIME content
>
> Is that correct?

It seems like we need more information here in order to make any
progress.  Mark, could you answer the questions above?

Thanks in advance.

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Stefan Kangas]

Stefan Kangas <stefankangas@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Mark Lillibridge <mdl@alum.mit.edu>
>>> Cc: eliz@gnu.org, larsi@gnus.org, 13329@debbugs.gnu.org
>>> Date: Mon, 10 Oct 2022 14:04:58 -0700
>>>
>>> >      I was using Emacs Rmail, but not the pure emacs 28.1 -q because I
>>> >  need to change settings to send email.  Let me see if I can try a purer
>>> >  reenactment.
>>>
>>>     Ok, I sent myself a message with last line "From nothing" (pure text
>>> body); fetchmail produced the attached incoming_sample.gz file, which
>>> Rmail 'g' added to an empty Rmail file to produce the attached
>>> RMAIL.sample.gz.  I did this using as pure an Rmail set up as I can with
>>> emacs 28.1.  My sending/receiving goes through MIT email forwarding and
>>> uses Gmail as a receiving service.
>>>
>>>    Both of these files have the proper mbox quoting, with a > before the
>>> >From in the body.  If I switch to emacs 28.1 -q and open that Rmail
>>> file, I see the extra > and pressing 'f' includes it in the forwarded
>>> message.  This shows both bugs (13328 and 13329).
>>>
>>>    See if you can open RMAIL.sample and reproduce the bugs on your end.
>>> If so, something must differ between our email sending/receiving before
>>> rmail processes.  If you can't reproduce the bugs given the Rmail file,
>>> I'm very confused.
>>
>> So, to summarize:
>>
>>   . the problem happens only in emails that have MIME-Version: header
>>   . the problem happens only in emails that have just the body, no
>>     attachments
>>   . the problem happens because rmail-show-mime doesn't remove the ">"
>>     from ">From" when it shows the plan-text MIME content
>>
>> Is that correct?
>
> It seems like we need more information here in order to make any
> progress.  Mark, could you answer the questions above?
>
> Thanks in advance.

More information was requested, but none was given within 4 months, so
I'm closing this bug.

If this is still an issue, please reply to this email (use "Reply to
all" in your email client) and we can reopen the bug report.

bug#13329: 24.2 post patch for #9521; Rmail forward does not correctly handle MIME messages containing "From " lines

[Eli Zaretskii]

> From: Mark Lillibridge <mdl@alum.mit.edu>
> Cc: larsi@gnus.org, 13329@debbugs.gnu.org
> Date: Mon, 10 Oct 2022 13:34:35 -0700
> 
> >  I just did, and I don't see the bug (in Emacs 28.2).  The mbox file I
> >  get does include a line that begins with ">From", but in the message
> >  buffer presented by Rmail to me I see just "From", with the ">"
> >  removed.
> >  
> >  I see the same in Emacs 28.1, so I'm unsure why you see the problem
> >  there.  Maybe the reproduction recipe above is too simple, and
> >  something else is missing to reproduce the problem?
> 
>     Hmm.  What mailer did you used to send the message?  Maybe it wasn't
> sending a MIME message?

Rmail, with sendmail-user-agent as the sending MUA.

> We know that #13328 doesn't show the extra > for non-MIME messages.

That wasn't clear originally, but later I sent "From" in an
attachment, and didn't see any ">From" in the received email.  I also
don't think there's a need to do anything with "From" in an
attachment, since attachments are encoded.


>     I was using Emacs Rmail, but not the pure emacs 28.1 -q because I
> need to change settings to send email.  Let me see if I can try a purer
> reenactment.
> 
>     Can you check the mbox I sent to see if you see extra >s for the
> last 2 messages there?

I see >From and >>From etc. in the attachments, but I don't understand
how the > part appeared there in the first place: attachments cannot
be possibly taken for the beginning of another email.