bug#23733: 24.5; Emacs sometimes inserts superfluous ">" when sending e-mails

bug#23733: 24.5; Emacs sometimes inserts superfluous ">" when sending e-mails

[Markus Triska]

Hi all,

please bear with me. For the first time, I do not have a recipe starting
from emacs -Q. I am trying to narrow down a mistake in Emacs that occurs
extremely rarely, when sending e-mails. In short, the mistake is that
Emacs, under very specific conditions, seems to insert a superfluous ">"
(greater than) character into e-mails before they are sent.

I first noticed the problem in 2008 when reporting two issues:

https://lists.gnu.org/archive/html/bug-gnu-emacs/2008-09/msg00606.html
https://lists.gnu.org/archive/html/bug-gnu-emacs/2008-12/msg00687.html

Please note the characteristic superfluous ">" in front of "From" in
both of these mails. I do not mean in the "From:" header, but within the
actual text of both mails, in particular:

 *) ">From machine B..."
 *) ">From OSX 10.4..."

I am 100% certain that I did not enter these ">" characters.

This issue has also happened to other users:

Example 1 (">From the documentation"):

    https://lists.gnu.org/archive/html/bug-gnu-emacs/2003-03/msg00013.html

Example 2 (">From man page"):

    https://lists.gnu.org/archive/html/bug-gnu-emacs/2013-09/msg00108.html

To find examples of this issue, I have analyzed 116 thousand messages
that were posted to the GNU Emacs bug tracker from 2002 to 2016.

Of these messages, I have restricted the search to those messages that
contained the regexp "User-Agent.*Gnus". There were 35 thousand such
messages. I have done this to limit the search to those messages that
were most likely really sent from within Emacs. There are several
further occurrences of the issue, but I could not tell with such
certainty whether those messages were sent with Emacs itself.

Interestingly this seems to happen only when "From" is used somewhere in
the mail, at the beginning of a line. Then, sometimes Emacs simply turns
"From" to ">From".

And it even gets somewhat stranger: In my ~/Mail/sent file, I have one
more occurrence of the issue:

  ">From what I understand..."

which arose when I posted (Wed, 21 Oct 2015 01:47:22 +0200) to:

   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21526

but in fact the superfluous ">" does not occur in the message as it is
shown in the tracker. Again, I am 100% certain that I did not manually
enter the ">" in front of "From", yet it appears in ~/Mail/sent.

At this point, it would already be extremely helpful to me if someone
could say "Yes, I also noticed this at some point in time."

If it helps, the value of my mail-user-agent is: message-user-agent,
and send-mail-function is: sendmail-send-it.

Thank you and all the best!
Markus



In GNU Emacs 24.5.1 (x86_64-apple-darwin15.5.0, X toolkit, Xaw scroll bars)
 of 2016-05-30 on mt-imac
Windowing system distributor `The X.Org Foundation', version 11.0.11502000
Configured using:
 `configure --prefix=/opt/local --without-dbus --without-gconf
 --without-libotf --without-m17n-flt --without-gpm --without-gnutls
 --with-xml2 --infodir /opt/local/share/info/emacs
 --with-x-toolkit=lucid --without-xaw3d --without-imagemagick --with-xpm
 --with-jpeg --with-tiff --with-gif --with-png --without-rsvg --with-xft
 'CFLAGS=-pipe -Os -arch x86_64' CPPFLAGS=-I/opt/local/include
 'LDFLAGS=-L/opt/local/lib -Wl,-headerpad_max_install_names -lfreetype
 -lfontconfig -Wl,-no_pie -arch x86_64''

bug#23733: 24.5; Emacs sometimes inserts superfluous ">" when sending e-mails

[Glenn Morris]

There's no mystery here, this is mbox from escaping.
https://en.wikipedia.org/wiki/Mbox

I don't think there is a real bug here, though I would not be
surprised to learn that Emacs sometimes leaves >From where strictly
speaking it should put From.

bug#23733: 24.5; Emacs sometimes inserts superfluous ">" when sending e-mails

[Markus Triska]

Glenn Morris <rgm@gnu.org> writes:

> this is mbox from escaping.  https://en.wikipedia.org/wiki/Mbox

OK, this explains the last occurrence I mentioned, which is the only one
taken from my ~/Mail/sent. But the other occurrences are part of the
mails that were actually sent to the mailing list, and certainly the ">"
should occur in any sent mails only if it was also intended. In other
words, there should be a difference whether I start a paragraph with
"From" or ">From" in terms of what is actually sent as e-mail.

Thank you for looking into this!

bug#23733: 24.5; Emacs sometimes inserts superfluous ">" when sending e-mails

[Glenn Morris]

FWIW I think the behaviour of Emacs in this area is unlikely to change.

bug#23733: 24.5; Emacs sometimes inserts superfluous ">" when sending e-mails

[Markus Triska]

Glenn Morris <rgm@gnu.org> writes:

> FWIW I think the behaviour of Emacs in this area is unlikely to change.

That's a pity because I would really like Emacs to send and receive
mails as intended. I have now constructed a self-contained test case:

a) $ emacs -Q
b) C-x m
c) write a mail with the following occurring in the message body,
   at the beginning of a line and without quotes: "From me to you!"
d) use any recipient and subject, and use C-c C-c to send the mail
e) select "transport" and hit RET (this requires sendmail)
f) the recipient will receive ">From me to you!".

Maybe there is any way to make this work is intended?

bug#23733: 24.5; Emacs sometimes inserts superfluous ">" when sending e-mails

[Andreas Schwab]

Markus Triska <triska@metalevel.at> writes:

> Maybe there is any way to make this work is intended?

Use quoted-printable encoding.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#23733: 24.5; Emacs sometimes inserts superfluous ">" when sending e-mails

[Markus Triska]

Andreas Schwab <schwab@linux-m68k.org> writes:

> Use quoted-printable encoding.

When the outgoing message is ASCII only, all ways I tried to enforce
quoted-printable encoding failed to work. I have in my .emacs:

 (setq mm-coding-system-priorities '(iso-8859-1 utf-8))

and I tried enforcing quoted-printable with:

 (us-ascii . quoted-printable)

in mm-body-charset-encoding-alist. Still, in the case I posted, the ">"
and the mail itself are sent without using quoted-printable encoding.

bug#23733: 24.5; Emacs sometimes inserts superfluous ">" when sending e-mails

[Markus Triska]

Glenn Morris <rgm@gnu.org> writes:

> I don't think there is a real bug here, though I would not be
> surprised to learn that Emacs sometimes leaves >From where strictly
> speaking it should put From.

I understand that it may be necessary to insert the ">" when Gnus sends
the message. But certainly the ">" that was inserted only to quote
"From" should not be shown on the *receiver* side. Instead, when Gnus
displays the received mail or (as in the case of the mailing list) a
HTML page is generated from received mails, the ">" should be stripped.

The main drawback I experience with the current behaviour is that even
if you send the mail with Gnus, and also read the received mail with
Gnus, the actual content of the mail is not preserved as intended.

Using quoted-printable (as Andreas suggested) may not help with this,
because this seems more of an issue with how Gnus or other clients
display received mails. It is only a minor issue, though I still would
appreciate if it could be fixed somehow in Gnus. Thank you!

bug#23733: 24.5; Emacs sometimes inserts superfluous ">" when sending e-mails

[Andreas Schwab]

Markus Triska <triska@metalevel.at> writes:

> I understand that it may be necessary to insert the ">" when Gnus sends
> the message. But certainly the ">" that was inserted only to quote
> "From" should not be shown on the *receiver* side. Instead, when Gnus
> displays the received mail or (as in the case of the mailing list) a
> HTML page is generated from received mails, the ">" should be stripped.

You can't do that because it's ambiguous.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#23733: 24.5; Emacs sometimes inserts superfluous ">" when sending e-mails

[Markus Triska]

Andreas Schwab <schwab@linux-m68k.org> writes:

> You can't do that because it's ambiguous.

OK, I see! I was surprised to see such an elementary flaw of the format.

Thank you and all the best,
Markus