bug#9521: 24.0.50; rmail-forward

bug#9521: 24.0.50; rmail-forward

[Kenichi Handa]

[-- Attachment #1: Type: text/plain, Size: 6411 bytes --]

rmail-forward doesn't handle a attachment file
correctly. For instance, when I have a message something
like this in RMAIL:

  ------------------------------------------------------------
  [...]
  [1:text/plain Hide]
  
  test of attachment
  
  [2:application/pdf Show Save:temp.pdf (2kB)]
  ------------------------------------------------------------

Typing f inserts just this in "part" part of *unsent mail to
...* buffer:

  ------------------------------------------------------------
  From: Kenichi Handa <handa@m17n.org>
  To: handa@m17n.org
  Subject: test from shatin
  Date: Thu, 15 Sep 2011 14:14:58 +0900
  Message-ID: <87aaa6xu7h.fsf@m17n.org>
  Content-Type: multipart/mixed; boundary="=-=-="


  [1:text/plain Hide]

  test of attachment


  [2:application/pdf Show Save:temp.pdf (2kB)]
  ------------------------------------------------------------

It's the content of rmail-view buffer and thus the outgoing
mail doesn't contain the correct attachment.

In Emacs 23.3, the content of "part" part was the original
whole message, and thus the outgoing mail surely contains an
attachment in a correct MIME form.

I'm attaching the same sample file so that you can see what
I described by typing 'f' in RMAIL.

---
Kenichi Handa
handa@m17n.org



In GNU Emacs 24.0.50.2 (i686-pc-linux-gnu, GTK+ Version 2.20.1)
 of 2011-09-16 on etlken
Windowing system distributor `The X.Org Foundation', version 11.0.10706000
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ja_JP.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: RMAIL

Minor modes in effect:
  diff-auto-refine-mode: t
  shell-dirtrack-mode: t
  display-time-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent input:
<backspace> <backspace> <backspace> C-\ c o m b <backspace> 
v e n i e n t <escape> $ 0 <escape> < C-c C-c h k <return> 
g n d d d d d d d d d d d d d d d d d d d d d d d d 
d d d d SPC <backspace> C-x o C-u C-v C-x o n d d d 
d d d d d d d s y C-x o C-h v r m a i l - m i <tab> 
<tab> <tab> <tab> C-g C-x C-f M-p <M-backspace> <M-backspace> 
<M-backspace> <M-backspace> <M-backspace> e m <tab> 
/ w o <tab> l i s p / m a i <tab> r m <tab> m <tab> 
m <tab> <return> C-s f o r w a r d C-w C-w C-s C-a 
C-n C-n C-n C-n C-n C-n C-n C-u C-v C-n C-n C-n C-n 
C-n C-n C-u C-v C-u C-v C-u C-v M-f M-f M-b M-b C-s 
C-w C-w C-w C-w C-w C-a C-x C-g <down-mouse-1> <mouse-1> 
C-a C-x b <return> C-x C-f M-p <return> C-M-a M-f M-f 
M-b C-s C-w C-w C-w C-w C-s C-s C-w C-a C-x C-f r m 
<tab> . <tab> <return> C-s C-s C-s C-w C-s C-a C-x 
b <return> C-r C-r C-a C-x k <return> C-x o j C-x o 
x C-g C-x o M-f M-f M-f M-f M-f M-f M-f M-f M-f M-b 
C-s C-w C-w C-r C-r C-a C-x o C-v C-p C-p C-p C-p C-p 
C-p C-n C-n C-n C-n C-n C-SPC C-v C-v C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-p C-p C-p <escape> w C-x C-x C-x C-x C-x C-g 
<escape> x r e p <tab> o <tab> r <tab> <return>

Recent messages:
Quit
Making completion list...
Mark saved where search started [2 times]
Mark set
Mark saved where search started [3 times]
Quit
Mark saved where search started
Mark set
Saved text from "rmail-forward still has a problem when a"
Making completion list... [2 times]

Load-path shadows:
/usr/local/share/emacs/site-lisp/evi-mule hides /usr/local/share/emacs/site-lisp/lookup/evi-mule
/usr/local/share/emacs/site-lisp/evi hides /usr/local/share/emacs/site-lisp/lookup/evi
/usr/local/share/emacs/site-lisp/anthy/anthy hides /usr/local/share/emacs/site-lisp/egg/egg/anthy
/usr/local/share/emacs/site-lisp/egg/its/thai hides /usr/local/work/emacs/stable/lisp/language/thai
/usr/local/share/emacs/site-lisp/egg/its/greek hides /usr/local/work/emacs/stable/lisp/language/greek
/usr/local/work/emacs/stable/lisp/textmodes/table hides ~/emacslisp/table
/usr/local/work/emacs/stable/lisp/language/thai-word hides ~/emacslisp/thai-word
/usr/local/work/emacs/stable/lisp/progmodes/prolog hides ~/emacslisp/prolog
/usr/local/work/emacs/stable/lisp/emacs-lisp/syntax hides ~/emacslisp/syntax
/usr/local/work/emacs/stable/lisp/textmodes/tex-mode hides ~/emacslisp/tex-mode

Features:
(iso-transl parse-time vc-cvs edmacro kmacro rect dabbrev
find-func etags warnings compile info diff-mode diff
thingatpt browse-url ind-util sh-script executable tar-mode
pcmpl-gnu pcmpl-unix ispell shadow emacsbug doc-view
image-mode dired nxml-uchnm rng-xsd xsd-regexp rng-cmpct
rng-nxml rng-valid rng-loc rng-uri rng-parse nxml-parse
rng-match rng-dt rng-util rng-pttrn nxml-ns nxml-mode
nxml-outln nxml-rap nxml-util nxml-glyph nxml-enc xmltok
help-fns ansi-color shell pcomplete comint ring add-log
vc-bzr pp wid-edit descr-text network-stream starttls tls
mailalias smtpmail auth-source eieio byte-opt bytecomp
byte-compile cconv macroexp assoc password-cache sendmail
regexp-opt jka-compr sort mailcap newcomment ja-dic
mule-util kkc ja-dic-utl quail help-mode view supercite
easy-mmode regi gnus-util mail-extr multi-isearch qp
rmailkwd rmailmm message format-spec rfc822 mml easymenu
mml-sec mm-decode mm-bodies mm-encode mailabbrev gmm-utils
mailheader mail-parse rfc2231 js2-mode-autoloads package
tabulated-list rmail-parse-url time rmail-sa rmailsum rmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
time-date japan-util tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset
image fringe lisp-mode register page menu-bar rfn-eshadow
timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer loaddefs button faces cus-face
files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable
backquote make-network-process dbusbind dynamic-setting
system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty emacs)


[-- Attachment #2: temp.pdf --]
[-- Type: application/pdf, Size: 2229 bytes --]

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Richard Stallman]

This bug report will be sent to the Bug-GNU-Emacs mailing list
and the GNU bug tracker at debbugs.gnu.org.  Please check that
the From: line contains a valid email address.  After a delay of up
to one day, you should receive an acknowledgement at that address.

Please write in English if possible, as the Emacs maintainers
usually do not have translators for other languages.

Please describe exactly what actions triggered the bug, and
the precise symptoms of the bug.  If you can, give a recipe
starting from `emacs -Q':

When I use the f command in Rmail, it generates a message like this:

======================================================================
From: Richard Stallman <rms@gnu.org>
To: rms
Subject: [muthu@member.fsf.org: Re: Who are the organizers of this event]
bcc: rms-outgoing@gnu.org
Reply-to: rms@gnu.org
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
--text follows this line--
--=-=-=
Content-Type: message/rfc822
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

X-Spam-Status: No, score=-0.7 required=5.0 tests=BAYES_20,RCVD_IN_DNSWL_LOW
	autolearn=ham version=3.3.1
Message-ID: <MTAwMDAxMi5tdXRodQ.1318317081@quikprotect>
In-Reply-To: <E1RCyk0-0007Qh-EF@fencepost.gnu.org>
Date: Tue, 11 Oct 2011 12:41:21 +0530 (IST)
Subject: Re: Who are the organizers of this event
From: "Muthu" <muthu@member.fsf.org>
To: rms@gnu.org
Content-Type: text/plain;charset=iso-8859-1

> Can you send me the email addresses of the organizers?
>
hackmed-info@mit.edu



--=-=-=
Content-Type: text/plain



-- 
Dr Richard Stallman

--=-=-=--
======================================================================

I have a friend who reads mail using IceDove 2.0.0.24, and these mime
parts appear empty.

Is there anything wrong with these mime parts?
Or is it a bug in that version of IceDove?


If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
/home/rms/emacs-bzr/trunk/etc/DEBUG.


In GNU Emacs 24.0.50.6 (mips64el-unknown-linux-gnu, GTK+ Version 2.12.12)
 of 2011-10-14 on theobromine2
configured using `configure  'CFLAGS=-g -O1''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Mail

Minor modes in effect:
  shell-dirtrack-mode: t
  gpm-mouse-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Recent input:
p p p n p p p p C-x b o u t g TAB RET g C-x b C-g C-x 
b C-g C-x d ESC DEL e m a c s - s r c / ESC DEL DEL 
- b z r / t r u n k / l i s p / m a i l RET C-s s e 
n d m a i l C-n C-n C-n C-n C-n C-n C-u C-p C-p C-p 
e C-v C-s p a r t C-n C-n C-s C-s C-s C-s C-a ESC v 
ESC < C-s m i m e C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-s C-s C-n C-n C-s C-s C-v C-s C-s ESC < C-s m 
a i l - e n c o d e - m m l C-s C-n C-s C-s C-s C-s 
C-a C-x C-f r m a i l / DEL . e l RET C-s C-s C-s C-s 
C-a C-u C-v C-u C-v ESC < C-s # p a r t C-s C-s C-a 
ESC < C-s p a r t C-s C-s C-s C-s C-s C-s C-s DEL C-s 
C-s C-s C-s C-s C-s C-g C-g C-u C-@ C-u C-@ C-u C-@ 
C-u C-v C-u C-v C-u C-u C-p C-u C-n C-n C-n C-n C-e 
C-n C-e C-h f C-g C-h v RET C-x o C-n C-n C-e C-p TAB 
C-u C-p C-p C-p C-p C-p TAB RET C-x 1 ESC f ESC f C-h 
v RET C-x o C-n C-n C-e C-h f RET C-n TAB RET C-x 1 
ESC f C-f C-@ ESC C-f ESC w C-x 4 m ESC x r e p o r 
t SPC e m a s SPC DEL c s SPC b u g RET

Recent messages:
Mark set
Mark saved where search started [3 times]
Mark set
Mark saved where search started
Mark set
Quit [2 times]
mouse-2, RET: describe this variable
mouse-2, RET: find variable's definition
mouse-2, RET: find function's definition
Mark set

Load-path shadows:
None found.

Features:
(wid-edit descr-text ind-util jka-compr etags cc-mode byte-opt
bytecomp byte-compile cconv macroexp cc-langs cl cc-fonts cc-guess
cc-bytecomp cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs find-func pp help-fns vc-bzr parse-time vc-cvs mule-util
cal-move cal-menu calendar cal-loaddefs ispell tmm electric dired-aux
epa-mail epa derived epg epg-config rmailout shell pcomplete grep
compile comint ring qp quail novice sgml-mode dabbrev multi-isearch
mail-extr newcomment help-mode view shadow mailalias emacsbug rmailmm
message sendmail format-spec rfc822 mml easymenu mml-sec mm-decode
mm-bodies mm-encode mailabbrev gmm-utils mailheader mail-parse rfc2231
rmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils dired
regexp-opt t-mouse time-date battery paren cus-start cus-load tooltip
ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd
fontset image fringe lisp-mode register page menu-bar rfn-eshadow
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Eli Zaretskii]

merge 9766 9521
stop

> Date: Sun, 16 Oct 2011 07:19:05 -0400
> From: Richard Stallman <rms@gnu.org>
> Reply-To: rms@gnu.org
> 
> When I use the f command in Rmail, it generates a message like this:
> [...]
> I have a friend who reads mail using IceDove 2.0.0.24, and these mime
> parts appear empty.
> 
> Is there anything wrong with these mime parts?
> Or is it a bug in that version of IceDove?

It's a missing feature in rmail-forward, see bug #9521 (now merged
with this one) and a short conversation that started here:

  http://lists.gnu.org/archive/html/bug-gnu-emacs/2011-09/msg00691.html

You can disable the MIME-attachment style of forwarding by setting
rmail-enable-mime-composing to nil, in which case Rmail will generate
an inline plain-text forwarded message, but I don't remember if that
fixes this particular problem.

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Lars Magne Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> Is there anything wrong with these mime parts?
> Or is it a bug in that version of IceDove?

The message looks valid to me, but some other mail programs have little
support for message/rfc822 parts, so it's often better to forward
messages using a non-MIME method.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Richard Stallman]

    It's a missing feature in rmail-forward, see bug #9521 (now merged
    with this one) and a short conversation that started here:

      http://lists.gnu.org/archive/html/bug-gnu-emacs/2011-09/msg00691.html

It is hard for me to read a thread of mail.
Can you show me the text that describes what is missing?
Maybe I can write it.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Eli Zaretskii]

> Date: Sun, 16 Oct 2011 17:40:12 -0400
> From: Richard Stallman <rms@gnu.org>
> CC: 9766@debbugs.gnu.org
> 
>     It's a missing feature in rmail-forward, see bug #9521 (now merged
>     with this one) and a short conversation that started here:
> 
>       http://lists.gnu.org/archive/html/bug-gnu-emacs/2011-09/msg00691.html
> 
> It is hard for me to read a thread of mail.
> Can you show me the text that describes what is missing?
> Maybe I can write it.

Here:

> > From: Kenichi Handa <address@hidden>
> > Cc: address@hidden, address@hidden
> > Date: Fri, 16 Sep 2011 12:41:37 +0900
> > 
> > rmail-forward still has a problem when a message has an
> > attachment file.

> Yes, I know.  But that's a different bug.  It happens because
> rmail-forward simply copies the text from the buffer where the
> forwarded message is displayed, which is not TRT when the message
> has
> attachments, because rmail-mime reformats it for display and adds
> buttons.

Actually that copying is done by
rmail-insert-mime-resent-message
(rmail-insert-mime-resent-message-function is set to it when
rmailmm.el is loaded), and that function was changed by RMS
as this.

2011-06-22  Richard Stallman  <address@hidden>

        * mail/sendmail.el (mail-bury): If Rmail is in use, return
          nicely
        to Rmail even if not started by a special Rmail command.

        * mail/rmailmm.el (rmail-insert-mime-forwarded-message):
        Copy the buffer currently showing just one message.

I don't know what it was going to fix.

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Richard Stallman]

    Actually that copying is done by
    rmail-insert-mime-resent-message
    (rmail-insert-mime-resent-message-function is set to it when
    rmailmm.el is loaded), and that function was changed by RMS
    as this.

My change was to forward the message as displayed, rather than the raw
text of the message.  I think my change eliminated the old bug.

Anyway, there is still a bug.  Some different bug.  Can someone tell
me what is wrong with the output that is generated now?  If I know
what output would be correct, I can fix this bug.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Eli Zaretskii]

> Date: Mon, 17 Oct 2011 09:14:44 -0400
> From: Richard Stallman <rms@gnu.org>
> CC: 9766@debbugs.gnu.org
> 
>     Actually that copying is done by
>     rmail-insert-mime-resent-message
>     (rmail-insert-mime-resent-message-function is set to it when
>     rmailmm.el is loaded), and that function was changed by RMS
>     as this.
> 
> My change was to forward the message as displayed, rather than the raw
> text of the message.

Was it for the MIME version of forwarding or for the inline version?

The problem we have now is only with the MIME version.

> Anyway, there is still a bug.  Some different bug.  Can someone tell
> me what is wrong with the output that is generated now?  If I know
> what output would be correct, I can fix this bug.

I think what needs to be done is to copy into the buffer where mail is
composed the message in the original form, not in the form we put into
the rmail-view-buffer.

For the definitive answer, I suggest to start Emacs with the default
value of mail-user-agent, start Rmail, forward (to yourself) a message
with attachments, and then look at hoe message.el did it.

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Richard Stallman]

    > My change was to forward the message as displayed, rather than the raw
    > text of the message.

    Was it for the MIME version of forwarding or for the inline version?

For the MIME version.

    I think what needs to be done is to copy into the buffer where mail is
    composed the message in the original form, not in the form we put into
    the rmail-view-buffer.

No.  That had the bug (the one reported before), and it also is an
ugly pain in the neck.  Anyway, if you want to forward the raw form,
all you have to do is type v.  That works, too.

    For the definitive answer, I suggest to start Emacs with the default
    value of mail-user-agent, start Rmail, forward (to yourself) a message
    with attachments, and then look at hoe message.el did it.

But that is not conclusive.  What we need is a person who knows MIME
well enough to look at the generated text, and tell us if it has an error.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Stefan Monnier]

> For the MIME version.
>     I think what needs to be done is to copy into the buffer where mail is
>     composed the message in the original form, not in the form we put into
>     the rmail-view-buffer.
> No.  That had the bug (the one reported before),

I'm not sure which bug this refers to.  But a MIME forward should
contain an "original form" email message.  You can strip some of the
headers if you want, but it should contain non-decoded content.

> and it also is an ugly pain in the neck.

Do you say that because it makes the underlying encoded message visible?
If so, it's a presentation problem: the content of the sent message
should be "ugly" but there's no reason it should ever be displayed in
this ugly form.

> Anyway, if you want to forward the raw form, all you have to do is
> type v.  That works, too.

Not using Rmail myself, I don't know what `v' runs.

>     For the definitive answer, I suggest to start Emacs with the default
>     value of mail-user-agent, start Rmail, forward (to yourself) a message
>     with attachments, and then look at hoe message.el did it.
> But that is not conclusive.  What we need is a person who knows MIME
> well enough to look at the generated text, and tell us if it has an error.

I haven't touched MIME in a while, but back when I was hacking on the
PGP support for ExMH, I was reasonably up-to-date in MIME.  To me the
message you showed looked OK except for the "--text follows this line--"
which should be an empty line, but I think this is just the "in buffer"
text and that line is turned into an empty line when you send the
message.  Other than that it also seems to lack another part: it has
a first part of type message/rfc822 and a second one of type text/plain,
but you'll usually want to add a first part of type text/plain where you
can put some text of your own making.


        Stefan


PS: Reminds me that I think this "--text follows this line--" should be
changed: instead of actual real text in the buffer, it should be an
overlay's after-string covering the empty line.

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Eli Zaretskii]

> Date: Mon, 17 Oct 2011 17:42:27 -0400
> From: Richard Stallman <rms@gnu.org>
> CC: 9766@debbugs.gnu.org
> 
>     For the definitive answer, I suggest to start Emacs with the default
>     value of mail-user-agent, start Rmail, forward (to yourself) a message
>     with attachments, and then look at hoe message.el did it.
> 
> But that is not conclusive.

Why not?  Last time I tried it, the result was correctly displayed by
Rmail.

> What we need is a person who knows MIME
> well enough to look at the generated text, and tell us if it has an error.

We have Lars and we have Rainer.  I wouldn't worry about that.

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Eli Zaretskii]

> Date: Mon, 17 Oct 2011 17:42:27 -0400
> From: Richard Stallman <rms@gnu.org>
> CC: 9766@debbugs.gnu.org
> 
> Anyway, if you want to forward the raw form, all you have to do is
> type v.

I thought so, but last time I tried, it didn't work well (the result
was unreadable in another mailer).  I will try again.

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  9766@debbugs.gnu.org
> Date: Mon, 17 Oct 2011 21:00:55 -0400
> 
> > Anyway, if you want to forward the raw form, all you have to do is
> > type v.  That works, too.
> 
> Not using Rmail myself, I don't know what `v' runs.

`v' shows the raw message as it came in the mbox file.

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> We have Lars and we have Rainer.  I wouldn't worry about that.

Actually, I'm not 100% sure that it was valid MIME.

The structure was basically

---------------------
Headers: Bla bla

--delimiter line
..
--end--
---------------------

Reading RFC2046, the CRLF preceding the delimiter line is part of the
delimiter line.  But at the same time, the blank line after the headers
is the thing that delimits the headers from the body, which means that
the same blank line has two different overlapping meanings, which may
make some very strict parsers misbehave.

Adding an additional blank line before the delimiter line should fix
this and make this unambiguously valid MIME.

Unless I'm missing something in RFC2046.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Richard Stallman]

    >     For the definitive answer, I suggest to start Emacs with the default
    >     value of mail-user-agent, start Rmail, forward (to yourself) a message
    >     with attachments, and then look at hoe message.el did it.
    > 
    > But that is not conclusive.

    Why not?  Last time I tried it, the result was correctly displayed by
    Rmail.

If the message displays badly in mail reader A, debugging that might
lead to some problem in how the message was written.

But if the message displays with no problem in mail reader A, that
doesn't tell us much about what it is about the message that causes
mail reader B to display it badly.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Stefan Monnier]

>> > Anyway, if you want to forward the raw form, all you have to do is
>> > type v.  That works, too.
>> Not using Rmail myself, I don't know what `v' runs.
> `v' shows the raw message as it came in the mbox file.

So it would be equivalent to "resend" rather than to "forward", right?


        Stefan

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rms@gnu.org,  9766@debbugs.gnu.org
> Date: Tue, 18 Oct 2011 09:01:21 -0400
> 
> >> > Anyway, if you want to forward the raw form, all you have to do is
> >> > type v.  That works, too.
> >> Not using Rmail myself, I don't know what `v' runs.
> > `v' shows the raw message as it came in the mbox file.
> 
> So it would be equivalent to "resend" rather than to "forward", right?

No, it's still "forward".  The raw message gets appended to whatever
you type in the mail composition buffer, before sending the result.

The difference is that what gets appended is the original mbox format,
complete with MIME headers and bodies of the attachments to that
original.

By contrast, forwarding without the preceding `v' appends the
formatted message, which includes buttons we create out of the MIME
attachments.  However, these buttons are copied as plain text, which
of course is useless to the addressee of the forwarded mail.

I think the result after `v' was also incorrect, perhaps because the
way sendmail.el calls mml-to-mime is not entirely correct when the
body already includes attachments.  I will have a second look when I
have time (unless Richard beats me to it).

bug#9521: This bug is still present in version 24.2; it prevents forwarding correctly essentially any MINE message

[Mark Lillibridge]

    In particular, it removes the MIME headers from the forwarded
message, making it undecodable.  This bug was not present in version
23.3.  It also forwards incorrectly non-MIME messages, leaving out
filtered headers, which may be wanted for diagnosing message bounces and
the like.

- Mark

bug#9521: PATCH for bug #9521, *not* bug #9766

[Mark Lillibridge]

[-- Attachment #1: Type: text/plain, Size: 3163 bytes --]


    This bug (#9521) was easy to fix.  The problem was with the
rmail-insert-mime-forwarded-message function in rmailmm.el:1355:

  (defun rmail-insert-mime-forwarded-message (forward-buffer)
    "Insert the message in FORWARD-BUFFER as a forwarded message.
  This is the usual value of `rmail-insert-mime-forwarded-message-function'."
    (let ((message-buffer
  	 (with-current-buffer forward-buffer
  	   (if rmail-buffer-swapped
  	       forward-buffer
  	     rmail-view-buffer))))
      (save-restriction
        (narrow-to-region (point) (point))
        (message-forward-make-body-mime message-buffer))))


    This does exactly the wrong thing by inserting the decoded version
of the message.  Swapping the two buffers (forward-buffer,
rmail-view-buffer) in the if expression fixes this:

  (defun rmail-insert-mime-forwarded-message (forward-buffer)
    "Insert the message in FORWARD-BUFFER as a forwarded message.
  This is the usual value of `rmail-insert-mime-forwarded-message-function'."
    (let ((message-buffer
  	 (with-current-buffer forward-buffer
  	   (if rmail-buffer-swapped
> 	       rmail-view-buffer
> 	     forward-buffer))))
      (save-restriction
        (narrow-to-region (point) (point))
        (message-forward-make-body-mime message-buffer))))


    Note that this does not fix bug #9766, which was incorrectly merged
with bug #9521.  The problem there (#9766) is that many email clients
including in particular, the iPad email app, do not properly display
RFC822 attachments or do not show it inline.  Fixing that problem
requires substantial work, including on the design front.

    One idea would be to generate the RFC822 attachment as now, which
preserves the full details of the message for competent email clients,
and also generate an abbreviated version in the message body for human
viewers of incompetent email clients.  The simplest approach would be to
just insert the Rmail decoded version:

    ====== forwarded message as seen by sender (full message attached) ====
    From: Kenichi Handa <handa <at> m17n.org>
    To: handa <at> m17n.org
    Subject: test from shatin
    Date: Thu, 15 Sep 2011 14:14:58 +0900
    Message-ID: <87aaa6xu7h.fsf <at> m17n.org>
    Content-Type: multipart/mixed; boundary="=-=-="

    [1:text/plain Hide]
    
    test of attachment
    
    
    [2:application/pdf Show Save:temp.pdf (2kB)]

Here, the message headers have been filtered by the users usual header
filtering rules and the body is as seen when the forwarding was done.
e.g., whatever message part toggling the user did is still visible.

    Drawbacks: the PDF attachment here is not accessible to the
incompetent email clients, this approach fails miserably for HTML-only
messages (distressingly common these days), and there is no way to
forward the HTML part instead of the text part inline (perhaps the HTML
part has the real content).  The first of these could be fixed by
attaching all of the original non-inline attachments to the new message;
this is what email clients like Outlook do when you forword a message.

    For the second and third parts, I have been experimenting with the
following:

    

[-- Attachment #2: Type: text/html, Size: 947 bytes --]

[-- Attachment #3: Type: text/plain, Size: 637 bytes --]


Note the need to use PRE or the equivalent to protect the headers.
Probably additional escaping of the headers are required and likely the
charset needs very careful handling.  Lots of corner cases here,
including what to do with multiple HTML parts.  


    At the moment, I have three versions of forward implemented, one
that sends the entire message as a RFC822 attachment, one that forwards
just the text part, and one that forwards just the HTML part.  The later
two work correctly when the entire message is one part.  I find each
useful in different circumstances so probably we don't want to support
only one of them.

- Mark

bug#9521: [RESEND] PATCH for bug #9521, *not* bug #9766

[Mark Lillibridge]

    This bug (#9521) was easy to fix.  The problem was with the
rmail-insert-mime-forwarded-message function in rmailmm.el:1355:

  (defun rmail-insert-mime-forwarded-message (forward-buffer)
    "Insert the message in FORWARD-BUFFER as a forwarded message.
  This is the usual value of `rmail-insert-mime-forwarded-message-function'."
    (let ((message-buffer
  	 (with-current-buffer forward-buffer
  	   (if rmail-buffer-swapped
  	       forward-buffer
  	     rmail-view-buffer))))
      (save-restriction
        (narrow-to-region (point) (point))
        (message-forward-make-body-mime message-buffer))))


    This does exactly the wrong thing by inserting the decoded version
of the message.  Swapping the two buffers (forward-buffer,
rmail-view-buffer) in the if expression fixes this:

  (defun rmail-insert-mime-forwarded-message (forward-buffer)
    "Insert the message in FORWARD-BUFFER as a forwarded message.
  This is the usual value of `rmail-insert-mime-forwarded-message-function'."
    (let ((message-buffer
  	 (with-current-buffer forward-buffer
  	   (if rmail-buffer-swapped
> 	       rmail-view-buffer
> 	     forward-buffer))))
      (save-restriction
        (narrow-to-region (point) (point))
        (message-forward-make-body-mime message-buffer))))


    Note that this does not fix bug #9766, which was incorrectly merged
with bug #9521.  The problem there (#9766) is that many email clients
including in particular, the iPad email app, do not properly display
RFC822 attachments or do not show it inline.  Fixing that problem
requires substantial work, including on the design front.

    One idea would be to generate the RFC822 attachment as now, which
preserves the full details of the message for competent email clients,
and also generate an abbreviated version in the message body for human
viewers of incompetent email clients.  The simplest approach would be to
just insert the Rmail decoded version:

    ====== forwarded message as seen by sender (full message attached) ====
    From: Kenichi Handa <handa <at> m17n.org>
    To: handa <at> m17n.org
    Subject: test from shatin
    Date: Thu, 15 Sep 2011 14:14:58 +0900
    Message-ID: <87aaa6xu7h.fsf <at> m17n.org>
    Content-Type: multipart/mixed; boundary="=-=-="

    [1:text/plain Hide]
    
    test of attachment
    
    
    [2:application/pdf Show Save:temp.pdf (2kB)]

Here, the message headers have been filtered by the users usual header
filtering rules and the body is as seen when the forwarding was done.
e.g., whatever message part toggling the user did is still visible.

    Drawbacks: the PDF attachment here is not accessible to the
incompetent email clients, this approach fails miserably for HTML-only
messages (distressingly common these days), and there is no way to
forward the HTML part instead of the text part inline (perhaps the HTML
part has the real content).  The first of these could be fixed by
attaching all of the original non-inline attachments to the new message;
this is what email clients like Outlook do when you forword a message.

    For the second and third parts, I have been experimenting with the
following (had to add " " after <'s to make < #parts escaped):

    < #part type=text/html disposition=inline raw=t>
    <PRE>
    ===== Forwarded message (HTML part only) follows =====
    Date: Fri, 28 Dec 2012 19:44:57 -0000
    From: "Hilton Hotels & Resorts" <hiltonhotels@h1.hilton.com>
    To: lillibridge@gmail.com
    Subject: Pick your paradise: choose from three unforgettable resort experiences
    Reply-To: "Hilton Hotels & Resorts" <support-b7j7kpqa20rdfhauv5u0bqcx2vkpsh@h1.hilton.com>
    </PRE>
    
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>Hilton Hotels &amp; Resorts</title><style type="text/css">
    body {
    	-webkit-text-size-adjust: none;
    }
    .ReadMsgBody {
    	width: 100%;
    }
    ...
    <img src="http://h1.hilton.com/a/hBQ3bP4AH$iOmB8v3UjNt2dv0P$/spacer.gif">
    </body></html>
    
    < #/part>

Note the need to use PRE or the equivalent to protect the headers.
Probably additional escaping of the headers are required and likely the
charset needs very careful handling.  Lots of corner cases here,
including what to do with multiple HTML parts.  


    At the moment, I have three versions of forward implemented, one
that sends the entire message as a RFC822 attachment, one that forwards
just the text part, and one that forwards just the HTML part.  The later
two work correctly when the entire message is one part.  I find each
useful in different circumstances so probably we don't want to support
only one of them.

- Mark

bug#9766: workaround for 9766

[Mark Lillibridge]

    It appears that resend (C-u f) does not have any of the problems
that forward does -- correctly resends the message in essentially its
current form so incompetent mail clients display it correctly -- so I
recommend using it plus a separate message telling the recipient that
you are resending the message and why for now.

- Mark

bug#9521: [RESEND] PATCH for bug #9521, *not* bug #9766

[Eli Zaretskii]

> From: Mark Lillibridge <mdl@alum.mit.edu>
> Date: Fri, 28 Dec 2012 12:55:25 -0800
> 
> 
>     This bug (#9521) was easy to fix.  The problem was with the
> rmail-insert-mime-forwarded-message function in rmailmm.el:1355:
> 
>   (defun rmail-insert-mime-forwarded-message (forward-buffer)
>     "Insert the message in FORWARD-BUFFER as a forwarded message.
>   This is the usual value of `rmail-insert-mime-forwarded-message-function'."
>     (let ((message-buffer
>   	 (with-current-buffer forward-buffer
>   	   (if rmail-buffer-swapped
>   	       forward-buffer
>   	     rmail-view-buffer))))
>       (save-restriction
>         (narrow-to-region (point) (point))
>         (message-forward-make-body-mime message-buffer))))
> 
> 
>     This does exactly the wrong thing by inserting the decoded version
> of the message.  Swapping the two buffers (forward-buffer,
> rmail-view-buffer) in the if expression fixes this:
> 
>   (defun rmail-insert-mime-forwarded-message (forward-buffer)
>     "Insert the message in FORWARD-BUFFER as a forwarded message.
>   This is the usual value of `rmail-insert-mime-forwarded-message-function'."
>     (let ((message-buffer
>   	 (with-current-buffer forward-buffer
>   	   (if rmail-buffer-swapped
> > 	       rmail-view-buffer
> > 	     forward-buffer))))
>       (save-restriction
>         (narrow-to-region (point) (point))
>         (message-forward-make-body-mime message-buffer))))

Thanks, I installed this simple change on the emacs-24 branch.

>     Note that this does not fix bug #9766, which was incorrectly merged
> with bug #9521.  The problem there (#9766) is that many email clients
> including in particular, the iPad email app, do not properly display
> RFC822 attachments or do not show it inline.  Fixing that problem
> requires substantial work, including on the design front.

I think #9766 is about both problems.  But I unmerged it anyway, and
am leaving it open for now.

bug#9521: this bug (#10080) also causes an extra blank line to be appended to forwarded messages

[Mark Lillibridge]

[-- Attachment #1: Type: text/plain, Size: 223 bytes --]


For example, when I forward the example message using f I get:

|To: 
|Subject: [mdl@alum.mit.edu: no blank lines follow]
|From: Mark Lillibridge <lillibridgem@build-debian-1.u.hpl.hp.com>
|--text follows this line--
|
|
|

[-- Attachment #2: Type: message/rfc822, Size: 178 bytes --]


|this is the only line
|
|

[-- Attachment #3: Type: text/plain, Size: 280 bytes --]


Notice the extra blank line at the end of the forwarded message.

    Fixing #9521 properly (switching to forwarding raw instead of
decoded messages) should fix this.  At the moment (post my 9521 patch),
it also appends a blank line to the end of the forwarded message.
 
- Mark

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Lars Ingebrigtsen]

Skimming this bug report, it seems like Eli and Richard fixed some
things, but it's not clear whether there was anything more to fix here.

Are you still seeing problems here in recent Emacs versions, Richard?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#9766: 24.0.50; rmail-forward makes attachment that doesn't read right

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Skimming this bug report, it seems like Eli and Richard fixed some
> things, but it's not clear whether there was anything more to fix here.
>
> Are you still seeing problems here in recent Emacs versions, Richard?

More information was requested, but no response was given within a
month, so I'm closing this bug report.  If the problem still exists,
please respond to this email and we'll reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no