bug#4623: 23.1.50; rmail changes encoding of characters on save

bug#4623: 23.1.50; rmail changes encoding of characters on save

[Markus Rost]

With rmail I observed random encoding changes in mbox files:  some
non-ascii characters are changed depending on which message you look
at while saving.

Providing a test case is a bit tiresome, so I omit one for now.

The reason for the problem is that rmail-expunge-and-save uses the
coding system of the current message when saving the file.  This is
certainly wrong:  the coding system for writing the mbox file must not
depend on the currently selected message.

Since rmail reads the mbox file with coding-system 'no-conversion, it
should save it with coding-system 'no-conversion.

Perhaps the problem sits in rmail-swap-buffers:  Shouldn't
rmail-swap-buffers exchange the value of buffer-file-coding-system in
both ways?  Or perhaps it should ensure that the buffer which contains
the full mbox file has always the same buffer-file-coding-system,
namely 'no-conversion.

Anyway, the following ad hoc patch fixes the problem for me.

*** rmail.el.~1.552.~	2009-09-26 21:21:07.000000000 +0200
--- rmail.el	2009-10-02 16:14:30.000000000 +0200
***************
*** 1458,1464 ****
    (rmail-expunge)
    ;; No need to swap buffers: rmail-write-region-annotate takes care of it.
    ;; (rmail-swap-buffers-maybe)
!   (save-buffer)
    (if (rmail-summary-exists)
        (rmail-select-summary (set-buffer-modified-p nil))))
  
--- 1458,1465 ----
    (rmail-expunge)
    ;; No need to swap buffers: rmail-write-region-annotate takes care of it.
    ;; (rmail-swap-buffers-maybe)
!   (let ((coding-system-for-write 'no-conversion))
!     (save-buffer))
    (if (rmail-summary-exists)
        (rmail-select-summary (set-buffer-modified-p nil))))


In GNU Emacs 23.1.50.1 (i686-pc-linux-gnu, GTK+ Version 2.16.1)
 of 2009-10-02 on laptop
Windowing system distributor `The X.Org Foundation', version 11.0.10600000
configured using `configure  '--prefix=/home/rost/local/cvs''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: C
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

bug#4623: 23.1.50; rmail changes encoding of characters on save

[Glenn Morris]

Thanks for the test case. I know very little about coding issues, but
if setting coding-system-for-write is the correct fix, I think it
should be done in a manner similar to the following, since saving an
rmail file with just "C-x C-s" should work.

*** rmail.el   6 Oct 2009 06:40:40 -0000  1.554
--- rmail.el   13 Oct 2009 01:17:20 -0000
***************
*** 4176,4181 ****
--- 4176,4182 ----
  (defun rmail-write-region-annotate (start end)
    (when (and (null start) (rmail-buffers-swapped-p))
      (set-buffer rmail-view-buffer)
+     (set (make-local-variable 'coding-system-for-write) 'no-conversion)
      (widen)
      nil))
  

bug#4623: 23.1.50; rmail changes encoding of characters on save

[Markus Rost]

The crucial thing is that the mbox file after reading and saving in
rmail differs from the original file only by some headers, usually
X-RMAIL-ATTRIBUTES, X-RMAIL-KEYWORDS.

Your patch fixes the problem.

> I think it
> should be done in a manner similar to the following, since saving an
> rmail file with just "C-x C-s" should work.

Yes, your patch is better than my attempts, as in

<URL:http://lists.gnu.org/archive/html/bug-gnu-emacs/2009-10/msg00052.html>

and at the end of

<URL:http://lists.gnu.org/archive/html/bug-gnu-emacs/2009-10/msg00182.html>

However all 3 patches leave the same problem, namely that the rmail
buffer gets the buffer-file-coding-system 'no-conversion.  This can be
easily seen from the mode line indicator ('=' instead of say '1' or
'u').  See the detailed discussion in the second part of

<URL:http://lists.gnu.org/archive/html/bug-gnu-emacs/2009-10/msg00182.html>

Now this problem is of minor importance, at least I don't know of any
serious effects, and refreshing the rmail buffer by hitting "Nj"
(where N is the message number you are looking at) resets the
buffer-file-coding-system (and the mode line indicator).

One could work around this by calling

(rmail-show-message rmail-current-message)

after the save, although this isn't satisfactory.

Anyway, I think that your patch should be installed.

bug#4623: marked as done (23.1.50; rmail changes encoding of characters on save)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 906 bytes --]

Your message dated Fri, 16 Oct 2009 23:12:23 -0400
with message-id <22r5t2emso.fsf@fencepost.gnu.org>
and subject line Re: bug#4623: 23.1.50; rmail changes encoding of characters on save
has caused the Emacs bug report #4623,
regarding 23.1.50; rmail changes encoding of characters on save
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@emacsbugs.donarmstrong.com
immediately.)


-- 
4623: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=4623
Emacs Bug Tracking System
Contact owner@emacsbugs.donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 4255 bytes --]

From: Markus Rost <rost@math.uni-bielefeld.de>
To: emacs-pretest-bug@gnu.org
Subject: 23.1.50; rmail changes encoding of characters on save
Date: Sat,  3 Oct 2009 16:54:35 +0200 (CEST)
Message-ID: <20091003145435.75D5D288A0AB@xen12.math.uni-bielefeld.de>


With rmail I observed random encoding changes in mbox files:  some
non-ascii characters are changed depending on which message you look
at while saving.

Providing a test case is a bit tiresome, so I omit one for now.

The reason for the problem is that rmail-expunge-and-save uses the
coding system of the current message when saving the file.  This is
certainly wrong:  the coding system for writing the mbox file must not
depend on the currently selected message.

Since rmail reads the mbox file with coding-system 'no-conversion, it
should save it with coding-system 'no-conversion.

Perhaps the problem sits in rmail-swap-buffers:  Shouldn't
rmail-swap-buffers exchange the value of buffer-file-coding-system in
both ways?  Or perhaps it should ensure that the buffer which contains
the full mbox file has always the same buffer-file-coding-system,
namely 'no-conversion.

Anyway, the following ad hoc patch fixes the problem for me.

*** rmail.el.~1.552.~	2009-09-26 21:21:07.000000000 +0200
--- rmail.el	2009-10-02 16:14:30.000000000 +0200
***************
*** 1458,1464 ****
    (rmail-expunge)
    ;; No need to swap buffers: rmail-write-region-annotate takes care of it.
    ;; (rmail-swap-buffers-maybe)
!   (save-buffer)
    (if (rmail-summary-exists)
        (rmail-select-summary (set-buffer-modified-p nil))))
  
--- 1458,1465 ----
    (rmail-expunge)
    ;; No need to swap buffers: rmail-write-region-annotate takes care of it.
    ;; (rmail-swap-buffers-maybe)
!   (let ((coding-system-for-write 'no-conversion))
!     (save-buffer))
    (if (rmail-summary-exists)
        (rmail-select-summary (set-buffer-modified-p nil))))


In GNU Emacs 23.1.50.1 (i686-pc-linux-gnu, GTK+ Version 2.16.1)
 of 2009-10-02 on laptop
Windowing system distributor `The X.Org Foundation', version 11.0.10600000
configured using `configure  '--prefix=/home/rost/local/cvs''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: C
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t


[-- Attachment #3: Type: message/rfc822, Size: 1697 bytes --]

From: Glenn Morris <rgm@gnu.org>
To: 4623-done@emacsbugs.donarmstrong.com
Subject: Re: bug#4623: 23.1.50; rmail changes encoding of characters on save
Date: Fri, 16 Oct 2009 23:12:23 -0400
Message-ID: <22r5t2emso.fsf@fencepost.gnu.org>

Markus Rost wrote:

> Anyway, I think that your patch should be installed.

In the absence of any other comments on this, I did so.