bug#935: 23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals

bug#935: 23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 1027 bytes --]

>>>>> Sven Joachim wrote:

> I have a problem with the gnus-mime-view-part-as-charset command that
> only shows up in Emacs 23, not sure whether this is a bug in Emacs or in
> Gnus.

> If an article has attachment with a wrongly specified charset, Gnus can
> work around that with the gnus-mime-view-part-as-charset command.  For
> instance, <news:20080908175652.GF4996@firemoth> on
> gmane.linux.debian.internationalization.german aka
> debian-l10n-german@lists.debian.org is such an article, it has an
> attachment that is incorrectly described as "Content-Type: text/x-diff;
> charset=utf-8" while it's actually encoded as iso-latin-1.  Now when I
> press C RET iso-latin-1, Emacs displays the German umlauts as octal
> sequences.  In Emacs 22.3, they are displayed correctly (Gnus version is
> the same, current No Gnus CVS).

That's exactly a bug.  Even in Emacs 22.3, it is not displayed
correctly if attachment contains CJK characters with wrong charset.
The patch is below.  I'll commit this after testing it further.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-diff, Size: 1404 bytes --]

--- gnus-art.el~	2008-08-11 22:24:20 +0000
+++ gnus-art.el	2008-09-09 00:56:42 +0000
@@ -5138,8 +5138,9 @@
 	   (mm-string-to-multibyte contents)))
 	(goto-char b)))))
 
-(defun gnus-mime-strip-charset-parameters (handle)
-  "Strip charset parameters from HANDLE."
+(defun gnus-mime-set-charset-parameters (handle)
+  "Set charset to parameters in HANDLE.
+The value of `gnus-newsgroup-charset' is used as a charset."
   (if (stringp (car handle))
       (mapc #'gnus-mime-strip-charset-parameters (cdr handle))
     (let* ((type (mm-handle-type (if (equal (mm-handle-media-type handle)
@@ -5150,8 +5151,10 @@
 				       (mm-handle-cache handle))
 				   handle)))
 	   (charset (assq 'charset (cdr type))))
-      (when charset
-	(delq charset type)))))
+      (if charset
+	  (setcdr charset gnus-newsgroup-charset)
+	(setcdr type (cons (cons 'charset gnus-newsgroup-charset)
+			   (cdr type)))))))
 
 (defun gnus-mime-view-part-as-charset (&optional handle arg)
   "Insert the MIME part under point into the current buffer using the
@@ -5172,7 +5175,7 @@
 			       (mm-read-coding-system "Charset: "))))
 	      (if (mm-handle-undisplayer handle)
 		  (mm-remove-part handle)))
-	(gnus-mime-strip-charset-parameters handle)
+	(gnus-mime-set-charset-parameters handle)
 	(when (and (consp (setq form (cdr-safe fun)))
 		   (setq form (ignore-errors
 				(assq 'gnus-mime-display-alternative form)))

bug#935: 23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals

[Daiki Ueno]

>>>>> In <b4mr67unnfk.fsf__39297.7221440107$1220923731$gmane$org@jpl.org> 
>>>>>	Katsumi Yamaoka <yamaoka@jpl.org> wrote:
> -(defun gnus-mime-strip-charset-parameters (handle)
> -  "Strip charset parameters from HANDLE."
> +(defun gnus-mime-set-charset-parameters (handle)
> +  "Set charset to parameters in HANDLE.
> +The value of `gnus-newsgroup-charset' is used as a charset."
>    (if (stringp (car handle))
>        (mapc #'gnus-mime-strip-charset-parameters (cdr handle))

Should this line also be changed to call
gnus-mime-set-charset-parameters?

I would also like to suggest that gnus-mime-set-charset-parameters
accept CHARSET as its 2nd arg to avoid dependency to the global variable
gnus-newsgroup-charset.

Regards,
-- 
Daiki Ueno

bug#935: 23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals

[Katsumi Yamaoka]

>>>>> Daiki Ueno wrote:
>> +(defun gnus-mime-set-charset-parameters (handle)
>> +  "Set charset to parameters in HANDLE.
>> +The value of `gnus-newsgroup-charset' is used as a charset."
>>    (if (stringp (car handle))
>>        (mapc #'gnus-mime-strip-charset-parameters (cdr handle))

> Should this line also be changed to call
> gnus-mime-set-charset-parameters?

Oops.  Thank you for pointing it out.

> I would also like to suggest that gnus-mime-set-charset-parameters
> accept CHARSET as its 2nd arg to avoid dependency to the global variable
> gnus-newsgroup-charset.

That's a good idea.  I also verified that to bind
`gnus-newsgroup-charset' is not necessary in the function
`gnus-mime-view-part-as-charset'.  Installed in the Emacs trunk.

Regards,

bug#935: 23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 1027 bytes --]

>>>>> Sven Joachim wrote:

> I have a problem with the gnus-mime-view-part-as-charset command that
> only shows up in Emacs 23, not sure whether this is a bug in Emacs or in
> Gnus.

> If an article has attachment with a wrongly specified charset, Gnus can
> work around that with the gnus-mime-view-part-as-charset command.  For
> instance, <news:20080908175652.GF4996@firemoth> on
> gmane.linux.debian.internationalization.german aka
> debian-l10n-german@lists.debian.org is such an article, it has an
> attachment that is incorrectly described as "Content-Type: text/x-diff;
> charset=utf-8" while it's actually encoded as iso-latin-1.  Now when I
> press C RET iso-latin-1, Emacs displays the German umlauts as octal
> sequences.  In Emacs 22.3, they are displayed correctly (Gnus version is
> the same, current No Gnus CVS).

That's exactly a bug.  Even in Emacs 22.3, it is not displayed
correctly if attachment contains CJK characters with wrong charset.
The patch is below.  I'll commit this after testing it further.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-diff, Size: 1404 bytes --]

--- gnus-art.el~	2008-08-11 22:24:20 +0000
+++ gnus-art.el	2008-09-09 00:56:42 +0000
@@ -5138,8 +5138,9 @@
 	   (mm-string-to-multibyte contents)))
 	(goto-char b)))))
 
-(defun gnus-mime-strip-charset-parameters (handle)
-  "Strip charset parameters from HANDLE."
+(defun gnus-mime-set-charset-parameters (handle)
+  "Set charset to parameters in HANDLE.
+The value of `gnus-newsgroup-charset' is used as a charset."
   (if (stringp (car handle))
       (mapc #'gnus-mime-strip-charset-parameters (cdr handle))
     (let* ((type (mm-handle-type (if (equal (mm-handle-media-type handle)
@@ -5150,8 +5151,10 @@
 				       (mm-handle-cache handle))
 				   handle)))
 	   (charset (assq 'charset (cdr type))))
-      (when charset
-	(delq charset type)))))
+      (if charset
+	  (setcdr charset gnus-newsgroup-charset)
+	(setcdr type (cons (cons 'charset gnus-newsgroup-charset)
+			   (cdr type)))))))
 
 (defun gnus-mime-view-part-as-charset (&optional handle arg)
   "Insert the MIME part under point into the current buffer using the
@@ -5172,7 +5175,7 @@
 			       (mm-read-coding-system "Charset: "))))
 	      (if (mm-handle-undisplayer handle)
 		  (mm-remove-part handle)))
-	(gnus-mime-strip-charset-parameters handle)
+	(gnus-mime-set-charset-parameters handle)
 	(when (and (consp (setq form (cdr-safe fun)))
 		   (setq form (ignore-errors
 				(assq 'gnus-mime-display-alternative form)))

bug#935: 23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals

[Katsumi Yamaoka]

>>>>> Katsumi Yamaoka wrote:
> The patch is below.  I'll commit this after testing it further.

> --- gnus-art.el~	2008-08-11 22:24:20 +0000
> +++ gnus-art.el	2008-09-09 00:56:42 +0000

I've committed it in the Emacs trunk, with a slight modification.

bug#935: 23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals

[Katsumi Yamaoka]

>>>>> Katsumi Yamaoka wrote:
> The patch is below.  I'll commit this after testing it further.

> --- gnus-art.el~	2008-08-11 22:24:20 +0000
> +++ gnus-art.el	2008-09-09 00:56:42 +0000

I've committed it in the Emacs trunk, with a slight modification.

bug#935: marked as done (23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 921 bytes --]


Your message dated Sun, 14 Sep 2008 18:48:06 +0200
with message-id <87k5den06x.fsf@gmx.de>
and subject line Re: bug#935: 23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals
has caused the Emacs bug report #935,
regarding 23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact don@donarmstrong.com
immediately.)


-- 
935: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=935
Emacs Bug Tracking System
Contact don@donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 5483 bytes --]

From: Sven Joachim <svenjoac@gmx.de>
To: emacs-pretest-bug@gnu.org
Subject: 23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals
Date: Mon, 08 Sep 2008 22:54:18 +0200
Message-ID: <877i9mmk91.fsf@gmx.de>


I have a problem with the gnus-mime-view-part-as-charset command that
only shows up in Emacs 23, not sure whether this is a bug in Emacs or in
Gnus.

If an article has attachment with a wrongly specified charset, Gnus can
work around that with the gnus-mime-view-part-as-charset command.  For
instance, <news:20080908175652.GF4996@firemoth> on
gmane.linux.debian.internationalization.german aka
debian-l10n-german@lists.debian.org is such an article, it has an
attachment that is incorrectly described as "Content-Type: text/x-diff;
charset=utf-8" while it's actually encoded as iso-latin-1.  Now when I
press C RET iso-latin-1, Emacs displays the German umlauts as octal
sequences.  In Emacs 22.3, they are displayed correctly (Gnus version is
the same, current No Gnus CVS).


In GNU Emacs 23.0.60.4 (i686-pc-linux-gnu, GTK+ Version 2.12.11)
 of 2008-09-08 on debian
Windowing system distributor `The X.Org Foundation', version 11.0.10402000
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: C
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: de_DE.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: Summary

Minor modes in effect:
  diff-auto-refine-mode: t
  gnus-mailing-list-mode: t
  display-time-mode: t
  auto-image-file-mode: t
  show-paren-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-x r b . e m <tab> <return> <C-home> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
C-e C-x C-e M-x g n u s <return> <down> L <next> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <up> 
<up> <up> <up> <up> <up> <up> <return> 2 <return> <down> 
SPC SPC C-x o <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> C i s o <tab> l 
<tab> 1 <return> C-h k C C-x o <tab> <return> <next> 
<prior> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <up> <up> <up> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <left> 
<left> <left> C-SPC M-f M-f M-f M-f M-f M-f M-w <next> 
q C-x 1 <f7> <down> <down> <down> <return> SPC <backspace> 
M-x r e p o r t - e m <tab> <return>

Recent messages:
Fetching headers for gmane.linux.debian.internationalization.german...done
Scoring...done
Generating summary...done
No more unread articles
Type C-x 4 o RET to restore the other window, C-M-v to scroll help.
mouse-2, RET: find function's definition
Note: file is write protected
Mark set
Buffer is read-only: #<buffer gnus-art.el>
Commands: d, s, x, u; f, o, 1, 2, m, v; ~, %; q to quit; ? for help.



[-- Attachment #3: Type: message/rfc822, Size: 1944 bytes --]

From: Sven Joachim <svenjoac@gmx.de>
To: Katsumi Yamaoka <yamaoka@jpl.org>
Cc: Daiki Ueno <ueno@unixuser.org>, 935-done@emacsbugs.donarmstrong.com, ding@gnus.org
Subject: Re: bug#935: 23.0.60; gnus-mime-view-part-as-charset displays non-ASCII characters as octals
Date: Sun, 14 Sep 2008 18:48:06 +0200
Message-ID: <87k5den06x.fsf@gmx.de>

Sorry for not following up earlier, I can confirm that the problem is
fixed, hence I'm closing the bug now.

Thank you,
          Sven