RMAIL bails out with coding-system-error

RMAIL bails out with coding-system-error

[David Kuehling]

I recently got a mail in my inbox, that contains the following
header-line:

X-Coding-System: japanese-shift-jisx0213-unix

Now whenever I want to read that mail using RMAIL, the following error
is signaled:

Debugger entered--Lisp error: 
  (coding-system-error japanese-shift-jisx0213-unix)
  check-coding-system(japanese-shift-jisx0213-unix)
  rmail-show-message(105)
  rmail-summary-goto-msg(nil)

I cannot even delete that message, since that will as well signal that
error.

Looks like someone forgot a `condition-case' ??

I'm using Emacs/Mule 21.1 with MULE-UCS extension installed, everything
as it came with my Debian/Woody.

David Kühling
-- 
GnuPG public key: http://user.cs.tu-berlin.de/~dvdkhlng/dk.gpg
Fingerprint: B17A DC95 D293 657B 4205  D016 7DEF 5323 C174 7D40

Re: RMAIL bails out with coding-system-error

[Richard Stallman]

Does this make it work right?

*** rmail.el.~1.357.~	Fri Mar 22 22:21:53 2002
--- rmail.el	Sat May  4 20:40:00 2002
***************
*** 2391,2398 ****
  	      (goto-char (point-min))
  	      (if (re-search-forward "^X-Coding-System: *\\(.*\\)$" nil t)
  		  (let ((coding-system (intern (match-string 1))))
! 		    (check-coding-system coding-system)
! 		    (setq buffer-file-coding-system coding-system))
  		(setq buffer-file-coding-system nil)))))
  	;; Clear the "unseen" attribute when we show a message.
  	(rmail-set-attribute "unseen" nil)
--- 2391,2402 ----
  	      (goto-char (point-min))
  	      (if (re-search-forward "^X-Coding-System: *\\(.*\\)$" nil t)
  		  (let ((coding-system (intern (match-string 1))))
! 		    (condition-case nil
! 			(progn
! 			  (check-coding-system coding-system)
! 			  (setq buffer-file-coding-system coding-system))
! 		      (error 
! 		       (setq buffer-file-coding-system nil))))
  		(setq buffer-file-coding-system nil)))))
  	;; Clear the "unseen" attribute when we show a message.
  	(rmail-set-attribute "unseen" nil)

Re: RMAIL bails out with coding-system-error

[David Kuehling]

>>>>> "Richard" == Richard Stallman <rms@gnu.org> writes:

> Does this make it work right?  
> [...]

Yes.  

BTW, the origin of the reported occurence of that bug seems to be the
use of Mule-UCS.  In rmail.el, line 1660, the following code is used to
add an X-Coding-System header to mails, using the last coding system
used:

	     (insert "X-Coding-System: "
		     (symbol-name last-coding-system-used)
		     "\n")))

When I received the mail, the last coding system I used was one of the
coding systems that came with Mule-UCS.  After temporaryly disabling
Mule-UCS, that coding system wasn't known any more...

Using the last coding system is after all a little confusing.  Maybe
`default-buffer-file-coding-system' would be a better (and more
predictable) choice?

David Kühling
-- 
GnuPG public key: http://user.cs.tu-berlin.de/~dvdkhlng/dk.gpg
Fingerprint: B17A DC95 D293 657B 4205  D016 7DEF 5323 C174 7D40

Re: RMAIL bails out with coding-system-error

[Simon Josefsson]

David Kuehling <dvdkhlng@gmx.de> writes:

> BTW, the origin of the reported occurence of that bug seems to be the
> use of Mule-UCS.  In rmail.el, line 1660, the following code is used to
> add an X-Coding-System header to mails, using the last coding system
> used:
>
> 	     (insert "X-Coding-System: "
> 		     (symbol-name last-coding-system-used)
> 		     "\n")))
>
> When I received the mail, the last coding system I used was one of the
> coding systems that came with Mule-UCS.  After temporaryly disabling
> Mule-UCS, that coding system wasn't known any more...
>
> Using the last coding system is after all a little confusing.  Maybe
> `default-buffer-file-coding-system' would be a better (and more
> predictable) choice?

Is this X-Coding-System header actually used by RMAIL for anything?

I think the header is a very bad idea and it should be removed.  There
is a standard for interchanging non-ASCII data using email, and it is
called MIME.  Inventing something new that is specific to emacs (and
can even depend on which additional packages are installed..) is a
perfect method to cause problems for users and make people reject the
entire software because it doesn't follow standards.  Gnus provides
two supposedly standalone packages called Message and Emacs MIME which
provides MIME encoding and decoding, can't RMAIL use them?

Re: RMAIL bails out with coding-system-error

[Eli Zaretskii]

On Sun, 5 May 2002, Simon Josefsson wrote:

> Is this X-Coding-System header actually used by RMAIL for anything?

Yes, it is used to set the value of buffer-file-coding-system when you 
read the same message later.  Without this, RMAIL would have to 
repeatedly decode messages each time you display it; this would be a 
nuisance with long messages.

In addition, recording an encoding allows features like 
rmail-redecode-body, that lets users override bogus MIME headers.

> I think the header is a very bad idea and it should be removed.  There
> is a standard for interchanging non-ASCII data using email, and it is
> called MIME.  Inventing something new that is specific to emacs (and
> can even depend on which additional packages are installed..) is a
> perfect method to cause problems for users and make people reject the
> entire software because it doesn't follow standards.

IIRC, the X-Coding-System header is only written to the BABYL-formatted 
files Emacs keeps for itself, and then only in the summary part.  What 
kinds of problems do you envision with this, given that headers beginning 
with "X-" can be used by applications for their own purposes?

However, I'm not saying that this is the only possible way of recording 
the message encoding.  If there are more standard methods that can 
support the same features, we could consider switching to them.

> Gnus provides
> two supposedly standalone packages called Message and Emacs MIME which
> provides MIME encoding and decoding, can't RMAIL use them?

The difference between RMAIL and Gnus is that RMAIL stores only the 
decoded messages.  It doesn't keep the undecoded copy around.  But I 
don't know if this should prevent the change of the kind you suggest.

Re: RMAIL bails out with coding-system-error

[Simon Josefsson]

Eli Zaretskii <eliz@is.elta.co.il> writes:

> On Sun, 5 May 2002, Simon Josefsson wrote:
>
>> Is this X-Coding-System header actually used by RMAIL for anything?
>
> Yes, it is used to set the value of buffer-file-coding-system when you 
> read the same message later.  Without this, RMAIL would have to 
> repeatedly decode messages each time you display it; this would be a 
> nuisance with long messages.
>
> In addition, recording an encoding allows features like 
> rmail-redecode-body, that lets users override bogus MIME headers.

I wasn't clear, I meant: Is it used for anything upon receiving a
message with that header from the network via email?

Using it internally after receiving the message is OK, I think.

>> I think the header is a very bad idea and it should be removed.  There
>> is a standard for interchanging non-ASCII data using email, and it is
>> called MIME.  Inventing something new that is specific to emacs (and
>> can even depend on which additional packages are installed..) is a
>> perfect method to cause problems for users and make people reject the
>> entire software because it doesn't follow standards.
>
> IIRC, the X-Coding-System header is only written to the BABYL-formatted 
> files Emacs keeps for itself, and then only in the summary part.  What 
> kinds of problems do you envision with this, given that headers beginning 
> with "X-" can be used by applications for their own purposes?

None.  I thought RMAIL decided how to display an incoming email
depending on the X-Coding-System header.  This is what would cause
problems.

Of course, RMAIL should probably make sure that X-Coding-System is
removed when forwarding messages, so other RMAIL instances isn't
confused.  Or at least remove it on incoming messages before adding
the new header locally.

> However, I'm not saying that this is the only possible way of recording 
> the message encoding.  If there are more standard methods that can 
> support the same features, we could consider switching to them.

For internal purposes, whatever works is OK.  Switching to MIME for
the internal representation seems like work with little gain.

>> Gnus provides
>> two supposedly standalone packages called Message and Emacs MIME which
>> provides MIME encoding and decoding, can't RMAIL use them?
>
> The difference between RMAIL and Gnus is that RMAIL stores only the 
> decoded messages.  It doesn't keep the undecoded copy around.  But I 
> don't know if this should prevent the change of the kind you suggest.

Ok.  I misunderstood the purposes of the header.

Re: RMAIL bails out with coding-system-error

[Eli Zaretskii]

> From: Simon Josefsson <jas@extundo.com>
> Date: Sun, 05 May 2002 17:11:50 +0200
> 
> I wasn't clear, I meant: Is it used for anything upon receiving a
> message with that header from the network via email?

The X-Coding-System header should be never seen in any message
received from the network.  It's something RMAIL adds when it
processes the message, but never sends it anywhere except storing it
in the processed mailbox.

> Of course, RMAIL should probably make sure that X-Coding-System is
> removed when forwarding messages, so other RMAIL instances isn't
> confused.

The X-Coding-System header is not sent with outgoing messages.  At
least it shouldn't; if you see one, it's some bug somewhere.

Re: RMAIL bails out with coding-system-error

[Kai Großjohann]

"Eli Zaretskii" <eliz@is.elta.co.il> writes:

>> From: Simon Josefsson <jas@extundo.com>
>> Date: Sun, 05 May 2002 17:11:50 +0200
>> 
>> I wasn't clear, I meant: Is it used for anything upon receiving a
>> message with that header from the network via email?
>
> The X-Coding-System header should be never seen in any message
> received from the network.

RMAIL should do something smart if it is.  (It could rename the
header, for instance.)  Otherwise, I see a DOS attack lurking...

kai
-- 
Silence is foo!

Re: RMAIL bails out with coding-system-error

[Eli Zaretskii]

On 5 May 2002, David Kuehling wrote:

> 	     (insert "X-Coding-System: "
> 		     (symbol-name last-coding-system-used)
> 		     "\n")))
> [...]
> Using the last coding system is after all a little confusing.  Maybe
> `default-buffer-file-coding-system' would be a better (and more
> predictable) choice?

last-coding-system-used is set by all the functions that decode text.  So 
the above code makes sure the encoding of the message is recorded in the 
message headers.  That encoding can come from any number of sources: the 
Charset= header, the current defaults taken from the language 
environment, or from the user (see, e.g., rmail-redecode-body).  Perhaps 
I'm missing something, but I fail to see how any global default setting 
such as default-buffer-file-coding-system, can be an alternative to 
last-coding-system-used.