Re: Asian characters in email corrupts all successive emails

[Kenichi Handa <handa@m17n.org>]

In article <xn6n0a8n0qt.fsf@txc.stpn.soft.net>, puneet <puneet@computer.org> writes:

>       I am using a recent cvs build of emacs. I have
> attached a small file with this email, which is part of my
> procmail.log.

>       As you can see on the line 14 of the attached file,
> it is the procmail log for some spam with asian
> characters. The problem is that in emacs, I am seeing all
> the lines after line 14 corrupted. It is not so if I use
> any other text editor (I have tried vim/vi).

Actually the file is already corrupted.  As the line 14
doesn't end with an ESC sequence to designate ASCII, the
bytes in the next lines are still considered as the lastly
designated charset (a Japanese charset in this case).

It seems that procmail trancates subject lines just by bytes
without considering the encoding.

And perhaps vim/vi assume ASCII designation sequence at line
end even if a line doesn't end with it, and perhaps Emacs
should do the same thing.  I'll work on it.

>       When I open the file in emacs and save it in another
> file, the saved file differs from the opened file.

Emacs can't preserve the exact byte sequence of the original
file on saving when the file is decoded by iso-2022 based
coding system, especially when the file contains incorrect
escape sequences.  Have you tried vim/vi to save the file?
Do they preserve the original byte sequence?

>       When such an email is coming to my mail
> folder/spool, emacs fails to recognize all the emails
> after the particular mail in the mail folder. Actually I
> am using gnus/nnml and emacs is not recognizing/getting
> any email after the particular email from the spool.

I think it's a bug of gnus/nnml.  From your description, I
suspect that it decodes all mails at once instead of
decoding each message one by one.  But, I'm not a Gnus user.
Could someone confirm it?

Anyway, please try this command to fix such kind of
incorrect ESC sequence.  It reads a file name FILENAME, and
write the fixed one into FILENAME.new.

(defun fix-iso2022 (file)
  (interactive "fFile to recover: ")
  (with-temp-file (concat file ".new")
    (set-buffer-multibyte nil)
    (let ((coding-system-for-read 'raw-text))
      (insert-file-contents file)
      (while (re-search-forward "\e$(?[@-~]" nil t)
	(or (search-forward "\e(B" (line-end-position) 'move)
	    (insert "\e(B"))))))

---
Ken'ichi HANDA
handa@m17n.org