Asian characters in email corrupts all successive emails

Asian characters in email corrupts all successive emails

[puneet]

[-- Attachment #1: Type: text/plain, Size: 989 bytes --]

Hello All

      I am using a recent cvs build of emacs. I have attached a small
      file with this email, which is part of my procmail.log.

      As you can see on the line 14 of the attached file, it is the
      procmail log for some spam with asian characters. The problem is
      that in emacs, I am seeing all the lines after line 14
      corrupted. It is not so if I use any other text editor (I have
      tried vim/vi).

      When I open the file in emacs and save it in another file, the
      saved file differs from the opened file.

      When such an email is coming to my mail folder/spool, emacs
      fails to recognize all the emails after the particular mail in
      the mail folder. Actually I am using gnus/nnml and emacs is not
      recognizing/getting any email after the particular email from
      the spool.

      I think it must be something to do with multi-language support,
      but it is corrupting a lot of emails in my mailbox.

Regards
- p
-- 


[-- Attachment #2: sample procmail log --]
[-- Type: application/octet-stream, Size: 1052 bytes --]

[-- Attachment #3: Type: text/plain, Size: 136 bytes --]



--
Puneet Goel

--
THE storm is like the cry of some god
in pain whose love the earth refuses.
-
Stray Birds, Rabindernath Tagore

--

[-- Attachment #4: Type: text/plain, Size: 141 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Asian characters in email corrupts all successive emails

[Kenichi Handa]

In article <xn6n0a8n0qt.fsf@txc.stpn.soft.net>, puneet <puneet@computer.org> writes:

>       I am using a recent cvs build of emacs. I have
> attached a small file with this email, which is part of my
> procmail.log.

>       As you can see on the line 14 of the attached file,
> it is the procmail log for some spam with asian
> characters. The problem is that in emacs, I am seeing all
> the lines after line 14 corrupted. It is not so if I use
> any other text editor (I have tried vim/vi).

Actually the file is already corrupted.  As the line 14
doesn't end with an ESC sequence to designate ASCII, the
bytes in the next lines are still considered as the lastly
designated charset (a Japanese charset in this case).

It seems that procmail trancates subject lines just by bytes
without considering the encoding.

And perhaps vim/vi assume ASCII designation sequence at line
end even if a line doesn't end with it, and perhaps Emacs
should do the same thing.  I'll work on it.

>       When I open the file in emacs and save it in another
> file, the saved file differs from the opened file.

Emacs can't preserve the exact byte sequence of the original
file on saving when the file is decoded by iso-2022 based
coding system, especially when the file contains incorrect
escape sequences.  Have you tried vim/vi to save the file?
Do they preserve the original byte sequence?

>       When such an email is coming to my mail
> folder/spool, emacs fails to recognize all the emails
> after the particular mail in the mail folder. Actually I
> am using gnus/nnml and emacs is not recognizing/getting
> any email after the particular email from the spool.

I think it's a bug of gnus/nnml.  From your description, I
suspect that it decodes all mails at once instead of
decoding each message one by one.  But, I'm not a Gnus user.
Could someone confirm it?

Anyway, please try this command to fix such kind of
incorrect ESC sequence.  It reads a file name FILENAME, and
write the fixed one into FILENAME.new.

(defun fix-iso2022 (file)
  (interactive "fFile to recover: ")
  (with-temp-file (concat file ".new")
    (set-buffer-multibyte nil)
    (let ((coding-system-for-read 'raw-text))
      (insert-file-contents file)
      (while (re-search-forward "\e$(?[@-~]" nil t)
	(or (search-forward "\e(B" (line-end-position) 'move)
	    (insert "\e(B"))))))

---
Ken'ichi HANDA
handa@m17n.org