detect big5, utf8, gb2312 as good as firefox

detect big5, utf8, gb2312 as good as firefox

[Dan Jacobson]

Gentlemen, with three plain text files of coded in my most commonly
encountered coding systems, I did this test:
$ for c in zh_CN.gb2312 zh_TW.utf8 zh_TW.big5;
do LC_CTYPE=$c LANG=$c LC_ALL=$c emacs -q gb2312 utf8 big5; done
Well, zh_CN.gb2312 guessed right each time!
With zh_TW.utf8, emacs guessed the big5 and gb files were latin-1, as
seen by the 1 in the modeline and the jumble on the screen.
With zh_TW.big5, the gb2312 file was seen jumbled as type big5.
In .emacs I can do
(set-language-environment "UTF-8")
(prefer-coding-system 'utf-8-unix)
(set-coding-priority ;So that big5 is still guessed right after utf-8.
 (reverse ;Found these lisp thingies and it works.
  (delete-duplicates
   (reverse;no lisp pro me
    (append(list 'coding-category-utf-8
		 'coding-category-big5)coding-category-list)))))
to detect all but gb2312 OK. What should I do, make my whole
environment CN even though I only visit those kind of files once a
week, and plan to live in UTF-8 / big5 land ... BTW, firefox guessed
right each time even though they were plain text files with no
charset= hints.

By the way the Info node "Language Environments"
only mentions
     The supported language environments include:
     Chinese-BIG5, Chinese-CNS, Chinese-GB, Cyrillic-ALT, Cyrillic-ISO,
     Cyrillic-KOI8, Czech, Devanagari, Dutch, English, Ethiopic, German,
     Greek, Hebrew, IPA, Japanese, Korean, Lao, Latin-1, Latin-2,
     Latin-3, Latin-4, Latin-5, Latin-8 (Celtic), Latin-9 (updated
     Latin-1, with the Euro sign), Polish, Romanian, Slovak, Slovenian,
     Spanish, Thai, Tibetan, Turkish, and Vietnamese.
But not utf-8 (or UTF-8, Utf-8?), (but maybe that's intentional?)

Re: detect big5, utf8, gb2312 as good as firefox

[Kevin Rodgers]

Dan Jacobson wrote:
> Gentlemen, with three plain text files of coded in my most commonly
> encountered coding systems, I did this test:
> $ for c in zh_CN.gb2312 zh_TW.utf8 zh_TW.big5;
> do LC_CTYPE=$c LANG=$c LC_ALL=$c emacs -q gb2312 utf8 big5; done
> Well, zh_CN.gb2312 guessed right each time!
> With zh_TW.utf8, emacs guessed the big5 and gb files were latin-1, as
> seen by the 1 in the modeline and the jumble on the screen.
> With zh_TW.big5, the gb2312 file was seen jumbled as type big5.
> In .emacs I can do
> (set-language-environment "UTF-8")
> (prefer-coding-system 'utf-8-unix)
> (set-coding-priority ;So that big5 is still guessed right after utf-8.
>  (reverse ;Found these lisp thingies and it works.
>   (delete-duplicates
>    (reverse;no lisp pro me
>     (append(list 'coding-category-utf-8
> 		 'coding-category-big5)coding-category-list)))))
> to detect all but gb2312 OK. What should I do, make my whole
> environment CN even though I only visit those kind of files once a
> week, and plan to live in UTF-8 / big5 land ... BTW, firefox guessed
> right each time even though they were plain text files with no
> charset= hints.

I would instead try this first:

;; From least- to most-preferred:
(prefer-coding-system 'gb2312)
(prefer-coding-system 'big5)
(prefer-coding-system 'utf-8)

If you insist on frobbing coding-category-list:

(set-coding-priority
  ;; Insert big5 after utf-8:
  (apply 'nconc
         (mapcar (lambda (coding-category)
                   (if (eq coding-category 'coding-category-utf-8)
                       (list 'coding-category-utf-8 'coding-category-big5)
                     (list coding-category)))
                 coding-category-list)))

Note that you can replace (apply 'nconc (mapcar ...)) with
(require 'cl)(mapcan ...)

Do you get better or worse results with those approaches?

-- 
Kevin Rodgers