Re: How to make emacs auto-recognize utf-8 encoded files upon visiting

[Dominic Cronin <dominic@ReplaceThisBitWithMySurname.co.uk>]

On 23 Sep 2002 18:39:19 +0200, Gerald Wildgruber
<gwil.remove.this.phrase@lrz.uni-muenchen.de> wrote:

>
>Hello,
>
>I'm trying to make my emacs (GNU Emacs 21.3.50.1 on linux) auto-recognize
>the right encoding when visiting files with utf-8 encoding. The emacs info
>help entry says on the topic:
>
>"Some coding systems can be recognized or distinguished by which byte
>sequences appear in the data. However, there are coding systems that cannot
>be distinguished, not even potentially."
>
>Does this also apply to utf-8 encoded files? Is it impossible for emacs to
>auto-recognize them (as for example the `file' command on the shell does)?

The RFC for UTF-8 (see http://www.ietf.org/rfc/rfc2279.txt) states: 

UTF-8 strings can be fairly reliably recognized as such by a simple
algorithm, i.e. the probability that a string of characters in any
other encoding appears as valid UTF-8 is low, diminishing with
increasing string length.

BTW - the RFC is quite an interesting read: an elegant solution to a
problem.
--  

Dominic Cronin
Amsterdam