Re: coding tags and utf-16

[Werner LEMBERG <wl@gnu.org>]

> There is a serious problem with coding tags and utf-16 encodings of
> any flavour: Emacs simply can't recognize the tag.  [...]


Surprisingly, I saw no response on the list which either means that my
mail hasn't come through, nobody is interested in this problem, or
that it is a non-issue.

In case it won't get fixed I suggest to add it to the TODO list,
together with a not in the emacs manual that coding tags don't work
with utf-16 encoding flavours.


    Werner


> This is a non-trivial problem.  Right now I'm working on a groff
> preprocessor which tries to handle this.  I'm doing the following to
> find the tag in an encoding-independent way:
> 
>   . Check whether the file starts with the BOM (Byte Order Mark) --
>     this is one of the following byte sequences:
> 
>       UTF-8:  0xEFBBBF
>       UTF-16: 0xFEFF or 0xFFFE
> 
>     Skip it.
> 
>   . Ignore zero bytes while looking for the -*- coding: ... -*-
>     stuff.
> 
> This heuristic algorithm might not give correct results in all cases
> but it should be sufficiently reliable for normal use.