Re: UCS-2BE

[Kenichi Handa <handa@m17n.org>]

In article <87ac5ko50j.fsf@jurta.org>, Juri Linkov <juri@jurta.org> writes:

> `UCS-2' is the fixed-length encoding of the BMP.  `UCS-2BE' is
> a big-endian version of the UCS-2 encoding without using a BOM.
> So as actually UCS-2 is a BMP subset of UTF-16, UCS-2BE is a BMP
> subset of UTF-16BE (and UCS-2LE is a BMP subset of UTF-16LE).

Where did you get that info?

The word "encoding" is ambiguous here.  There are "CEF
(Character Encoding Form)" and "CES (Character Encoding
Scheme)".  Unicode says (see Glossary):

Character Encoding Form: Mapping from a character set
definition to the actual code units used to represent the
data.

Character Encoding Scheme: A character encoding form plus
byte serialization. ...

UCS-XXX are CEF, and UTF-XXX are CES.  So, UCS-XXX are not
appropriate lavel names for specifing how to byte-serialize
characters (i.e. on saving characters in a file).  At least,
that is the official definition in Unicode.

And, as you see now, there's is a contradition in the term
"UCS-2BE" because "BE" is information about
byte-serialization.  But the term "UCS-2BE" itself is not
defined in Unicode.  So, there are two possibilities:

(1) It's just a mis-label of something.
(2) It's defined somewhere else.

Which is the case?

> The encodings `UCS-2' and `UCS-2BE' are implemented in iconv
> (http://www.gnu.org/software/libiconv/), so you could look
> at the implementation of UCS-2BE:

Does it mean that it's an invention of iconv to use those
names as CES?

> Does the Emacs implementation of UTF-16 output the BOM?

Yes.

> > (at least, we should not select it by select-safe-coding-system on
> > saving a buffer that contains non-BMP characters).

> What do you think is the right way to deal with non-BMP characters
> when the user will try to save a UTF-16(BE) buffer in the UCS-2(BE)
> encoding?

It depends on how UCS-2BE is defined.  If we follow the
implementation of iconv (and if the buffer contains non-BMP
characters), we should ask the user to select a proper
coding system.

---
Kenichi Handa
handa@m17n.org