Re: Emacs 23 character code space

[Kenichi Handa <handa@m17n.org>]

In article <uk5aviv36.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:

>       Emacs can convert unibyte text to multibyte; it can also convert
>     multibyte text to unibyte provided that the multibyte text contains
>     only @acronym{ASCII} and 8-bit characters.

> What exactly is meant here by ``8-bit characters''?  Do you mean
> eight-bit raw bytes, or do you mean Unicode characters whose
> codepoints are below 256?

The former; more precisely, characters representing
eight-bit raw bytes.  They have different character codes in
multibyte text (#x3FFF80..#x3FFFFF) and unibyte text
(#x80..#xFF).

>       Converting unibyte text to multibyte text leaves @acronym{ASCII} characters
>     unchanged, and converts 8-bit characters (codes 128 through 159) to
>     the corresponding representation for multibyte text.

> Again, by ``8-bit characters'' you mean raw 8-bit bytes here, right?

Yes.

>     @defun string-to-multibyte string
>     This function returns a multibyte string containing the same sequence
>     of characters as @var{string}.  If @var{string} is a multibyte string,
>     it is returned unchanged.
>     @end defun

> I'm not sure I understand the effect of this function.  Does it decode
> its argument, converting each byte to the corresponding internal
> representation of the encoded single-byte character?  I think this is
> not what it does, but then what does it do?

No, all 8-bit characters (#x80..#xFF) in the source unibyte
string is converted to the multibyte representation of those
8-bit characters (#x3FFF80..#x3FFFFF).

>     @defun string-to-unibyte string
>     This function returns a unibyte string containing the same sequence of
>     characters as @var{string}.  It signals an error if @var{string}
>     contains a non-@acronym{ASCII} character.  If @var{string} is a
>     unibyte string, it is returned unchanged.
>     @end defun

> Since this function handles any non-ASCII characters lossily, when
> would it be useful?

If you know that a string containts only ASCII or 8-bit
characters, you can use it to get a unibyte string without
loosing information.

>     @defun multibyte-char-to-unibyte char
>     This convert the multibyte character @var{char} to a unibyte
>     character.  If @var{char} is a non-@acronym{ASCII} character, the
>     value is -1.
>     @end defun

>     @defun unibyte-char-to-multibyte char
>     This convert the unibyte character @var{char} to a multibyte
>     character.
>     @end defun

> Again, when are these functions useful?

Perhaps, we don't need them anymore.  We can use get-byte.
Anyway, the relationship of they and
string-to-unibyte/multibyte is this:

(defun string-to-unibyte (str)
  (let ((new (make-string (length str) 0)))
    (dotimes (i (length str))
      (let ((byte (multibyte-char-to-unibyte (aref str i))))
	(if (< byte 0)
	    (error))
	(aset new i byte)))
    new))

(defun string-to-multibyte (str)
  (let ((new (make-string (length str) 0)))
    (dotimes (i (length str))
      (aset new i (unibyte-char-to-multibyte (aref str i))))
    new))

---
Kenichi Handa
handa@ni.aist.go.jp