Re: Emacs 23 character code space

[Stefan Monnier <monnier@iro.umontreal.ca>]

> What exactly is meant here by ``8-bit characters''?  Do you mean
> eight-bit raw bytes, or do you mean Unicode characters whose
> codepoints are below 256?

It should be eight-bit raw bytes.  In some cases it's difficult to tell
the difference, so Emacs may occasionally accept latin-1 chars as stand
ins for eight-bit raw bytes.

>     Converting unibyte text to multibyte text leaves @acronym{ASCII}
>     characters unchanged, and converts 8-bit characters (codes 128
>     through 159) to the corresponding representation for
>     multibyte text.
> Again, by ``8-bit characters'' you mean raw 8-bit bytes here, right?

Yes.

I think we should state somewhere that unibyte strings and buffers
contain bytes only.  And that multibyte strings and buffers contain
chars.  And that bytes are a subset of chars.

>     @defun string-to-multibyte string
>     This function returns a multibyte string containing the same sequence
>     of characters as @var{string}.  If @var{string} is a multibyte string,
>     it is returned unchanged.
>     @end defun

> I'm not sure I understand the effect of this function.

It returns a string containing the same bytes (in the sense of
ASCII+eight-bit, not in the sense of the underlying internal
representation, which we should as much as possible not mention
anywhere) but in a multibyte string instead.  I.e. the output is
a multibyte string of the same length whose chars are bytes.

>     @defun string-to-unibyte string
>     This function returns a unibyte string containing the same sequence of
>     characters as @var{string}.  It signals an error if @var{string}
>     contains a non-@acronym{ASCII} character.  If @var{string} is a
>     unibyte string, it is returned unchanged.
>     @end defun

> Since this function handles any non-ASCII characters lossily, when
> would it be useful?

I think the "non-ASCII" part is incorrect.  It probably should say
"non-byte char" instead.  It's useful when you have a multibyte string
which you (think you) know only holds bytes.

In 99% (actually 99.99999% for the `as' case) of the cases you shouldn't
use string-{as/make/to}-{uni/multi}byte.  Instead you should use
{en/de}code-coding-string.

>     @defun multibyte-char-to-unibyte char
>     This convert the multibyte character @var{char} to a unibyte
>     character.  If @var{char} is a non-@acronym{ASCII} character, the
>     value is -1.
>     @end defun

>     @defun unibyte-char-to-multibyte char
>     This convert the unibyte character @var{char} to a multibyte
>     character.
>     @end defun

> Again, when are these functions useful?

Rarely.


        Stefan