bug#5700: emacs-23 and 8-bit characters in 128..255

[Stefan Monnier <monnier@IRO.UMontreal.CA>]

> This morning, I set out to track down the source of one of the
> problems in a function that I use a lot, and eventually narrowed it to
> the failure of functions like these:

>     (string-equal (buffer-substring (point) (1+ (point))) "\377")

Indeed, we have a problem:

  (string-equal "\377" (string-to-multibyte "\377"))

returned t in Emacs-22 but returns nil in Emacs-23.  Another (somewhat
related) problem is that under Emacs-22, we had:

  "\377"                        prints as    "\377"
  "\xff"                        prints as    "\xff"
  (multibyte-string-p "\377")   prints as    "\xff"

which seems acceptable, whereas under Emacs-23 we have:

  "\377"                        prints as    "ÿ"
  "\xff"                        prints as    "ÿ"
  (multibyte-string-p "\377")   prints as    "\377"

which looks rather confusing.

>     (looking-at "\377")

This is probably a separate bug.

>     The internal encoding used for buffers and strings is now
>     Unicode-based and called `utf-8-emacs' (`emacs-internal' is an alias

It is related, but only to the extent that a lot of the code that
handles multibyte chars (and especially "eight-bit chars") was
completely rewritten, and this is a very delicate area.


        Stefan