mapconcat + format problem

mapconcat + format problem

[Peter Tury]

Hi,

I found that

(mapconcat (lambda (n)
             (format "%c" n))
           (number-sequence start end)
           "")

gives different results for charcters after ~160 depending on `start':
if `start' is 0 then I get "readable" results, but otherwise I get the
characters' octal code. E.g. after

(defun to-string (start end)
  (mapconcat (lambda (n)
               (format "%c" n))
             (number-sequence start end)
             ""))

(equal (substring (to-string 0 190) 1)
       (substring (to-string 1 190) 0))

results nil. Why?

I don't see differences if I use `concat' instead of (mapconcat (...
(format:

(concat (number-sequence 0 190))
and
(concat (number-sequence 1 190))

seems to work consistently (gives octal codes).

What is the problem with the mapconcat solution?

Thanks,
P

Re: mapconcat + format problem

[Sebastian Tennant]

Quoth Peter Tury <tury.peter@gmail.com>:
> Hi,
>
> I found that
>
> (mapconcat (lambda (n)
>              (format "%c" n))
>            (number-sequence start end)
>            "")
>
> gives different results for charcters after ~160 depending on `start':
> if `start' is 0 then I get "readable" results, but otherwise I get the
> characters' octal code. E.g. after
>
> (defun to-string (start end)
>   (mapconcat (lambda (n)
>                (format "%c" n))
>              (number-sequence start end)
>              ""))
>
> (equal (substring (to-string 0 190) 1)
>        (substring (to-string 1 190) 0))
>
> results nil. Why?

(set-buffer-multibyte nil)
=> nil

(to-string 0 190)
"...{|}~\x7f\x80\x81..."

(to-string 1 190)
"... {|}~\x7f\200\201..."

It seems to me the output begins to differ after character code 127,
rather than ~ 160:

 (format "%c" 127)
 => "^?"

which suggests the issue has something to do with the way mapconcat
is handling ASCII versus non-ASCII characters.

However, this is interesting:

  (info "(elisp)Text Representations")

     In multibyte representation, a character may occupy more than one
  byte, and as a result, the full range of Emacs character codes can be
  stored.  The first byte of a multibyte character is always in the range
  128 through 159 (octal 0200 through 0237).  These values are called
  "leading codes".  The second and subsequent bytes of a multibyte
  character are always in the range 160 through 255 (octal 0240 through
  0377); these values are "trailing codes".

     Some sequences of bytes are not valid in multibyte text: for example,
  a single isolated byte in the range 128 through 159 is not allowed.  But
  character codes 128 through 159 can appear in multibyte text,
  represented as two-byte sequences.  All the character codes 128 through
  255 are possible (though slightly abnormal) in multibyte text; they
  appear in multibyte buffers and strings when you do explicit encoding
  and decoding (*note Explicit Encoding::).

But this is all about text representation in buffers and won't explain
why your equality test:

  (equal (substring (to-string 0 190) 1)
         (substring (to-string 1 190) 0))

fails.

Sorry!  Not much help :-/

Sebastian