bug#1187: 23.0.60; Cannot read vline.el - invalid read syntax

[Lawrence Mitchell <wence@gmx.li>]

Drew Adams wrote:
> Library vline.el can be read fine in Emacs 22, but Emacs 23 raises an
> error, saying Invalid read syntax: "?".

The following sequence of events works for me in

GNU Emacs 23.0.60.4 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll
bars) of 2008-10-07 on lamacq.ph.ed.ac.uk

$ wget http://www.emacswiki.org/cgi-bin/emacs/download/vline.el \
-O /tmp/vline.el

$ LANG=C emacs -Q --eval '(load-file "/tmp/vline.el")'
LANG=C so that current-language-environment is "ASCII"
and default-buffer-file-coding-system is nil.

Successfully prints:
Loading /tmp/vline.el (source)...done

> In GNU Emacs 23.0.60.1 (i386-mingw-nt5.1.2600)
>  of 2008-10-03 on LENNART-69DE564
> Windowing system distributor `Microsoft Corp.', version 5.1.2600
> configured using `configure --with-gcc (3.4) --no-opt --cflags -Ic:/g/include
> -fno-crossjumping'

> Important settings:
>   value of $LC_ALL: nil
>   value of $LC_COLLATE: nil
>   value of $LC_CTYPE: nil
>   value of $LC_MESSAGES: nil
>   value of $LC_MONETARY: nil
>   value of $LC_NUMERIC: nil
>   value of $LC_TIME: nil
>   value of $LANG: ENU
                    ^^^ this seems to be the culprit.

LANG=ENU is an unknown locale here but if LANG=en_GB such that
default-buffer-file-coding-system is iso-latin-1-unix then the
section of the file that causes problems is:
(memq char '(?\t ?　))

where the second character is:

|         character: 　 (12288, #o30000, #x3000)
| preferred charset: unicode (Unicode (ISO10646))
|        code point: 0x3000
|            syntax: _ 	which means: symbol
|          category: c:Chinese h:Korean j:Japanese
| 		   |:While filling, we can break a line at this character.
|       buffer code: #xE3 #x80 #x80
|         file code: #xE3 #x80 #x80 (encoded by coding system utf-8-unix)
|           display: by this font (glyph code)
|     xft:-unknown-Kochi Gothic-normal-normal-normal-*-15-*-*-*-*-0-iso10646-1 (#x6F3)

| Character code properties: customize what to show
|   name: IDEOGRAPHIC SPACE
|   general-category: Zs (Separator, Space)
|   decomposition: (<wide> 32) (<wide> ' ')


The difference between emacs-21.3 and emacs 23 appears to be that
the former ignores the two extra bytes when reading in latin-1
but emacs 23 does not:

$ od -c /tmp/f
0000000   ? 343 200 200
0000004

Emacs 23:

(let ((coding-system-for-read 'iso-latin-1)
      ret)
  (with-temp-buffer
    (insert-file-contents "/tmp/f")
    (setq ret (read (current-buffer))))
  (insert ret))

=>

(invalid-read-syntax "?")
  read(#<buffer  *temp*>)

----
(let ((coding-system-for-read 'utf-8)
      ret)
  (with-temp-buffer
    (insert-file-contents "/tmp/f")
    (setq ret (read (current-buffer))))
  (insert ret))

=>

IDEOGRAPHIC SPACE is inserted correctly


Emacs 21.3

(let ((coding-system-for-read 'iso-latin-1)
      ret)
  (with-temp-buffer
    (insert-file-contents "/tmp/f")
    (setq ret (read (current-buffer))))
  (insert ret))

=>

ã ; LATIN SMALL LETTER A WITH TILDE is inserted.

-----
(let ((coding-system-for-read 'utf-8)
      ret)
  (with-temp-buffer
    (insert-file-contents "/tmp/f")
    (setq ret (read (current-buffer))))
  (insert ret))

=>

IDEOGRAPHIC SPACE is inserted correctly
-- 
Lawrence Mitchell <wence@gmx.li>