Re: utf-16le vs utf-16-le

[Eli Zaretskii <eliz@gnu.org>]

> From: "Stephen J. Turnbull" <turnbull@sk.tsukuba.ac.jp>
> Cc: emacs-devel@gnu.org
> Date: Wed, 16 Apr 2008 01:51:50 +0900
> 
> Eli Zaretskii writes:
> 
>  > > BOM-{prohibited,auto,required}.
>  > 
>  > But we don't have these in Emacs, do we?
> 
> Huh?  We don't have the full suite, but we do have -signature variants.

Bot for UTF-8, we don't, at least not in GNU Emacs 23.  When I type
"C-x RET c utf-8 TAB TAB TAB", I see this in the completions buffer:

    Click <mouse-2> on a completion to select it.
    In this buffer, type RET to select the completion near point.

    Possible completions are:
    utf-8            utf-8-dos         utf-8-emacs  utf-8-emacs-dos
    utf-8-emacs-mac  utf-8-emacs-unix  utf-8-mac    utf-8-unix

No -signature variants.

>  > >  > Don't forget that en/decoding is used on strings as well, not only on
>  > >  > buffers.  Buffer-local variables won't cut it, I think.
>  > > 
>  > > Strings don't have encoding signatures or newline variants
>  > 
>  > ??? Of course, they do.
> 
> Indeed?  Suppose I have a string as the value of the symbol `s'
> containing the octets "\r\n".  Please explain to me how to compute
> whether that is the value 0x0D0A from a network stream prepared using
> htons(3), or a line ending suitable for appending to a Windows file.

The Lisp code that created the string knows what it is and how to deal
with it.  But you already know that, so I probably simply fail to
follow your reasoning.

>  > A string can be sent to a process, for example, so we must have some
>  > way of generating an external representation for it.
> 
> Well, of course we must.  But the right generalization of "buffer file
> coding system" is not to apply en/decoding to strings, but rather to
> give processes and sockets, etc, coding system properties equivalent
> to my proposed buffer-local variables.

I'm afraid that this will be very hard to implement in Emacs, since
the internals are very much exposed and we are used to copy strings to
and fro freely.  I think we also don't have sockets and other similar
interfaces as Lisp object to which we could give properties.