Re: decode-coding-string gone awry?

[David Kastrup <dak@gnu.org>]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> The problem is that by passing `output' to decode-coding-string you
>>> clearly consider `output' to be a sequence of bytes.  But to
>>> construct `output' you use pieces of `string' so you have to make
>>> sure that `string' is also a sequence of bytes.  Assuming `string'
>>> comes from the TeX process, you can do that by making sure that that
>>> process's output coding system is `binary' (or `raw-text' if you
>>> want EOL-conversion).
>
>> I already mentioned that this _is_ exactly what we do already: the
>> problem is that some TeX systems are set up to quote _some_ bytes from
>> utf-8 in the ^^xx hexadecimal notation, and let some bytes through
>> unchanged.
>
> I'm not sure I understand.  What I meant above is not "make sure the
> TeX process only outputs binary", but really set the
> `process-coding-system' of the TeX process such that its output
> coding-system is `raw-text' or `binary'.  This *should* (aka "module
> bugs") encusre that the strings passed to the process filter are
> unibyte.
>
> If the string goes through a buffer

Yes.

> instead of being processed directly from the process filter, then
> you should also ensure that this buffer is unibyte.

Yuk.  The problem is that this buffer is not only processed by
preview-latex, but also by AUCTeX, and the versions that get combined
may be different.  AUCTeX uses the source code buffer's file encoding
by default, which is fine for basically unibyte based coding systems.

If a buffer is unibyte, how will its characters get displayed?  In
particular, on a system that has all its language-environment set to
accommodate utf-8?  At what time does the decision whether a buffer is
unibyte or multibyte get made?

I guess that in the long run we will have to install something
directly at filter level, with some CCL program processing the TeX
output.  But at the moment I am trying to stumble along in the context
we have now.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum