Re: utf8 char display in buffer

[ken <gebser@mousecar.com>]

On 06/12/2009 06:27 PM Lennart Borgman wrote:
> Ken, I think this is a good idea so I have sent this along to Emacs devel.
> 
> On Sat, Jun 13, 2009 at 12:23 AM, ken<gebser@mousecar.com> wrote:
>> Yet emacs puts a little box in the place of a character it cannot find
>> (or, per your explanation) possibly confused about.  The fact remains
>> that the little box is not a correct rendering of the code.  It is an
>> error... at least it is for me, because that's not what I typed in.  So
>> it is an error.  As an error, there should be a corresponding error
>> message, hopefully one (or more) which would help diagnose the problem.
>>  It seems obvious that, given the long thread on this issue with no
>> resolution, we could use some help-- like an error message-- which would
>> help in diagnosis.

Thank you, Lennart!  To give the people at emacs-devel some context to
the issue, the salient portion of the previous post is pasted below:

0) Some others and myself want to include some non-English characters in
a file being edited in emacs. Problems arise, however:

1) In a buffer which is already utf-8 encoded, I set the appropriate
input method, type in the desired characters. They display just peachy
and there is happiness in EmacsLand.

2) I save the buffer to a file, then close the buffer.

3) I visit the same file (i.e., load it again into emacs). Because it
has <!-- -*- coding: utf-8; -*- --> as the first line, it opens
utf-8 encoded. This is confirmed by the presence of a 'u' as the second
character in the status bar.

4) The text in the buffer displays fine, except that in place of each of
those non-English characters is a little empty box. With the cursor on
one of those boxes, an 'a' with a horizontal bar above it, doing "C-x
=", emacs returns "Char: ā (01210041, 331809, 0x51021, file ...)".
(While, in emacs the character after "Char:" is a little box, if I load
this same file into Firefox, that same character appears as it should,
as an 'a' with a horizontal bar above it. How it appears in your email
client will depend upon your email client.)

A) The fact that, as described in (4), the characters display correctly
in Firefox, but not in emacs indicates that emacs is not drawing on the
needed character set. Yet, the fact that in (1) the characters initially
display correctly (when first input) indicates that the needed character
set is present on the system and emacs can find it and has permission
access it. Further, we would think that emacs would throw out an error
message if either of these conditions were not met... and it doesn't. We
can only assume that, when visiting and then decoding a file and pulling
into a buffer for display, emacs is not even asking for the proper
character set when encountering a non-English character. This is where I
would start to look for the error.

B) It would be helpful if the code which does the decoding of a file and
renders it into the buffer display, if that part of it would throw an
error message when it encounters a character it doesn't know how to
display, i.e., when a little box character is displayed. After all,
isn't it an error when a little box is displayed in lieu of the correct
character? Possible error messages would be something like: "decoding
process can't find /path/to/charset.file" or "decoding process doesn't
have requisite permission to read /path/to/charset.file" or "invalid
character: [hex/decimal value]" or other.

###

Thanks much,
ken