Re: File names with accented Latin characters are not displayed correctly

[Kenichi Handa <handa@m17n.org>]

In article <0ed07c7b9fe05cb0334850bba636bd40@Web.DE>, Peter Dyballa <Peter_Dyballa@Web.DE> writes:

>>  When I use 'ls -lw' to display the file names in xterm, I get:
>> 
>>  	-rw-r--r--    1 pete  pete        62 25 Mär  2005 áÛïǓà.txt
>>  	-rw-r--r--    1 pete  pete        62 25 Mär  2005 äÖüÄöÜ.txt
>>  	-rw-r--r--    1 pete  pete       107  2 Dez  2004 äöüßÜÖÄ€
>>  	
>>  Doing the same in Emacs' *shell* buffer I get:
>> 
>>  	-rw-r--r--    1 pete  pete        62 25 Mär  2005 áÛïǓà.txt
>>  	-rw-r--r--    1 pete  pete        62 25 Mär  2005 äÖüÄöÜ.txt
>>  	-rw-r--r--    1 pete  pete       107  2 Dez  2004 äöüßÜÖÄ€

You are using Unicode Emacs on Mac Darwin.  I heard that all
file names on that system are treated in a decomposed form
(normalization form NFD or NFKD, see UAX15 of Unicode).

But, currently emacs doesn't have a converter between each
normalization form.  So, a-umlaut in a file name is actually
the two characters seqeunce "a" and "umlaut" (U+0308) on
that system, but when you type that character in Emacs,
Emacs produces U+00E4.

[...]
> OK, now an explanation is given: no font. The question is: do I need to 
> supply a font? If so: how? Hitting C-h v on that 
> `reference-point-alist' gives a reference to a variable (I think: too 
> big to cite it here) defined in `composite'. There I found a reference 
> to the function toggle-auto-composition. When I apply this function to 
> the *Buffers List* I can see that it "changes" one file name: obviously 
> one which is the exact copy of the entry in dired-mode!

You said that Emacs used to show those characters in an ugly
but correct way, and stopped displaying in that way
recently.  I think that is because some font/fontset related
code was changed for Mac recently, and I have no idea why
Emacs can't find a correct font on Mac now.  Could someone
who is working on Mac port help him?

> And I now recognised too that I when I open a file with the ä in the 
> name, it appears in mode-line correct. In the pop-up buffers menu I see 
> its name printed in normal UTF-8 representation, i.e. C3 A4 = ä. OK, I 
> can guess the right name. When I open such a file from dired-mode by 
> pressing the mouse, the ä is represented by a hollow box in the 
> mode-line. This hollow box is "translated" in pop-up buffers menu to 
> "Ì▢." OK, I am cheating a bit: when I open the file with C-x f or 
> change any name to a name with ä, then the name is correct in 
> mode-line. In *Buffer List* this name is displayed correctly too. The 
> other file name, which I open 'with the mouse,' has the de-composed ä 
> glyph which is described by C-u C-x = as the ä in dired mode. And in 
> this file name I can toggle the representation between "a" and "a▢" -- 
> but no change in pop-up buffers menu!

All those confusions are because of normalization form used
on your system.  Emacs and the system don't agree with file
name encoding.

Mr. Kawabata is now working on implementing converters
between all normalization forms.  He has already finished
writing a code, sent assignment paper to FSF, and is now
waiting for a reply.  As soon as his contribution is
accepted, I'll install it.  Then I think you problem is
solved.

---
Kenichi Handa
handa@m17n.org