bug#3325: 23.0.93; Unexpected font for composed character

bug#3325: 23.0.93; Unexpected font for composed character

[Markus Triska]

I have a file ~/Downloads/Büroanwendungen.zip. When I visit ~/Downloads/
in dired (C-x d ~/Downloads/ RET) and press C-u x = on the "ü", I get:

           character: u (117, #o165, #x75)
   preferred charset: ascii (ASCII (ISO646 IRV))
          code point: 0x75
              syntax: w 	which means: word
            category: .:Base, a:ASCII, l:Latin, r:Roman
         buffer code: #x75
           file code: #x75 (encoded by coding system utf-8-unix)
             display: composed to form "ü" (see below)

   Composed with the following character(s) "̈" using this font:
     xft:-itc-American Typewriter-normal-normal-normal-*-20-*-*-*-*-0-iso10646-1
   by these glyphs:
     [0 1 117 93 12 -1 12 11 0 nil]
     [0 1 776 241 0 -8 -2 14 -11 [-2 -2 0]]

   Character code properties: customize what to show
     name: LATIN SMALL LETTER U
     general-category: Ll (Letter, Lowercase)

   There are text properties here:
     dired-filename       t
     fontified            t
     help-echo            "mouse-2: visit this file in other window"
     mouse-face           highlight

This font differs unexpectedly (for me) from the one used for the "r":

             character: r (114, #o162, #x72)
   preferred charset: ascii (ASCII (ISO646 IRV))
          code point: 0x72
              syntax: w 	which means: word
            category: .:Base, a:ASCII, l:Latin, r:Roman
         buffer code: #x72
           file code: #x72 (encoded by coding system utf-8-unix)
             display: by this font (glyph code)
       xft:-bitstream-Bitstream Vera Sans Mono-normal-normal-normal-*-20-*-*-*-m-0-iso10646-1 (#x55)

   Character code properties: customize what to show
     name: LATIN SMALL LETTER R
     general-category: Ll (Letter, Lowercase)

   There are text properties here:
     dired-filename       t
     fontified            t
     help-echo            "mouse-2: visit this file in other window"
     mouse-face           highlight


In GNU Emacs 23.0.93.3 (i386-apple-darwin9.6.1, GTK+ Version 2.14.7)
 of 2009-05-11 on mt-imac.local
Windowing system distributor `The X.Org Foundation', version 11.0.10402000
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

bug#3325: 23.0.93; Unexpected font for composed character

[Kenichi Handa]

In article <m2skj2s2w7.fsf@gmx.at>, Markus Triska <markus.triska@gmx.at> writes:

> I have a file ~/Downloads/Büroanwendungen.zip. When I visit ~/Downloads/
> in dired (C-x d ~/Downloads/ RET) and press C-u x = on the "ü", I get:

>            character: u (117, #o165, #x75)
>    preferred charset: ascii (ASCII (ISO646 IRV))
>           code point: 0x75
>               syntax: w 	which means: word
>             category: .:Base, a:ASCII, l:Latin, r:Roman
>          buffer code: #x75
>            file code: #x75 (encoded by coding system utf-8-unix)
>              display: composed to form "ü" (see below)

>    Composed with the following character(s) "̈" using this font:
>      xft:-itc-American Typewriter-normal-normal-normal-*-20-*-*-*-*-0-iso10646-1
[...]
> This font differs unexpectedly (for me) from the one used for the "r":
[...]
>        xft:-bitstream-Bitstream Vera Sans Mono-normal-normal-normal-*-20-*-*-*-m-0-iso10646-1 (#x55)

In your file, `ü" is acually not a signle character but two
characters `u' and U+308 (COMBINING DIAERESIS), and it seems
that the above bitstream font doesn't contain a glyph of
U+308.  So, Emacs searches for a font that has that glyph.
The found font in your case was "American Typewriter".

It may be good that Emacs knows that `u'+U+308 = `ü', but
that kind of normalization is not yet supported.

---
Kenichi Handa
handa@m17n.org

bug#3325: 23.0.93; Unexpected font for composed character

[Alan Third]

Kenichi Handa <handa@m17n.org> writes:

> In article <m2skj2s2w7.fsf@gmx.at>, Markus Triska <markus.triska@gmx.at> writes:
>
>> I have a file ~/Downloads/Büroanwendungen.zip. When I visit ~/Downloads/
>> in dired (C-x d ~/Downloads/ RET) and press C-u x = on the "ü", I get:
>
>>            character: u (117, #o165, #x75)
>>    preferred charset: ascii (ASCII (ISO646 IRV))
>>           code point: 0x75
>>               syntax: w 	which means: word
>>             category: .:Base, a:ASCII, l:Latin, r:Roman
>>          buffer code: #x75
>>            file code: #x75 (encoded by coding system utf-8-unix)
>>              display: composed to form "ü" (see below)
>
>>    Composed with the following character(s) "̈" using this font:
>>      xft:-itc-American Typewriter-normal-normal-normal-*-20-*-*-*-*-0-iso10646-1
> [...]
>> This font differs unexpectedly (for me) from the one used for the "r":
> [...]
>>        xft:-bitstream-Bitstream Vera Sans Mono-normal-normal-normal-*-20-*-*-*-m-0-iso10646-1 (#x55)
>
> In your file, `ü" is acually not a signle character but two
> characters `u' and U+308 (COMBINING DIAERESIS), and it seems
> that the above bitstream font doesn't contain a glyph of
> U+308.  So, Emacs searches for a font that has that glyph.
> The found font in your case was "American Typewriter".

This still happens in Emacs 25, but from the above description it
doesn't really sound like a bug. It's Emacs working around the fact that
the font doesn't have the glyph.

> It may be good that Emacs knows that `u'+U+308 = `ü', but
> that kind of normalization is not yet supported.

I'm changing this bug report to wishlist.
-- 
Alan Third

bug#3325: 23.0.93; Unexpected font for composed character

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 624 bytes --]

Hi!

In Dired, this should not happen any more (on OS X, using the "ns"
interface). Emacs is now using a file-name coding system (utf-8-hfs) that
converts the file names to non-decomposed names, before presenting them to
the user.

However, in other contexts where decomposed characters are used, the
problem is still present. The problem seems to be font-related. I tested
all my installed fonts using `ns-popup-font-panel', and only a handful of
them worked, mainly those that were provided by the OS: Menlo, Monaco,
Courier, Courier New. Included in the fonts that didn't work were Vera and
Hack.

    -- Anders Lindgren

[-- Attachment #2: Type: text/html, Size: 746 bytes --]

bug#3325: 23.0.93; Unexpected font for composed character

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

> This still happens in Emacs 25, but from the above description it
> doesn't really sound like a bug. It's Emacs working around the fact that
> the font doesn't have the glyph.
>
>> It may be good that Emacs knows that `u'+U+308 = `ü', but
>> that kind of normalization is not yet supported.
>
> I'm changing this bug report to wishlist.

If I try this (i.e., load a file with ü in it, or say (insert ?u
#x308)), I get the following:

             position: 1 of 2 (0%), column: 0
            character: u (displayed as u) (codepoint 117, #o165, #x75)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x75
               script: latin
               syntax: w 	which means: word
             category: .:Base, L:Left-to-right (strong), a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 75" or "C-x 8 RET LATIN SMALL LETTER U"
          buffer code: #x75
            file code: #x75 (encoded by coding system utf-8)
              display: composed to form "ü" (see below)

Composed with the following character(s) "̈" using this font:
  xfthb:-PfEd-DejaVu Sans Mono-normal-normal-normal-*-25-*-*-*-m-0-iso10646-1
by these glyphs:
  [0 1 117 190 15 2 13 19 0 nil]

Character code properties: customize what to show
  name: LATIN SMALL LETTER U
  general-category: Ll (Letter, Lowercase)
  decomposition: (117) ('u')

And the display looks correct.  So it seems like this has been fixed
now?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#3325: 23.0.93; Unexpected font for composed character

[Stefan Kangas]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Alan Third <alan@idiocy.org> writes:
>
>> This still happens in Emacs 25, but from the above description it
>> doesn't really sound like a bug. It's Emacs working around the fact that
>> the font doesn't have the glyph.
>>
>>> It may be good that Emacs knows that `u'+U+308 = `ü', but
>>> that kind of normalization is not yet supported.
>>
>> I'm changing this bug report to wishlist.
>
> If I try this (i.e., load a file with ü in it, or say (insert ?u
> #x308)), I get the following:
>
>              position: 1 of 2 (0%), column: 0
>             character: u (displayed as u) (codepoint 117, #o165, #x75)
>               charset: ascii (ASCII (ISO646 IRV))
> code point in charset: 0x75
>                script: latin
>                syntax: w 	which means: word
>              category: .:Base, L:Left-to-right (strong), a:ASCII, l:Latin, r:Roman
>              to input: type "C-x 8 RET 75" or "C-x 8 RET LATIN SMALL LETTER U"
>           buffer code: #x75
>             file code: #x75 (encoded by coding system utf-8)
>               display: composed to form "ü" (see below)
>
> Composed with the following character(s) "̈" using this font:
>   xfthb:-PfEd-DejaVu Sans Mono-normal-normal-normal-*-25-*-*-*-m-0-iso10646-1
> by these glyphs:
>   [0 1 117 190 15 2 13 19 0 nil]
>
> Character code properties: customize what to show
>   name: LATIN SMALL LETTER U
>   general-category: Ll (Letter, Lowercase)
>   decomposition: (117) ('u')
>
> And the display looks correct.  So it seems like this has been fixed
> now?

Since there has been no further update here within 40 weeks, I'm going
to assume yes and close this bug report.

If this conclusion is incorrect, please reply to this email (use "Reply
to all" in your email client) and we can reopen the bug report.

Best regards,
Stefan Kangas