Font selection weirdness

Font selection weirdness

[Juanma Barranquero]

In my setup, there are a few characters in etc/HELLO (four
mathematical symbols and two extended latin chars from vietnamese, at
least) which are not displayed; I mean, they appear as whitespace, not
the "no font available" empty box. However, describe-char says that
there's a font for them:

--------------------------------------------------------------------------------
        character: ∀ (8704, #o21000, #x2200)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x2200
           syntax: . 	which means: punctuation
         category: h:Korean j:Japanese
      buffer code: #xE2 #x88 #x80
        file code: ESC #x24 #x42 #x22 #x4F (encoded by coding system
iso-2022-7bit-dos)
          display: by this font (glyph code)
     -outline-Symbol-normal-normal-normal-decorative-13-*-*-*-p-*-ms-symbol
(#x2200)

Character code properties: customize what to show
  name: FOR ALL
  general-category: Sm (Symbol, Math)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: ON (Other Neutrals)
  mirrored: N

There are text properties here:
  auto-composed        t
  charset              japanese-jisx0208
--------------------------------------------------------------------------------
        character: ∈ (8712, #o21010, #x2208)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x2208
           syntax: . 	which means: punctuation
         category: c:Chinese h:Korean j:Japanese
      buffer code: #xE2 #x88 #x88
        file code: ESC #x24 #x41 #x21 #x4A (encoded by coding system
iso-2022-7bit-dos)
          display: by this font (glyph code)
     -outline-Symbol-normal-normal-normal-decorative-13-*-*-*-p-*-ms-symbol
(#x2208)

Character code properties: customize what to show
  name: ELEMENT OF
  general-category: Sm (Symbol, Math)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: ON (Other Neutrals)
  mirrored: Y

There are text properties here:
  auto-composed        t
  charset              chinese-gb2312
--------------------------------------------------------------------------------
        character: • (8226, #o20042, #x2022)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x2022
           syntax: _ 	which means: symbol
         category: j:Japanese
      buffer code: #xE2 #x80 #xA2
        file code: ESC #x24 #x28 #x4F #x23 #x40 (encoded by coding
system iso-2022-7bit-dos)
          display: by this font (glyph code)
     -outline-Symbol-normal-normal-normal-decorative-13-*-*-*-p-*-ms-symbol
(#x2022)

Character code properties: customize what to show
  name: BULLET
  general-category: Po (Punctuation, Other)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: ON (Other Neutrals)
  mirrored: N

There are text properties here:
  auto-composed        t
  charset              japanese-jisx0213-1
--------------------------------------------------------------------------------
        character: □ (9633, #o22641, #x25a1)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x25A1
           syntax: _ 	which means: symbol
         category: c:Chinese h:Korean j:Japanese
      buffer code: #xE2 #x96 #xA1
        file code: ESC #x24 #x41 #x21 #x75 (encoded by coding system
iso-2022-7bit-dos)
          display: by this font (glyph code)
     -outline-Symbol-normal-normal-normal-decorative-13-*-*-*-p-*-ms-symbol
(#x25A1)

Character code properties: customize what to show
  name: WHITE SQUARE
  general-category: So (Symbol, Other)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: ON (Other Neutrals)
  mirrored: N

There are text properties here:
  auto-composed        t
  charset              chinese-gb2312
--------------------------------------------------------------------------------
        character: ế (7871, #o17277, #x1ebf)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x1EBF
           syntax: w 	which means: word
         category: l:Latin v:Vietnamese
      buffer code: #xE1 #xBA #xBF
        file code: ESC #x2C #x31 #x2A (encoded by coding system
iso-2022-7bit-dos)
          display: by this font (glyph code)
     -outline-DejaVu Sans
Mono-normal-normal-normal-mono-13-*-*-*-m-*-iso8859-5 (#x03)

Character code properties: customize what to show
  name: LATIN SMALL LETTER E WITH CIRCUMFLEX AND ACUTE
  general-category: Ll (Letter, Lowercase)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: L (Left-to-Right)
  decomposition: (234 769) ('ê' '́')
  mirrored: N
  uppercase: 7870 (Ế)
  titlecase: 7870 (Ế)

There are text properties here:
  auto-composed        t
  charset              vietnamese-viscii-lower
--------------------------------------------------------------------------------
        character: ệ (7879, #o17307, #x1ec7)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x1EC7
           syntax: w 	which means: word
         category: l:Latin v:Vietnamese
      buffer code: #xE1 #xBB #x87
        file code: ESC #x2C #x31 #x2E (encoded by coding system
iso-2022-7bit-dos)
          display: by this font (glyph code)
     -outline-DejaVu Sans
Mono-normal-normal-normal-mono-13-*-*-*-m-*-iso8859-5 (#x03)

Character code properties: customize what to show
  name: LATIN SMALL LETTER E WITH CIRCUMFLEX AND DOT BELOW
  general-category: Ll (Letter, Lowercase)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: L (Left-to-Right)
  decomposition: (7865 770) ('ẹ' '̂')
  mirrored: N
  uppercase: 7878 (Ệ)
  titlecase: 7878 (Ệ)

There are text properties here:
  auto-composed        t
  charset              vietnamese-viscii-lower
--------------------------------------------------------------------------------



If I type these characters in *scratch*, via ucs-insert, three of them
are displayed correctly (U+2208, U+2022 and U+25A1), and the other
three appear as empty boxes (though Emacs still thinks it has a font
for them).

This is what describe-char says for the three non-visible characters
in *scratch*:



--------------------------------------------------------------------------------
        character: ∀ (8704, #o21000, #x2200)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x2200
           syntax: . 	which means: punctuation
         category: h:Korean j:Japanese
      buffer code: #xE2 #x88 #x80
        file code: #xE2 #x88 #x80 (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
     -outline-DejaVu Sans
Mono-normal-normal-normal-mono-13-*-*-*-m-*-iso10646-1 (#x2200)

Character code properties: customize what to show
  name: FOR ALL
  general-category: Sm (Symbol, Math)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: ON (Other Neutrals)
  mirrored: N

There are text properties here:
  auto-composed        t
  fontified            t
--------------------------------------------------------------------------------
        character: ∈ (8712, #o21010, #x2208)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x2208
           syntax: . 	which means: punctuation
         category: c:Chinese h:Korean j:Japanese
      buffer code: #xE2 #x88 #x88
        file code: #xE2 #x88 #x88 (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
     -outline-DejaVu Sans
Mono-normal-normal-normal-mono-13-*-*-*-m-*-iso10646-1 (#x2208)

Character code properties: customize what to show
  name: ELEMENT OF
  general-category: Sm (Symbol, Math)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: ON (Other Neutrals)
  mirrored: Y

There are text properties here:
  auto-composed        t
  fontified            t
--------------------------------------------------------------------------------
        character: • (8226, #o20042, #x2022)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x2022
           syntax: _ 	which means: symbol
         category: j:Japanese
      buffer code: #xE2 #x80 #xA2
        file code: #xE2 #x80 #xA2 (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
     -outline-DejaVu Sans
Mono-normal-normal-normal-mono-13-*-*-*-m-*-iso10646-1 (#x2022)

Character code properties: customize what to show
  name: BULLET
  general-category: Po (Punctuation, Other)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: ON (Other Neutrals)
  mirrored: N

There are text properties here:
  auto-composed        t
  fontified            t
--------------------------------------------------------------------------------
        character: □ (9633, #o22641, #x25a1)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x25A1
           syntax: _ 	which means: symbol
         category: c:Chinese h:Korean j:Japanese
      buffer code: #xE2 #x96 #xA1
        file code: #xE2 #x96 #xA1 (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
     -outline-DejaVu Sans
Mono-normal-normal-normal-mono-13-*-*-*-m-*-iso10646-1 (#x25A1)

Character code properties: customize what to show
  name: WHITE SQUARE
  general-category: So (Symbol, Other)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: ON (Other Neutrals)
  mirrored: N

There are text properties here:
  auto-composed        t
  fontified            t
--------------------------------------------------------------------------------
        character: ế (7871, #o17277, #x1ebf)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x1EBF
           syntax: w 	which means: word
         category: l:Latin v:Vietnamese
      buffer code: #xE1 #xBA #xBF
        file code: #xE1 #xBA #xBF (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
     -outline-DejaVu Sans
Mono-normal-normal-normal-mono-13-*-*-*-m-*-iso10646-1 (#x1EBF)

Character code properties: customize what to show
  name: LATIN SMALL LETTER E WITH CIRCUMFLEX AND ACUTE
  general-category: Ll (Letter, Lowercase)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: L (Left-to-Right)
  decomposition: (234 769) ('ê' '́')
  mirrored: N
  uppercase: 7870 (Ế)
  titlecase: 7870 (Ế)

There are text properties here:
  auto-composed        t
  fontified            t
--------------------------------------------------------------------------------
        character: ệ (7879, #o17307, #x1ec7)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x1EC7
           syntax: w 	which means: word
         category: l:Latin v:Vietnamese
      buffer code: #xE1 #xBB #x87
        file code: #xE1 #xBB #x87 (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
     -outline-DejaVu Sans
Mono-normal-normal-normal-mono-13-*-*-*-m-*-iso10646-1 (#x1EC7)

Character code properties: customize what to show
  name: LATIN SMALL LETTER E WITH CIRCUMFLEX AND DOT BELOW
  general-category: Ll (Letter, Lowercase)
  canonical-combining-class: 0 (Spacing, split, enclosing, reordrant,
and Tibetan subjoined)
  bidi-class: L (Left-to-Right)
  decomposition: (7865 770) ('ẹ' '̂')
  mirrored: N
  uppercase: 7878 (Ệ)
  titlecase: 7878 (Ệ)

There are text properties here:
  auto-composed        t
  fontified            t
--------------------------------------------------------------------------------



 Juanma

Re: Font selection weirdness

[Juanma Barranquero]

> This is what describe-char says for the three non-visible characters
> in *scratch*:

Sorry. I wrote that, and then decided to paste info for all six chars.

             Juanma

Re: Font selection weirdness

[Jason Rumney]

Juanma Barranquero wrote:
> --------------------------------------------------------------------------------
>         character: ế (7871, #o17277, #x1ebf)
> preferred charset: unicode (Unicode (ISO10646))
>        code point: 0x1EBF
>            syntax: w 	which means: word
>          category: l:Latin v:Vietnamese
>       buffer code: #xE1 #xBA #xBF
>         file code: ESC #x2C #x31 #x2A (encoded by coding system
> iso-2022-7bit-dos)
>           display: by this font (glyph code)
>      -outline-DejaVu Sans
> Mono-normal-normal-normal-mono-13-*-*-*-m-*-iso8859-5 (#x03)
>   

The choice of iso8859-5 is strange, but it doesn't actually matter on 
Windows, as all truetype fonts are treated as Unicode, and the different 
character sets reported as supported for a font are only to assist the 
font matching in finding the right fonts. It may just be that this font 
was first opened when displaying a Cyrillic character, so subsequent 
uses reflect that.

Character at glyph index 3 in most fonts is space, which is consistent 
with what you see, but it is strange that DejaVu Sans Mono maps this 
character to that and not glyph index 0 (.notdef) if it doesn't have a 
glyph  for it.


> If I type these characters in *scratch*, via ucs-insert, three of them
> are displayed correctly (U+2208, U+2022 and U+25A1), and the other
> three appear as empty boxes (though Emacs still thinks it has a font
> for them).
>   

>         character: ∀ (8704, #o21000, #x2200)
> preferred charset: unicode (Unicode (ISO10646))
>        code point: 0x2200
>            syntax: . 	which means: punctuation
>          category: h:Korean j:Japanese
>       buffer code: #xE2 #x88 #x80
>         file code: #xE2 #x88 #x80 (encoded by coding system utf-8-unix)
>           display: by this font (glyph code)
>      -outline-DejaVu Sans Mono-normal-normal-normal-mono-13-*-*-*-m-*-iso10646-1 (#x2200)
>   

Now the same font is being used (but with a iso10646-1 charset, so 
probably a different font_object pointing to the same font). But this 
time, it seems unicode code points are being used instead of glyph 
index, which indicates that an error has been returned by 
GetCharacterPlacementW or GetGlyphOutlineW.

To see why the error is occuring, set a breakpoint on 
clear_cached_metrics and when it is hit, go back up the stack (in gdb: 
frame 1) to see what function it has failed from. Then get information 
such as the return code of the failing function, the character or glyph 
code that it is failing on, and the font, tyo help diagnose why it has 
failed.

Re: Font selection weirdness

[Juanma Barranquero]

> To see why the error is occuring, set a breakpoint on clear_cached_metrics
> and when it is hit, go back up the stack (in gdb: frame 1) to see what
> function it has failed from. Then get information such as the return code of
> the failing function, the character or glyph code that it is failing on, and
> the font, tyo help diagnose why it has failed.

Inserting the problematic chars into *scratch* does not trigger
clear_cached_metrics. C-h h does trigger it, but I suppose that's
reasonable because there are many characters in etc/HELLO for which I
do not have a suitable font.

Anything else I can try?

             Juanma

Re: Font selection weirdness

[Jason Rumney]

Juanma Barranquero wrote:
> Inserting the problematic chars into *scratch* does not trigger
> clear_cached_metrics. C-h h does trigger it, but I suppose that's
> reasonable because there are many characters in etc/HELLO for which I
> do not have a suitable font.
>   
No, it's not reasonable. If you don't have a suitable font, then Emacs 
should not be trying to display those characters.

Re: Font selection weirdness

[Juanma Barranquero]

> No, it's not reasonable. If you don't have a suitable font, then Emacs
> should not be trying to display those characters.

I'll try to debug it. But, are you saying that clear_cached_metrics
should never be called, or that it should not be called for
TrueType/OpenFont fonts?

Meanwhile, a curious thing: if, on the HELLO buffer, I remove the
charset property in the "Mathematics" line, the characters are
displayed right. (Well, strictly speaking U+2200 FOR ALL is not
displayed because DejaVu Sans Mono has no glyph for it, but at least
it does correctly appear as an empty box).

 Juanma

Re: Font selection weirdness

[Jason Rumney]

Juanma Barranquero wrote:
>> No, it's not reasonable. If you don't have a suitable font, then Emacs
>> should not be trying to display those characters.
>>     
>
> I'll try to debug it. But, are you saying that clear_cached_metrics
> should never be called, or that it should not be called for
> TrueType/OpenFont fonts?
>   

I thought I'd initialised things so that bitmap fonts would default to 
using unicode code points without needing to invalidate the metric cache 
when we change our mind later, but it turns out I hadn't. I made a 
change today that should mean that the function should never be called 
on Windows 2000/XP/Vista, unless there is a bug in the font drawing that 
continues to draw after finding the character is not supported, or a 
buggy truetype font that uses a glyph index other than 0 to indicate 
that a character is not supported.

Re: Font selection weirdness

[Juanma Barranquero]

On Tue, May 27, 2008 at 6:10 PM, Jason Rumney <jasonr@gnu.org> wrote:

> I made a change today that
> should mean that the function should never be called on Windows
> 2000/XP/Vista, unless there is a bug in the font drawing that continues to
> draw after finding the character is not supported, or a buggy truetype font
> that uses a glyph index other than 0 to indicate that a character is not
> supported.

With the current CVS, clear_cached_metrics is being called every time
I try to display etc/HELLO on my work computer (it doesn't happen in
my laptop, but the installed fonts are quite different). I'll try to
debug it to find what's happening.

 Juanma

Re: Font selection weirdness

[Juanma Barranquero]

On Mon, May 26, 2008 at 2:36 PM, Jason Rumney <jasonr@gnu.org> wrote:

> To see why the error is occuring, set a breakpoint on clear_cached_metrics
> and when it is hit, go back up the stack (in gdb: frame 1) to see what
> function it has failed from. Then get information such as the return code of
> the failing function, the character or glyph code that it is failing on, and
> the font, tyo help diagnose why it has failed.

I'm getting this backtrace (the build is optimized; I'll rebuild if needed):

#1  0x01202aa0 in clear_cached_metrics (w32_font=0x1f93300) at w32font.c:1899
#2  0x012072c0 in w32font_text_extents (font=0x1f93300, code=0x82deb0,
nglyphs=1, metrics=0x142fdb8) at w32font.c:1885
#3  0x01024116 in get_per_char_metric (f=0x190cc00, font=0x1f93300,
char2b=0x82dfd6) at xdisp.c:19453
#4  0x0104c538 in x_produce_glyphs (it=0x82e0c4) at xdisp.c:21000
#5  0x0103a5bd in display_line (it=0x82e0c4) at xdisp.c:16295
#6  0x0103e954 in try_window (window=40862212, pos={charpos = 1,
bytepos = 1}, check_margins=1) at xdisp.c:13832
#7  0x01052be7 in redisplay_window (window=40862212,
just_this_one_p=0) at xdisp.c:13450
#8  0x01054ca1 in redisplay_window_0 (window=40862212) at xdisp.c:12041
#9  0x01018c51 in internal_condition_case_1 (bfun=0x1054c7e
<redisplay_window_0>, arg=40862212, handlers=25687301,
    hfun=0x1023c5a <redisplay_window_error>) at eval.c:1554
#10 0x0102d71d in redisplay_windows (window=33108736) at xdisp.c:12020
#11 0x010461ed in redisplay_internal (preserve_echo_area=1) at xdisp.c:11586
#12 0x01088249 in read_char (commandflag=1, nmaps=3, maps=0x82fb70,
prev_event=25704449, used_mouse_menu=0x82fc34,
    end_time=0x0) at keyboard.c:2685
#13 0x0108c89a in read_key_sequence (keybuf=0x82fcd4, bufsize=30,
prompt=25704449, dont_downcase_last=0,
    can_return_switch_frame=1, fix_current_buffer=1) at keyboard.c:9443
#14 0x0108fa0d in command_loop_1 () at keyboard.c:1651
#15 0x01018ed6 in internal_condition_case (bfun=0x108f77f
<command_loop_1>, handlers=25768129,
    hfun=0x10869e6 <cmd_error>) at eval.c:1506
#16 0x01085cef in command_loop_2 () at keyboard.c:1367
#17 0x01018f80 in internal_catch (tag=25764201, func=0x1085ccc
<command_loop_2>, arg=25704449) at eval.c:1242
#18 0x0108682b in command_loop () at keyboard.c:1346
#19 0x01086b7f in recursive_edit_1 () at keyboard.c:955
#20 0x01086cea in Frecursive_edit () at keyboard.c:1017
#21 0x01002c3c in main (argc=8585136, argv=0xa941f8) at emacs.c:1770

So the clear_cached_metrics call that triggers is the one in
compute_metrics (which has been optimized away).

In frame 1, w32_font has

$14 = {
  font = {
    size = 1075838994,
    next = 0x2071440,
    props = {25990081, 25992433, 30970217, 25992313, 31041897, 205440,
205056, 205312, 104, 25704449, 0, 25704449,
      49267133, 49267181, 31652435, 31652419, 25704449, 31041969},
    max_width = 14,
    pixel_size = 13,
    height = 17,
    space_width = 7,
    average_width = 7,
    min_width = 7,
    ascent = 13,
    descent = 4,
    underline_thickness = 1,
    underline_position = 1,
    vertical_centering = 0,
    encoding_type = 0 '\0',
    baseline_offset = 0,
    relative_compose = 0,
    default_ascent = 13,
    font_encoder = 0x0,
    driver = 0x137d2c0,
    encoding_charset = -1,
    repertory_charset = -1
  },
  metrics = {
    tmHeight = 17,
    tmAscent = 13,
    tmDescent = 4,
    tmInternalLeading = 4,
    tmExternalLeading = 0,
    tmAveCharWidth = 7,
    tmMaxCharWidth = 14,
    tmWeight = 400,
    tmOverhang = 0,
    tmDigitizedAspectX = 1001,
    tmDigitizedAspectY = 1001,
    tmFirstChar = 32 ' ',
    tmLastChar = 255 '\377',
    tmDefaultChar = 32 ' ',
    tmBreakChar = 32 ' ',
    tmItalic = 0 '\0',
    tmUnderlined = 0 '\0',
    tmStruckOut = 0 '\0',
    tmPitchAndFamily = 91 '[',
    tmCharSet = 2 '\002'
  },
  glyph_idx = 0,
  cached_metrics = 0x1aaa600,
  n_cache_blocks = 69,
  compat_w32_font = 0x2f09400
}

where the props are

type: gdi
foundry: outline
family: Symbol
additional style: decorative
registry: ms-symbol
weight: 25680
slant: 25632
width: 25664
size: 13
dpi: nil
spacing: 0
average width: nil
extra: ((:format . type1))
font objects: (#<font-object
"-outline-Symbol-normal-normal-normal-decorative-13-*-*-*-p-*-ms-symbol">)
name: "-outline-Symbol-normal-normal-normal-decorative-13-*-*-*-p-*-ms-symbol"
full name: "Symbol-10.0"
file: nil
format: type1

A weird thing: w32_font->font.glyph_index is 16 when entering
compute_metrics, however code = 8704 (U+2200), which is not a glyph.
Is that correct?

The call to GetGlyphOutlineW returns GDI_ERROR, but GetLastError () == 0.

Do you need additional info?

   Juanma

Re: Font selection weirdness

[Jason Rumney]

Juanma Barranquero wrote:

> where the props are [abbreviated]
>
> type: gdi
> foundry: outline
> family: Symbol
> name: "-outline-Symbol-normal-normal-normal-decorative-13-*-*-*-p-*-ms-symbol"
> format: type1
>   

On my system, Symbol shows up as an Opentype font, but it does not 
appear to contain a Unicode mapping table - the character set selector 
in Windows Character Map is greyed out, and selecting the "FOR ALL" 
character in Character Map (0x22) and copying it and pasting elsewhere 
results in ASCII 0x22 (") being pasted.
If I can figure out a way to determine such fonts when enumerating we 
can avoid tagging them as supporting Unicode and glyph points, the same 
way we do now with bitmap fonts.

> A weird thing: w32_font->font.glyph_index is 16 when entering
> compute_metrics, however code = 8704 (U+2200), which is not a glyph.
> Is that correct?
>   

I think that is due to the same problems as above. Perhaps 
GetCharacterPlacementW in w32font_encode_char did not return an error as 
expected, and just silently copied the Unicode code point to the glyph 
code point. Another posibility is that w32font_encode_char has not been 
called for that character yet, which would be a bug somewhere in Emacs.

Re: Font selection weirdness

[Juanma Barranquero]

On Thu, May 29, 2008 at 1:36 PM, Jason Rumney <jasonr@gnu.org> wrote:

> On my system, Symbol shows up as an Opentype font, but it does not appear to
> contain a Unicode mapping table - the character set selector in Windows
> Character Map is greyed out, and selecting the "FOR ALL" character in
> Character Map (0x22) and copying it and pasting elsewhere results in ASCII
> 0x22 (") being pasted.

Once removed the Adobe Type 1 Symbol font, the problem disappears. The
0x22 "FOR ALL" in the OpenType Symbol font does not preclude Emac
selecting a suitable U+2200 in DejaVu Sans Mono.

In fact, removing all Type 1 fonts fixes all three problems I was
having with font selection:

  - The incorrect display of space for some missing characters (i.e.,
the problem in this thread)
  - The garbage in the startup screen in place of "Meta-x recover-session RET"
  - The screwing of *all* glyphs when visiting HELLO using the default
startup font.

> I think that is due to the same problems as above. Perhaps
> GetCharacterPlacementW in w32font_encode_char did not return an error as
> expected, and just silently copied the Unicode code point to the glyph code
> point. Another posibility is that w32font_encode_char has not been called
> for that character yet, which would be a bug somewhere in Emacs.

I'll try to debug it over the weekend; perhaps I can find some clue.

   Juanma

Re: Font selection weirdness

[Juanma Barranquero]

On Thu, May 29, 2008 at 5:19 PM, Juanma Barranquero <lekktu@gmail.com> wrote:

> In fact, removing all Type 1 fonts fixes all three problems I was
> having with font selection:
>
>  - The incorrect display of space for some missing characters (i.e.,
> the problem in this thread)

Well, removing the Type 1 fonts and compiling with your fixes has in
fact been a great help, but I still see some weirdness.

- I start Emacs using my preferred font, DejaVu Sans Mono-10.

- I display HELLO: the U+2200 char is shown correctly.

- I cut&paste it to *scratch*. Still correct. describe-char says:

        character: ∀ (8704, #o21000, #x2200)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x2200
           syntax: . 	which means: punctuation
         category: h:Korean j:Japanese
      buffer code: #xE2 #x88 #x80
        file code: not encodable by coding system iso-latin-1-dos
          display: by this font (glyph code)
     -outline-DejaVu
Sans-normal-normal-normal-sans-13-*-*-*-p-*-ms-symbol (#xC28)

There are text properties here:
  auto-composed        t
  charset              japanese-jisx0208
  fontified            t
  rear-nonsticky       t

- I insert another U+2200, with ucs-insert. Shown as space (*not* as
an empty box). describe-char for this U+2200 says:

        character: ∀ (8704, #o21000, #x2200)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x2200
           syntax: . 	which means: punctuation
         category: h:Korean j:Japanese
      buffer code: #xE2 #x88 #x80
        file code: not encodable by coding system iso-latin-1-dos
          display: by this font (glyph code)
     -outline-DejaVu Sans
Mono-normal-normal-normal-mono-13-*-*-*-c-*-iso10646-1 (#x03)

There are text properties here:
  auto-composed        t
  fontified            t

The obvious differences are the charset text property, and that the
correct FOR ALL is found in DejaVu Sans (ms-symbol), while the
"spaced" one comes from DejaVu Sans Mono (iso10646-1). Interestingly
enough, clear_cached_metrics is not being called.

Now, if I repeat the test with the default fontset (Emacs starts with
Courier New), the following differences happen:

 - clear_cached_metrics is triggered when displaying HELLO, though it
is for another character (U+0AA0) AFAICS.

 - The cut&pasted FOR ALL is still correct, and describe-char output
is like above.

 - The FOR ALL inserted with ucs-insert is shown as an empty box
(*not* space). Its describe-char output is this:

        character: ∀ (8704, #o21000, #x2200)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x2200
           syntax: . 	which means: punctuation
         category: h:Korean j:Japanese
      buffer code: #xE2 #x88 #x80
        file code: not encodable by coding system iso-latin-1-dos
          display: by this font (glyph code)
     -outline-Courier
New-normal-normal-normal-mono-13-*-*-*-c-*-iso10646-1 (#x2200)

There are text properties here:
  auto-composed        t
  fontified            t

Hope this helps,

 Juanma