bug#3174: NS font selection still broken

bug#3174: NS font selection still broken

[David Reitter]

Adrian, readers of bugs 3174 and 3588,

There are still some bad issues with the font selection in the NS port.

For example, take the following unicode characters:

[∃]  2203  THERE EXISTS
[∄]  2204  THERE DOES NOT EXIST
[∈]  2208  ELEMENT OF

They are rendered perfectly in the Carbon port (at least in Aquamacs).

In NS, it uses Symbol for 2203, even though the glyph is available in  
the default font (like Monaco or Lucida Grande).
As a consequence, the symbol looks very small and is unreadable.

As for 2203, it also uses Symbol, even though Symbol doesn't even have  
the glyph.

This is on Snow Leopard.

I'm wondering if this is this the same bug as #3588.  I would write to  
emacs-devel to ask for help, but I don't even know what exactly the  
problem is in the underlying code, and whether this is NS specific (I  
guess so)...   I can happily file another bug report so we've got  
something to ignore :)

Let me know what if I can help.

Thanks
- David

bug#3174: NS font selection still broken

[Adrian Robert]

Hi,

For debugging it will be more useful to compare with another port on  
Emacs-23 (e.g., X11) than Carbon Emacs 22, as character rendering and  
font selection was changed completely.

For differences from w32 or x11 that show up on Emacs 23, font  
selection is handled through ns_findfonts() in nsfont.m.  The rewrite  
of a few months ago brought this mostly in line with other ports, but  
some crucial differences may still remain (though I'm not sure what  
they are).  I'm afraid it will take a fairly significant effort at  
turning on tracing in that file (NSFONT_TRACE at the top), and working  
through the logic of what font.c is doing and what nsfont.m is doing,  
for the characters in question.  I can't think of any shortcuts, and I  
have to admit these mathematical symbol ranges have long been a  
problem eluding my efforts, though I believe the rewrite lessened the  
number of erroneously-handled characters.

The actual "bugs" could be either in nsfont.m or font.c -- sometimes  
in the past nsfont's different implementation sometimes revealed  
problems in font.c that did not affect other ports.  (Of course the  
nsfont.m impl should continue to be normalized to the other ports so  
such bugs can remain mercifully hidden.)

I'm not sure #3588 is related, as the 'italic' property is a confound  
there.

-Adrian



On Oct 29, 2009, at 2:18 PM, David Reitter wrote:

> Adrian, readers of bugs 3174 and 3588,
>
> There are still some bad issues with the font selection in the NS  
> port.
>
> For example, take the following unicode characters:
>
> [∃]  2203  THERE EXISTS
> [∄]  2204  THERE DOES NOT EXIST
> [∈]  2208  ELEMENT OF
>
> They are rendered perfectly in the Carbon port (at least in Aquamacs).
>
> In NS, it uses Symbol for 2203, even though the glyph is available  
> in the default font (like Monaco or Lucida Grande).
> As a consequence, the symbol looks very small and is unreadable.
>
> As for 2203, it also uses Symbol, even though Symbol doesn't even  
> have the glyph.
>
> This is on Snow Leopard.
>
> I'm wondering if this is this the same bug as #3588.  I would write  
> to emacs-devel to ask for help, but I don't even know what exactly  
> the problem is in the underlying code, and whether this is NS  
> specific (I guess so)...   I can happily file another bug report so  
> we've got something to ignore :)
>
> Let me know what if I can help.
>
> Thanks
> - David

bug#3174: NS font selection still broken

[David Reitter]

Adrian,

> For differences from w32 or x11 that show up on Emacs 23, font  
> selection is handled through ns_findfonts() in nsfont.m.  The  
> rewrite of a few months ago brought this mostly in line with other  
> ports, but some crucial differences may still remain (though I'm not  
> sure what they are).  I'm afraid it will take a fairly significant  
> effort at turning on tracing in that file (NSFONT_TRACE at the top),  
> and working through the logic of what font.c is doing and what  
> nsfont.m is doing, for the characters in question.  I can't think of  
> any shortcuts, and I have to admit these mathematical symbol ranges  
> have long been a problem eluding my efforts, though I believe the  
> rewrite lessened the number of erroneously-handled characters.
>
> The actual "bugs" could be either in nsfont.m or font.c -- sometimes  
> in the past nsfont's different implementation sometimes revealed  
> problems in font.c that did not affect other ports.  (Of course the  
> nsfont.m impl should continue to be normalized to the other ports so  
> such bugs can remain mercifully hidden.)

I've been looking at the trace from nsfont and, more importantly, ` 
(reverse font-log)' from font.c.

To display [“] 201C  LEFT DOUBLE QUOTATION MARK, we get the log below  
(after adding a few items to log).

At first (top of list), it tries to find "Apple Lucida Grande" and  
"Lucida Grande" with a particular "registry" and for the symbol  
script.  This fails, because the `val' variable in font_list_entities  
ends up being Qnil.

I don't quite see, why.  Possibilities are that the [part of the] font  
isn't loaded and/or caching fails, but I don't really understand what  
the code after the call to font_get_cache is supposed to be doing.    
Can you see the problem?  If not, feel free to say so and I'll ask - 
devel.

I did try adjusting script-representative-chars to make sure that the  
example characters listed for `symbol' are all in Lucida.  This had no  
effect.

Thanks
- D




((default\ fontset:\ font\ for 8220 nil)
  (finding nil nil)
  (ASIZE\ \(val\)\ =<\ 0\ not\ adding\ to\ list
   []
   nil)
  (list "-apple-Lucida Grande-*-iso10646-1:script=symbol" nil)
  (font_list_entities\ returned\ nil nil nil)
  (ASIZE\ \(val\)\ =<\ 0\ not\ adding\ to\ list
   []
   nil)
  (list "-*-Lucida Grande-*-iso10646-1:script=symbol" nil)
  (font_list_entities\ returned\ nil nil nil)
  (ASIZE\ \(val\)\ >\ 0\ adding\ to\ list
   [#<font-entity ns apple Apple_Symbols nil iso10646-1 medium normal  
normal 0 nil 0 0
		 ((:script . symbol))
		 > #<font-entity ns apple Apple_Symbols nil iso10646-1 medium normal  
normal 0 nil 0 0
		 ((:script . symbol))
		 > #<font-entity ns apple MS_PMincho nil iso10646-1 medium normal  
normal 0 nil 0 0
		 ((:script . symbol))
		 > #<font-entity ns apple MS_PGothic nil iso10646-1 medium normal  
normal 0 nil 0 0
		 ((:script . symbol))
		 > #<font-entity ns apple MS_Mincho nil iso10646-1 medium normal  
normal 0 nil 0 0
		 ((:script . symbol))
		 > #<font-entity ns apple MS_Gothic nil iso10646-1 medium normal  
normal 0 nil 0 0
		 ((:script . symbol))
		 > #<font-entity ns apple Menlo nil iso10646-1 bold italic normal 0  
nil 100 0
		 ((:script . symbol))
		 > #<font-entity ns apple Menlo nil iso10646-1 bold normal normal 0  
nil 100 0
		 ((:script . symbol))
		 > #<font-entity ns apple Menlo nil iso10646-1 medium italic normal  
0 nil 100 0
		 ((:script . symbol))
		 > #<font-entity ns apple Menlo nil iso10646-1 medium normal normal  
0 nil 100 0
		 ((:script . symbol))
		 > #<font-entity ns apple Code2000 nil iso10646-1 medium normal  
normal 0 nil 0 0
		 ((:script . symbol))
		 > #<font-entity ns apple Arial_Unicode_MS nil iso10646-1 medium  
normal normal 0 nil 0 0
		 ((:script . symbol))
		 >]
   nil)
  (list "-apple-*-iso10646-1:script=symbol"
        ["-apple-Apple_Symbols-medium-normal-normal-*-p-0-iso10646-1"  
"-apple-Apple_Symbols-medium-normal-normal-*-p-0-iso10646-1" "-apple- 
MS_PMincho-medium-normal-normal-*-p-0-iso10646-1" "-apple-MS_PGothic- 
medium-normal-normal-*-p-0-iso10646-1" "-apple-MS_Mincho-medium-normal- 
normal-*-p-0-iso10646-1" "-apple-MS_Gothic-medium-normal-normal-*-p-0- 
iso10646-1" "-apple-Menlo-bold-italic-normal-*-m-0-iso10646-1" "-apple- 
Menlo-bold-normal-normal-*-m-0-iso10646-1" "-apple-Menlo-medium-italic- 
normal-*-m-0-iso10646-1" "-apple-Menlo-medium-normal-normal-*-m-0- 
iso10646-1" "-apple-Code2000-medium-normal-normal-*-p-0-iso10646-1" "- 
apple-Arial_Unicode_MS-medium-normal-normal-*-p-0-iso10646-1"])
  (sort-by "-*-medium-normal-normal-*-13-*" "ns:-apple-Apple_Symbols- 
medium-normal-normal-*-p-0-iso10646-1")
  (finding\.\.\.finished\ early nil nil)
  (open "-apple-Apple_Symbols-medium-normal-normal-*-p-0- 
iso10646-1:script=symbol" "nil:-apple-Apple_Symbols-medium-normal- 
normal-*-13-*-p-0-iso10646-1"))

bug#3174: NS font selection still broken

[Adrian Robert]

>
> I've been looking at the trace from nsfont and, more importantly, ` 
> (reverse font-log)' from font.c.
>
> To display [“] 201C  LEFT DOUBLE QUOTATION MARK, we get the log  
> below (after adding a few items to log).
>
> At first (top of list), it tries to find "Apple Lucida Grande" and  
> "Lucida Grande" with a particular "registry" and for the symbol  
> script.  This fails, because the `val' variable in  
> font_list_entities ends up being Qnil.

This is as expected.  The first query asks for fonts with  
family=Lucida Grande, that cover script=symbol.  The way the second  
criterion is handled is the issue.  ns_findfonts() has no idea whether  
the font is being requested just for some particular character or for  
rendering the script in general, so it tries to deliver a font that  
has "good coverage" of the requested script.  This is handled by  
ns_get_covering_families() for the script, which looks at the whole  
set of characters provided by the font and returns those that cover a  
certain percentage of the characters in the unicode range for the  
script.

I don't see what other behavior would be reasonable here.

Perhaps the 90% criterion (nsfont.m line 507) could be lowered; this  
might lead to other problems though -- I did experiment with this when  
setting it.  The best thing at this point might be to do a similar  
trace against X11 or w32 and see where things differ.  Of course, the  
available fonts will be different, so it is impossible to duplicate  
the conditions.

bug#3174: NS font selection still broken

[Kenichi Handa]

In article <2FBE059F-5AE1-4DB7-A325-80F49ACF2490@gmail.com>, Adrian Robert <adrian.b.robert@gmail.com> writes:

> > At first (top of list), it tries to find "Apple Lucida Grande" and  
> > "Lucida Grande" with a particular "registry" and for the symbol  
> > script.  This fails, because the `val' variable in  
> > font_list_entities ends up being Qnil.

> This is as expected.  The first query asks for fonts with  
> family=Lucida Grande, that cover script=symbol.  The way the second  
> criterion is handled is the issue.  ns_findfonts() has no idea whether  
> the font is being requested just for some particular character or for  
> rendering the script in general, so it tries to deliver a font that  
> has "good coverage" of the requested script.  This is handled by  
> ns_get_covering_families() for the script, which looks at the whole  
> set of characters provided by the font and returns those that cover a  
> certain percentage of the characters in the unicode range for the  
> script.

> I don't see what other behavior would be reasonable here.

> Perhaps the 90% criterion (nsfont.m line 507) could be lowered; this  
> might lead to other problems though -- I did experiment with this when  
> setting it.  The best thing at this point might be to do a similar  
> trace against X11 or w32 and see where things differ.  Of course, the  
> available fonts will be different, so it is impossible to duplicate  
> the conditions.

For selecting a font that supports a specific script, xft
backend utilizes script-representative-chars which maps a
script to a few representative characters of the script.
From a set of fonts found that way, the code in fontset.c
chooses the best font that actually supports the target
character.

---
Kenichi Handa
handa@m17n.org