bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[Steve Purcell]

Starting a few months ago, I've observed a regression in Emacs HEAD
relative to Emacs 23.3.1.

Steps to reproduce:
===============

* Open Emacs HEAD on OS X
* In *scratch*, evaluate:
  (set-face-attribute 'default nil :foundry "apple" :family "Menlo" :height 120)
* Then use "M-x ucs-insert" to insert the "EMPTY SET" char (∅)
* Use "M-x describe-char" to inspect the properties of the displayed character

Expected results:
==============

This char is available in the Menlo font, so the expected results are as
for Emacs 23.3:

        character: ∅ (8709, #o21005, #x2205)
preferred charset: unicode (Unicode (ISO10646))
       code point: 0x2205
           syntax: . 	which means: punctuation
         category: .:Base, j:Japanese
      buffer code: #xE2 #x88 #x85
        file code: #xE2 #x88 #x85 (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
    nil:-apple-Menlo-medium-normal-normal-*-12-*-*-*-m-0-iso10646-1 (#x7B4)

Character code properties: customize what to show
  name: EMPTY SET
  general-category: Sm (Symbol, Math)


Actual results:
===========

Instead, Emacs HEAD now displays the character from a fallback font, STIXGeneral:

             position: 275 of 275 (100%), column: 0
            character: ∅ (displayed as ∅) (codepoint 8709, #o21005, #x2205)
    preferred charset: unicode (Unicode (ISO10646))
code point in charset: 0x2205
               syntax: . 	which means: punctuation
             category: .:Base, j:Japanese
          buffer code: #xE2 #x88 #x85
            file code: #xE2 #x88 #x85 (encoded by coding system utf-8-unix)
              display: by this font (glyph code)
    nil:-apple-STIXGeneral-medium-normal-normal-*-14-*-*-*-p-0-iso10646-1 (#x41C)

Character code properties: customize what to show
  name: EMPTY SET
  general-category: Sm (Symbol, Math)
  decomposition: (8709) ('∅')






In GNU Emacs 24.1.50.2 (i386-apple-darwin11.3.0, NS apple-appkit-1138.32)
of 2012-05-10 on mandala.home.gateway
Windowing system distributor `Apple', version 10.3.1138
Configured using:
`configure '--with-ns' '--without-dbus' '--without-x' 'CFLAGS=3D-g''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: en_US.UTF-8
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<up> <up> <right> <right> <right> M-x d e s c - c a=20
h <backspace> <backspace> h a r <tab> <return> <help-echo>=20
<help-echo> <down-mouse-1> <mouse-1> M-x u c s - i=20
n <tab> <return> e m p t y SPC s e t <tab> <return>=20
<left> M-x d e s c - c h a r <return> <up> <return>=20
<return> <up> C-y C-x C-e <down> <down> <right> <left>=20
M-x d e s c - c h a r <return> C-x 5 2 M-x e m a c=20
s - b <tab> <s-backspace> <M-backspace> <M-backspace>=20
<M-backspace> r e p <tab> <M-backspace> <M-backspace>=20
r e p o r <tab> <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Type C-x 1 to delete the help window, C-M-v to scroll help.
You can run the command `ucs-insert' with C-x 8 RET

Mark set
nil


Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils pp wid-edit descr-text help-mode easymenu view
time-date tooltip ediff-hook vc-hooks lisp-float-type mwheel ns-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process ns multi-tty
emacs)

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[Alp Aker]

I cannot reproduce this on the latest trunk (the empty set character
is rendered using Menlo, as expected).

Could you try again starting Emacs as "Emacs -Q"?

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[Alp Aker]

On Fri, May 11, 2012 at 12:45 PM, Steve Purcell <steve@sanityinc.com> wrote:
> I *had* started Emacs with -Q in order to be sure the bug report was valid.
>
> The trunk version I have is current up to the following commit:
>
>   Author: Chong Yidong <cyd@gnu.org>
>   Date:   Thu May 10 14:27:12 2012 +0800
>   Cleanups and improvements for FFAP and URL.
>
> I'm on a Lion machine (10.7.4), with the latest XCode and Emacs compiled locally to 64bit in the regular manner.
>
> What else can I try to help pin down the problem?

I can't reproduce the problem building that rev, either, so I'm not
sure I can be of further help.  That said, try the following:

(1) If you haven't already, do "make extraclean" and rebuild from a clean slate.
(2) If that doesn't work, go to src/nsfont.m:47 and change it to
define NSFONT_TRACE as 1.  Then rebuild, follow your original recipe,
and report back with the results of stderr.  (This probably won't tell
us more than we already know, but it's worth a shot.)

PS:  Please keep the bug tracker cc'ed on follow ups.

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[Steve Purcell]

On 11 May 2012, at 21:43, Alp Aker wrote:
> I can't reproduce the problem building that rev, either, so I'm not
> sure I can be of further help.  That said, try the following:
> 
> (1) If you haven't already, do "make extraclean" and rebuild from a clean slate.
> (2) If that doesn't work, go to src/nsfont.m:47 and change it to
> define NSFONT_TRACE as 1.  Then rebuild, follow your original recipe,
> and report back with the results of stderr.  (This probably won't tell
> us more than we already know, but it's worth a shot.)
> 
> PS:  Please keep the bug tracker cc'ed on follow ups.



Thanks for the extraclean tip, though that ultimately didn't make any difference.

The font trace info is below, from startup with the default "Monaco", through setting the default face to "Menlo", and then on to displaying the EMPTY SET char:



nsfont: list for fontspec:
    #<font-spec ns nil Monaco nil iso8859-1 nil nil nil nil nil nil nil ((:name . "Monaco"))>
2012-05-13 11:21:47.852 Emacs[94679:e03] Request covering families for script: 'latin'
(0 . 127)
(160 . 591)
(688 . 879)
(7616 . 7935)
(11360 . 11391)
(42752 . 43007)
2012-05-13 11:21:49.399 Emacs[94679:e03]     returning 1 families
2012-05-13 11:21:49.400 Emacs[94679:e03] Got desc NSCTFontDescriptor <0x102e1ae90> = {
    NSFontFamilyAttribute = Monaco;
} and found 1 matching fonts from it: 
    Returning 0 entities.
nsfont: list for fontspec:
    #<font-spec ns nil Monaco nil ascii-0 nil nil nil nil nil nil nil ((:name . "Monaco"))>
2012-05-13 11:21:49.401 Emacs[94679:e03] Request covering families for script: ''
2012-05-13 11:21:49.402 Emacs[94679:e03]     returning 691 families
2012-05-13 11:21:49.403 Emacs[94679:e03] Got desc NSCTFontDescriptor <0x102e1b510> = {
    NSFontFamilyAttribute = Monaco;
} and found 1 matching fonts from it: 
created font_entity:
    #<font-entity ns apple Monaco nil iso10646-1 medium normal normal 0 nil 100 0 ((:name . "Monaco"))>
created font_entity:
    #<font-entity ns apple Monaco synthItal iso10646-1 medium italic normal 0 nil 0 0 ((:name . "Monaco"))>
    Returning 2 entities.
nsfont: open size 0 of fontentity:
    #<font-entity ns apple Monaco nil iso10646-1 medium normal normal 0 nil 100 0 ((:name . "Monaco"))>
2012-05-13 11:21:49.436 Emacs[94679:e03] "Monaco 12.00 pt. P [] (0x102e1cd30) fobj=0x102b96da0, spc=7.20"
nsfont: list for fontspec:
    #<font-spec ns apple Monaco nil iso10646-1 nil nil nil nil nil 100 nil ((:name . "Monaco") (user-spec . "Monaco"))>
2012-05-13 11:21:49.615 Emacs[94679:e03] Request covering families for script: ''
2012-05-13 11:21:49.615 Emacs[94679:e03]     returning 691 families
2012-05-13 11:21:49.616 Emacs[94679:e03] Got desc NSCTFontDescriptor <0x102e2cd40> = {
    NSFontFamilyAttribute = Monaco;
} and found 1 matching fonts from it: 
created font_entity:
    #<font-entity ns apple Monaco nil iso10646-1 medium normal normal 0 nil 100 0 ((:name . "Monaco") (user-spec . "Monaco"))>
created font_entity:
    #<font-entity ns apple Monaco synthItal iso10646-1 medium italic normal 0 nil 0 0 ((:name . "Monaco") (user-spec . "Monaco"))>
    Returning 2 entities.
nsfont: open size 12 of fontentity:
    #<font-entity ns apple Monaco nil iso10646-1 medium normal normal 0 nil 100 0 ((:name . "Monaco") (user-spec . "Monaco"))>
2012-05-13 11:21:49.617 Emacs[94679:e03] "Monaco 12.00 pt. P [] (0x102e1cd30) fobj=0x102b96da0, spc=7.20"
*** nsfont_open CACHE HIT!
nsfont: list for fontspec:
    #<font-spec ns apple Monaco nil iso10646-1 nil nil nil nil nil 100 nil ((user-spec . "Monaco"))>
2012-05-13 11:21:49.618 Emacs[94679:e03] Request covering families for script: ''
2012-05-13 11:21:49.619 Emacs[94679:e03]     returning 691 families
2012-05-13 11:21:49.620 Emacs[94679:e03] Got desc NSCTFontDescriptor <0x102e2e4e0> = {
    NSFontFamilyAttribute = Monaco;
} and found 1 matching fonts from it: 
created font_entity:
    #<font-entity ns apple Monaco nil iso10646-1 medium normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
created font_entity:
    #<font-entity ns apple Monaco synthItal iso10646-1 medium italic normal 0 nil 0 0 ((user-spec . "Monaco"))>
    Returning 2 entities.
nsfont: open size 12 of fontentity:
    #<font-entity ns apple Monaco synthItal iso10646-1 medium italic normal 0 nil 0 0 ((user-spec . "Monaco"))>
2012-05-13 11:21:49.621 Emacs[94679:e03] "Monaco 12.00 pt. P [] (0x102e1cd30) fobj=0x102b96da0, spc=7.20"
nsfont: open size 12 of fontentity:
    #<font-entity ns apple Monaco nil iso10646-1 medium normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
2012-05-13 11:21:49.785 Emacs[94679:e03] "Monaco 12.00 pt. P [] (0x102e1cd30) fobj=0x102b96da0, spc=7.20"
*** nsfont_open CACHE HIT!
nsfont: open size 12 of fontentity:
    #<font-entity ns apple Monaco nil iso10646-1 medium normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
2012-05-13 11:21:50.055 Emacs[94679:e03] "Monaco 12.00 pt. P [] (0x102e1cd30) fobj=0x102b96da0, spc=7.20"
*** nsfont_open CACHE HIT!
0x102e1d4d0	Finding glyphs for glyphs in block 0
0x102e1d4d0	Computing metrics for glyphs in block 0
nsfont: open size 12 of fontentity:
    #<font-entity ns apple Monaco nil iso10646-1 medium normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
2012-05-13 11:21:50.070 Emacs[94679:e03] "Monaco 12.00 pt. P [] (0x102e1cd30) fobj=0x102b96da0, spc=7.20"
*** nsfont_open CACHE HIT!
nsfont: list for fontspec:
    #<font-spec ns apple Menlo nil iso8859-1 nil nil nil nil nil nil nil ((user-spec . "Monaco"))>
2012-05-13 11:21:55.722 Emacs[94679:e03] Request covering families for script: 'latin'
2012-05-13 11:21:55.723 Emacs[94679:e03]     returning 1 families
2012-05-13 11:21:55.724 Emacs[94679:e03] Got desc NSCTFontDescriptor <0x102e71f30> = {
    NSFontFamilyAttribute = Menlo;
} and found 4 matching fonts from it: 
    Returning 0 entities.
nsfont: list for fontspec:
    #<font-spec ns apple Menlo nil ascii-0 nil nil nil nil nil nil nil ((user-spec . "Monaco"))>
2012-05-13 11:21:55.726 Emacs[94679:e03] Request covering families for script: ''
2012-05-13 11:21:55.726 Emacs[94679:e03]     returning 691 families
2012-05-13 11:21:55.727 Emacs[94679:e03] Got desc NSCTFontDescriptor <0x102e72dd0> = {
    NSFontFamilyAttribute = Menlo;
} and found 4 matching fonts from it: 
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 medium normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 medium italic normal 0 nil 100 0 ((user-spec . "Monaco"))>
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 bold normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 bold italic normal 0 nil 100 0 ((user-spec . "Monaco"))>
    Returning 4 entities.
nsfont: open size 12 of fontentity:
    #<font-entity ns apple Menlo nil iso10646-1 medium normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
2012-05-13 11:21:55.763 Emacs[94679:e03] "Menlo-Regular 12.00 pt. P [] (0x102e17460) fobj=0x102c622e0, spc=7.22"
nsfont: list for fontspec:
    #<font-spec ns apple Menlo nil iso10646-1 nil nil nil nil nil 100 nil ((user-spec . "Monaco"))>
2012-05-13 11:21:55.788 Emacs[94679:e03] Request covering families for script: ''
2012-05-13 11:21:55.788 Emacs[94679:e03]     returning 691 families
2012-05-13 11:21:55.789 Emacs[94679:e03] Got desc NSCTFontDescriptor <0x102e59c90> = {
    NSFontFamilyAttribute = Menlo;
} and found 4 matching fonts from it: 
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 medium normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 medium italic normal 0 nil 100 0 ((user-spec . "Monaco"))>
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 bold normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 bold italic normal 0 nil 100 0 ((user-spec . "Monaco"))>
    Returning 4 entities.
nsfont: open size 12 of fontentity:
    #<font-entity ns apple Menlo nil iso10646-1 medium normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
2012-05-13 11:21:55.790 Emacs[94679:e03] "Menlo-Regular 12.00 pt. P [] (0x102e17460) fobj=0x102c622e0, spc=7.22"
*** nsfont_open CACHE HIT!
0x102e737e0	Finding glyphs for glyphs in block 0
0x102e737e0	Computing metrics for glyphs in block 0
nsfont: open size 12 of fontentity:
    #<font-entity ns apple Menlo nil iso10646-1 medium normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
2012-05-13 11:21:55.807 Emacs[94679:e03] "Menlo-Regular 12.00 pt. P [] (0x102e17460) fobj=0x102c622e0, spc=7.22"
*** nsfont_open CACHE HIT!
nsfont: open size 12 of fontentity:
    #<font-entity ns apple Menlo nil iso10646-1 bold normal normal 0 nil 100 0 ((user-spec . "Monaco"))>
2012-05-13 11:21:55.812 Emacs[94679:e03] "Menlo-Bold 12.00 pt. P [] (0x102dbd490) fobj=0x102dbc060, spc=7.22"
0x102e7bcb0	Finding glyphs for glyphs in block 0
0x102e7bcb0	Computing metrics for glyphs in block 0
nsfont: list for fontspec:
    #<font-spec ns apple Menlo nil iso10646-1 nil nil nil nil nil nil nil ((:script . symbol))>
2012-05-13 11:22:11.194 Emacs[94679:e03] Request covering families for script: 'symbol'
(8192 . 10239)
(10496 . 11263)
(11776 . 11903)
(127232 . 127487)
(127744 . 128591)
(128640 . 128895)
2012-05-13 11:22:11.254 Emacs[94679:e03]     returning 1 families
2012-05-13 11:22:11.254 Emacs[94679:e03] Got desc NSCTFontDescriptor <0x102586c70> = {
    NSFontFamilyAttribute = Menlo;
} and found 4 matching fonts from it: 
    Returning 0 entities.
nsfont: list for fontspec:
    #<font-spec ns nil Menlo nil iso10646-1 nil nil nil nil nil nil nil ((:script . symbol))>
2012-05-13 11:22:11.255 Emacs[94679:e03] Request covering families for script: 'symbol'
2012-05-13 11:22:11.255 Emacs[94679:e03]     returning 1 families
2012-05-13 11:22:11.256 Emacs[94679:e03] Got desc NSCTFontDescriptor <0x102deaf30> = {
    NSFontFamilyAttribute = Menlo;
} and found 4 matching fonts from it: 
    Returning 0 entities.
nsfont: list for fontspec:
    #<font-spec ns apple nil nil iso10646-1 nil nil nil nil nil nil nil ((:script . symbol))>
2012-05-13 11:22:11.256 Emacs[94679:e03] Request covering families for script: 'symbol'
2012-05-13 11:22:11.257 Emacs[94679:e03]     returning 1 families
2012-05-13 11:22:11.601 Emacs[94679:e03] Got desc NSCTFontDescriptor <0x102ddcd80> = {
} and found 1171 matching fonts from it: 
created font_entity:
    #<font-entity ns apple STIXGeneral nil iso10646-1 medium normal normal 0 nil 0 0 ((:script . symbol))>
created font_entity:
    #<font-entity ns apple STIXGeneral nil iso10646-1 medium italic normal 0 nil 0 0 ((:script . symbol))>
created font_entity:
    #<font-entity ns apple STIXGeneral nil iso10646-1 bold normal normal 0 nil 0 0 ((:script . symbol))>
created font_entity:
    #<font-entity ns apple STIXGeneral nil iso10646-1 bold italic normal 0 nil 0 0 ((:script . symbol))>
    Returning 4 entities.
nsfont: open size 12 of fontentity:
    #<font-entity ns apple STIXGeneral nil iso10646-1 medium normal normal 0 nil 0 0 ((:script . symbol))>
2012-05-13 11:22:11.668 Emacs[94679:e03] "STIXGeneral-Regular 12.00 pt. P [] (0x102dbd490) fobj=0x102cc6c20, spc=3.00"
0x102e96420	Finding glyphs for glyphs in block 34
0x102e96420	Computing metrics for glyphs in block 4

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[Steve Purcell]

It's probably unrelated, but I also wanted to note the following font-relating compilation warnings I noticed:

gcc -std=gnu99 -c  -Demacs -DHAVE_CONFIG_H  -I. -I/Users/steve/Projects/External/emacs/src -I../lib -I/Users/steve/Projects/External/emacs/src/../lib               -MMD -MF deps/font.d -MP     -g font.c
font.c: In function ‘Ffont_variation_glyphs’:
font.c:4388: warning: comparison is always true due to limited range of data type
font.c:4388: warning: comparison is always true due to limited range of data type
font.c:4388: warning: comparison is always true due to limited range of data type
font.c:4388: warning: comparison is always true due to limited range of data type
font.c: In function ‘Ffont_get_glyphs’:
font.c:4721: warning: comparison is always true due to limited range of data type
font.c:4721: warning: comparison is always true due to limited range of data type
font.c:4721: warning: comparison is always true due to limited range of data type
font.c:4721: warning: comparison is always true due to limited range of data type

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[Alp Aker]

On Mon, May 14, 2012 at 4:39 AM, Steve Purcell <steve@sanityinc.com> wrote:
> It's probably unrelated, but I also wanted to note the following font-relating
> compilation warnings I noticed:

Those aren't related.

The trace you posted looks odd.  It says you have 671 (!) different
font families; does that sound right for your system?  Also, when
Emacs looks for a family covering the character range for the 'symbol'
script it should, before it finds STIXGeneral, find several other
fonts that are installed by default on OS X, but that's not what the
trace seems to show.  Have you removed or disabled, e.g., Apple
Symbol, AppleMyungjo, or AppleGothic? If not, it looks like the Cocoa
font api is giving Emacs bad info on your system, and you have a
corrupt font cache or the like.

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[David Leatherman]

I managed to isolate the issue to the commit in which it was introduced
as well as the offending lines in the commit. The git commit when the
bug first started happening is 97665d8445c7f8b142cae9db2d81bdcafa310243
which was pushed on Sat, Feb 18 with the following commit message:

"Update char-script-table from the latest Unicode Database.

lisp/international/characters.el (script-list): Sync with the latest
Unicode Character Database."

In lisp/international/characters.el the addition of either of the
following two symbol character ranges will cause the issue: (taken out
of context here)

(#x2900 #x2BFF symbol)
(#x2E00 #x2E7F symbol)

If either of these lines remain, then the font will be incorrectly
chosen. I had to rebuild Emacs to test this despite the file being an
Emacs Lisp file, but it could be that I simply failed to figure out how
to change this file and make it work (I changed it, byte compiled it,
etc. with no effect). YMMV.

I turned on NSFONT_TRACE as suggested earlier and the below is from
starting emacs -Q, changing the font to Menlo, and inserting the EMPTY
SET character. The first trace is from the code with the two lines of
code removed (i.e., will produce the desired behavior). The second is
from the incorrect behavior. 

I've trimmed out non-essential differences. The first substantial
difference to me is the additional two lines in the second diff (the
second and third dotted pair at the beginning). This addition seems to
prevent nsfont.m from returning the 4 entities that it returns in the
correct case.

**  First - correct font chosen **

Emacs[73145:307] Request covering families for script: 'symbol'
(8192 . 10239)
(127232 . 127487)
(127744 . 128591)
(128640 . 128895)
Emacs[73145:307]     returning 4 families
Emacs[73145:307] Got desc NSCTFontDescriptor <0x1011e2e50> = {
    NSFontFamilyAttribute = Menlo;
} and found 4 matching fonts from it: 
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 medium normal normal 0 nil 100 0 ((:script . symbol))>
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 medium italic normal 0 nil 100 0 ((:script . symbol))>
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 bold normal normal 0 nil 100 0 ((:script . symbol))>
created font_entity:
    #<font-entity ns apple Menlo nil iso10646-1 bold italic normal 0 nil 100 0 ((:script . symbol))>
    Returning 4 entities.
nsfont: open size 12 of fontentity:
    #<font-entity ns apple Menlo nil iso10646-1 medium normal normal 0 nil 100 0 ((:script . symbol))>
Emacs[73145:307] "Menlo-Regular 12.00 pt. P [] (0x100ae1270) fobj=0x101169740, spc=7.22"
*** nsfont_open CACHE HIT!
0x101819188	Finding glyphs for glyphs in block 34
0x101819188	Computing metrics for glyphs in block 7


** Second - incorrect font chosen **

Emacs[36212:307] Request covering families for script: 'symbol'
(8192 . 10239)
(10496 . 11263)
(11776 . 11903)
(127232 . 127487)
(127744 . 128591)
(128640 . 128895)
Emacs[36212:307]     returning 1 families
Emacs[36212:307] Got desc NSCTFontDescriptor <0x100adf8b0> = {
    NSFontFamilyAttribute = Menlo;
} and found 4 matching fonts from it: 
    Returning 0 entities.
nsfont: list for fontspec:
    #<font-spec ns nil Menlo nil iso10646-1 nil nil nil nil nil nil nil ((:script . symbol))>
Emacs[36212:307] Request covering families for script: 'symbol'
Emacs[36212:307]     returning 1 families
Emacs[36212:307] Got desc NSCTFontDescriptor <0x104408050> = {
    NSFontFamilyAttribute = Menlo;
} and found 4 matching fonts from it: 
    Returning 0 entities.
nsfont: list for fontspec:
    #<font-spec ns apple nil nil iso10646-1 nil nil nil nil nil nil nil ((:script . symbol))>
Emacs[36212:307] Request covering families for script: 'symbol'
Emacs[36212:307]     returning 1 families
Emacs[36212:307] Got desc NSCTFontDescriptor <0x104405dd0> = {
} and found 330 matching fonts from it: 
created font_entity:
    #<font-entity ns apple STIXGeneral nil iso10646-1 medium normal normal 0 nil 0 0 ((:script . symbol))>
created font_entity:
    #<font-entity ns apple STIXGeneral nil iso10646-1 medium italic normal 0 nil 0 0 ((:script . symbol))>
created font_entity:
    #<font-entity ns apple STIXGeneral nil iso10646-1 bold normal normal 0 nil 0 0 ((:script . symbol))>
created font_entity:
    #<font-entity ns apple STIXGeneral nil iso10646-1 bold italic normal 0 nil 0 0 ((:script . symbol))>
    Returning 4 entities.
nsfont: open size 12 of fontentity:
    #<font-entity ns apple STIXGeneral nil iso10646-1 medium normal normal 0 nil 0 0 ((:script . symbol))>
Emacs[36212:307] "STIXGeneral-Regular 12.00 pt. P [] (0x100ae4ec0) fobj=0x100aac290, spc=3.00"
0x1049bde68	Finding glyphs for glyphs in block 34
0x1049bde68	Computing metrics for glyphs in block 4

*****

I hope this is helpful. I don't know the cause to these effects except
to observe that somehow the extra two character ranges in characters.el
end up causing nsfont.m to look up the wrong font.

Let me know if I can help or answer any questions.


--
David

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[David Leatherman]

Just a quick update; The notes I added before seem to remain valid for
24.2. I added the CCs back to the message just to see if anyone had any
interest. As I forgot to CC anyone last time, the details are here:

http://debbugs.gnu.org/cgi/bugreport.cgi?msg=23;bug=11450

Steve, I created a small patch that I am using with Homebrew on the Mac
when building Emacs. It might be of use to you and while it likely isn't
the "right" fix, it'll do for now.

diff --git a/lisp/international/characters.el b/lisp/international/characters.el
index c75ca81..01eb036 100644
--- a/lisp/international/characters.el
+++ b/lisp/international/characters.el
@@ -1215,7 +1215,6 @@
 	 (#x1F00 #x1FFF greek)
 	 (#x2000 #x27FF symbol)
 	 (#x2800 #x28FF braille)
-	 (#x2900 #x2BFF symbol)
 	 (#x2C00 #x2C5F glagolitic)
 	 (#x2C60 #x2C7F latin)
 	 (#x2C80 #x2CFF coptic)
@@ -1223,7 +1222,6 @@
 	 (#x2D30 #x2D7F tifinagh)
 	 (#x2D80 #x2DDF ethiopic)
 	 (#x2DE0 #x2DFF cyrillic)
-	 (#x2E00 #x2E7F symbol)
 	 (#x2E80 #x2FDF han)
 	 (#x2FF0 #x2FFF ideographic-description)
 	 (#x3000 #x303F cjk-misc)

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[Steve Purcell]

On 4 Sep 2012, at 20:09, David Leatherman <leathekd@gmail.com> wrote:

> Just a quick update; The notes I added before seem to remain valid for
> 24.2. I added the CCs back to the message just to see if anyone had any
> interest. As I forgot to CC anyone last time, the details are here:
> 
> http://debbugs.gnu.org/cgi/bugreport.cgi?msg=23;bug=11450
> 
> Steve, I created a small patch that I am using with Homebrew on the Mac
> when building Emacs. It might be of use to you and while it likely isn't
> the "right" fix, it'll do for now.
> 
> diff --git a/lisp/international/characters.el b/lisp/international/characters.el
> index c75ca81..01eb036 100644
> --- a/lisp/international/characters.el
> +++ b/lisp/international/characters.el
> @@ -1215,7 +1215,6 @@
> 	 (#x1F00 #x1FFF greek)
> 	 (#x2000 #x27FF symbol)
> 	 (#x2800 #x28FF braille)
> -	 (#x2900 #x2BFF symbol)
> 	 (#x2C00 #x2C5F glagolitic)
> 	 (#x2C60 #x2C7F latin)
> 	 (#x2C80 #x2CFF coptic)
> @@ -1223,7 +1222,6 @@
> 	 (#x2D30 #x2D7F tifinagh)
> 	 (#x2D80 #x2DDF ethiopic)
> 	 (#x2DE0 #x2DFF cyrillic)
> -	 (#x2E00 #x2E7F symbol)
> 	 (#x2E80 #x2FDF han)
> 	 (#x2FF0 #x2FFF ideographic-description)
> 	 (#x3000 #x303F cjk-misc)


David; thanks, that's interesting, but there's still a fundamental issue that the fix doesn't work around.

For example, the "GREEK SMALL LETTER LAMDA" (sic) character is available in my favoured primary fonts (Pragmata Pro or Anonymous Pro), but Emacs 24 still displays the Times New Roman version of the character. It turns out that this mistake is also made by Emacs 23.3.1.

Meanwhile, OS X's Textedit app renders that character correctly in any font which contains it.

Strangely, if Emacs' default font is set to Menlo, the lambda character from that font *is* used. The same applies to the "EMPTY SET" character.

-Steve

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[David Leatherman]

Ugh. That's too bad, but hopefully useful to someone who knows more
about fonts on the Mac.

You are right, of course, that the patch doesn't fix the underlying
issue. My theory is that there is some odd interaction between the added
character ranges an Apple font selection, but I don't know what it could
be. Perhaps Menlo, being a built-in font, appears earlier in the font
list and that is why it works.

There may have been other entries made in that commit that you will need
to revert. Or perhaps reverting the entire commit will work for you.

The commit, if you're interested: 97665d8445c7f8b142cae9db2d81bdcafa310243

Good luck!

> David; thanks, that's interesting, but there's still a fundamental issue that the fix doesn't work around.
>
> For example, the "GREEK SMALL LETTER LAMDA" (sic) character is available in my favoured primary fonts (Pragmata Pro or Anonymous Pro), but Emacs 24 still displays the Times New Roman version of the character. It turns out that this mistake is also made by Emacs 23.3.1.
>
> Meanwhile, OS X's Textedit app renders that character correctly in any font which contains it.
>
> Strangely, if Emacs' default font is set to Menlo, the lambda character from that font *is* used. The same applies to the "EMPTY SET" character.
>
> -Steve

--
David

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[Steve Purcell]

Hah! I think I made progress. I believe the commit to which you (David) referred is basically helpful, in that it more correctly catalogues the various character ranges.

On a whim, I tried adding a "set-fontset-font" call to my usual font setup, as shown below:

  (set-fontset-font "fontset-default" 'symbol "PragmataPro")
  (set-face-attribute 'default nil :foundry "apple" :family "PragmataPro" :height 140)

And that actually causes Emacs to use the default font for both the "GREEK SMALL LETTER LAMDA" and "EMPTY SET" characters. 

I guess that in the absence of this line, Emacs asks the system for a font which supports the "symbol" character range, without first trying the default font.

As for why Menlo works like this out of the box and the other fonts don't, that's still a mystery to me.

-Steve

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[David Leatherman]

Nicely done, works for me, as well - without my patch.  The less patches I
have to remember the better.

Thanks.

--
David

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[Steve Purcell]

One caveat with the "set-fontset-font" approach is that the given font is used unconditionally for all characters in the 'symbol ranges; if the font doesn't contain glyphs for a certain symbol character, Emacs will display the empty box character rather than look for an alternative font containing that character.

So this is something of a workaround, rather than a definitive solution.

-Steve

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[Alan Third]

Steve Purcell <steve@sanityinc.com> writes:

> For example, the "GREEK SMALL LETTER LAMDA" (sic) character is
> available in my favoured primary fonts (Pragmata Pro or Anonymous
> Pro), but Emacs 24 still displays the Times New Roman version of the
> character. It turns out that this mistake is also made by Emacs
> 23.3.1.
>
> Meanwhile, OS X's Textedit app renders that character correctly in any
> font which contains it.
>
> Strangely, if Emacs' default font is set to Menlo, the lambda
> character from that font *is* used. The same applies to the "EMPTY
> SET" character.

Hi, are either of you still seeing the problems with the incorrect fonts
being used? The entire font back-end for Emacs on OS X was replaced some
time ago, so hopefully if the problem wasn't sorted before then, that
should have done it.
-- 
Alan Third

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[Steve Purcell]

On 18 Jul 2016, at 08:41, Alan Third <alan@idiocy.org> wrote:
> 
> Hi, are either of you still seeing the problems with the incorrect fonts
> being used? The entire font back-end for Emacs on OS X was replaced some
> time ago, so hopefully if the problem wasn't sorted before then, that
> should have done it.


Thanks Alan. This appears to work okay for me with a recent Emacs 25 snapshot. I’ve had a workaround in place for a long time now, which no longer appears to be necessary. Can’t easily confirm in which Emacs version it became redundant, sorry.

bug#11450: 24.1.50; NS regression: fallback font used even when glyph available in primary font

[David Leatherman]

[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]

Hi Alan,

I hadn't noticed due to some workarounds but this bug appears to be fixed.

Great success!

Thanks,
David

On Sun, Jul 17, 2016 at 4:41 PM, Alan Third <alan@idiocy.org> wrote:

> Steve Purcell <steve@sanityinc.com> writes:
>
> > For example, the "GREEK SMALL LETTER LAMDA" (sic) character is
> > available in my favoured primary fonts (Pragmata Pro or Anonymous
> > Pro), but Emacs 24 still displays the Times New Roman version of the
> > character. It turns out that this mistake is also made by Emacs
> > 23.3.1.
> >
> > Meanwhile, OS X's Textedit app renders that character correctly in any
> > font which contains it.
> >
> > Strangely, if Emacs' default font is set to Menlo, the lambda
> > character from that font *is* used. The same applies to the "EMPTY
> > SET" character.
>
> Hi, are either of you still seeing the problems with the incorrect fonts
> being used? The entire font back-end for Emacs on OS X was replaced some
> time ago, so hopefully if the problem wasn't sorted before then, that
> should have done it.
> --
> Alan Third
>

[-- Attachment #2: Type: text/html, Size: 1671 bytes --]