bug#19993: 25.0.50; Unicode fonts defective on Windows

[Eli Zaretskii <eliz@gnu.org>]

> Date: Thu, 12 Mar 2015 18:52:15 -0700
> From: Ilya Zakharevich <ilya@math.berkeley.edu>
> Cc: 19993@debbugs.gnu.org
> 
> > 	  /* Find a font best-matching with the spec without checking
> > 	     the support of the character C.  That checking is costly,
> > 	     and even without the checking, the found font supports C
> > 	     in high possibility.  */
> 
> So, this explains why U+2099, U+27e8, U+27e9 are not shown here (while
> supported by a lot of fonts).

I do have these displayed (by STIX, not Symbola, but that's just
because STIX was probably examined first).  Did you try that with the
latest master?  Or maybe it's DejaVu at work, see below.

> Did you look into the link I provided (about how Firefox does it)?
> 
>   http://search.cpan.org/~ilyaz/UI-KeyboardLayout/lib/UI/KeyboardLayout.pm#There_is_no_way_to_show_Unicode_contents_on_Windows
> 
> As my experiments show (I did not try to read the source code) the
> logic of falling back goes this way:
> 
>   • If document’s fonts can show a char, stop;
>   • If (user-configurable) fallback fonts for a Subset can show a
>     char, stop; 
>   • If (user-configurable) universal fallback fonts can show a
>     char, stop;
>   • Otherwise, scan all fonts to find one supporting a char.

Emacs already does the first 3 of these.  It also does the 4th, but
without actually opening the fonts.  Changing that to actually open
the fonts is beyond my pay grade.  Given that the fontsets machinery
is so easy to tweak to fix specific problems (once you understand how
to use fontsets), it sounds to me that providing a smarter default
fontset is a much easier way of having Emacs DTRT out of the box.
Also faster, because fontests are checked before scanning all the
fonts and need much less processing.  After all, free fonts with good
coverage of the Unicode codepoints are very few, well-known, and
unlikely to change much in the future.

> Emacs:
>   • Supports different fallbacks for different subsets;
>   • But it supports only one fallback font per character.

Not per character, per script, or Unicode block.  And it's not really
a fallback, it's the result of searching all the installed fonts.
"Fallback" in Emacs is defined by fontsets; a fontset can in principle
define a different fallback font for each codepoint.

>     (Well, it allows specifying more than one font, but as you saw,
>     only one of them will be actually used — at least in the case
>     when the fonts would claim having chars in all the ranges — as
>     most of “good universal fonts” would do.)

There's no problem to specify different fonts for specific characters
belonging to the same Unicode block.  So I don't see any problem here,
not with what can be defined using fontsets.

> The second one is a significant show-stopper, since it is very hard to
> boil down things to one font.

It can be handled in one of the two ways I suggested:

  1) Have the default fontset do this, based on expert knowledge of
     problems with existing fonts and characters.  This can be done
     today by simply augmenting the default fontset.

  2) Add a data base of "bad" fonts that should be rejected for
     certain ranges of codepoints, even if they claim support for
     the corresponding Unicode blocks.  This might need additional
     code to be added (although it's possible that fontsets already
     support that as well -- see the ADD argument of set-fontset-font,
     whose semantics is not explained well enough).

> Myself, I only use scripts with “simple shaping”, so all of my needs
> are covered by 4 fonts:
>    DejaVu *
>    Symbola
>    Junicode
>    Unifont Smooth

I tried DejaVu, but recently uninstalled it, because it masked too
many characters for which it falsely claimed support (including the
Mathematical Alphanumeric Symbols, I think).  What do you need it for?

Also, Quivira has a very good coverage of rare blocks.

> BTW, is font-family search caseless?

I think so.

> > One problem with both of these is that it's hard to recommend fonts
> > because many good fonts are non-free.
> 
> For simple rendering (no shaping), there is a lot of possibilities.

We only need a few popular ones.