bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Dmitry Alexandrov]

[-- Attachment #1: Type: text/plain, Size: 415 bytes --]

GNU Emacs 26.0.50.1 (x86_64-unknown-linux-gnu)
Repository revision: 6bf83218314db1c63ce34564edfb994d9431b28a
Debian GNU/Linux 8.6 Jessie
xfonts-terminus 4.40-2 as shipped in Debian

Steps to reproduce:

   $ emacs --quick --font '-*-terminus-medium-r-*-*-20-*-*-*-*-*-*-*'

Find some document with curly quotes: ‘’ and “”.  GNU Emacs Manual (‘<f1> r’) uses them heavily, for instance.

It looks wrong:


[-- Attachment #2: 91.png --]
[-- Type: image/png, Size: 5247 bytes --]

[-- Attachment #3: Type: text/plain, Size: 775 bytes --]


‘C-u C-x =’ says that instead of Terminus ‘x:-misc-fixed-medium-r-normal--20-200-75-75-c-100-iso10646-1’ font is used for rendering quotes.  When I request 24th size, I got yet another font — ‘x:-efont-fixed-medium-r-normal--24-240-75-75-c-240-iso10646-1’.

In a strange way there is no such a problem with some other sizes:

   $ emacs --quick --font '-*-terminus-medium-r-*-*-16-*-*-*-*-*-*-*'

is okay, for example.  ‘C-u C-x =’ says that ‘xft:-xos4-Terminus-normal-normal-normal-*-16-*-*-*-c-80-iso10646-1’ is used in that case.

The reason is *not* that Terminus just does not have these glyphs — it does.  Moreover — GNU Emacs 24.4.1 finds and renders them correctly:

   $ emacs24 --quick --font '-*-terminus-medium-r-*-*-20-*-*-*-*-*-*-*'


[-- Attachment #4: 90.png --]
[-- Type: image/png, Size: 5227 bytes --]

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Eli Zaretskii]

> From: Dmitry Alexandrov <321942@gmail.com>
> Date: Sat, 17 Dec 2016 04:03:07 +0300
> 
> Steps to reproduce:
> 
>    $ emacs --quick --font '-*-terminus-medium-r-*-*-20-*-*-*-*-*-*-*'
> 
> Find some document with curly quotes: ‘’ and “”.  GNU Emacs Manual (‘<f1> r’) uses them heavily, for instance.
> 
> It looks wrong:

What is wrong with how it looks?  Info mode uses several fonts anyway,
so seeing a different font used for quotes shouldn't be a disaster.

> ‘C-u C-x =’ says that instead of Terminus ‘x:-misc-fixed-medium-r-normal--20-200-75-75-c-100-iso10646-1’ font is used for rendering quotes.  When I request 24th size, I got yet another font — ‘x:-efont-fixed-medium-r-normal--24-240-75-75-c-240-iso10646-1’.
> 
> In a strange way there is no such a problem with some other sizes:
> 
>    $ emacs --quick --font '-*-terminus-medium-r-*-*-16-*-*-*-*-*-*-*'
> 
> is okay, for example.  ‘C-u C-x =’ says that ‘xft:-xos4-Terminus-normal-normal-normal-*-16-*-*-*-c-80-iso10646-1’ is used in that case.

Emacs 25 should prefer the default font for symbol and punctuation
characters, as long as the default font supports them.  So I don't
understand why you see these character displayed using another font,
if the default one has the corresponding glyphs.

> The reason is *not* that Terminus just does not have these glyphs — it does.  Moreover — GNU Emacs 24.4.1 finds and renders them correctly:

Emacs 25 has different settings of the default fontset, see
fontset.el.  It instructs Emacs to use fixed-medium fonts for symbols
and punctuation characters that are absent from the default font.
Emacs 24 didn't have those settings.

I think you should be able to force Emacs to use your font for these
characters by using set-fontset-font for the range of codepoints you
want to be displayed by the Terminus font.

Of course, if you or someone can explain why Emacs doesn't use the
default font, as it should have, for these characters, that would be
even better, since we might then be able to fix the problem without
any need for user customizations.

One idea I have is that the way you specify the font on the Emacs
command line doesn't specify a registry.  Did you try using iso10646-1
as the registry?

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Dmitry Alexandrov]

>> Steps to reproduce:
>> 
>>    $ emacs --quick --font '-*-terminus-medium-r-*-*-20-*-*-*-*-*-*-*'
>> 
>> Find some document with curly quotes: ‘’ and “”.  GNU Emacs Manual (‘<f1> r’) uses them heavily, for instance.
>> 
>> It looks wrong:
>
> What is wrong with how it looks?

That’s apparently another font.

> Info mode uses several fonts anyway,

‘GNU Emacs Manual’ was a mere example of an easy accessible page that uses that symbols.  And the quotes are not the only problematic glyphs, just the most notable ones.  Now I noticed the same issue with arrows: ↑ ← ↓ and →, I guess there are others.

> so seeing a different font used for quotes shouldn't be a disaster.
>
>> ‘C-u C-x =’ says that instead of Terminus
>> ‘x:-misc-fixed-medium-r-normal--20-200-75-75-c-100-iso10646-1’ font
>> is used for rendering quotes. When I request 24th size, I got yet
>> another font —
>> ‘x:-efont-fixed-medium-r-normal--24-240-75-75-c-240-iso10646-1’.
>> 
>> In a strange way there is no such a problem with some other sizes:
>> 
>>    $ emacs --quick --font '-*-terminus-medium-r-*-*-16-*-*-*-*-*-*-*'
>> 
>> is okay, for example. ‘C-u C-x =’ says that
>> ‘xft:-xos4-Terminus-normal-normal-normal-*-16-*-*-*-c-80-iso10646-1’
>> is used in that case.
>
> Emacs 25 should prefer the default font for symbol and punctuation
> characters, as long as the default font supports them.  So I don't
> understand why you see these character displayed using another
> if the default one has the corresponding glyphs.
>
>> The reason is *not* that Terminus just does not have these glyphs —
>> it does. Moreover — GNU Emacs 24.4.1 finds and renders them
>> correctly:
>
> Emacs 25 has different settings of the default fontset, see
> fontset.el.

Aha!  Thanks.  The exact problematic commit is ‘2276a19d’ [0].

By the way, how could I prove that without rebuilding the whole Emacs?  I tried:

   $ git show 2276a19^:lisp/international/fontset.el > /tmp/lisp/fontset.el
   $ emacs --directory /tmp/lisp --quick --font '-*-terminus-medium-r-*-*-20-*-*-*-*-*-*-*'

Despite that:

   (locate-library "fontset")
        ⇒ "/tmp/lisp/fontset.el"

I did not see any effect.

[0] https://lists.gnu.org/archive/html/emacs-diffs/2015-06/msg00150.html

> It instructs Emacs to use fixed-medium fonts for symbols
> and punctuation characters that are absent from the default font.
> Emacs 24 didn't have those settings.

> I think you should be able to force Emacs to use your font for these
> characters by using set-fontset-font for the range of codepoints you
> want to be displayed by the Terminus font.

Yes, indeed.  Many thanks.

   (set-fontset-font "fontset-default" '(#x2000 . #x2067) "Terminus")

However, It might be even better if I could erase that rule, as if there were no lines about that range in ‘fontsel.el’.  Is it possible?

> Of course, if you or someone can explain why Emacs doesn't use the
> default font, as it should have, for these characters, that would be
> even better, since we might then be able to fix the problem without
> any need for user customizations.
>
> One idea I have is that the way you specify the font on the Emacs
> command line doesn't specify a registry.  Did you try using iso10646-1
> as the registry?

I’ve just tried — no effect.

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Eli Zaretskii]

> From: Dmitry Alexandrov <321942@gmail.com>
> Cc: 25216@debbugs.gnu.org
> Date: Sat, 17 Dec 2016 12:33:01 +0300
> 
> > What is wrong with how it looks?
> 
> That’s apparently another font.
> 
> > Info mode uses several fonts anyway,
> 
> ‘GNU Emacs Manual’ was a mere example of an easy accessible page that uses that symbols.  And the quotes are not the only problematic glyphs, just the most notable ones.  Now I noticed the same issue with arrows: ↑ ← ↓ and →, I guess there are others.

It doesn't surprise me.  What I don't understand is why this happens.
Emacs explicitly tests the default font for supporting any symbol or
punctuation character it needs to display, before it looks for another
font.  See the function face_for_char in fontset.c.

> Aha!  Thanks.  The exact problematic commit is ‘2276a19d’ [0].
> 
> By the way, how could I prove that without rebuilding the whole Emacs?  I tried:
> 
>    $ git show 2276a19^:lisp/international/fontset.el > /tmp/lisp/fontset.el
>    $ emacs --directory /tmp/lisp --quick --font '-*-terminus-medium-r-*-*-20-*-*-*-*-*-*-*'
> 
> Despite that:
> 
>    (locate-library "fontset")
>         ⇒ "/tmp/lisp/fontset.el"
> 
> I did not see any effect.

You must rebuild Emacs or manually load fontset.el after changing it.
fontset.el is preloaded when Emacs is built.

> > I think you should be able to force Emacs to use your font for these
> > characters by using set-fontset-font for the range of codepoints you
> > want to be displayed by the Terminus font.
> 
> Yes, indeed.  Many thanks.
> 
>    (set-fontset-font "fontset-default" '(#x2000 . #x2067) "Terminus")
> 
> However, It might be even better if I could erase that rule, as if there were no lines about that range in ‘fontsel.el’.  Is it possible?

That would adversely affect all the users.  We arrived at that rule
after many complaints that symbol and punctuation characters are not
displayed at all, or displayed using variable-pitch fonts, so we made
the default fontset "smarter" about that by using fonts people
generally have installed and which are the default fonts in Emacs.

> > Of course, if you or someone can explain why Emacs doesn't use the
> > default font, as it should have, for these characters, that would be
> > even better, since we might then be able to fix the problem without
> > any need for user customizations.
> >
> > One idea I have is that the way you specify the font on the Emacs
> > command line doesn't specify a registry.  Did you try using iso10646-1
> > as the registry?
> 
> I’ve just tried — no effect.

You don't need to change fontset.el for that, just add the
set-fontset-font line to your .emacs.

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Dmitry Alexandrov]

> It doesn't surprise me.  What I don't understand is why this happens.
> Emacs explicitly tests the default font for supporting any symbol or
> punctuation character it needs to display, before it looks for another
> font.  See the function face_for_char in fontset.c.
>
>> Aha!  Thanks.  The exact problematic commit is ‘2276a19d’ [0].
>> 
>> By the way, how could I prove that without rebuilding the whole Emacs?  I tried:
>> 
>>    $ git show 2276a19^:lisp/international/fontset.el > /tmp/lisp/fontset.el
>>    $ emacs --directory /tmp/lisp --quick --font '-*-terminus-medium-r-*-*-20-*-*-*-*-*-*-*'
>> 
>> Despite that:
>> 
>>    (locate-library "fontset")
>>         ⇒ "/tmp/lisp/fontset.el"
>> 
>> I did not see any effect.
>
> You must rebuild Emacs or manually load fontset.el after changing it.
> fontset.el is preloaded when Emacs is built.

So I need to use ‘temacs’?  Yes, that works, thank you.

>> > I think you should be able to force Emacs to use your font for these
>> > characters by using set-fontset-font for the range of codepoints you
>> > want to be displayed by the Terminus font.
>> 
>> Yes, indeed.  Many thanks.
>> 
>>    (set-fontset-font "fontset-default" '(#x2000 . #x2067) "Terminus")
>> 
>> However, It might be even better if I could erase that rule, as if
>> there were no lines about that range in ‘fontsel.el’. Is it
>> possible?
>
> That would adversely affect all the users.  We arrived at that rule
> after many complaints that symbol and punctuation characters are not
> displayed at all, or displayed using variable-pitch fonts, so we made
> the default fontset "smarter" about that by using fonts people
> generally have installed and which are the default fonts in Emacs.

Well, I’m the only user on most my machines, so that’s probably bearable.  However, we apparently misunderstood each other — I did not ask how to patch ‘fontsel.el’ machine-wide (I hope, I do understand how to do that).  I meant, is it possible to remove elements from the default fontset *user-wide*?  ‘Terminus’ is not the only font I use, so that would be cleaner than appending it there.

I read (info "(emacs) Fontsets"), it did not help.  I could not even get yet how to show the current value of a certain fontset.

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Eli Zaretskii]

> From: Dmitry Alexandrov <321942@gmail.com>
> Cc: 25216@debbugs.gnu.org
> Date: Mon, 19 Dec 2016 06:17:57 +0300
> 
> > It doesn't surprise me.  What I don't understand is why this happens.
> > Emacs explicitly tests the default font for supporting any symbol or
> > punctuation character it needs to display, before it looks for another
> > font.  See the function face_for_char in fontset.c.
> >
> >> Aha!  Thanks.  The exact problematic commit is ‘2276a19d’ [0].
> >> 
> >> By the way, how could I prove that without rebuilding the whole Emacs?  I tried:
> >> 
> >>    $ git show 2276a19^:lisp/international/fontset.el > /tmp/lisp/fontset.el
> >>    $ emacs --directory /tmp/lisp --quick --font '-*-terminus-medium-r-*-*-20-*-*-*-*-*-*-*'
> >> 
> >> Despite that:
> >> 
> >>    (locate-library "fontset")
> >>         ⇒ "/tmp/lisp/fontset.el"
> >> 
> >> I did not see any effect.
> >
> > You must rebuild Emacs or manually load fontset.el after changing it.
> > fontset.el is preloaded when Emacs is built.
> 
> So I need to use ‘temacs’?  Yes, that works, thank you.

If you want to rebuild Emacs, then yes, you will need to run the
command which dumps it after changing fontset.el.

> >> However, It might be even better if I could erase that rule, as if
> >> there were no lines about that range in ‘fontsel.el’. Is it
> >> possible?
> >
> > That would adversely affect all the users.  We arrived at that rule
> > after many complaints that symbol and punctuation characters are not
> > displayed at all, or displayed using variable-pitch fonts, so we made
> > the default fontset "smarter" about that by using fonts people
> > generally have installed and which are the default fonts in Emacs.
> 
> Well, I’m the only user on most my machines, so that’s probably bearable.

No, I meant all the users of Emacs, world wide.

> However, we apparently misunderstood each other — I did not ask how to patch ‘fontsel.el’ machine-wide (I hope, I do understand how to do that).  I meant, is it possible to remove elements from the default fontset *user-wide*?  ‘Terminus’ is not the only font I use, so that would be cleaner than appending it there.

You don't need (and cannot, AFAIK) to remove that.  What you need is
to prepend your rules, which will override what's there.  Like this:

 (set-fontset-font "fontset-default" '(#x2000 . #x2067) "Terminus" 'prepend)

Note the "prepend" part.

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Dmitry Alexandrov]

>> > It doesn't surprise me.  What I don't understand is why this happens.
>> > Emacs explicitly tests the default font for supporting any symbol or
>> > punctuation character it needs to display, before it looks for another
>> > font.  See the function face_for_char in fontset.c.
>> >
>> >> Aha!  Thanks.  The exact problematic commit is ‘2276a19d’ [0].
>> >> 
>> >> By the way, how could I prove that without rebuilding the whole Emacs?  I tried:
>> >> 
>> >>    $ git show 2276a19^:lisp/international/fontset.el > /tmp/lisp/fontset.el
>> >>    $ emacs --directory /tmp/lisp --quick --font '-*-terminus-medium-r-*-*-20-*-*-*-*-*-*-*'
>> >> 
>> >> Despite that:
>> >> 
>> >>    (locate-library "fontset")
>> >>         ⇒ "/tmp/lisp/fontset.el"
>> >> 
>> >> I did not see any effect.
>> >
>> > You must rebuild Emacs or manually load fontset.el after changing it.
>> > fontset.el is preloaded when Emacs is built.
>> 
>> So I need to use ‘temacs’?  Yes, that works, thank you.
>
> If you want to rebuild Emacs, then yes, you will need to run the
> command which dumps it after changing fontset.el.

So there is another way?  Mere (load "/tmp/lisp/fontset.el") have no use after built-in ‘fontset’ is already loaded of course.  I’m just curious.

>> >> However, It might be even better if I could erase that rule, as if
>> >> there were no lines about that range in ‘fontsel.el’. Is it
>> >> possible?
>> >
>> > That would adversely affect all the users.  We arrived at that rule
>> > after many complaints that symbol and punctuation characters are not
>> > displayed at all, or displayed using variable-pitch fonts, so we made
>> > the default fontset "smarter" about that by using fonts people
>> > generally have installed and which are the default fonts in Emacs.
>> 
>> Well, I’m the only user on most my machines, so that’s probably bearable.
>
> No, I meant all the users of Emacs, world wide.
>
>> However, we apparently misunderstood each other — I did not ask how
>> to patch ‘fontsel.el’ machine-wide (I hope, I do understand how to
>> do that). I meant, is it possible to remove elements from the
>> default fontset *user-wide*? ‘Terminus’ is not the only font I use,
>> so that would be cleaner than appending it there.
>
> You don't need (and cannot, AFAIK) to remove that.  What you need is
> to prepend your rules, which will override what's there.  Like this:
>
>  (set-fontset-font "fontset-default" '(#x2000 . #x2067) "Terminus" nil 'prepend)
>
> Note the "prepend" part.

Okay, if I can’t, I can’t.  As for ‘prepend’ it actually works all three ways for me: obviously ‘nil’ (for override, not what I want), ‘prepend’ (as expected) and ‘append’ (no idea why).

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Eli Zaretskii]

> From: Dmitry Alexandrov <321942@gmail.com>
> Cc: 25216@debbugs.gnu.org
> Date: Mon, 19 Dec 2016 07:52:42 +0300
> 
> >> > You must rebuild Emacs or manually load fontset.el after changing it.
> >> > fontset.el is preloaded when Emacs is built.
> >> 
> >> So I need to use ‘temacs’?  Yes, that works, thank you.
> >
> > If you want to rebuild Emacs, then yes, you will need to run the
> > command which dumps it after changing fontset.el.
> 
> So there is another way?  Mere (load "/tmp/lisp/fontset.el") have no
> use after built-in ‘fontset’ is already loaded of course.

It should, AFAIK.

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Eli Zaretskii]

> From: Dmitry Alexandrov <321942@gmail.com>
> Cc: 25216@debbugs.gnu.org
> Date: Mon, 19 Dec 2016 06:17:57 +0300
> 
> I could not even get yet how to show the current value of a certain
> fontset.

"M-x describe-fontset" is what you want.

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Dmitry Alexandrov]

[-- Attachment #1: Type: text/plain, Size: 204 bytes --]

>> I could not even get yet how to show the current value of a certain
>> fontset.
>
> "M-x describe-fontset" is what you want.

Nice.  It worth brief mentioning in (info "(emacs) Fontsets"), I believe.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: describe-fontset.patch --]
[-- Type: text/x-diff, Size: 599 bytes --]

diff --git a/doc/emacs/mule.texi b/doc/emacs/mule.texi
index 80b1384..e92aad1 100644
--- a/doc/emacs/mule.texi
+++ b/doc/emacs/mule.texi
@@ -1370,6 +1370,11 @@ display that character properly.  It will display that character as a
 hex code or thin space or an empty box instead.  (@xref{Text Display, ,
 glyphless characters}, for details.)
 
+@findex describe-fontset
+  @kbd{M-x describe-fontset} lists the complete range of characters
+and fonts assigned to them in a given fontset.  Its default is the
+fontset used by the current frame.
+
 @node Defining Fontsets
 @section Defining fontsets
 

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Eli Zaretskii]

> From: Dmitry Alexandrov <321942@gmail.com>
> Cc: 25216@debbugs.gnu.org
> Date: Mon, 19 Dec 2016 07:53:28 +0300
> 
> > "M-x describe-fontset" is what you want.
> 
> Nice.  It worth brief mentioning in (info "(emacs) Fontsets"), I believe.

Thanks, I added a slightly different description.

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Dmitry Alexandrov]

> "M-x describe-fontset" is what you want.

By studying what it says, I accidentally found a workaround, that does not imply nor careful enumeration of characters that ‘Terminus’ support in order to add them to ‘fontset-default’ nor clogging ‘fontset-default’ with erroneous rules.

   $ emacs --quick --font '-*-terminus-medium-r-*-*-20-*-*-*-*-*-*-*'

   (set-fontset-font "-*-Terminus-*-fontset-startup"
                     '(0 . #x3FFFFF)
                     "Terminus")

Looks very much like (setq a a), but works like a charm!

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Eli Zaretskii]

> From: Dmitry Alexandrov <321942@gmail.com>
> Cc: 25216@debbugs.gnu.org
> Date: Mon, 19 Dec 2016 09:14:44 +0300
> 
>    (set-fontset-font "-*-Terminus-*-fontset-startup"
>                      '(0 . #x3FFFFF)
>                      "Terminus")
> 
> Looks very much like (setq a a), but works like a charm!

Are you sure this doesn't get in the way when you want to display a
character not supported by that font?

And I still don't understand why you need that.  AFAICS, Emacs
explicitly tries to use the default font for punctuation and symbol
characters before falling back on looking up other fonts.

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Dmitry Alexandrov]

[-- Attachment #1: Type: text/plain, Size: 667 bytes --]

>>    (set-fontset-font "-*-Terminus-*-fontset-startup"
>>                      '(0 . #x3FFFFF)
>>                      "Terminus")
>> 
>> Looks very much like (setq a a), but works like a charm!
>
> Are you sure this doesn't get in the way when you want to display a
> character not supported by that font?

No, I am not.  But how it could if I did mess with ‘fontset-default’?  Anyway, let us consider that example:

| Unicode charset: “Eĥoŝanĝo ĉiuĵaŭde”, “Γειά σας”, “שלום”, “Здравствуйте!”

Before (note that the fallback font is inconsistent even linewise — the first opening quote is different from others):


[-- Attachment #2: 93.png --]
[-- Type: image/png, Size: 1694 bytes --]

[-- Attachment #3: Type: text/plain, Size: 9 bytes --]


After:


[-- Attachment #4: 94.png --]
[-- Type: image/png, Size: 1678 bytes --]

[-- Attachment #5: Type: text/plain, Size: 572 bytes --]


It looks like the way it should be — first try the current fonset, only then fallback to ‘fontset-default’.

> And I still don't understand why you need that.  AFAICS, Emacs
> explicitly tries to use the default font for punctuation and symbol
> characters before falling back on looking up other fonts.

Do you mean that you could not reproduce this?

I’ve installed the lastest upstream version of Terminus [0] — and got the same issue.  So it does not seem to be Debian-specific.

[0] https://sourceforge.net/projects/terminus-font/files/terminus-font-4.40/

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Eli Zaretskii]

> From: Dmitry Alexandrov <321942@gmail.com>
> Cc: 25216@debbugs.gnu.org
> Date: Tue, 20 Dec 2016 05:51:04 +0300
> 
> > And I still don't understand why you need that.  AFAICS, Emacs
> > explicitly tries to use the default font for punctuation and symbol
> > characters before falling back on looking up other fonts.
> 
> Do you mean that you could not reproduce this?

It means I don't understand how it could happen.  The function
face_for_char (defined in fontset.c) explicitly tries the current
default font for any punctuation and symbol character, before it
starts looking in other fonts.  I don't understand why this fails for
you, and only in certain font sizes on top of that.  Perhaps you could
step through that code with a debugger and see what happens there?

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Dmitry Alexandrov]

>> > And I still don't understand why you need that.  AFAICS, Emacs
>> > explicitly tries to use the default font for punctuation and symbol
>> > characters before falling back on looking up other fonts.
>> 
>> Do you mean that you could not reproduce this?
>
> It means I don't understand how it could happen.  The function
> face_for_char (defined in fontset.c) explicitly tries the current
> default font for any punctuation and symbol character, before it
> starts looking in other fonts.  I don't understand why this fails for
> you, and only in certain font sizes on top of that.  Perhaps you could
> step through that code with a debugger and see what happens there?

Well.  font_has_char of font.c:2989 returns 0 ← xftfont_has_char of xftfont.c:532 returns 0 ← XftCharExists of libXft.so.2 returns 0.

So the problem goes into libxft2.

Thus the possible workaround may be to prefer ‘x’ backend for font rendering:

   $ emacs -q -fn '-*-terminus-*-18-*-iso10646-1' -xrm 'emacs.fontBackend: x,xft'

However, by a fluke dropping ‘x’ entirely also conceals the issue at least on my system:

   $ emacs -q -fn '-*-terminus-*-18-*-iso10646-1' -xrm 'emacs.fontBackend: xft'

Only that (which is presumably the default) leads to the problem:

   $ emacs -q -fn '-*-terminus-*-18-*-iso10646-1' -xrm 'emacs.fontBackend: xft,x'

At least it’s now clear what’s so special about combination of Terminus and GNU Emacs — the former it is a rare example of a font that is available through both of font engines, the latter — of an application that uses both of them.

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Eli Zaretskii]

> From: Dmitry Alexandrov <321942@gmail.com>
> Cc: 25216@debbugs.gnu.org
> Date: Sat, 31 Dec 2016 08:56:21 +0300
> 
> >> > And I still don't understand why you need that.  AFAICS, Emacs
> >> > explicitly tries to use the default font for punctuation and symbol
> >> > characters before falling back on looking up other fonts.
> >> 
> >> Do you mean that you could not reproduce this?
> >
> > It means I don't understand how it could happen.  The function
> > face_for_char (defined in fontset.c) explicitly tries the current
> > default font for any punctuation and symbol character, before it
> > starts looking in other fonts.  I don't understand why this fails for
> > you, and only in certain font sizes on top of that.  Perhaps you could
> > step through that code with a debugger and see what happens there?
> 
> Well.  font_has_char of font.c:2989 returns 0 ← xftfont_has_char of xftfont.c:532 returns 0 ← XftCharExists of libXft.so.2 returns 0.
> 
> So the problem goes into libxft2.

Thanks for digging into this.  So in this case libxft2 says that
character is not available, whereas in fact it is?  Or did I
misunderstand?

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Thanks for digging into this.  So in this case libxft2 says that
> character is not available, whereas in fact it is?  Or did I
> misunderstand?

There was no response to this -- was any further progress made?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Dmitry Alexandrov]

[-- Attachment #1: Type: text/plain, Size: 397 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Thanks for digging into this.  So in this case libxft2 says that character is not available, whereas in fact it is?  Or did I misunderstand?
>
> There was no response to this -- was any further progress made?

On my side: sorry, no.  I am fine with the workaround:

	Emacs.fontBackend: xft

(in ~/.Xresources).

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Eli Zaretskii]

> From: Dmitry Alexandrov <321942@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  25216@debbugs.gnu.org
> Date: Sun, 17 Nov 2019 11:38:26 +0300
> 
> > There was no response to this -- was any further progress made?
> 
> On my side: sorry, no.  I am fine with the workaround:
> 
> 	Emacs.fontBackend: xft
> 
> (in ~/.Xresources).

Could you try the master branch, after making sure you have HarfBuzz
installed?  The xft font backend is going to be much less important
starting with Emacs 27, so I'd like to know whether this problerm
exists with HarfBuzz, which is the default font backend if you have it
installed.

Thanks.

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Dmitry Alexandrov]

[-- Attachment #1: Type: text/plain, Size: 825 bytes --]

Eli Zaretskii <eliz@gnu.org> wrote:
>> > There was no response to this -- was any further progress made?
>> 
>> On my side: sorry, no.  I am fine with the workaround:
>> 
>> 	Emacs.fontBackend: xft
>> 
>> (in ~/.Xresources).
>
> Could you try the master branch, after making sure you have HarfBuzz installed?  The xft font backend is going to be much less important starting with Emacs 27, so I'd like to know whether this problerm exists with HarfBuzz

Yes, the problem is still here, symptoms are the same, except that Terminus is now loaded via harfbuzz:

	xfthb:-xos4-Terminus-normal-normal-normal-*-18-*-*-*-c-100-iso10646-1
	   ^^

The same workaround works as well:

	Emacs.fontBackend: xfthb

> which is the default font backend if you have it installed.

Ah, thanks, I perhaps have to switch.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

bug#25216: 26.0.50 [regression]; Curly quotes are not found in some sizes of ‘Terminus’ font

[Eli Zaretskii]

> From: Dmitry Alexandrov <321942@gmail.com>
> Cc: larsi@gnus.org,  25216@debbugs.gnu.org
> Date: Wed, 20 Nov 2019 18:01:48 +0300
> 
> Eli Zaretskii <eliz@gnu.org> wrote:
> >> > There was no response to this -- was any further progress made?
> >> 
> >> On my side: sorry, no.  I am fine with the workaround:
> >> 
> >> 	Emacs.fontBackend: xft
> >> 
> >> (in ~/.Xresources).
> >
> > Could you try the master branch, after making sure you have HarfBuzz installed?  The xft font backend is going to be much less important starting with Emacs 27, so I'd like to know whether this problerm exists with HarfBuzz
> 
> Yes, the problem is still here, symptoms are the same, except that Terminus is now loaded via harfbuzz:
> 
> 	xfthb:-xos4-Terminus-normal-normal-normal-*-18-*-*-*-c-100-iso10646-1
> 	   ^^
> 
> The same workaround works as well:
> 
> 	Emacs.fontBackend: xfthb

I've built the latest version 4.48 of the Terminus font on MS-Windows
and tried it with Emacs 27.  I couldn't reproduce the problem reported
with the xft or xfthb font backends: no matter what size of the font I
tried, the curly quote characters came from the same font.  I do see
that, since this font is bitmapped, i.e. not TrueType, Emacs on
Windows uses the GDI font backend to display it, not the default
HarfBuzz backend.

So I think what you see is something specific to the libxft2 library.
And since this discussion identified at least 2 workarounds: either
customize the fontset, or force the single xft backend in .Xresources,
I think this bug can be closed now.

Thanks.