bug#58184: Faulty font selection for Latin characters

bug#58184: Faulty font selection for Latin characters

[समीर सिंह Sameer Singh]

[-- Attachment #1.1: Type: text/plain, Size: 767 bytes --]

When a font is set for the "default" face in the init.el file.
For example, like this:

(set-face-attribute 'default nil
   :font "Fira Code"
   :weight 'regular
   :height 170)

This messes up the font selection for various latin and all ipa characters.
Despite the configured font supporting the characters which are typed,
Emacs selects a different font for them, this results in visually jarring
text or sometimes failed composition.

For example see in 1.png
All of the letters except ṇ (#x1E47) are in Fira Code while it is in Latin
Modern Mono despite Fira Code supporting it.
Below it t̪ (t + #x32A) is not composed properly because while 't' is in
FiraCode #x32A is in Step Regular.

Note: This does not happen with emacs -Q

Thanks

[-- Attachment #1.2: Type: text/html, Size: 1017 bytes --]

[-- Attachment #2: 1.png --]
[-- Type: image/png, Size: 4972 bytes --]

bug#58184: Faulty font selection for Latin characters

[समीर सिंह Sameer Singh]

[-- Attachment #1.1: Type: text/plain, Size: 1116 bytes --]

Please replace Fira Code in the above mail with JetBrains Mono and
disregard the IPA stuff.
I have attached the emacs and hb-view output.

I have noticed that this does not happen with Iosevka

On Fri, Sep 30, 2022 at 4:24 AM समीर सिंह Sameer Singh <
lumarzeli30@gmail.com> wrote:

> When a font is set for the "default" face in the init.el file.
> For example, like this:
>
> (set-face-attribute 'default nil
>    :font "Fira Code"
>    :weight 'regular
>    :height 170)
>
> This messes up the font selection for various latin and all ipa characters.
> Despite the configured font supporting the characters which are typed,
> Emacs selects a different font for them, this results in visually jarring
> text or sometimes failed composition.
>
> For example see in 1.png
> All of the letters except ṇ (#x1E47) are in Fira Code while it is in Latin
> Modern Mono despite Fira Code supporting it.
> Below it t̪ (t + #x32A) is not composed properly because while 't' is in
> FiraCode #x32A is in Step Regular.
>
> Note: This does not happen with emacs -Q
>
> Thanks
>

[-- Attachment #1.2: Type: text/html, Size: 1644 bytes --]

[-- Attachment #2: emacs-jetbrains.png --]
[-- Type: image/png, Size: 10769 bytes --]

[-- Attachment #3: hb-view-jet.png --]
[-- Type: image/png, Size: 14896 bytes --]

bug#58184: Faulty font selection for Latin characters

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 1358 bytes --]

codepoints of faulty characters in the above mail:
ṣ #x1e63
ṇ #x1e47
ḍ #x1e0d

On Fri, Sep 30, 2022 at 4:46 AM समीर सिंह Sameer Singh <
lumarzeli30@gmail.com> wrote:

> Please replace Fira Code in the above mail with JetBrains Mono and
> disregard the IPA stuff.
> I have attached the emacs and hb-view output.
>
> I have noticed that this does not happen with Iosevka
>
> On Fri, Sep 30, 2022 at 4:24 AM समीर सिंह Sameer Singh <
> lumarzeli30@gmail.com> wrote:
>
>> When a font is set for the "default" face in the init.el file.
>> For example, like this:
>>
>> (set-face-attribute 'default nil
>>    :font "Fira Code"
>>    :weight 'regular
>>    :height 170)
>>
>> This messes up the font selection for various latin and all ipa
>> characters.
>> Despite the configured font supporting the characters which are typed,
>> Emacs selects a different font for them, this results in visually jarring
>> text or sometimes failed composition.
>>
>> For example see in 1.png
>> All of the letters except ṇ (#x1E47) are in Fira Code while it is in
>> Latin Modern Mono despite Fira Code supporting it.
>> Below it t̪ (t + #x32A) is not composed properly because while 't' is in
>> FiraCode #x32A is in Step Regular.
>>
>> Note: This does not happen with emacs -Q
>>
>> Thanks
>>
>

[-- Attachment #2: Type: text/html, Size: 2178 bytes --]

bug#58184: Faulty font selection for Latin characters

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh
>  <lumarzeli30@gmail.com>
> Date: Fri, 30 Sep 2022 04:49:37 +0530
> 
> codepoints of faulty characters in the above mail:
> ṣ #x1e63
> ṇ #x1e47
> ḍ #x1e0d

Please _always_ show the actual text, not just an image of its
display.  It is practically impossible to know what text you used
given just its display screenshot, especially when character
compositions are involved.

In any case, we will need a reproducible recipe starting from
"emacs -Q".  Also, please include in your report the output of

   M-: (font-at (point)) RET

for the "good" and the "faulty" characters.

In general, reporting issues with fonts need as many specific details
as you can collect, because these issues are inherently
system-specific, due to different sets of fonts installed on each
system, and due to font customizations that are outside of Emacs (on
the Fontconfig level).  The probability of getting unreproducible
results is very high, and each detail helps.

bug#58184: Faulty font selection for Latin characters

[समीर सिंह Sameer Singh]

[-- Attachment #1.1: Type: text/plain, Size: 2746 bytes --]

Here are the steps to reproduce:
1. emacs -Q

2. enter these three words:
hṛṣyatha (h#x1E5B#x1E63yatha)
kṣiptaḥ (k#x1E63ipta#x1E25)
viśāmaḥ (vi#x015B#x0101ma#x1E25)

At this stage everything is fine, all of the characters use the same font
therefore all of them return the same font after evaluating (font-at
(point)) on them:
#<font-object "-ADBO-Source Code
Pro-regular-normal-normal-*-32-*-*-*-m-0-iso10646-1">

3. Now eval the following in the buffer
(set-face-attribute 'default nil
   :font "JetBrains Mono"
   :weight 'regular
   :height 170)

Now you'll notice that the font for some of the characters above are
different from JetBrainsMono, these characters are:
ṛ (#x1e5b)
ṣ (#x1e63)
ḥ (#x1e25)
(font-at (point)) returns #<font-object "-UKWN-Latin Modern
Mono-regular-normal-normal-*-57-*-*-*-*-0-iso10646-1"> on all of them
while for the rest of the characters gives #<font-object "-JB-JetBrains
Mono-regular-normal-normal-*-57-*-*-*-m-0-iso10646-1">

Changing the weight from regular to medium also does not help, the
offending characters instead of being displayed with Latin Modern Mono are
now displayed with: #<font-object "-GOOG-Noto Serif
Display-medium-normal-normal-*-57-*-*-*-*-0-iso10646-1">
while the rest of the characters are displayed with #<font-object
"-JB-JetBrains Mono-medium-normal-normal-*-57-*-*-*-m-0-iso10646-1">

I have also included screenshots from emacs -Q before, emacs -Q after and
hb-view from JetBrains

Hope this makes things clear, if something is still missing please tell me.
Thanks


On Fri, Sep 30, 2022 at 11:33 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh
> >  <lumarzeli30@gmail.com>
> > Date: Fri, 30 Sep 2022 04:49:37 +0530
> >
> > codepoints of faulty characters in the above mail:
> > ṣ #x1e63
> > ṇ #x1e47
> > ḍ #x1e0d
>
> Please _always_ show the actual text, not just an image of its
> display.  It is practically impossible to know what text you used
> given just its display screenshot, especially when character
> compositions are involved.
>
> In any case, we will need a reproducible recipe starting from
> "emacs -Q".  Also, please include in your report the output of
>
>    M-: (font-at (point)) RET
>
> for the "good" and the "faulty" characters.
>
> In general, reporting issues with fonts need as many specific details
> as you can collect, because these issues are inherently
> system-specific, due to different sets of fonts installed on each
> system, and due to font customizations that are outside of Emacs (on
> the Fontconfig level).  The probability of getting unreproducible
> results is very high, and each detail helps.
>

[-- Attachment #1.2: Type: text/html, Size: 3602 bytes --]

[-- Attachment #2: emacs-jetbrains.png --]
[-- Type: image/png, Size: 36978 bytes --]

[-- Attachment #3: emacs-after.png --]
[-- Type: image/png, Size: 23070 bytes --]

[-- Attachment #4: emacs-before.png --]
[-- Type: image/png, Size: 23140 bytes --]

bug#58184: Faulty font selection for Latin characters

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Fri, 30 Sep 2022 16:53:01 +0530
> Cc: 58184@debbugs.gnu.org
> 
> Here are the steps to reproduce:
> 1. emacs -Q
> 
> 2. enter these three words:
> hṛṣyatha (h#x1E5B#x1E63yatha)
> kṣiptaḥ (k#x1E63ipta#x1E25)
> viśāmaḥ (vi#x015B#x0101ma#x1E25)
> 
> At this stage everything is fine, all of the characters use the same font therefore all of them return the same
> font after evaluating (font-at (point)) on them:
> #<font-object "-ADBO-Source Code Pro-regular-normal-normal-*-32-*-*-*-m-0-iso10646-1">
> 
> 3. Now eval the following in the buffer
> (set-face-attribute 'default nil
>    :font "JetBrains Mono"
>    :weight 'regular
>    :height 170)
> 
> Now you'll notice that the font for some of the characters above are different from JetBrainsMono, these
> characters are:
> ṛ (#x1e5b)
> ṣ (#x1e63)
> ḥ (#x1e25)
> (font-at (point)) returns #<font-object "-UKWN-Latin Modern
> Mono-regular-normal-normal-*-57-*-*-*-*-0-iso10646-1"> on all of them
> while for the rest of the characters gives #<font-object "-JB-JetBrains
> Mono-regular-normal-normal-*-57-*-*-*-m-0-iso10646-1">

I downloaded the JetBrainsMono font, and I see that it doesn't have
glyphs for these characters.  Its coverage of the Latin Extended
Additional block is only partial: only 97 out of 256 characters.

So I think Emacs does TRT here, at least with this font I have here.

In your original report you said the characters were supported by the
new default font, but that is not so in this detailed recipe.

Or maybe you are using a different version of JetBrainsMono?  I have
v2.242 here.

bug#58184: Faulty font selection for Latin characters

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 2624 bytes --]

I think I may have found the problem here, JetBrains Mono does not have the
glyphs for these
"faulty" characters that is why Emacs chooses a different font for them,
but the thing is these characters
can still be displayed in the correct font i.e. JetBrains Mono by combining
the glyphs which made up the
unsupported glyph, this is why hb-view was able to display them I guess.
For example entering ṃ (#x1e43 Latin small letter m with a dot below) will
result in it being displayed in a different font,
but entering ṃ (m + #x323 Combining dot below) will result in it being
displayed with JetBrains Mono.

So now the question is should these characters be decomposed to better fit
with other characters when the font does not support them?

On Fri, Sep 30, 2022 at 5:18 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Fri, 30 Sep 2022 16:53:01 +0530
> > Cc: 58184@debbugs.gnu.org
> >
> > Here are the steps to reproduce:
> > 1. emacs -Q
> >
> > 2. enter these three words:
> > hṛṣyatha (h#x1E5B#x1E63yatha)
> > kṣiptaḥ (k#x1E63ipta#x1E25)
> > viśāmaḥ (vi#x015B#x0101ma#x1E25)
> >
> > At this stage everything is fine, all of the characters use the same
> font therefore all of them return the same
> > font after evaluating (font-at (point)) on them:
> > #<font-object "-ADBO-Source Code
> Pro-regular-normal-normal-*-32-*-*-*-m-0-iso10646-1">
> >
> > 3. Now eval the following in the buffer
> > (set-face-attribute 'default nil
> >    :font "JetBrains Mono"
> >    :weight 'regular
> >    :height 170)
> >
> > Now you'll notice that the font for some of the characters above are
> different from JetBrainsMono, these
> > characters are:
> > ṛ (#x1e5b)
> > ṣ (#x1e63)
> > ḥ (#x1e25)
> > (font-at (point)) returns #<font-object "-UKWN-Latin Modern
> > Mono-regular-normal-normal-*-57-*-*-*-*-0-iso10646-1"> on all of them
> > while for the rest of the characters gives #<font-object "-JB-JetBrains
> > Mono-regular-normal-normal-*-57-*-*-*-m-0-iso10646-1">
>
> I downloaded the JetBrainsMono font, and I see that it doesn't have
> glyphs for these characters.  Its coverage of the Latin Extended
> Additional block is only partial: only 97 out of 256 characters.
>
> So I think Emacs does TRT here, at least with this font I have here.
>
> In your original report you said the characters were supported by the
> new default font, but that is not so in this detailed recipe.
>
> Or maybe you are using a different version of JetBrainsMono?  I have
> v2.242 here.
>

[-- Attachment #2: Type: text/html, Size: 3361 bytes --]

bug#58184: Faulty font selection for Latin characters

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Fri, 30 Sep 2022 18:05:02 +0530
> Cc: 58184@debbugs.gnu.org
> 
> I think I may have found the problem here, JetBrains Mono does not have the glyphs for these
> "faulty" characters that is why Emacs chooses a different font for them, but the thing is these characters
> can still be displayed in the correct font i.e. JetBrains Mono by combining the glyphs which made up the
> unsupported glyph, this is why hb-view was able to display them I guess.
> For example entering ṃ (#x1e43 Latin small letter m with a dot below) will result in it being displayed in a
> different font,
> but entering ṃ (m + #x323 Combining dot below) will result in it being displayed with JetBrains Mono.
> 
> So now the question is should these characters be decomposed to better fit with other characters when the
> font does not support them? 

We cannot do that in the buffer text, because that would mean
modifying the text behind user's back.  And doing this in display code
woul mean activating character composition where none should happen.

I think fonts that don't have glyphs for precomposed characters
shouldn't be used in Emacs for text that could have the codepoints of
those characters.  Emacs doesn't pass every character to the shaping
engine, and so the tricks of decomposing characters to get them
displayed are something we cannot be expected to do.  Sorry.

bug#58184: Faulty font selection for Latin characters

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 1693 bytes --]

It's ok, at least I found the cause of my problem.
You can close the bug report.

Thanks

On Fri, Sep 30, 2022 at 6:22 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Fri, 30 Sep 2022 18:05:02 +0530
> > Cc: 58184@debbugs.gnu.org
> >
> > I think I may have found the problem here, JetBrains Mono does not have
> the glyphs for these
> > "faulty" characters that is why Emacs chooses a different font for them,
> but the thing is these characters
> > can still be displayed in the correct font i.e. JetBrains Mono by
> combining the glyphs which made up the
> > unsupported glyph, this is why hb-view was able to display them I guess.
> > For example entering ṃ (#x1e43 Latin small letter m with a dot below)
> will result in it being displayed in a
> > different font,
> > but entering ṃ (m + #x323 Combining dot below) will result in it being
> displayed with JetBrains Mono.
> >
> > So now the question is should these characters be decomposed to better
> fit with other characters when the
> > font does not support them?
>
> We cannot do that in the buffer text, because that would mean
> modifying the text behind user's back.  And doing this in display code
> woul mean activating character composition where none should happen.
>
> I think fonts that don't have glyphs for precomposed characters
> shouldn't be used in Emacs for text that could have the codepoints of
> those characters.  Emacs doesn't pass every character to the shaping
> engine, and so the tricks of decomposing characters to get them
> displayed are something we cannot be expected to do.  Sorry.
>

[-- Attachment #2: Type: text/html, Size: 2238 bytes --]

bug#58184: Faulty font selection for Latin characters

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Fri, 30 Sep 2022 18:25:31 +0530
> Cc: 58184@debbugs.gnu.org
> 
> It's ok, at least I found the cause of my problem.
> You can close the bug report.

Thanks, done.

bug#58184: Faulty font selection for Latin characters

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh
>  <lumarzeli30@gmail.com>
> Date: Fri, 30 Sep 2022 04:23:28 +0530
> 
> When a font is set for the "default" face in the init.el file.
> For example, like this:
> 
> (set-face-attribute 'default nil
>    :font "Fira Code"
>    :weight 'regular
>    :height 170)
> 
> This messes up the font selection for various latin and all ipa characters.

What happens if you use 'medium instead of 'regular?

> Despite the configured font supporting the characters which are typed, Emacs selects a different font for
> them, this results in visually jarring text or sometimes failed composition.
> 
> For example see in 1.png
> All of the letters except ṇ (#x1E47) are in Fira Code while it is in Latin Modern Mono despite Fira Code
> supporting it.
> Below it t̪ (t + #x32A) is not composed properly because while 't' is in FiraCode #x32A is in Step Regular.
> 
> Note: This does not happen with emacs -Q

I don't understand: what doesn't happen in "emacs -Q"?  Can you show a
recipe starting from "emacs -Q" that reproduces the problem?