Incorrect font weight selected

Incorrect font weight selected

[Yuri D'Elia]

Rebuilt today emacs on 36d873bf0d3f1185d0090e4b506a6a726476aec6, I
noticed when I start a fresh new instance of emacs the selected weight
for my default font is 'normal instead of 'medium.

I'm currently using Iosevka, which provides both regular and medium
weights, which I suspect interacts with the recent commit
1b2511fa2aed460120a36765ba16c14e355eef1d.

This is why I currently do in my init.el:

(set-face-attribute 'default nil :family "Iosevka"
                    :height 140 :weight 'medium)

If I re-evaluate this _after_ the frame has been created though, the
correct weight is selected.

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Date: Fri, 17 Dec 2021 15:51:58 +0100
> 
> Rebuilt today emacs on 36d873bf0d3f1185d0090e4b506a6a726476aec6, I
> noticed when I start a fresh new instance of emacs the selected weight
> for my default font is 'normal instead of 'medium.

Emacs by default requests the normal (a.k.a. "regular") weight, so
what you see now is the correct expected behavior.  If previously
Emacs selected the "medium" weight, that was a subtle bug.

> This is why I currently do in my init.el:
> 
> (set-face-attribute 'default nil :family "Iosevka"
>                     :height 140 :weight 'medium)
> 
> If I re-evaluate this _after_ the frame has been created though, the
> correct weight is selected.

This is the intended behavior, so I don't think there's anything we
need to fix here.

Re: Incorrect font weight selected

[Yuri D'Elia]

On Fri, Dec 17 2021, Eli Zaretskii wrote:
> Emacs by default requests the normal (a.k.a. "regular") weight, so
> what you see now is the correct expected behavior.  If previously
> Emacs selected the "medium" weight, that was a subtle bug.

But as you saw, I'm explicitly setting the default face, and that
includes selecting 'medium as the font weight.

>> (set-face-attribute 'default nil :family "Iosevka"
>>                     :height 140 :weight 'medium)
>> 
>> If I re-evaluate this _after_ the frame has been created though, the
>> correct weight is selected.
>
> This is the intended behavior, so I don't think there's anything we
> need to fix here.

Why this call should select a different font when executed during init
vs. when called interactively?

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Date: Fri, 17 Dec 2021 20:47:44 +0100
> 
> On Fri, Dec 17 2021, Eli Zaretskii wrote:
> > Emacs by default requests the normal (a.k.a. "regular") weight, so
> > what you see now is the correct expected behavior.  If previously
> > Emacs selected the "medium" weight, that was a subtle bug.
> 
> But as you saw, I'm explicitly setting the default face, and that
> includes selecting 'medium as the font weight.

Sorry, maybe I didn't understand your report, chronology-wise.  What
happened before and what happened after the change?  And when did you
add the set-face-attribute call to your init.el file?

> >> (set-face-attribute 'default nil :family "Iosevka"
> >>                     :height 140 :weight 'medium)
> >> 
> >> If I re-evaluate this _after_ the frame has been created though, the
> >> correct weight is selected.
> >
> > This is the intended behavior, so I don't think there's anything we
> > need to fix here.
> 
> Why this call should select a different font when executed during init
> vs. when called interactively?

Now I'm totally confused: what does "interactive" has to do with this?
If you put the same into your init.el, in a
after-make-frame-functions, doesn't it produce the same effect as
invoking it interactively after Emacs started?

Re: Incorrect font weight selected

[Yuri D'Elia]

On Fri, Dec 17 2021, Eli Zaretskii wrote:
> Now I'm totally confused: what does "interactive" has to do with this?
> If you put the same into your init.el, in a
> after-make-frame-functions, doesn't it produce the same effect as
> invoking it interactively after Emacs started?

I debugged a little bit further with some extra details.
To recap, I have:

(set-face-attribute 'default nil :family "Iosevka"
                     :height 140 :weight 'medium)

somewhere at the end of my init.el. It has always been there.
It is not in after-make-frame-functions.

If I start emacs normally (not in daemon mode), the 'medium weight is
selected correctly.

When I start emacs with --daemon and then create a new frame via
emacsclient, the 'regular weight is used for the new frame instead.

This is new. Until the build from ~4 days ago, 'medium was selected
correctly in both cases. In both cases, the correct font family and size
is used: only the weight is incorrect.

When I revert commit 1b2511fa2aed460120a36765ba16c14e355eef1d then the
'medium weight is selected correctly in both scenarios.

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 17 Dec 2021 22:25:32 +0100
> 
> (set-face-attribute 'default nil :family "Iosevka"
>                      :height 140 :weight 'medium)
> 
> somewhere at the end of my init.el. It has always been there.
> It is not in after-make-frame-functions.
> 
> If I start emacs normally (not in daemon mode), the 'medium weight is
> selected correctly.

OK, so this use case works correctly and as intended.

> When I start emacs with --daemon and then create a new frame via
> emacsclient, the 'regular weight is used for the new frame instead.
> 
> This is new. Until the build from ~4 days ago, 'medium was selected
> correctly in both cases. In both cases, the correct font family and size
> is used: only the weight is incorrect.
> 
> When I revert commit 1b2511fa2aed460120a36765ba16c14e355eef1d then the
> 'medium weight is selected correctly in both scenarios.

In a daemon session, you are supposed to put such customizations in
the server-after-make-frame-hook.  If you do that, does it work as
expected?  Before a GUI frame is created, customizations of faces and
fonts doesn't work as you'd expect because the GUI system may not be
available yet (the frame which the daemon has is basically a surrogate
TTY frame).

The commit you point to made font selection more lenient wrt medium vs
regular, so I'd be surprised if it prevented some weight selection.  I
believe it previously worked for you by sheer luck.

Re: Incorrect font weight selected

[Yuri D'Elia]

On Sat, Dec 18 2021, Eli Zaretskii wrote:
>> When I revert commit 1b2511fa2aed460120a36765ba16c14e355eef1d then the
>> 'medium weight is selected correctly in both scenarios.
>
> In a daemon session, you are supposed to put such customizations in
> the server-after-make-frame-hook.  If you do that, does it work as
> expected?  Before a GUI frame is created, customizations of faces and
> fonts doesn't work as you'd expect because the GUI system may not be
> available yet (the frame which the daemon has is basically a surrogate
> TTY frame).

I understand this, however I really fail to see how this explains the
behavior.

I would understand if it didn't resolve the font family at all,
or it didn't set the correct resolution according to the current
fontconfig settings for example, but it does both correctly.

Why would it change only the weight selection?

I assumed that when setting the default font (by calling
set-face-attribute with FRAME set to nil) the actual face selection
would be effectively delayed until frame creation time. I guess this is
incorrect - i/e font selection is done immediately?

> The commit you point to made font selection more lenient wrt medium vs
> regular, so I'd be surprised if it prevented some weight selection.  I
> believe it previously worked for you by sheer luck.

As I was adjusting the code to move my setup into an after-make-frame
hook, I noticed another subtlety:

Assuming the following skeleton:

(add-hook 'server-after-make-frame-hook
  (lambda ()
    (set-face-attribute 'default FRAME ...)))

I get different results if FRAME is t or nil.

Ideally I'd want nil (default). If I do that, it seems to work
correctly. However in such case I'd also need to ensure I call this hook
once, not for every frame (the main reason I didn't bother calling this
in a make-frame hook) and gets slower as I add frames.

If I use "t" to set the current frame only, I get a completely different
font size: smaller in fact. However, this only holds if I _never_ set
the "default frame".

If I evaluate instead:

(set-face-attribute 'default nil ...)
(set-face-attribute 'default t ...)

where "..." is the same face selection. In such case, setting the
default frame somewhat alters permanently what setting the current frame
will do. The second call to set-face-attribute doesn't give me the
smaller face, instead it gives me the same face twice.

I'm totally confused as of why this happens.

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 18 Dec 2021 11:43:25 +0100
> 
> or it didn't set the correct resolution according to the current
> fontconfig settings for example, but it does both correctly.
> 
> Why would it change only the weight selection?

I don't know.

I don't have access to any system where this happens, so I cannot step
through the code to see what happens there.  You (or someone else) are
welcome to do that and see what happens there.  Maybe we will then be
able to make this less surprising and confusing.

> I assumed that when setting the default font (by calling
> set-face-attribute with FRAME set to nil) the actual face selection
> would be effectively delayed until frame creation time. I guess this is
> incorrect - i/e font selection is done immediately?

Yes.

> Ideally I'd want nil (default). If I do that, it seems to work
> correctly. However in such case I'd also need to ensure I call this hook
> once, not for every frame (the main reason I didn't bother calling this
> in a make-frame hook) and gets slower as I add frames.

You could use a simple flag variable for that.

> If I use "t" to set the current frame only, I get a completely different
> font size: smaller in fact. However, this only holds if I _never_ set
> the "default frame".
> 
> If I evaluate instead:
> 
> (set-face-attribute 'default nil ...)
> (set-face-attribute 'default t ...)
> 
> where "..." is the same face selection. In such case, setting the
> default frame somewhat alters permanently what setting the current frame
> will do. The second call to set-face-attribute doesn't give me the
> smaller face, instead it gives me the same face twice.
> 
> I'm totally confused as of why this happens.

Welcome to the club.  The code which selects fonts in Emacs is quite
complex and notoriously under-documented.  On top of that, we don't
have any experts on board who are familiar with that code and can
readily answer questions such as this one.  The only way to
investigate is to step through the code, starting in xfaces.c (where
we process set-face-attribute), and following into font.c and
fontset.c, where the font selection actually happens.

Re: Incorrect font weight selected

[Yuri D'Elia]

On Sat, Dec 18 2021, Eli Zaretskii wrote:
>> I assumed that when setting the default font (by calling
>> set-face-attribute with FRAME set to nil) the actual face selection
>> would be effectively delayed until frame creation time. I guess this is
>> incorrect - i/e font selection is done immediately?
>
> Yes.

Just out of curiosity, if you happen to know it, how does the "custom"
machinery does this then regarding to faces? It it also done in a frame
hook?

As it looks like we're able to alter the face
colors/parameters/inheritance at will, but still have to delay setting
the font family until the frame is setup. This is kind of ugly.

> You could use a simple flag variable for that.

Regarding that...

> Welcome to the club.  The code which selects fonts in Emacs is quite
> complex and notoriously under-documented.  On top of that, we don't
> have any experts on board who are familiar with that code and can
> readily answer questions such as this one.  The only way to
> investigate is to step through the code, starting in xfaces.c (where
> we process set-face-attribute), and following into font.c and
> fontset.c, where the font selection actually happens.

So I now moved the (set-face-attribute 'default nil ...) call to run
once after the first frame is created.

It works. But for the first frame only (!).

When I create a new frame, the second new frame still selects the wrong
weight.

Duh...

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 18 Dec 2021 13:00:52 +0100
> 
> On Sat, Dec 18 2021, Eli Zaretskii wrote:
> >> I assumed that when setting the default font (by calling
> >> set-face-attribute with FRAME set to nil) the actual face selection
> >> would be effectively delayed until frame creation time. I guess this is
> >> incorrect - i/e font selection is done immediately?
> >
> > Yes.
> 
> Just out of curiosity, if you happen to know it, how does the "custom"
> machinery does this then regarding to faces? It it also done in a frame
> hook?

You mean, how it makes the customization be in effect for future
frames?  No, it just records the change in the default faces for new
frames.

Re: Incorrect font weight selected

[Sean Whitton]

Hello,

On Fri 17 Dec 2021 at 03:51PM +01, Yuri D'Elia wrote:

> Rebuilt today emacs on 36d873bf0d3f1185d0090e4b506a6a726476aec6, I
> noticed when I start a fresh new instance of emacs the selected weight
> for my default font is 'normal instead of 'medium.
>
> I'm currently using Iosevka, which provides both regular and medium
> weights, which I suspect interacts with the recent commit
> 1b2511fa2aed460120a36765ba16c14e355eef1d.
>
> This is why I currently do in my init.el:
>
> (set-face-attribute 'default nil :family "Iosevka"
>                     :height 140 :weight 'medium)
>
> If I re-evaluate this _after_ the frame has been created though, the
> correct weight is selected.

I am seeing something similar but distinct, I think.  I'm using pgtk.
If I use M-: to evaluate this form

    (set-frame-font (font-spec :name "Inconsolata-13" :weight 'medium))

then what I get is the regular weight, not the medium weight.  It is as
though the :weight parameter is being completely ignored.

-- 
Sean Whitton

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Sean Whitton <spwhitton@spwhitton.name>
> Date: Sat, 18 Dec 2021 16:26:16 -0700
> 
> I am seeing something similar but distinct, I think.  I'm using pgtk.
> If I use M-: to evaluate this form
> 
>     (set-frame-font (font-spec :name "Inconsolata-13" :weight 'medium))
> 
> then what I get is the regular weight, not the medium weight.  It is as
> though the :weight parameter is being completely ignored.

Does that font have the 'medium' weight?

Re: Incorrect font weight selected

[Yuri D'Elia]

On Sat, Dec 18 2021, Eli Zaretskii wrote:
>> Just out of curiosity, if you happen to know it, how does the "custom"
>> machinery does this then regarding to faces? It it also done in a frame
>> hook?
>
> You mean, how it makes the customization be in effect for future
> frames?  No, it just records the change in the default faces for new
> frames.

Which is ok, but then again does it also delay the font selection until
the first graphical frame is selected? Otherwise it would also suffer
from selecting the font too early when used in daemon mode.

On my side, I've tried a few other approaches by never setting the
default font (all frames), and only performing lookup/setting
frame-specific fonts in the after-make-frame hooks. I'm running into all
sort of quirks. Aside from the completely incorrect size (which I've
just bumped up for the sake of testing), it looks like a lot of other
things get broken. For example the bold version of the 'default face
shows undefined glypths (the unicode square thingy), but only when the
second frame is created. The second frame gets an identical treatment in
the hook. Again, all these issues disappear if I set the default font
for all frames at least once, and looks like I can set _any_ font. This
seems to trigger some sort of one-time setup.

Re: Incorrect font weight selected

[Yuri D'Elia]

On Sun, Dec 19 2021, Eli Zaretskii wrote:
>>     (set-frame-font (font-spec :name "Inconsolata-13" :weight 'medium))
>>
>> then what I get is the regular weight, not the medium weight.  It is as
>> though the :weight parameter is being completely ignored.
>
> Does that font have the 'medium' weight?

$ fc-list 'Inconsolata' | grep Medium
/usr/share/fonts/truetype/inconsolata/Inconsolata.otf: Inconsolata:style=Medium

However:

$ fc-list 'Iosevka' -f '%{style}\n' | grep Medium
Medium Extended Oblique,Regular
Medium Extended,Regular
Medium Oblique,Regular
Medium Italic,Italic
Medium Extended Italic,Italic
Medium,Regular
^^^^^^^^^^^^^^

Note that "Regular" exists as a style on it's own.

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: emacs-devel@gnu.org
> Date: Sun, 19 Dec 2021 12:14:28 +0100
> 
> On Sat, Dec 18 2021, Eli Zaretskii wrote:
> >> Just out of curiosity, if you happen to know it, how does the "custom"
> >> machinery does this then regarding to faces? It it also done in a frame
> >> hook?
> >
> > You mean, how it makes the customization be in effect for future
> > frames?  No, it just records the change in the default faces for new
> > frames.
> 
> Which is ok, but then again does it also delay the font selection until
> the first graphical frame is selected? Otherwise it would also suffer
> from selecting the font too early when used in daemon mode.

No, AFAIK font selection happens as soon as Emacs processes the
set-face-attribute call (or its equivalent) which sets the font or its
family for the default face.

> On my side, I've tried a few other approaches by never setting the
> default font (all frames), and only performing lookup/setting
> frame-specific fonts in the after-make-frame hooks. I'm running into all
> sort of quirks. Aside from the completely incorrect size (which I've
> just bumped up for the sake of testing), it looks like a lot of other
> things get broken. For example the bold version of the 'default face
> shows undefined glypths (the unicode square thingy), but only when the
> second frame is created. The second frame gets an identical treatment in
> the hook. Again, all these issues disappear if I set the default font
> for all frames at least once, and looks like I can set _any_ font. This
> seems to trigger some sort of one-time setup.

I'm not sure, after reading this, whether or not you found a
satisfactory solution, but if you are still looking for a hook to run
this kind of customization, you may wish trying emacs-startup-hook and
window-setup-hook.

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: Sean Whitton <spwhitton@spwhitton.name>, emacs-devel@gnu.org
> Date: Sun, 19 Dec 2021 12:29:10 +0100
> 
> On Sun, Dec 19 2021, Eli Zaretskii wrote:
> >>     (set-frame-font (font-spec :name "Inconsolata-13" :weight 'medium))
> >>
> >> then what I get is the regular weight, not the medium weight.  It is as
> >> though the :weight parameter is being completely ignored.
> >
> > Does that font have the 'medium' weight?
> 
> $ fc-list 'Inconsolata' | grep Medium
> /usr/share/fonts/truetype/inconsolata/Inconsolata.otf: Inconsolata:style=Medium
> 
> However:
> 
> $ fc-list 'Iosevka' -f '%{style}\n' | grep Medium
> Medium Extended Oblique,Regular
> Medium Extended,Regular
> Medium Oblique,Regular
> Medium Italic,Italic
> Medium Extended Italic,Italic
> Medium,Regular
> ^^^^^^^^^^^^^^

I asked about the weight, not the style.

Emacs 29 distinguishes between 'medium' and 'regular'/'normal' weight,
and I don't think I see 'medium' in the above output (or maybe I
misunderstand it).

Can you show the numerical value of the weight for each one of the
above font varieties?  (Sorry, I don't know how to ask fc to report
that.)

Re: Incorrect font weight selected

[Yuri D'Elia]

On Sun, Dec 19 2021, Eli Zaretskii wrote:
> I asked about the weight, not the style.

Good point, got confused :/

> Emacs 29 distinguishes between 'medium' and 'regular'/'normal' weight,
> and I don't think I see 'medium' in the above output (or maybe I
> misunderstand it).
>
> Can you show the numerical value of the weight for each one of the
> above font varieties?  (Sorry, I don't know how to ask fc to report
> that.)

$ fc-list Iosevka -f '%{weight} %{style}\n' | egrep -i 'Medium|Regular' | sort -n
0 Thin Extended Oblique,Regular
0 Thin Extended,Regular
0 Thin Oblique,Regular
0 Thin,Regular
40 Extralight Extended Oblique,Regular
40 Extralight Extended,Regular
40 Extralight Oblique,Regular
40 Extralight,Regular
50 Light Extended Oblique,Regular
50 Light Extended,Regular
50 Light Oblique,Regular
50 Light,Regular
80 Extended Oblique,Regular
80 Extended,Regular
80 Oblique,Regular
80 Regular
100 Medium Extended Italic,Italic
100 Medium Extended Oblique,Regular
100 Medium Extended,Regular
100 Medium Italic,Italic
100 Medium Oblique,Regular
100 Medium,Regular
180 Semibold Extended Oblique,Regular
180 Semibold Extended,Regular
180 Semibold Oblique,Regular
180 Semibold,Regular
205 Extrabold Extended Oblique,Regular
205 Extrabold Extended,Regular
205 Extrabold Oblique,Regular
205 Extrabold,Regular
210 Heavy Extended Oblique,Regular
210 Heavy Extended,Regular
210 Heavy Oblique,Regular
210 Heavy,Regular

Keep in mind I'm also not a font guru.

$ fc-list Inconsolata -f '%{weight} %{style}\n'
100 Medium

So Inconsolata _only_ has Medium on my system (this is Inconsolata
001.010-6 from the debian unstable package). I assume the other weights
are synthesized via freetype in this case.

For comparison:

$ fc-list 'Noto Sans' -f '%{weight} %{style}\n' | sort -n
80 Italic
80 Regular
200 Bold
200 Bold Italic

So the weight mapping in Iosevka seems at least superficially correct.
Note that Iosevka is a open font, just in case you want to test this:

https://github.com/be5invis/Iosevka/

The version I've been using for these tests:

https://github.com/be5invis/Iosevka/releases/download/v11.2.0/super-ttc-iosevka-11.2.0.zip

it's a massive font due to all the style/weight combinations.

Re: Incorrect font weight selected

[Yuri D'Elia]

On Sun, Dec 19 2021, Eli Zaretskii wrote:
> I'm not sure, after reading this, whether or not you found a
> satisfactory solution, but if you are still looking for a hook to run
> this kind of customization, you may wish trying emacs-startup-hook and
> window-setup-hook.

Not found a satisfactory solution yet. I was trying to understand how
and what currently works before attempting to go deeper with gdb. It
looks like that even if I do things "properly", I'm running into subtle
font selection issues.

Re: Incorrect font weight selected

[Lars Ingebrigtsen]

Yuri D'Elia <wavexx@thregr.org> writes:

> Not found a satisfactory solution yet. I was trying to understand how
> and what currently works before attempting to go deeper with gdb. It
> looks like that even if I do things "properly", I'm running into subtle
> font selection issues.

There was a long thread touching upon these issues in the bug tracker
somewhere.  Basically, Gtk tells us the font weights in PANGO_WEIGHT_*,
and we map these to symbols according to spec, see xg_weight_to_symbol.
And then we map these to a 0-255 scale in weight_table, and it's these
weights that are really in the fonts.

And PANGO and the fonts don't always agree about these things.

If I remember correctly; it's been several months since I looked at this
last, and I've already managed to suppress most of it.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: spwhitton@spwhitton.name, emacs-devel@gnu.org
> Date: Sun, 19 Dec 2021 13:57:06 +0100
> 
> On Sun, Dec 19 2021, Eli Zaretskii wrote:
> > I asked about the weight, not the style.
> 
> Good point, got confused :/
> 
> > Emacs 29 distinguishes between 'medium' and 'regular'/'normal' weight,
> > and I don't think I see 'medium' in the above output (or maybe I
> > misunderstand it).
> >
> > Can you show the numerical value of the weight for each one of the
> > above font varieties?  (Sorry, I don't know how to ask fc to report
> > that.)
> 
> $ fc-list Iosevka -f '%{weight} %{style}\n' | egrep -i 'Medium|Regular' | sort -n
> 0 Thin Extended Oblique,Regular
> 0 Thin Extended,Regular
> 0 Thin Oblique,Regular
> 0 Thin,Regular
> 40 Extralight Extended Oblique,Regular
> 40 Extralight Extended,Regular
> 40 Extralight Oblique,Regular
> 40 Extralight,Regular
> 50 Light Extended Oblique,Regular
> 50 Light Extended,Regular
> 50 Light Oblique,Regular
> 50 Light,Regular
> 80 Extended Oblique,Regular
> 80 Extended,Regular
> 80 Oblique,Regular
> 80 Regular
> 100 Medium Extended Italic,Italic
> 100 Medium Extended Oblique,Regular
> 100 Medium Extended,Regular
> 100 Medium Italic,Italic
> 100 Medium Oblique,Regular
> 100 Medium,Regular
> 180 Semibold Extended Oblique,Regular
> 180 Semibold Extended,Regular
> 180 Semibold Oblique,Regular
> 180 Semibold,Regular
> 205 Extrabold Extended Oblique,Regular
> 205 Extrabold Extended,Regular
> 205 Extrabold Oblique,Regular
> 205 Extrabold,Regular
> 210 Heavy Extended Oblique,Regular
> 210 Heavy Extended,Regular
> 210 Heavy Oblique,Regular
> 210 Heavy,Regular
> 
> Keep in mind I'm also not a font guru.
> 
> $ fc-list Inconsolata -f '%{weight} %{style}\n'
> 100 Medium
> 
> So Inconsolata _only_ has Medium on my system (this is Inconsolata
> 001.010-6 from the debian unstable package). I assume the other weights
> are synthesized via freetype in this case.
> 
> For comparison:
> 
> $ fc-list 'Noto Sans' -f '%{weight} %{style}\n' | sort -n
> 80 Italic
> 80 Regular
> 200 Bold
> 200 Bold Italic
> 
> So the weight mapping in Iosevka seems at least superficially correct.

There are two different people in this discussion with two different
font families and with potentially different setups of their systems.
I no longer understand what questions am I supposed to answer.

> Note that Iosevka is a open font, just in case you want to test this:
> 
> https://github.com/be5invis/Iosevka/

I cannot test this on machines to which I have access, sorry.

Anyway, I asked about Inconsolata, not Iosevka, and I asked Sean about
his setup.

Re: Incorrect font weight selected

[Sean Whitton]

[-- Attachment #1: Type: text/plain, Size: 1758 bytes --]

Hello,

On Sun 19 Dec 2021 at 02:52PM +02, Eli Zaretskii wrote:

>> From: Yuri D'Elia <wavexx@thregr.org>
>> Cc: Sean Whitton <spwhitton@spwhitton.name>, emacs-devel@gnu.org
>> Date: Sun, 19 Dec 2021 12:29:10 +0100
>> 
>> On Sun, Dec 19 2021, Eli Zaretskii wrote:
>> >>     (set-frame-font (font-spec :name "Inconsolata-13" :weight 'medium))
>> >>
>> >> then what I get is the regular weight, not the medium weight.  It is as
>> >> though the :weight parameter is being completely ignored.
>> >
>> > Does that font have the 'medium' weight?

I think so, though see fc-list output below.

I customized the 'default' face to medium Inconsolata, and commented out
my code which directly calls set-frame-font, and then I get something
which looks the same as it used to (I compared some screenshots).

> Emacs 29 distinguishes between 'medium' and 'regular'/'normal' weight,
> and I don't think I see 'medium' in the above output (or maybe I
> misunderstand it).
>
> Can you show the numerical value of the weight for each one of the
> above font varieties?  (Sorry, I don't know how to ask fc to report
> that.)

Here is what I get:

% fc-list Inconsolata family weight | grep -i medium

Inconsolata,Inconsolata ExpandedMedium:weight=100
Inconsolata,Inconsolata UltraExpandedMedium:weight=100
Inconsolata,Inconsolata CondensedMedium:weight=100
Inconsolata,Inconsolata UltraCondensedMedium:weight=100
Inconsolata,Inconsolata SemiCondensedMedium:weight=100
Inconsolata,Inconsolata ExtraExpandedMedium:weight=100
Inconsolata,Inconsolata Medium:weight=100
Inconsolata,Inconsolata ExtraCondensedMedium:weight=100
Inconsolata,Inconsolata SemiExpandedMedium:weight=100

I'm attaching the un-grepped output so you can see the other numerical
weights.

-- 
Sean Whitton

[-- Attachment #2: fc-list-Inconsolata --]
[-- Type: application/octet-stream, Size: 3116 bytes --]

Inconsolata,Inconsolata UltraExpandedExtraLight:weight=45.5
Inconsolata,Inconsolata CondensedLight:weight=50
Inconsolata,Inconsolata ExtraExpandedExtraLight:weight=45.5
Inconsolata,Inconsolata CondensedExtraLight:weight=45.5
Inconsolata,Inconsolata UltraExpandedBlack:weight=210
Inconsolata,Inconsolata Black:weight=210
Inconsolata,Inconsolata ExtraCondensedBlack:weight=210
Inconsolata,Inconsolata ExpandedMedium:weight=100
Inconsolata,Inconsolata ExpandedExtraLight:weight=45.5
Inconsolata,Inconsolata ExtraBold:weight=205
Inconsolata,Inconsolata UltraExpandedMedium:weight=100
Inconsolata,Inconsolata ExtraExpandedSemiBold:weight=180
Inconsolata,Inconsolata UltraCondensedBlack:weight=210
Inconsolata,Inconsolata UltraExpandedSemiBold:weight=180
Inconsolata,Inconsolata UltraCondensedSemiBold:weight=180
Inconsolata,Inconsolata ExtraExpandedBlack:weight=210
Inconsolata,Inconsolata UltraCondensedExtraLight:weight=45.5
Inconsolata,Inconsolata CondensedMedium:weight=100
Inconsolata,Inconsolata ExtraLight:weight=45.5
Inconsolata,Inconsolata SemiCondensedBlack:weight=210
Inconsolata,Inconsolata ExtraCondensedExtraBold:weight=205
Inconsolata,Inconsolata CondensedSemiBold:weight=180
Inconsolata,Inconsolata ExtraCondensedSemiBold:weight=180
Inconsolata,Inconsolata SemiExpandedBlack:weight=210
Inconsolata,Inconsolata SemiCondensedExtraBold:weight=205
Inconsolata,Inconsolata UltraExpandedLight:weight=50
Inconsolata,Inconsolata UltraCondensedMedium:weight=100
Inconsolata,Inconsolata SemiCondensedMedium:weight=100
Inconsolata,Inconsolata ExpandedSemiBold:weight=180
Inconsolata,Inconsolata SemiCondensedSemiBold:weight=180
Inconsolata,Inconsolata ExtraCondensedLight:weight=50
Inconsolata,Inconsolata ExtraCondensedExtraLight:weight=45.5
Inconsolata,Inconsolata ExpandedExtraBold:weight=205
Inconsolata,Inconsolata UltraCondensedLight:weight=50
Inconsolata,Inconsolata ExpandedLight:weight=50
Inconsolata,Inconsolata CondensedBlack:weight=210
Inconsolata,Inconsolata ExtraExpandedLight:weight=50
Inconsolata,Inconsolata CondensedExtraBold:weight=205
Inconsolata,Inconsolata ExtraExpandedMedium:weight=100
Inconsolata,Inconsolata UltraCondensedExtraBold:weight=205
Inconsolata,Inconsolata SemiExpandedExtraLight:weight=45.5
Inconsolata,Inconsolata ExtraExpandedExtraBold:weight=205
Inconsolata,Inconsolata SemiExpandedLight:weight=50
Inconsolata,Inconsolata SemiCondensedLight:weight=50
Inconsolata,Inconsolata Light:weight=50
Inconsolata:weight=50
Inconsolata:weight=40
Inconsolata:weight=100
Inconsolata:weight=80
Inconsolata:weight=180
Inconsolata,Inconsolata Medium:weight=100
Inconsolata,Inconsolata SemiBold:weight=180
Inconsolata,Inconsolata SemiCondensedExtraLight:weight=45.5
Inconsolata:weight=210
Inconsolata:weight=205
Inconsolata:weight=200
Inconsolata,Inconsolata UltraExpandedExtraBold:weight=205
Inconsolata,Inconsolata SemiExpandedExtraBold:weight=205
Inconsolata,Inconsolata ExtraCondensedMedium:weight=100
Inconsolata,Inconsolata SemiExpandedSemiBold:weight=180
Inconsolata:weight=[40 210]
Inconsolata,Inconsolata SemiExpandedMedium:weight=100
Inconsolata,Inconsolata ExpandedBlack:weight=210

Re: Incorrect font weight selected

[Yuri D'Elia]

On Sun, Dec 19 2021, Sean Whitton wrote:
> I think so, though see fc-list output below.
>
> I customized the 'default' face to medium Inconsolata, and commented out
> my code which directly calls set-frame-font, and then I get something
> which looks the same as it used to (I compared some screenshots).

So you switched to customize?
Could you provide some screenshots before/after?

Also, what happens when you create a new frame? (M-x make-frame)

Re: Incorrect font weight selected

[Sean Whitton]

Hello,

On Sun 19 Dec 2021 at 11:04PM +01, Yuri D'Elia wrote:

> On Sun, Dec 19 2021, Sean Whitton wrote:
>> I think so, though see fc-list output below.
>>
>> I customized the 'default' face to medium Inconsolata, and commented out
>> my code which directly calls set-frame-font, and then I get something
>> which looks the same as it used to (I compared some screenshots).
>
> So you switched to customize?

Well, I'd rather keep my programmatic font setting, as it does a few
things which are useful to me.

What's weird is how the weight seems to be ignored in the set-face-font
call but not in the customisation.

> Could you provide some screenshots before/after?

Before/after what, sorry?

> Also, what happens when you create a new frame? (M-x make-frame)

Do you mean with my original programmatic font setting?

It gives me (what looks like) a regular rather than medium weight.

-- 
Sean Whitton

Re: Incorrect font weight selected

[Yuri D'Elia]

[-- Attachment #1: Type: text/plain, Size: 964 bytes --]

On Fri, Dec 17 2021, Yuri D'Elia wrote:
> When I start emacs with --daemon and then create a new frame via
> emacsclient, the 'regular weight is used for the new frame instead.
>
> This is new. Until the build from ~4 days ago, 'medium was selected
> correctly in both cases. In both cases, the correct font family and size
> is used: only the weight is incorrect.
>
> When I revert commit 1b2511fa2aed460120a36765ba16c14e355eef1d then the
> 'medium weight is selected correctly in both scenarios.

After debugging a little bit on this, I came up with a relatively short
test to reproduce the issue.

Consider the following test.el:

  (set-face-attribute 'default nil :family "Iosevka SS08" :height 140 :weight 'bold)
  (make-frame)

started with emacs -q -l test.el.

This starts emacs, sets the default face with the iosevka family with
bold, then creates a second frame. The original frame is at the top. The
second frame at the bottom. master.png following:


[-- Attachment #2: master.png --]
[-- Type: image/png, Size: 56090 bytes --]

[-- Attachment #3: Type: text/plain, Size: 237 bytes --]


Notice how I'm using "bold" now, but the resulting face in the new frame
is still just regular. Turns out medium has nothing to do with this.

Now with commit 1b2511fa2aed460120a36765ba16c14e355eef1d reverted
(reverted.png following):


[-- Attachment #4: reverted.png --]
[-- Type: image/png, Size: 56766 bytes --]

[-- Attachment #5: Type: text/plain, Size: 550 bytes --]


Note how both frames are now bold. Please note the mode-line face as
well. I'm not changing the face there. The incorrect weight selection is
also affecting the face there.

Indeed, it seems that any font weight which is within "100" in the
weight table can be susceptible to bad selection. Commit
1b2511fa2aed460120a36765ba16c14e355eef1d by itself makes sense. We
likely have a problem elsewhere.

Maybe font_sort_entities is not sorting the matches correctly by weight,
or we are calling it wrong in some places.

Cannot go further at the moment.

Re: Incorrect font weight selected

[Lars Ingebrigtsen]

Yuri D'Elia <wavexx@thregr.org> writes:

> Notice how I'm using "bold" now, but the resulting face in the new frame
> is still just regular. Turns out medium has nothing to do with this.

I can reproduce this behaviour on the current trunk, too.

> Indeed, it seems that any font weight which is within "100" in the
> weight table can be susceptible to bad selection. Commit
> 1b2511fa2aed460120a36765ba16c14e355eef1d by itself makes sense. We
> likely have a problem elsewhere.
>
> Maybe font_sort_entities is not sorting the matches correctly by weight,
> or we are calling it wrong in some places.

Sounds like it.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 20 Dec 2021 00:24:20 +0100
> 
> Note how both frames are now bold. Please note the mode-line face as
> well. I'm not changing the face there. The incorrect weight selection is
> also affecting the face there.
> 
> Indeed, it seems that any font weight which is within "100" in the
> weight table can be susceptible to bad selection. Commit
> 1b2511fa2aed460120a36765ba16c14e355eef1d by itself makes sense. We
> likely have a problem elsewhere.
> 
> Maybe font_sort_entities is not sorting the matches correctly by weight,
> or we are calling it wrong in some places.
> 
> Cannot go further at the moment.

Thanks, I think you are getting closer to the root cause.  Please keep
going.

I'm not wedded to the change in 1b2511f: it was originally concocted
for MS-Windows, so maybe it is not right for Fontconfig-based
platforms.

Re: Incorrect font weight selected

[Stefan Monnier]

> I can reproduce this behaviour on the current trunk, too.

Another way to reproduce the problem (which corresponds to the way the
problem affects my setup):

    emacs -Q --eval '(setq minibuffer-frame-alist `((font . "-misc-fixed-bold-r-normal-*-13-*-*-*-*-*-*-*")) default-frame-alist `((minibuffer . nil)))'

with the old code (e.g. Emacs-28, Emacs-27, ...) the minibuffer frame
gets a "misc fixed bold" font, whereas with the current code on `master`
the minibuffer frame gets a "misc fixed non-bold" font.

I see the same with "DejaVu Sans Bold" instead of
"-misc-fixed-bold-r-normal-*-13-*-*-*-*-*-*-*", so you can probably
reproduce it with "any" other font.

[ "misc-fixed" was a widespread monospaced font in X11 in the days
  before anti-aliased fonts.  ]


        Stefan

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Yuri D'Elia <wavexx@thregr.org>,  Eli Zaretskii <eliz@gnu.org>,
>  emacs-devel@gnu.org
> Date: Mon, 20 Dec 2021 14:43:42 -0500
> 
>     emacs -Q --eval '(setq minibuffer-frame-alist `((font . "-misc-fixed-bold-r-normal-*-13-*-*-*-*-*-*-*")) default-frame-alist `((minibuffer . nil)))'
> 
> with the old code (e.g. Emacs-28, Emacs-27, ...) the minibuffer frame
> gets a "misc fixed bold" font, whereas with the current code on `master`
> the minibuffer frame gets a "misc fixed non-bold" font.
> 
> I see the same with "DejaVu Sans Bold" instead of
> "-misc-fixed-bold-r-normal-*-13-*-*-*-*-*-*-*", so you can probably
> reproduce it with "any" other font.

And if you revert commit 1b2511f?

Re: Incorrect font weight selected

[Stefan Monnier]

> I see the same with "DejaVu Sans Bold" instead of
> "-misc-fixed-bold-r-normal-*-13-*-*-*-*-*-*-*", so you can probably
> reproduce it with "any" other font.

Hm.. scratch that: the problem doesn't seem to appear with
"DejaVu Sans Bold".


        Stefan

Re: Incorrect font weight selected

[Stefan Monnier]

> And if you revert commit 1b2511f?

Yes, reverting that commit brings back the bold font in the
minibuffer-only frame.


        Stefan

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: wavexx@thregr.org,  larsi@gnus.org,  emacs-devel@gnu.org
> Date: Mon, 20 Dec 2021 15:14:39 -0500
> 
> > And if you revert commit 1b2511f?
> 
> Yes, reverting that commit brings back the bold font in the
> minibuffer-only frame.

Then, barring any insights, I will revert it soon.  Dmitry will not
like that, but...

Re: Incorrect font weight selected

[Sean Whitton]

Hello,

On Sun 19 Dec 2021 at 02:03PM -07, Sean Whitton wrote:

> Hello,
>
> On Sun 19 Dec 2021 at 02:52PM +02, Eli Zaretskii wrote:
>
>>> From: Yuri D'Elia <wavexx@thregr.org>
>>> Cc: Sean Whitton <spwhitton@spwhitton.name>, emacs-devel@gnu.org
>>> Date: Sun, 19 Dec 2021 12:29:10 +0100
>>>
>>> On Sun, Dec 19 2021, Eli Zaretskii wrote:
>>> >>     (set-frame-font (font-spec :name "Inconsolata-13" :weight 'medium))
>>> >>
>>> >> then what I get is the regular weight, not the medium weight.  It is as
>>> >> though the :weight parameter is being completely ignored.
>>> >
>>> > Does that font have the 'medium' weight?
>
> I think so, though see fc-list output below.
>
> I customized the 'default' face to medium Inconsolata, and commented out
> my code which directly calls set-frame-font, and then I get something
> which looks the same as it used to (I compared some screenshots).

Reverting 1b2511f fixes the problem.

-- 
Sean Whitton

Re: Incorrect font weight selected

[Dmitry Gutov]

On 20.12.2021 23:19, Eli Zaretskii wrote:
>> From: Stefan Monnier<monnier@iro.umontreal.ca>
>> Cc:wavexx@thregr.org,larsi@gnus.org,emacs-devel@gnu.org
>> Date: Mon, 20 Dec 2021 15:14:39 -0500
>>
>>> And if you revert commit 1b2511f?
>> Yes, reverting that commit brings back the bold font in the
>> minibuffer-only frame.
> Then, barring any insights, I will revert it soon.  Dmitry will not
> like that, but...
> 


As long as some other ("proper"?) fix is designed instead, I can wait.

Re: Incorrect font weight selected

[Eli Zaretskii]

> Cc: wavexx@thregr.org, larsi@gnus.org, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 21 Dec 2021 07:22:18 +0300
> 
> On 20.12.2021 23:19, Eli Zaretskii wrote:
> >> From: Stefan Monnier<monnier@iro.umontreal.ca>
> >> Cc:wavexx@thregr.org,larsi@gnus.org,emacs-devel@gnu.org
> >> Date: Mon, 20 Dec 2021 15:14:39 -0500
> >>
> >>> And if you revert commit 1b2511f?
> >> Yes, reverting that commit brings back the bold font in the
> >> minibuffer-only frame.
> > Then, barring any insights, I will revert it soon.  Dmitry will not
> > like that, but...
> > 
> 
> 
> As long as some other ("proper"?) fix is designed instead, I can wait.

I still hope Yuri will provide more insights from his debugging, and
we will then be able to do something smarter.

Re: Incorrect font weight selected

[Yuri D'Elia]

On Tue, Dec 21 2021, Eli Zaretskii wrote:
>> >>> And if you revert commit 1b2511f?
>> >> Yes, reverting that commit brings back the bold font in the
>> >> minibuffer-only frame.
>> > Then, barring any insights, I will revert it soon.  Dmitry will not
>> > like that, but...
>>
>> As long as some other ("proper"?) fix is designed instead, I can wait.
>
> I still hope Yuri will provide more insights from his debugging, and
> we will then be able to do something smarter.

I'll be looking at this in a few days.

Raising the weight threshold in 1b2511f to 200 makes the problem
much more apparent as it selects always at least two weights.

The annoying fact is that font selection is called in so many contexts
it's just time consuming to drill down and/or set a good breakpoint
condition.

Re: Incorrect font weight selected

[John ff]

[-- Attachment #1: Type: text/plain, Size: 626 bytes --]



⁣Get TypeApp for Android ​

On 21 Dec 2021, 04:24, at 04:24, Dmitry Gutov <dgutov@yandex.ru> wrote:
>On 20.12.2021 23:19, Eli Zaretskii wrote:
>>> From: Stefan Monnier<monnier@iro.umontreal.ca>
>>> Cc:wavexx@thregr.org,larsi@gnus.org,emacs-devel@gnu.org
>>> Date: Mon, 20 Dec 2021 15:14:39 -0500
>>>
>>>> And if you revert commit 1b2511f?
>>> Yes, reverting that commit brings back the bold font in the
>>> minibuffer-only frame.
>> Then, barring any insights, I will revert it soon.  Dmitry will not
>> like that, but...
>>
>
>
>As long as some other ("proper"?) fix is designed instead, I can wait.

[-- Attachment #2: Type: text/html, Size: 1547 bytes --]

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: Dmitry Gutov <dgutov@yandex.ru>, monnier@iro.umontreal.ca,
>  larsi@gnus.org, emacs-devel@gnu.org
> Date: Tue, 21 Dec 2021 13:27:03 +0100
> 
> The annoying fact is that font selection is called in so many contexts
> it's just time consuming to drill down and/or set a good breakpoint
> condition.

I usually set a breakpoint in xfaces.c where the face attribute(s)
is/are being changed, and then step into the functions it calls to
select the font.  It's a bit slow, since you have a lot to step
through, but it makes sure it's the right calls you are tracing.

HTH

Re: Incorrect font weight selected

[Yuri D'Elia]

[-- Attachment #1: Type: text/plain, Size: 1973 bytes --]

On Tue, Dec 21 2021, Eli Zaretskii wrote:
>> From: Yuri D'Elia <wavexx@thregr.org>
>> Cc: Dmitry Gutov <dgutov@yandex.ru>, monnier@iro.umontreal.ca,
>>  larsi@gnus.org, emacs-devel@gnu.org
>> Date: Tue, 21 Dec 2021 13:27:03 +0100
>>
>> The annoying fact is that font selection is called in so many contexts
>> it's just time consuming to drill down and/or set a good breakpoint
>> condition.
>
> I usually set a breakpoint in xfaces.c where the face attribute(s)
> is/are being changed, and then step into the functions it calls to
> select the font.  It's a bit slow, since you have a lot to step
> through, but it makes sure it's the right calls you are tracing.

After looking at this through a couple of times, I came up with the
attached patch. When changing the default face, the resulting font
name/pattern is stored as a string. Whenever a new frame is created, the
string is used to re-open the face again. I guess this is done to
support creating frames across different terminal types preserving the
closest available font.

This is done by calling font_open_by_name, which builds up a font spec
again from the stored string (which at this point _should_ be fully
specified), then calls the font_open_by_spec. The problem seems that
font_open_by_spec is _explicitly_ requesting a normal
weight/slant/width. So if multiple candidates were available while
enumerating fonts, the regular variant was always picked irregardless of
our preference.

This used to work before since only a single variant was generally
enumerated. In the patch, instead of just resetting/overriding the spec,
we just preset a normal variant if the spec is incomplete. I included
also the family/foundry in the spec, which can be used during selection
instead of blindly ignoring it.

This seems to work correctly for me now, however I'm only testing on
x11. font_open_by_name is not used in many places aside frame creation.
This change seems coherent to me looking at the other calls.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Do-not-override-the-requested-spec-in-font_open_by_s.patch --]
[-- Type: text/x-diff, Size: 1857 bytes --]

From 1e9e66541553b2a439f8748b1ae12f398040d7e5 Mon Sep 17 00:00:00 2001
From: Yuri D'Elia <wavexx@thregr.org>
Date: Wed, 5 Jan 2022 17:11:23 +0100
Subject: [PATCH] Do not override the requested spec in font_open_by_spec

* src/font.c (font_open_by_spec): Do not override the requested spec
when opening a font. Only provide default values if the spec is
underspecified.

This ensures that, when loading fonts with multiple matching
alternatives, font_load_for_lface will pick the requested variant.
---
 src/font.c | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/src/font.c b/src/font.c
index 58ff1a7981..1cb20f5796 100644
--- a/src/font.c
+++ b/src/font.c
@@ -3544,11 +3544,22 @@ font_open_by_spec (struct frame *f, Lisp_Object spec)
 {
   Lisp_Object attrs[LFACE_VECTOR_SIZE];
 
-  /* We set up the default font-related attributes of a face to prefer
-     a moderate font.  */
-  attrs[LFACE_FAMILY_INDEX] = attrs[LFACE_FOUNDRY_INDEX] = Qnil;
-  attrs[LFACE_SWIDTH_INDEX] = attrs[LFACE_WEIGHT_INDEX]
-    = attrs[LFACE_SLANT_INDEX] = Qnormal;
+  /* Filter according to spec.  */
+  attrs[LFACE_FAMILY_INDEX] = Ffont_get (spec, QCfamily);
+  attrs[LFACE_FOUNDRY_INDEX] = Ffont_get (spec, QCfoundry);
+  attrs[LFACE_SWIDTH_INDEX] = Ffont_get (spec, QCwidth);
+  attrs[LFACE_WEIGHT_INDEX] = Ffont_get (spec, QCweight);
+  attrs[LFACE_SLANT_INDEX] = Ffont_get (spec, QCslant);
+
+  /* We set up the fallback font-related attributes of a face to
+     prefer a moderate font.  */
+  if (NILP (attrs[LFACE_SWIDTH_INDEX]))
+    attrs[LFACE_SWIDTH_INDEX] = Qnormal;
+  if (NILP (attrs[LFACE_WEIGHT_INDEX]))
+    attrs[LFACE_WEIGHT_INDEX] = Qnormal;
+  if (NILP (attrs[LFACE_WEIGHT_INDEX]))
+    attrs[LFACE_WEIGHT_INDEX] = Qnormal;
+
 #ifndef HAVE_NS
   attrs[LFACE_HEIGHT_INDEX] = make_fixnum (120);
 #else
-- 
2.34.1

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: dgutov@yandex.ru, monnier@iro.umontreal.ca, larsi@gnus.org,
>  emacs-devel@gnu.org
> Date: Wed, 05 Jan 2022 17:19:35 +0100
> 
> After looking at this through a couple of times, I came up with the
> attached patch. When changing the default face, the resulting font
> name/pattern is stored as a string. Whenever a new frame is created, the
> string is used to re-open the face again. I guess this is done to
> support creating frames across different terminal types preserving the
> closest available font.
> 
> This is done by calling font_open_by_name, which builds up a font spec
> again from the stored string (which at this point _should_ be fully
> specified), then calls the font_open_by_spec. The problem seems that
> font_open_by_spec is _explicitly_ requesting a normal
> weight/slant/width. So if multiple candidates were available while
> enumerating fonts, the regular variant was always picked irregardless of
> our preference.
> 
> This used to work before since only a single variant was generally
> enumerated. In the patch, instead of just resetting/overriding the spec,
> we just preset a normal variant if the spec is incomplete. I included
> also the family/foundry in the spec, which can be used during selection
> instead of blindly ignoring it.
> 
> This seems to work correctly for me now, however I'm only testing on
> x11. font_open_by_name is not used in many places aside frame creation.
> This change seems coherent to me looking at the other calls.

Thanks, but this is a scary change.  I have no idea what unintended
consequences it could cause.

Can we back up a bit, and understand how the change in 1b2511f
triggered the problems you see?  If you already understand that, can
you describe it?  Maybe we could then come up with a safer, better
understood change.  If nothing better comes to mind, I'd prefer to
revert 1b2511f than doing something like what you propose.

Re: Incorrect font weight selected

[Yuri D'Elia]

On Wed, Jan 05 2022, Eli Zaretskii wrote:
> Thanks, but this is a scary change.  I have no idea what unintended
> consequences it could cause.

At least on X11, I've tried creating frames both normally, using the
daemon (across x11 and terminal sessions), and generally I've been using
this for about a week.

> Can we back up a bit, and understand how the change in 1b2511f
> triggered the problems you see?  If you already understand that, can
> you describe it?  Maybe we could then come up with a safer, better
> understood change.  If nothing better comes to mind, I'd prefer to
> revert 1b2511f than doing something like what you propose.

Overall, the change in 1b2511f seems correct to me. font_list_entities
(and the downstream font_delete_unmatched as touched by 1b2511f) can
return a list of candidates which can be used subsequently for
refinement.

If you look at the main font_find_for_lface which is where the action
takes place, this is done by using font_list_entities first, then using
font_select_entity to narrow it down further. This works as intended
already, and that's why the problem only showed itself on new frames
and/or with the daemon (which doesn't immediately create a frame).

When creating a new frame though, the attributes do not come from the
existing face spec the user supplied, but are are reconstructed from the
full font name that matched in the first frame instead.

I wrote above my guess about why this seems to be done in this way. We
could probably do better by re-using the attributes in the frame that
called make-frame (if that frame is of the same type as the new frame),
but that requires a bit more work.

So in the end we're left with font_open_by_name, which is passed a fully
qualified font pattern/name, parses it, but then resets the attributes
for selection, discarding the font attributes as set by the user. It
used to work as the returned entities only matched the requested font
exactly, so no selection ever took place.

The original commit introducing font_open_by_name also has the attribute
override, so that commit doesn't really tell me much about it. Maybe
font_list_entities never returned multiple weights/slants as we're doing
now, so the problem went unnoticed for a long time.

Or something else I don't see in the other font backends. Still, I do
support the change in 1b2511f despite the breakage. I'd rather fight
with the breakage introduced by this change.

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: dgutov@yandex.ru, monnier@iro.umontreal.ca, larsi@gnus.org,
>  emacs-devel@gnu.org
> Date: Wed, 05 Jan 2022 18:11:50 +0100
> 
> On Wed, Jan 05 2022, Eli Zaretskii wrote:
> > Thanks, but this is a scary change.  I have no idea what unintended
> > consequences it could cause.
> 
> At least on X11, I've tried creating frames both normally, using the
> daemon (across x11 and terminal sessions), and generally I've been using
> this for about a week.

I believe you.  But the problem is that there are so many different
use patterns with fonts and faces that it is impossible to test all
the consequences in such a short time, certainly by a single
individual.

> > Can we back up a bit, and understand how the change in 1b2511f
> > triggered the problems you see?  If you already understand that, can
> > you describe it?  Maybe we could then come up with a safer, better
> > understood change.  If nothing better comes to mind, I'd prefer to
> > revert 1b2511f than doing something like what you propose.
> 
> Overall, the change in 1b2511f seems correct to me. font_list_entities
> (and the downstream font_delete_unmatched as touched by 1b2511f) can
> return a list of candidates which can be used subsequently for
> refinement.
> 
> If you look at the main font_find_for_lface which is where the action
> takes place, this is done by using font_list_entities first, then using
> font_select_entity to narrow it down further. This works as intended
> already, and that's why the problem only showed itself on new frames
> and/or with the daemon (which doesn't immediately create a frame).

Then maybe I didn't realize what problem are you trying to solve.
Does the problem happen only with new frames?  At least Sean Whitton
said in this thread that his problems caused by 1b2511f are not with
new frames, AFAIU.

Could you please describe again the problem you are debugging?

> When creating a new frame though, the attributes do not come from the
> existing face spec the user supplied, but are are reconstructed from the
> full font name that matched in the first frame instead.

But that sounds like TRT to me, at least in general, because an
existing frame could have all kinds of frame parameters that don't
necessarily apply to new frames.

> I wrote above my guess about why this seems to be done in this way. We
> could probably do better by re-using the attributes in the frame that
> called make-frame (if that frame is of the same type as the new frame),
> but that requires a bit more work.

I don't think this would be correct, since frames are supposed to be
independent wrt faces.

> So in the end we're left with font_open_by_name, which is passed a fully
> qualified font pattern/name, parses it, but then resets the attributes
> for selection, discarding the font attributes as set by the user. It
> used to work as the returned entities only matched the requested font
> exactly, so no selection ever took place.

Ah, so this is where 1b2511f comes into the play...

In any case, if 1b2511f causes other problems, then reverting it would
solve those problems and this one as well.  We are then left with the
problem reported by Dmitry.

> The original commit introducing font_open_by_name also has the attribute
> override, so that commit doesn't really tell me much about it. Maybe
> font_list_entities never returned multiple weights/slants as we're doing
> now, so the problem went unnoticed for a long time.
> 
> Or something else I don't see in the other font backends. Still, I do
> support the change in 1b2511f despite the breakage. I'd rather fight
> with the breakage introduced by this change.

The problem is we are fighting blind, without real understanding of
the overall logic and the implementation details of this part of
Emacs.  And users are extremely annoyed when their beloved font
suddenly fails to be selected.  I'd rather avoid the waste of time and
energy on this kind of problems.  At least until someone comes aboard
who will know enough about this part to fix any problems quickly and
efficiently.

And I'm still not sure your change is correct in principle, as it
makes faces on frame N depend on faces on some frame N-m, which was
the selected one when frame N was created.  That doesn't sound right
to me: each new frame should start from the same default starting
point.  I'd even go as far as repeating in the new frame the same
steps of selecting the font that were done initially, without any
shortcuts, so that we won't need anything from the previous frame(s).

Re: Incorrect font weight selected

[Yuri D'Elia]

On Wed, Jan 05 2022, Eli Zaretskii wrote:
> I believe you.  But the problem is that there are so many different
> use patterns with fonts and faces that it is impossible to test all
> the consequences in such a short time, certainly by a single
> individual.

Absolutely.

> Then maybe I didn't realize what problem are you trying to solve.
> Does the problem happen only with new frames?  At least Sean Whitton
> said in this thread that his problems caused by 1b2511f are not with
> new frames, AFAIU.

Before guessing too much, can we get some comments from Sean and Dmitry?
Any change either can comment on the patch and/or behavior?

I think this was simply confounded by the fact that starting with
--daemon changes the startup semantics and immediately results in a new
graphical frame being created. Took me a while to notice/debug the exact
behavior, and the possibilities setting up the default faces during
startup are numerous.

> I don't think this would be correct, since frames are supposed to be
> independent wrt faces.

MMh, yes, and no? I definitely understand this reasoning, and we need to
support that no questions here. But purely as as user, when I'm editing
with multiple frames and I change the default font I certainly want all
old and new frames to change as a result, which is why we have this odd
dance. Maybe the user interface to do that is odd. I totally agree that
looking at the big picture, it's... a mess. I wrote some toughs on how
this plays with themes and customize in "custom-theme-set-faces/face
inheritance and delayed face initialization".

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: dgutov@yandex.ru, monnier@iro.umontreal.ca, larsi@gnus.org,
>  emacs-devel@gnu.org
> Date: Wed, 05 Jan 2022 19:08:21 +0100
> 
> > I don't think this would be correct, since frames are supposed to be
> > independent wrt faces.
> 
> MMh, yes, and no? I definitely understand this reasoning, and we need to
> support that no questions here. But purely as as user, when I'm editing
> with multiple frames and I change the default font I certainly want all
> old and new frames to change as a result, which is why we have this odd
> dance.

Your expectations are in general incorrect.  They are due to the fact
that calling set-face-font and similar APIs without the optional FRAME
argument arranges for the change to affect all frames.  But the
low-level code which implements set-face-font only changes the face of
a single frame, and doesn't touch the face definition on other frames.
And the default font generally stands for the font of the default face
of the frame on which you change that face, so it, too, doesn't
necessarily affect other frames.  When we look at low-level code such
as this one, we should always keep that in mind, and avoid leaking the
face definitions to other frames.  The trick of making the change take
effect on other frames is implemented by higher levels.

This is the general background that makes me uneasy to accept the
changes which you suggested, because they seem to violate that general
principle on a very low level.

Re: Incorrect font weight selected

[Dmitry Gutov]

On 05.01.2022 21:08, Yuri D'Elia wrote:
>> Then maybe I didn't realize what problem are you trying to solve.
>> Does the problem happen only with new frames?  At least Sean Whitton
>> said in this thread that his problems caused by 1b2511f are not with
>> new frames, AFAIU.
> Before guessing too much, can we get some comments from Sean and Dmitry?

I don't mind trying the patch, if you can reach an agreement with Eli 
that it is a somewhat reasonable change.

FWIW, I have switched to a different font a week or so ago, so I 
wouldn't particularly object to the revert of the fix either.

I do have the previous font available for testing, though.

Re: Incorrect font weight selected

[Yuri D'Elia]

On Wed, Jan 05 2022, Dmitry Gutov wrote:
> I don't mind trying the patch, if you can reach an agreement with Eli
> that it is a somewhat reasonable change.
>
> FWIW, I have switched to a different font a week or so ago, so I
> wouldn't particularly object to the revert of the fix either.
>
> I do have the previous font available for testing, though.

Testing would still be nice though, to understand if the problem is the
same.

Re: Incorrect font weight selected

[Yuri D'Elia]

On Wed, Jan 05 2022, Dmitry Gutov wrote:
> I don't mind trying the patch, if you can reach an agreement with Eli
> that it is a somewhat reasonable change.
>
> FWIW, I have switched to a different font a week or so ago, so I
> wouldn't particularly object to the revert of the fix either.
>
> I do have the previous font available for testing, though.

Testing would still be nice though, to understand if the problem is the
has the same origin.

Re: Incorrect font weight selected

[Yuri D'Elia]

On Wed, Jan 05 2022, Eli Zaretskii wrote:
> When we look at low-level code such as this one, we should always keep
> that in mind, and avoid leaking the face definitions to other frames.
> The trick of making the change take effect on other frames is
> implemented by higher levels.

Do you have a more concrete overview of how the change _should_ at least
ideally be propagated for me to get a better overview?

One side-effect of pushing a few set-face-font calls in a
after-make-frame hook in my setup (as opposed to calling set-face-font
once) was a noticeable increase of time required to show the resulting
frame, which is not too nice to have.

> This is the general background that makes me uneasy to accept the
> changes which you suggested, because they seem to violate that general
> principle on a very low level.

In this sense, the change only fixes the lower-level font selection when
opening a face in a single frame, so it doesn't alter how we're
propagating the change at all at the moment.

Re: Incorrect font weight selected

[Po Lu]

Yuri D'Elia <wavexx@thregr.org> writes:

> This seems to work correctly for me now, however I'm only testing on
> x11. font_open_by_name is not used in many places aside frame creation.

FWIW, this breaks the Haiku font driver (displaying etc/HELLO becomes
very slow afterwards) -- if someone wants this installed, no matter, I
will fix it afterwards, but this could be indicative of some larger
problems.

Thanks.

Re: Incorrect font weight selected

[Po Lu]

Po Lu <luangruo@yahoo.com> writes:

> FWIW, this breaks the Haiku font driver (displaying etc/HELLO becomes
> very slow afterwards) -- if someone wants this installed, no matter, I
> will fix it afterwards, but this could be indicative of some larger
> problems.

It also breaks nsfont with the same symptoms.

Re: Incorrect font weight selected

[Sean Whitton]

Hello Eli, Yuri,

On Wed 05 Jan 2022 at 07:08PM +01, Yuri D'Elia wrote:

> On Wed, Jan 05 2022, Eli Zaretskii wrote:
>> I believe you.  But the problem is that there are so many different
>> use patterns with fonts and faces that it is impossible to test all
>> the consequences in such a short time, certainly by a single
>> individual.
>
> Absolutely.
>
>> Then maybe I didn't realize what problem are you trying to solve.
>> Does the problem happen only with new frames?  At least Sean Whitton
>> said in this thread that his problems caused by 1b2511f are not with
>> new frames, AFAIU.

Well, my code which sets the font is in after-focus-change-function and
also in window-size-change-functions, and I could avoid the problem, in
a sense, by replacing that code with just customising the default face.

> Before guessing too much, can we get some comments from Sean and
> Dmitry?  Any change either can comment on the patch and/or behavior?

Thank you for working on this.  I dropped my local revert of 1b2511f and
applied your patch, but unfortunately the problem then reoccurs.

Happy to do more testing etc. -- let me know.

>> I don't think this would be correct, since frames are supposed to be
>> independent wrt faces.
>
> MMh, yes, and no? I definitely understand this reasoning, and we need to
> support that no questions here. But purely as as user, when I'm editing
> with multiple frames and I change the default font I certainly want all
> old and new frames to change as a result, which is why we have this odd
> dance. Maybe the user interface to do that is odd. I totally agree that
> looking at the big picture, it's... a mess. I wrote some toughs on how
> this plays with themes and customize in "custom-theme-set-faces/face
> inheritance and delayed face initialization".

In my case I have the code in window-size-change-functions precisely
because I want some frames of a particular size to use a different font.

-- 
Sean Whitton

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: dgutov@yandex.ru, monnier@iro.umontreal.ca, larsi@gnus.org,
>  emacs-devel@gnu.org
> Date: Thu, 06 Jan 2022 00:16:15 +0100
> 
> On Wed, Jan 05 2022, Eli Zaretskii wrote:
> > When we look at low-level code such as this one, we should always keep
> > that in mind, and avoid leaking the face definitions to other frames.
> > The trick of making the change take effect on other frames is
> > implemented by higher levels.
> 
> Do you have a more concrete overview of how the change _should_ at least
> ideally be propagated for me to get a better overview?

Sorry, I don't think I understand what kind of overview are you asking
for.  Please tell more.

> > This is the general background that makes me uneasy to accept the
> > changes which you suggested, because they seem to violate that general
> > principle on a very low level.
> 
> In this sense, the change only fixes the lower-level font selection when
> opening a face in a single frame, so it doesn't alter how we're
> propagating the change at all at the moment.

But in doing so, it uses a font spec that came from another frame,
does it not?

Re: Incorrect font weight selected

[Yuri D'Elia]

On Thu, Jan 06 2022, Eli Zaretskii wrote:
>> In this sense, the change only fixes the lower-level font selection when
>> opening a face in a single frame, so it doesn't alter how we're
>> propagating the change at all at the moment.
>
> But in doing so, it uses a font spec that came from another frame,
> does it not?

Yes, but this font spec is the default spec for the frame applied by
upper lisp code already. Not at my desk right now, but I'll try to get a
lisp backtrace of the call involved.

Re: Incorrect font weight selected

[Yuri D'Elia]

On Thu, Jan 06 2022, Po Lu wrote:
> Yuri D'Elia <wavexx@thregr.org> writes:
>
>> This seems to work correctly for me now, however I'm only testing on
>> x11. font_open_by_name is not used in many places aside frame creation.
>
> FWIW, this breaks the Haiku font driver (displaying etc/HELLO becomes
> very slow afterwards) -- if someone wants this installed, no matter, I
> will fix it afterwards, but this could be indicative of some larger
> problems.

Uuh, didn't think of trying etc/HELLO.

However after this and report from Sean, that I agree that at least for
the moment reverting 1b2511f seems the best course of action.

But this all smells of some deeper underlying problem to me.

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, larsi@gnus.org, emacs-devel@gnu.org,
>  monnier@iro.umontreal.ca, dgutov@yandex.ru
> Date: Thu, 06 Jan 2022 10:49:55 +0100
> 
> Uuh, didn't think of trying etc/HELLO.
> 
> However after this and report from Sean, that I agree that at least for
> the moment reverting 1b2511f seems the best course of action.

Done now.

> But this all smells of some deeper underlying problem to me.

It's quite clear there is a deeper problem.  We should understand this
better, and then perhaps we could come up with the idea for a
solution.

Re: Incorrect font weight selected

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Cc: dgutov@yandex.ru, monnier@iro.umontreal.ca, larsi@gnus.org,
>  emacs-devel@gnu.org
> Date: Thu, 06 Jan 2022 10:46:23 +0100
> 
> On Thu, Jan 06 2022, Eli Zaretskii wrote:
> >> In this sense, the change only fixes the lower-level font selection when
> >> opening a face in a single frame, so it doesn't alter how we're
> >> propagating the change at all at the moment.
> >
> > But in doing so, it uses a font spec that came from another frame,
> > does it not?
> 
> Yes, but this font spec is the default spec for the frame applied by
> upper lisp code already. Not at my desk right now, but I'll try to get a
> lisp backtrace of the call involved.

Yes, please show that backtrace, I'd like to look into this some more.