Different/incorrect font chosen -- fontconfig not deterministic?

Different/incorrect font chosen -- fontconfig not deterministic?

[Simen Endsjø]

[-- Attachment #1: Type: text/plain, Size: 5866 bytes --]


I’m having problems where the wrong/different font is chosen on one
computer even though my setup should be the same.

On my first system, Material Icons font is chosen, but on the second
system, Symbols Nerd Font and GNU Unifont is chosen.

I copied my old operating-system configuration and only did minor tweaks
to support the new system (e.g. change hostname), but the font setup is
the same. I also use the exact same home-environment and the same emacs
configuration.

Material Icons is added by emacs `all-the-icons-install-fonts' and added
to `~/.local/share/fonts/material-design-icons.ttf'. This file exist on
both systems. fc-list shows this font too.

So I’m guessing fontconfig is not deterministic in some way here.
`fc-cache -fr' doesn’t help, and I’ve tried rebuilding the system, home
and rebooted too.

Char 1 () shows a different icon than intended as Nerd and Material
uses different symbols for the character.

First system - correct:
┌────
│              position: 57 of 1659 (3%), column: 0
│             character:  (displayed as ) (codepoint 59576, #o164270, #xe8b8)
│               charset: unicode-bmp (Unicode Basic Multilingual Plane (U+0000..U+FFFF))
│ code point in charset: 0xE8B8
│                syntax: w 	which means: word
│              category: L:Strong L2R
│              to input: type "C-x 8 RET e8b8"
│           buffer code: #xEE #xA2 #xB8
│             file code: #xEE #xA2 #xB8 (encoded by coding system utf-8-unix)
│               display: by this font (glyph code):
│     ftcrhb:-unknown-Material Icons-regular-normal-normal-*-24-*-*-*-m-0-iso10646-1 (#x34A)
│
│ Character code properties: customize what to show
│   general-category: Co (Other, Private Use)
│   decomposition: (59576) ('')
└────

Second system - incorrect/different:
┌────
│              position: 50 of 53 (92%), column: 0
│             character:  (displayed as ) (codepoint 59576, #o164270, #xe8b8)
│               charset: unicode-bmp (Unicode Basic Multilingual Plane (U+0000..U+FFFF))
│ code point in charset: 0xE8B8
│                syntax: w 	which means: word
│              category: L:Strong L2R
│              to input: type "C-x 8 RET e8b8"
│           buffer code: #xEE #xA2 #xB8
│             file code: #xEE #xA2 #xB8 (encoded by coding system utf-8-unix)
│               display: by this font (glyph code):
│     ftcrhb:-PfEd-Symbols Nerd Font Mono-regular-normal-normal-*-24-*-*-*-m-0-iso10646-1 (#x448)
│
│ Character code properties: customize what to show
│   general-category: Co (Other, Private Use)
│   decomposition: (59576) ('')
└────

Char 2 () choose a font which doesn’t have this character at all, so it
renders as E5 D9.

First system – correct:
┌────
│              position: 1649 of 2449 (67%), column: 0
│             character:  (displayed as ) (codepoint 58841, #o162731, #xe5d9)
│               charset: unicode-bmp (Unicode Basic Multilingual Plane (U+0000..U+FFFF))
│ code point in charset: 0xE5D9
│                syntax: w 	which means: word
│              category: L:Strong L2R, j:Japanese
│              to input: type "C-x 8 RET e5d9"
│           buffer code: #xEE #x97 #x99
│             file code: #xEE #x97 #x99 (encoded by coding system utf-8-unix)
│               display: by this font (glyph code):
│     ftcrhb:-unknown-Material Icons-regular-normal-normal-*-24-*-*-*-m-0-iso10646-1 (#x27D)
│
│ Character code properties: customize what to show
│   general-category: Co (Other, Private Use)
│   decomposition: (58841) ('')
└────

Second system – incorrect font chosen:
┌────
│              position: 859 of 891 (96%), column: 0
│             character:  (displayed as ) (codepoint 58841, #o162731, #xe5d9)
│               charset: unicode-bmp (Unicode Basic Multilingual Plane (U+0000..U+FFFF))
│ code point in charset: 0xE5D9
│                syntax: w 	which means: word
│              category: L:Strong L2R, j:Japanese
│              to input: type "C-x 8 RET e5d9"
│           buffer code: #xEE #x97 #x99
│             file code: #xEE #x97 #x99 (encoded by coding system utf-8-unix)
│               display: by this font (glyph code):
│     ftcrhb:-GNU -Unifont Sample-regular-normal-normal-*-24-*-*-*-d-0-iso10646-1 (#xDDD8)
│
│ Character code properties: customize what to show
│   general-category: Co (Other, Private Use)
│   decomposition: (58841) ('')
└────

The fonts in fc-list lists the fonts in different order on the two
systems, which is probably the reason why it picks different fonts on
the two systems, but it doesn’t explain why it picks a font which
doesn’t have the symbol…?

┌────
│ fc-list | grep -i material
└────

First (correct) system:
┌────
│ ~/.local/share/fonts/material-design-icons.ttf: Material Icons:style=Regular
│ ~/.guix-home/profile/share/fonts/web/MaterialIcons-Regular.woff: Material Icons:style=Regular
│ ~/.guix-home/profile/share/fonts/truetype/MaterialIcons-Regular.ttf: Material Icons:style=Regular
└────

Second (incorrect) system:
┌────
│ ~/.guix-home/profile/share/fonts/web/MaterialIcons-Regular.woff: Material Icons:style=Regular
│ ~/.local/share/fonts/material-design-icons.ttf: Material Icons:style=Regular
│ ~/.guix-home/profile/share/fonts/truetype/MaterialIcons-Regular.ttf: Material Icons:style=Regular
└────

What is the correct way to make the systems deterministic? And only
choose a font which has symbols defined? And choose the preferred fonts
in different situations?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 515 bytes --]

Re: Different/incorrect font chosen -- fontconfig not deterministic?

[Marek Paśnikowski]

[-- Attachment #1: Type: text/plain, Size: 9573 bytes --]

Good morning Simen.

I am also offended by the incomprehensible documentation of Fontconfig on 
their own website.  For the goal learning more about this crucial library, I 
would like to help explore your problem.

From your message, I have clearly understood that your Fontconfig 
configuration has some variable which you fail to find, despite very similar 
system configuration declarations.  Unfortunately, you provided neither the 
relevant sections of the system configurations, nor the fontconfig 
configuration files.

I will use the concept of functional impurity.  In this instance, I mean by 
this that Guix is vulnerable to initial data in storage regardless of its 
promise of pure reproducibility of build.  The important factor to consider is 
that Guix promises only to reproduce builds of program binaries, and not 
everything that programs could ever utilise to change operational logic.

Sunday, 5 January 2025 09:39:28 CET Simen Endsjø <contact@simendsjo.me> wrote:
> I’m having problems where the wrong/different font is chosen on one
> computer even though my setup should be the same.
> 
> On my first system, Material Icons font is chosen, but on the second
> system, Symbols Nerd Font and GNU Unifont is chosen.
> 
> I copied my old operating-system configuration and only did minor tweaks
> to support the new system (e.g. change hostname), but the font setup is
> the same. I also use the exact same home-environment and the same emacs
> configuration.
> 
> Material Icons is added by emacs `all-the-icons-install-fonts' and added
> to `~/.local/share/fonts/material-design-icons.ttf'. This file exist on
> both systems. fc-list shows this font too.
> 
> So I’m guessing fontconfig is not deterministic in some way here.
> `fc-cache -fr' doesn’t help, and I’ve tried rebuilding the system, home
> and rebooted too.
> 

There is also a possibility, that default Fontconfig settings had changed 
between the two system installations.  Or you had some pre-existing user 
configuration on one of the systems which introduced impurity to the system 
installation.  There are three configuration spaces needing alignment here:
1. what you want to see;
2. what you request the system to build;
3. what impurity differences exist between the computers.
You have only roughly described the space number one.  On its own it is 
completely useless, because there is no way to tell why the problem manifests 
and what to change.

Sunday, 5 January 2025 09:39:28 CET Simen Endsjø <contact@simendsjo.me> wrote:
> Char 1 () shows a different icon than intended as Nerd and Material
> uses different symbols for the character.
> 
> First system - correct:
> ┌────
> │              position: 57 of 1659 (3%), column: 0
> │             character:  (displayed as ) (codepoint 59576, #o164270,
> #xe8b8) │               charset: unicode-bmp (Unicode Basic Multilingual
> Plane (U+0000..U+FFFF)) │ code point in charset: 0xE8B8
> │                syntax: w 	which means: word
> │              category: L:Strong L2R
> │              to input: type "C-x 8 RET e8b8"
> │           buffer code: #xEE #xA2 #xB8
> │             file code: #xEE #xA2 #xB8 (encoded by coding system
> utf-8-unix) │               display: by this font (glyph code):
> │     ftcrhb:-unknown-Material
> Icons-regular-normal-normal-*-24-*-*-*-m-0-iso10646-1 (#x34A) │
> │ Character code properties: customize what to show
> │   general-category: Co (Other, Private Use)
> │   decomposition: (59576) ('')
> └────
> 
> Second system - incorrect/different:
> ┌────
> │              position: 50 of 53 (92%), column: 0
> │             character:  (displayed as ) (codepoint 59576, #o164270,
> #xe8b8) │               charset: unicode-bmp (Unicode Basic Multilingual
> Plane (U+0000..U+FFFF)) │ code point in charset: 0xE8B8
> │                syntax: w 	which means: word
> │              category: L:Strong L2R
> │              to input: type "C-x 8 RET e8b8"
> │           buffer code: #xEE #xA2 #xB8
> │             file code: #xEE #xA2 #xB8 (encoded by coding system
> utf-8-unix) │               display: by this font (glyph code):
> │     ftcrhb:-PfEd-Symbols Nerd Font
> Mono-regular-normal-normal-*-24-*-*-*-m-0-iso10646-1 (#x448) │
> │ Character code properties: customize what to show
> │   general-category: Co (Other, Private Use)
> │   decomposition: (59576) ('')
> └────
> 
> Char 2 () choose a font which doesn’t have this character at all, so it
> renders as E5 D9.
> 
> First system – correct:
> ┌────
> │              position: 1649 of 2449 (67%), column: 0
> │             character:  (displayed as ) (codepoint 58841, #o162731,
> #xe5d9) │               charset: unicode-bmp (Unicode Basic Multilingual
> Plane (U+0000..U+FFFF)) │ code point in charset: 0xE5D9
> │                syntax: w 	which means: word
> │              category: L:Strong L2R, j:Japanese
> │              to input: type "C-x 8 RET e5d9"
> │           buffer code: #xEE #x97 #x99
> │             file code: #xEE #x97 #x99 (encoded by coding system
> utf-8-unix) │               display: by this font (glyph code):
> │     ftcrhb:-unknown-Material
> Icons-regular-normal-normal-*-24-*-*-*-m-0-iso10646-1 (#x27D) │
> │ Character code properties: customize what to show
> │   general-category: Co (Other, Private Use)
> │   decomposition: (58841) ('')
> └────
> 
> Second system – incorrect font chosen:
> ┌────
> │              position: 859 of 891 (96%), column: 0
> │             character:  (displayed as ) (codepoint 58841, #o162731,
> #xe5d9) │               charset: unicode-bmp (Unicode Basic Multilingual
> Plane (U+0000..U+FFFF)) │ code point in charset: 0xE5D9
> │                syntax: w 	which means: word
> │              category: L:Strong L2R, j:Japanese
> │              to input: type "C-x 8 RET e5d9"
> │           buffer code: #xEE #x97 #x99
> │             file code: #xEE #x97 #x99 (encoded by coding system
> utf-8-unix) │               display: by this font (glyph code):
> │     ftcrhb:-GNU -Unifont
> Sample-regular-normal-normal-*-24-*-*-*-d-0-iso10646-1 (#xDDD8) │
> │ Character code properties: customize what to show
> │   general-category: Co (Other, Private Use)
> │   decomposition: (58841) ('')
> └────
> 
> The fonts in fc-list lists the fonts in different order on the two
> systems, which is probably the reason why it picks different fonts on
> the two systems, but it doesn’t explain why it picks a font which
> doesn’t have the symbol…?
> 
> ┌────
> │ fc-list | grep -i material
> └────
> 
> First (correct) system:
> ┌────
> │ ~/.local/share/fonts/material-design-icons.ttf: Material
> Icons:style=Regular │
> ~/.guix-home/profile/share/fonts/web/MaterialIcons-Regular.woff: Material
> Icons:style=Regular │
> ~/.guix-home/profile/share/fonts/truetype/MaterialIcons-Regular.ttf:
> Material Icons:style=Regular └────
> 
> Second (incorrect) system:
> ┌────
> │ ~/.guix-home/profile/share/fonts/web/MaterialIcons-Regular.woff: Material
> Icons:style=Regular │ ~/.local/share/fonts/material-design-icons.ttf:
> Material Icons:style=Regular │
> ~/.guix-home/profile/share/fonts/truetype/MaterialIcons-Regular.ttf:
> Material Icons:style=Regular └────
> 

I appreciate that you took your time to prove that you have this problem.  I 
am afraid though that this information alonw will not bring any results.  
Fontconfig uses a complicated formula to sort the fonts for display, making it 
impossible to reverse engineer the problem source from the output.

Sunday, 5 January 2025 09:39:28 CET Simen Endsjø <contact@simendsjo.me> wrote:
> What is the correct way to make the systems deterministic? And only
> choose a font which has symbols defined? And choose the preferred fonts
> in different situations?

The only way to truly ensure deterministc behaviour of Fontconfig is to build 
both systems from scratch on storage without any font configuration files, and 
introduce any configuration changes solely through the Guix configuration 
system.  This strategy has a very high cost of implementation, as it requires 
the complicated process of backing up data, removing existing systems and then 
reconstructing everything.

The next best strategy is to remove all existing Fontconfig configuration 
files and reconfigure everything to prevent impurities related to pre-existing 
files.  It is relatively cheap to implement, but it does not guarantee purity, 
as there are many places where font configuration adjustment can take place.  
If you do not find them all, you will not achieve determinism.

For further inspiration, take a look at my Fontconfig configuration:
https://git.marekpasnikowski.pl/distribution.git/tree/suweren/home/services/
fontutils.scm
I am running out of time to figure out how to prevent the link from being 
wrapped to a newline, so please bear with it.

If you send system or home configuration snippets relating to font 
configuration and the resulting Fontconfig configuration files, I might be 
able to provide better commentary on the problem at hand.

-----

Guix Help Member,
Marek Paśnikowski

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]