bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[xuan]

[-- Attachment #1: Type: text/plain, Size: 5970 bytes --]


Setup:

1. install `ligature.el` from melpa.

2. start `emacs -Q` and execute the attached script or simply use the attached script as the `init.el`.

3. open the attached `init.el` file and execute `init-faces` or hit "C-#" to start randomizing the font faces,

4. eventually you will see some characters gets rendered with extra spaces (sample screenshots provided below), hit "C-!" to stop the randomization,



5. hitting "C-return" will randomize font once, which might be useful for you find debug,



Expected behavior:

the fonts are rendered consistently without extra spacing,



Observed behavior:

the fonts are rendered with extra spacing randomly,



Related information:

I previously reported this bug here: https://github.com/mickeynp/ligature.el/issues/56.



In GNU Emacs 29.4 (build 1, x86_64-redhat-linux-gnu, GTK+ Version

3.24.42, cairo version 1.18.0) of 2024-07-16 built on 
27527c2e06f843c0962737354e0b3cf7 
System Description: Fedora Linux 40 (Sway) 

Configured using: 
'configure --build=x86_64-redhat-linux-gnu 
--host=x86_64-redhat-linux-gnu --program-prefix= 
--disable-dependency-tracking --prefix=/usr --exec-prefix=/usr 
--bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc 
--datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 
--libexecdir=/usr/libexec --localstatedir=/var --runstatedir=/run 
--sharedstatedir=/var/lib --mandir=/usr/share/man 
--infodir=/usr/share/info --with-cairo --with-dbus --with-gif 
--with-gpm=no --with-harfbuzz --with-jpeg --with-json --with-modules 
--with-native-compilation=aot --with-pgtk --with-png --with-rsvg 
--with-sqlite3 --with-tiff --with-tree-sitter --with-webp --with-xpm 
--with-xwidgets build_alias=x86_64-redhat-linux-gnu 
host_alias=x86_64-redhat-linux-gnu CC=gcc 'CFLAGS=-DMAIL_USE_LOCKF -O2 
-flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches 
-pipe -Wall -Werror=format-security 
-Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS 
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong 
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64 
-mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection 
-fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer ' 
LDFLAGS=-Wl,-z,relro 
PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig CXX=g++ 
'CXXFLAGS=-O2 -flto=auto -ffat-lto-objects -fexceptions -g 
-grecord-gcc-switches -pipe -Wall -Werror=format-security 
-Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS 
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong 
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64 
-mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection 
-fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer '' 

Configured features: 
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON 
LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY 
PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF 
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XIM XWIDGETS GTK3 ZLIB 

Important settings: 
value of $LANG: en_US.UTF-8 
locale-coding-system: utf-8-unix 

Major mode: ELisp/l 

Minor modes in effect: 
global-ligature-mode: t 
ligature-mode: t 
tooltip-mode: t 
global-eldoc-mode: t 
eldoc-mode: t 
show-paren-mode: t 
electric-indent-mode: t 
mouse-wheel-mode: t 
tool-bar-mode: t 
menu-bar-mode: t 
file-name-shadow-mode: t 
global-font-lock-mode: t 
font-lock-mode: t 
blink-cursor-mode: t 
line-number-mode: t 
indent-tabs-mode: t 
transient-mark-mode: t 
auto-composition-mode: t 
auto-encryption-mode: t 
auto-compression-mode: t 

Load-path shadows: 
None found. 

Features: 
(shadow sort mail-extr emacsbug message yank-media puny dired 
dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068 epg-config 
gnus-util text-property-search time-date mm-decode mm-bodies mm-encode 
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 
rfc2045 ietf-drums mm-util mail-prsvr mail-utils comp comp-cstr warnings 
icons rx ligature cl-extra help-mode use-package-core ligature-autoloads 
package browse-url url url-proxy url-privacy url-expand url-methods 
url-history url-cookie generate-lisp-file url-domsuf url-util mailcap 
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs 
password-cache json subr-x map byte-opt gv bytecomp byte-compile 
url-vars cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren 
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel 
term/pgtk-win pgtk-win term/common-win pgtk-dnd tool-bar dnd fontset 
image regexp-opt fringe tabulated-list replace newcomment text-mode 
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch 
easymenu timer select scroll-bar mouse jit-lock font-lock syntax 
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic 
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese 
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek 
romanian slovak czech european ethiopic indian cyrillic chinese 
composite emoji-zwj charscript charprop case-table epa-hook 
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs 
theme-loaddefs faces cus-face macroexp files window text-properties 
overlay sha1 md5 base64 format env code-pages mule custom widget keymap 
hashtable-print-readable backquote threads xwidget-internal dbusbind 
inotify dynamic-setting system-font-setting font-render-setting cairo 
gtk pgtk multi-tty make-network-process native-compile emacs) 

Memory information: 
((conses 16 108675 14724) 
(symbols 48 8830 0) 
(strings 32 27784 2658) 
(string-bytes 1 906120) 
(vectors 16 22224) 
(vector-slots 8 441624 17378) 
(floats 8 45 30) 
(intervals 56 736 1) 
(buffers 984 12))  

[-- Attachment #2.1: Type: text/html, Size: 748 bytes --]

[-- Attachment #2.2: 83k2iR0RkF0vUnlj.png --]
[-- Type: image/png, Size: 5035 bytes --]

[-- Attachment #2.3: Type: text/html, Size: 189 bytes --]

[-- Attachment #2.4: vj0A4vBxR2UgLHU2.png --]
[-- Type: image/png, Size: 1199 bytes --]

[-- Attachment #2.5: Type: text/html, Size: 189 bytes --]

[-- Attachment #2.6: dknFd3G9YDiGAGQ9.png --]
[-- Type: image/png, Size: 1323 bytes --]

[-- Attachment #2.7: Type: text/html, Size: 189 bytes --]

[-- Attachment #2.8: uM000GtbhXhDurze.png --]
[-- Type: image/png, Size: 1330 bytes --]

[-- Attachment #2.9: Type: text/html, Size: 6959 bytes --]

[-- Attachment #2.10: init.el --]
[-- Type: application/octet-stream, Size: 3667 bytes --]

;; -*- lexical-binding: t; -*-
(setq inhibit-startup-screen t)

;; starting font
(setq default-font (font-spec :family "JetBrains Mono" :size 16 :weight 'regular))
(set-face-attribute 'default nil :width 'normal :weight 'normal :slant 'normal :font default-font)

(use-package ligature
  :config
  (ligature-set-ligatures 'prog-mode '("--" "---" "==" "===" "!=" "!==" "=!="
                              "=:=" "=/=" "<=" ">=" "&&" "&&&" "&=" "++" "+++" "***" ";;" "!!"
                              "??" "???" "?:" "?." "?=" "<:" ":<" ":>" ">:" "<:<" "<>" "<<<" ">>>"
                              "<<" ">>" "||" "-|" "_|_" "|-" "||-" "|=" "||=" "##" "###" "####"
                              "#{" "#[" "]#" "#(" "#?" "#_" "#_(" "#:" "#!" "#=" "^=" "<$>" "<$"
                              "$>" "<+>" "<+" "+>" "<*>" "<*" "*>" "</" "</>" "/>" "<!--" "<#--"
                              "-->" "->" "->>" "<<-" "<-" "<=<" "=<<" "<<=" "<==" "<=>" "<==>"
                              "==>" "=>" "=>>" ">=>" ">>=" ">>-" ">-" "-<" "-<<" ">->" "<-<" "<-|"
                              "<=|" "|=>" "|->" "<->" "<~~" "<~" "<~>" "~~" "~~>" "~>" "~-" "-~"
                              "~@" "[||]" "|]" "[|" "|}" "{|" "[<" ">]" "|>" "<|" "||>" "<||"
                              "|||>" "<|||" "<|>" "..." ".." ".=" "..<" ".?" "::" ":::" ":=" "::="
                              ":?" ":?>" "//" "///" "/*" "*/" "/=" "//=" "/==" "@_" "__" "???"
                              "<:<" ";;;"))
  (global-ligature-mode t))

;; ("\\\"o" ?ö)
;; ("^^+" ?⁺) ("__+" ?₊) ("^^-" ?⁻)
;; ("__0" ?₀) ("__1" ?₁) ("__2" ?₂) ("__3" ?₃) ("__4" ?₄)
;; ("__5" ?₅) ("__6" ?₆) ("__7" ?₇) ("__8" ?₈) ("__9" ?₉)
;; ("__a" ?ₐ) ("__e" ?ₑ) ("__h" ?ₕ) ("__i" ?ᵢ) ("__k" ?ₖ)
;; ("__l" ?ₗ) ("__m" ?ₘ) ("__n" ?ₙ) ("__o" ?ₒ) ("__p" ?ₚ)
;; ("__r" ?ᵣ) ("__s" ?ₛ) ("__t" ?ₜ) ("__u" ?ᵤ) ("__v" ?ᵥ) ("__x" ?ₓ))

(setq faces '((face1 . "--")
	      (face2 . "__")
	      (face3 . "=:=")
	      (face4 . "??")
	      (face5 . "<<")
	      (face6 . "#{")
	      (face7 . "$>")
	      (face8 . "-->")
	      (face9 . "==>")
	      (face10 . "<=|")
	      (face11 . "~@")
	      (face12 . "||>")
	      (face13 . "<:<")))

(defun init-faces ()
  (interactive)
  (dolist (face faces)
    (make-face (car face))
    (highlight-lines-matching-regexp (cdr face) (car face))
    (run-at-time nil 1 #'random-faces)))
  
(defun random-from-list (l)
  (nth (random (length l)) l))

(defun random-face (face)
  (interactive)
  (let* ((weight (random-from-list '(light normal medium bold)))
	 (slant (random-from-list '(normal italic)))
	 (height (random-from-list '(120 125 115 110 130)))
	 )
    (set-face-attribute face nil :weight weight :slant slant :height height)))

(defun random-faces ()
  (interactive)
  (dolist (face faces)
    (random-face (car face))))

(global-set-key (kbd "C-#") #'init-faces)
(global-set-key (kbd "C-!") (lambda () (interactive) (cancel-function-timers #'random-faces)))
(global-set-key (kbd "<C-return>") #'random-faces)

(custom-set-variables
 ;; custom-set-variables was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 '(package-selected-packages '(ligature)))
(custom-set-faces
 ;; custom-set-faces was added by Custom.
 ;; If you edit it by hand, you could mess it up, so be careful.
 ;; Your init file should contain only one such instance.
 ;; If there is more than one, they won't work right.
 )

[-- Attachment #2.11: Type: text/html, Size: 162 bytes --]

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: xuan@xlk.me
> Date: Fri, 11 Oct 2024 12:18:57 -0400
> 
> Setup:
> 
> 1. install `ligature.el` from melpa.
> 
> 2. start `emacs -Q` and execute the attached script or simply use the attached script as the `init.el`.
> 
> 3. open the attached `init.el` file and execute `init-faces` or hit "C-#" to start randomizing the font faces,
> 
> 4. eventually you will see some characters gets rendered with extra spaces (sample screenshots provided below), hit "C-!" to stop the randomization,
> 
> 5. hitting "C-return" will randomize font once, which might be useful for you find debug,

I cannot reproduce the problem on my system.  I've let Emacs randomize
the fonts for quite some time, and everything keeps rendering
correctly without the extra spaces.  When you say "eventually", how
long do you typically wait until the problem appears?

(As two deviations from your init.el, I used a different font
(Cascadia Code), since I don't have JetBrains Mono installed; and I
replaces use-package with the equivalent Lisp, since I don't want to
install ligatures.el on this system.  I don't think these two
deviations should matter, but I mention them FTR.)

I think this might be system-dependent.  Can someone try this in a
Cairo build on GNU/Linux and see if the problem reproduces?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

On 10/12/24 04:02, Eli Zaretskii wrote:
> I cannot reproduce the problem on my system.  I've let Emacs randomize
> the fonts for quite some time, and everything keeps rendering
> correctly without the extra spaces.  When you say "eventually", how
> long do you typically wait until the problem appears?

It took me 5 seconds at most, and usually happened immediately.

> (As two deviations from your init.el, I used a different font
> (Cascadia Code), since I don't have JetBrains Mono installed; and I
> replaces use-package with the equivalent Lisp, since I don't want to
> install ligatures.el on this system.  I don't think these two
> deviations should matter, but I mention them FTR.)

I can replicate the same problem (and even worse in some cases) using 
Cascadia Code.

The `use-package` should be replaceable, except you want to keep the 
list of ligature strings formatted in the same way. I'm not fluent in 
Elisp or emacs API, so my dumb way of randomization is achieved through 
regexp matching over the ligature strings. If they are re-organized in 
some way (say merged into one huge line), it might break the randomization.

> I think this might be system-dependent.  Can someone try this in a
> Cairo build on GNU/Linux and see if the problem reproduces?

Potentially, I'm using emacs-pgtk on wayland.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

Ping!  Unless someone can debug this, I'm going to close this bug as
unreproducible.

> Date: Sat, 12 Oct 2024 12:09:02 -0400
> Cc: 73752@debbugs.gnu.org
> From: Yixuan Chen <xuan@xlk.me>
> 
> On 10/12/24 04:02, Eli Zaretskii wrote:
> > I cannot reproduce the problem on my system.  I've let Emacs randomize
> > the fonts for quite some time, and everything keeps rendering
> > correctly without the extra spaces.  When you say "eventually", how
> > long do you typically wait until the problem appears?
> 
> It took me 5 seconds at most, and usually happened immediately.
> 
> > (As two deviations from your init.el, I used a different font
> > (Cascadia Code), since I don't have JetBrains Mono installed; and I
> > replaces use-package with the equivalent Lisp, since I don't want to
> > install ligatures.el on this system.  I don't think these two
> > deviations should matter, but I mention them FTR.)
> 
> I can replicate the same problem (and even worse in some cases) using 
> Cascadia Code.
> 
> The `use-package` should be replaceable, except you want to keep the 
> list of ligature strings formatted in the same way. I'm not fluent in 
> Elisp or emacs API, so my dumb way of randomization is achieved through 
> regexp matching over the ligature strings. If they are re-organized in 
> some way (say merged into one huge line), it might break the randomization.
> 
> > I think this might be system-dependent.  Can someone try this in a
> > Cairo build on GNU/Linux and see if the problem reproduces?
> 
> Potentially, I'm using emacs-pgtk on wayland.
> 

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Visuwesh]

[-- Attachment #1: Type: text/plain, Size: 1145 bytes --]

[ஞாயிறு அக்டோபர் 27, 2024] Eli Zaretskii wrote:

> Ping!  Unless someone can debug this, I'm going to close this bug as
> unreproducible.

I can reproduce this with Cascadia Code but it took a lot more than "5
seconds."  I see no extra spaces but I see misalignment, it is not as
bad as it was reported in

    https://github.com/mickeynp/ligature.el/issues/56

by others.  I tried it with JetBrains Mono but failed to reproduce.  I
have a suspicion that it might be related to bug#54646 where I face(d) a
similar issue with Tamil text.  FWIW, I experienced a similar
misalignment (exactly) once with Julia Mono for the sequence

    LATIN SMALL LETTER A (a) + COMBINING CIRCUMFLEX ACCENT

where the accent was a little off to the side.  Since I run Emacs as a
daemon, closing all GUI frames and opening a fresh new one made the
misalignment go away.

I am attaching two images "good.png" and "misaligned.png" to illustrate
the issue.  [ I hope the images aren't too big.  ]

In the "misaligned" Emacs instance, some combination of font
weight+size+slant do not exhibit the misalignment.


[-- Attachment #2: misaligned.png --]
[-- Type: image/png, Size: 104838 bytes --]

[-- Attachment #3: good.png --]
[-- Type: image/png, Size: 105566 bytes --]

[-- Attachment #4: Type: text/plain, Size: 2471 bytes --]


M-x report-emacs-bug RET reports:

In GNU Emacs 31.0.50 (build 15, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.18.0, Xaw scroll bars) of 2024-10-17 built on astatine
Repository revision: fada04cfc788a486265c9da6636611986b48ae49
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101011
System Description: Debian GNU/Linux trixie/sid

Configured using:
 'configure --with-sound=alsa --with-x-toolkit=lucid --without-xaw3d
 --without-gconf --without-libsystemd --with-cairo CFLAGS=-g3'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
LCMS2 LIBOTF LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS WEBP X11 XDBE XIM
XINPUT2 XPM LUCID ZLIB

Important settings:
  value of $LC_MONETARY: ta_IN.UTF-8
  value of $LC_NUMERIC: ta_IN.UTF-8
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix


>> Date: Sat, 12 Oct 2024 12:09:02 -0400
>> Cc: 73752@debbugs.gnu.org
>> From: Yixuan Chen <xuan@xlk.me>
>> 
>> On 10/12/24 04:02, Eli Zaretskii wrote:
>> > I cannot reproduce the problem on my system.  I've let Emacs randomize
>> > the fonts for quite some time, and everything keeps rendering
>> > correctly without the extra spaces.  When you say "eventually", how
>> > long do you typically wait until the problem appears?
>> 
>> It took me 5 seconds at most, and usually happened immediately.
>> 
>> > (As two deviations from your init.el, I used a different font
>> > (Cascadia Code), since I don't have JetBrains Mono installed; and I
>> > replaces use-package with the equivalent Lisp, since I don't want to
>> > install ligatures.el on this system.  I don't think these two
>> > deviations should matter, but I mention them FTR.)
>> 
>> I can replicate the same problem (and even worse in some cases) using 
>> Cascadia Code.
>> 
>> The `use-package` should be replaceable, except you want to keep the 
>> list of ligature strings formatted in the same way. I'm not fluent in 
>> Elisp or emacs API, so my dumb way of randomization is achieved through 
>> regexp matching over the ligature strings. If they are re-organized in 
>> some way (say merged into one huge line), it might break the randomization.
>> 
>> > I think this might be system-dependent.  Can someone try this in a
>> > Cairo build on GNU/Linux and see if the problem reproduces?
>> 
>> Potentially, I'm using emacs-pgtk on wayland.
>> 

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Visuwesh <visuweshm@gmail.com>
> Cc: Yixuan Chen <xuan@xlk.me>,  Po Lu <luangruo@yahoo.com>,
>   73752@debbugs.gnu.org
> Date: Sun, 27 Oct 2024 21:49:32 +0530
> 
> [ஞாயிறு அக்டோபர் 27, 2024] Eli Zaretskii wrote:
> 
> > Ping!  Unless someone can debug this, I'm going to close this bug as
> > unreproducible.
> 
> I can reproduce this with Cascadia Code but it took a lot more than "5
> seconds."  I see no extra spaces but I see misalignment, it is not as
> bad as it was reported in
> 
>     https://github.com/mickeynp/ligature.el/issues/56
> 
> by others.  I tried it with JetBrains Mono but failed to reproduce.  I
> have a suspicion that it might be related to bug#54646 where I face(d) a
> similar issue with Tamil text.  FWIW, I experienced a similar
> misalignment (exactly) once with Julia Mono for the sequence
> 
>     LATIN SMALL LETTER A (a) + COMBINING CIRCUMFLEX ACCENT
> 
> where the accent was a little off to the side.  Since I run Emacs as a
> daemon, closing all GUI frames and opening a fresh new one made the
> misalignment go away.
> 
> I am attaching two images "good.png" and "misaligned.png" to illustrate
> the issue.  [ I hope the images aren't too big.  ]

It's quite clear from the image that the "misaligned" line uses a font
with a different slant/weight/height value.  If that is the reason, I
guess the problem is with composition caching, but why is that an
issue in real life?  Do real-life Lisp programs modify face font
attributes so frequently?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Cc: luangruo@yahoo.com, 73752@debbugs.gnu.org, xuan@xlk.me
> Date: Sun, 27 Oct 2024 19:19:48 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> It's quite clear from the image that the "misaligned" line uses a font
> with a different slant/weight/height value.  If that is the reason, I
> guess the problem is with composition caching, but why is that an
> issue in real life?  Do real-life Lisp programs modify face font
> attributes so frequently?

Or maybe I don't understand the original problem.  The bug report says
"extra spaces", but eacg font has its own metrics of the SPC glyph, so
highlighting a line with a given face can affect the metrics of the
whitespace.  Why is this surprising?  Or what did I miss?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

On 10/27/24 13:19, Eli Zaretskii wrote:
> It's quite clear from the image that the "misaligned" line uses a font
> with a different slant/weight/height value.  If that is the reason, I
> guess the problem is with composition caching, but why is that an
> issue in real life?  Do real-life Lisp programs modify face font
> attributes so frequently?

It's not about frequency, but probably total number of font face 
attribute changes. The lisp code I submitted was changing the attributes 
rapidly just to recreate the problem faster. In real-life, this problem 
happens with normal emacs configuration, usually within half an hour.

For reference, here 
(https://github.com/LukeXuan/dot-files/tree/e6067ec232ea34e01fb9a1f1a358d1c328d47c50/.config/doom) 
is my emacs configuration on top of doom. I primarily write LaTeX and 
Coq and this problem occurs every 30 minutes or so. The solution is to 
either restart emacs (what I used to do) or disable ligature altogether 
(what I do now).

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

On 10/27/24 13:27, Eli Zaretskii wrote:
> Or maybe I don't understand the original problem.  The bug report says
> "extra spaces", but eacg font has its own metrics of the SPC glyph, so
> highlighting a line with a given face can affect the metrics of the
> whitespace.  Why is this surprising?  Or what did I miss?

The problem is that the "space" being rendered is different in the 
following two case:

- if the default font is set to a specific combination of attributes 
(say 16, bold, normal) from start, it renders all the characters with 
one version of spacing. Let's call this version of spacing the 
"expected" spacing,
- if the default font is some other combination of attributes (say 16, 
regular, normal), and one line is set to the attribute of (16, bold, 
normal). There is a **probability** that this line will render with a 
different version of spacing than the "expected" spacing. In my setup, 
this different spacing is extra wide.

Ultimately, I believe the following should be true: if one line is 
rendered with **the same font face attributes**, it should always be 
rendered in **the same way**, regardless when, where, and how its 
attributes are assigned. This is not the case for emacs right now, and 
thus I consider it a bug.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Date: Sun, 27 Oct 2024 13:39:56 -0400
> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
> From: Yixuan Chen <xuan@xlk.me>
> 
> On 10/27/24 13:27, Eli Zaretskii wrote:
> > Or maybe I don't understand the original problem.  The bug report says
> > "extra spaces", but eacg font has its own metrics of the SPC glyph, so
> > highlighting a line with a given face can affect the metrics of the
> > whitespace.  Why is this surprising?  Or what did I miss?
> 
> The problem is that the "space" being rendered is different in the 
> following two case:
> 
> - if the default font is set to a specific combination of attributes 
> (say 16, bold, normal) from start, it renders all the characters with 
> one version of spacing. Let's call this version of spacing the 
> "expected" spacing,
> - if the default font is some other combination of attributes (say 16, 
> regular, normal), and one line is set to the attribute of (16, bold, 
> normal). There is a **probability** that this line will render with a 
> different version of spacing than the "expected" spacing. In my setup, 
> this different spacing is extra wide.
> 
> Ultimately, I believe the following should be true: if one line is 
> rendered with **the same font face attributes**, it should always be 
> rendered in **the same way**, regardless when, where, and how its 
> attributes are assigned. This is not the case for emacs right now, and 
> thus I consider it a bug.

Sorry, I still don't understand.  Your code does

  (highlight-lines-matching-regexp (cdr face) (car face))

This potentially shows each line in a different face, and thus can
affect the metrics of the SPC character glyph which is what the
indentation is made of.  So why is this a problem, let alone a bug?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

> Sorry, I still don't understand.  Your code does
> 
>    (highlight-lines-matching-regexp (cdr face) (car face))
> 
> This potentially shows each line in a different face, and thus can
> affect the metrics of the SPC character glyph which is what the
> indentation is made of.  So why is this a problem, let alone a bug?

The problem is executing that line at 10PM today may render the font one 
way, while executing that line at 6AM tomorrow (with exactly the same 
"face" variable) may render the font a different way, even if all the 
other variables remains the same.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Date: Sun, 27 Oct 2024 13:46:55 -0400
> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
> From: Yixuan Chen <xuan@xlk.me>
> 
> > Sorry, I still don't understand.  Your code does
> > 
> >    (highlight-lines-matching-regexp (cdr face) (car face))
> > 
> > This potentially shows each line in a different face, and thus can
> > affect the metrics of the SPC character glyph which is what the
> > indentation is made of.  So why is this a problem, let alone a bug?
> 
> The problem is executing that line at 10PM today may render the font one 
> way, while executing that line at 6AM tomorrow (with exactly the same 
> "face" variable) may render the font a different way, even if all the 
> other variables remains the same.

But you modify the faces in a random fashion, so why is this a
surprise that it is not deterministic?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

On 10/27/24 15:14, Eli Zaretskii wrote:
>> Date: Sun, 27 Oct 2024 13:46:55 -0400
>> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
>> From: Yixuan Chen <xuan@xlk.me>
>>
>>> Sorry, I still don't understand.  Your code does
>>>
>>>     (highlight-lines-matching-regexp (cdr face) (car face))
>>>
>>> This potentially shows each line in a different face, and thus can
>>> affect the metrics of the SPC character glyph which is what the
>>> indentation is made of.  So why is this a problem, let alone a bug?
>>
>> The problem is executing that line at 10PM today may render the font one
>> way, while executing that line at 6AM tomorrow (with exactly the same
>> "face" variable) may render the font a different way, even if all the
>> other variables remains the same.
> 
> But you modify the faces in a random fashion, so why is this a
> surprise that it is not deterministic?

I said,

 > ... (with exactly the same "face" variable) ... even if all the
 > other variables remains the same.

Yes, this script is modifying the scripts in a random fashion, but let's 
imagine this:

1. the first time I open emacs, start the script and it randomizes the 
first line to a combination of face attributes (let's say (16, bold, 
regular)) in 5 seconds after going through 50 different combinations, it 
looks one way.

2. I restart emacs, start the script and it randomizes the first line to 
the same combination of face attributes (16, bold, regular) in 5 minutes 
after going through 3000 combinations, it looks differently from the 
first time,

Do you agree that in both cases that single line should look the same? 
And they should always look the same as long as the randomization hits 
(16, bold, regular) on that line regardless of time and previous history?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

On 10/27/24 15:14, Eli Zaretskii wrote:
>> Date: Sun, 27 Oct 2024 13:46:55 -0400
>> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
>> From: Yixuan Chen <xuan@xlk.me>
>>
>>> Sorry, I still don't understand.  Your code does
>>>
>>>     (highlight-lines-matching-regexp (cdr face) (car face))
>>>
>>> This potentially shows each line in a different face, and thus can
>>> affect the metrics of the SPC character glyph which is what the
>>> indentation is made of.  So why is this a problem, let alone a bug?
>>
>> The problem is executing that line at 10PM today may render the font one
>> way, while executing that line at 6AM tomorrow (with exactly the same
>> "face" variable) may render the font a different way, even if all the
>> other variables remains the same.
> 
> But you modify the faces in a random fashion, so why is this a
> surprise that it is not deterministic?

Just in case my last explanation wasn't clear enough: by "with exactly 
the same face variable", I'm saying that when the randomizer picks the 
same font attribute (the same everything), there is a chance it will 
render that line differently.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Date: Sun, 27 Oct 2024 15:36:06 -0400
> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
> From: Yixuan Chen <xuan@xlk.me>
> 
> On 10/27/24 15:14, Eli Zaretskii wrote:
> >> Date: Sun, 27 Oct 2024 13:46:55 -0400
> >> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
> >> From: Yixuan Chen <xuan@xlk.me>
> >>
> >>> Sorry, I still don't understand.  Your code does
> >>>
> >>>     (highlight-lines-matching-regexp (cdr face) (car face))
> >>>
> >>> This potentially shows each line in a different face, and thus can
> >>> affect the metrics of the SPC character glyph which is what the
> >>> indentation is made of.  So why is this a problem, let alone a bug?
> >>
> >> The problem is executing that line at 10PM today may render the font one
> >> way, while executing that line at 6AM tomorrow (with exactly the same
> >> "face" variable) may render the font a different way, even if all the
> >> other variables remains the same.
> > 
> > But you modify the faces in a random fashion, so why is this a
> > surprise that it is not deterministic?
> 
> I said,
> 
>  > ... (with exactly the same "face" variable) ... even if all the
>  > other variables remains the same.
> 
> Yes, this script is modifying the scripts in a random fashion, but let's 
> imagine this:
> 
> 1. the first time I open emacs, start the script and it randomizes the 
> first line to a combination of face attributes (let's say (16, bold, 
> regular)) in 5 seconds after going through 50 different combinations, it 
> looks one way.
> 
> 2. I restart emacs, start the script and it randomizes the first line to 
> the same combination of face attributes (16, bold, regular) in 5 minutes 
> after going through 3000 combinations, it looks differently from the 
> first time,
> 
> Do you agree that in both cases that single line should look the same? 
> And they should always look the same as long as the randomization hits 
> (16, bold, regular) on that line regardless of time and previous history?

That depends when redisplay kicks in, between your face
randomizations.  Among other factors.

And I still fail to understand why this largely artificial program is
important to investigate.  Are there any real-life cases where you see
the same face displayed differently when ligatures are involved?  If
not, what useful stuff will we learn by digging into this, especially
since reproducing the problem is so hard?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

On 10/27/24 15:44, Eli Zaretskii wrote:
>> Date: Sun, 27 Oct 2024 15:36:06 -0400
>> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
>> From: Yixuan Chen <xuan@xlk.me>
>>
>> On 10/27/24 15:14, Eli Zaretskii wrote:
>>>> Date: Sun, 27 Oct 2024 13:46:55 -0400
>>>> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
>>>> From: Yixuan Chen <xuan@xlk.me>
>>>>
>>>>> Sorry, I still don't understand.  Your code does
>>>>>
>>>>>      (highlight-lines-matching-regexp (cdr face) (car face))
>>>>>
>>>>> This potentially shows each line in a different face, and thus can
>>>>> affect the metrics of the SPC character glyph which is what the
>>>>> indentation is made of.  So why is this a problem, let alone a bug?
>>>>
>>>> The problem is executing that line at 10PM today may render the font one
>>>> way, while executing that line at 6AM tomorrow (with exactly the same
>>>> "face" variable) may render the font a different way, even if all the
>>>> other variables remains the same.
>>>
>>> But you modify the faces in a random fashion, so why is this a
>>> surprise that it is not deterministic?
>>
>> I said,
>>
>>   > ... (with exactly the same "face" variable) ... even if all the
>>   > other variables remains the same.
>>
>> Yes, this script is modifying the scripts in a random fashion, but let's
>> imagine this:
>>
>> 1. the first time I open emacs, start the script and it randomizes the
>> first line to a combination of face attributes (let's say (16, bold,
>> regular)) in 5 seconds after going through 50 different combinations, it
>> looks one way.
>>
>> 2. I restart emacs, start the script and it randomizes the first line to
>> the same combination of face attributes (16, bold, regular) in 5 minutes
>> after going through 3000 combinations, it looks differently from the
>> first time,
>>
>> Do you agree that in both cases that single line should look the same?
>> And they should always look the same as long as the randomization hits
>> (16, bold, regular) on that line regardless of time and previous history?
> 
> That depends when redisplay kicks in, between your face
> randomizations.  Among other factors.
> 
> And I still fail to understand why this largely artificial program is
> important to investigate.  Are there any real-life cases where you see
> the same face displayed differently when ligatures are involved?  If
> not, what useful stuff will we learn by digging into this, especially
> since reproducing the problem is so hard?

Maybe you missed one of my previous email, let me copy paste what I said 
again,

> It's not about frequency, but probably total number of font face 
> attribute changes. The lisp code I submitted was changing the attributes 
> rapidly just to recreate the problem faster. In real-life, this problem 
> happens with normal emacs configuration, usually within half an hour.
> 
> For reference, here 
> (https://github.com/LukeXuan/dot-files/tree/e6067ec232ea34e01fb9a1f1a358d1c328d47c50/.config/doom) 
> is my emacs configuration on top of doom. I primarily write LaTeX and 
> Coq and this problem occurs every 30 minutes or so. The solution is to 
> either restart emacs (what I used to do) or disable ligature altogether 
> (what I do now).
> 

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Date: Sun, 27 Oct 2024 15:41:06 -0400
> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
> From: Yixuan Chen <xuan@xlk.me>
> 
> On 10/27/24 15:14, Eli Zaretskii wrote:
> >> Date: Sun, 27 Oct 2024 13:46:55 -0400
> >> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
> >> From: Yixuan Chen <xuan@xlk.me>
> >>
> >>> Sorry, I still don't understand.  Your code does
> >>>
> >>>     (highlight-lines-matching-regexp (cdr face) (car face))
> >>>
> >>> This potentially shows each line in a different face, and thus can
> >>> affect the metrics of the SPC character glyph which is what the
> >>> indentation is made of.  So why is this a problem, let alone a bug?
> >>
> >> The problem is executing that line at 10PM today may render the font one
> >> way, while executing that line at 6AM tomorrow (with exactly the same
> >> "face" variable) may render the font a different way, even if all the
> >> other variables remains the same.
> > 
> > But you modify the faces in a random fashion, so why is this a
> > surprise that it is not deterministic?
> 
> Just in case my last explanation wasn't clear enough: by "with exactly 
> the same face variable", I'm saying that when the randomizer picks the 
> same font attribute (the same everything), there is a chance it will 
> render that line differently.

That's not what I see in the images I've been shown.  The lines that
have different indentation have either different slant or different
weight.

To convince me that this is really happening (although I'm unable to
understand how it could, given how Emacs faces work), you will need to
show some code which generates such a situation in a reproducible
manner, and then show me by using "M-x describe-text-properties" and
"C-u C-x =" that indeed the same characters in the same face are shown
on different lines with different metrics.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Date: Sun, 27 Oct 2024 15:47:57 -0400
> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
> From: Yixuan Chen <xuan@xlk.me>
> 
> > It's not about frequency, but probably total number of font face 
> > attribute changes. The lisp code I submitted was changing the attributes 
> > rapidly just to recreate the problem faster. In real-life, this problem 
> > happens with normal emacs configuration, usually within half an hour.

What is "the problem which happens within half an hour" in real-life?

And what causes "font face attribute changes" in that case?

Also, can you reproduce the problem at will, and then leave that
session running so I could ask you to report some details and perform
some experiments?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

[-- Attachment #1: Type: text/plain, Size: 3949 bytes --]

On 10/27/24 16:07, Eli Zaretskii wrote:
> To convince me that this is really happening (although I'm unable to
> understand how it could, given how Emacs faces work), you will need to
> show some code which generates such a situation in a reproducible
> manner, and then show me by using "M-x describe-text-properties" and
> "C-u C-x =" that indeed the same characters in the same face are shown
> on different lines with different metrics.

OK, here you go.

> you will need to show some code which generates such a situation in a reproducible manner

The same script I submitted in this bug.

"screenshot1.png" shows the bugged display. Here's the result of 
"describe-text-properties",
> There are text properties here:
>   face                 (face12 font-lock-string-face)
>   fontified            t
In case you want to know what the face12 is,
> Face: face12 (sample) (customize this face)
> 
> Documentation:
> Not documented as a face.
> 
> 
>            Family: unspecified
>           Foundry: unspecified
>             Width: unspecified
>            Height: 125
>            Weight: medium
>             Slant: italic
>        Foreground: unspecified
> DistantForeground: unspecified
>        Background: unspecified
>         Underline: unspecified
>          Overline: unspecified
>    Strike-through: unspecified
>               Box: unspecified
>           Inverse: unspecified
>           Stipple: unspecified
>              Font: unspecified
>           Fontset: unspecified
>            Extend: unspecified
>           Inherit: unspecified
and here's the result of "C-x C-u =":
>              position: 1277 of 3527 (36%), column: 34
>             character: > (displayed as >) (codepoint 62, #o76, #x3e)
>               charset: ascii (ASCII (ISO646 IRV))
> code point in charset: 0x3E
>                script: latin
>                syntax: _ 	which means: symbol
>              category: .:Base, a:ASCII, l:Latin, r:Roman
>              to input: type "C-x 8 RET 3e" or "C-x 8 RET GREATER-THAN SIGN"
>           buffer code: #x3E
>             file code: #x3E (encoded by coding system utf-8-unix)
>               display: by this font (glyph code):
>     ftcrhb:-SAJA-Cascadia Code-regular-italic-normal-*-17-*-*-*-m-0-iso10646-1 (#x3AC)
> 
> Character code properties: customize what to show
>   name: GREATER-THAN SIGN
>   general-category: Sm (Symbol, Math)
>   decomposition: (62) ('>')
> 
> There are text properties here:
>   face                 (face12 font-lock-string-face)
>   fontified            t

In a separate fresh new emacs window, I execute the following command:

     (set-face-attribute 'default nil :weight 'medium :slant 'italic 
:height 125)

"screenshot2.png" shows how it looks this time (the normal display).

the result of "describe-text-properties":
> There are text properties here:
>   face                 font-lock-string-face
>   fontified            nil
and "C-u C-x =":
>              position: 1277 of 3527 (36%), column: 34
>             character: > (displayed as >) (codepoint 62, #o76, #x3e)
>               charset: ascii (ASCII (ISO646 IRV))
> code point in charset: 0x3E
>                script: latin
>                syntax: _ 	which means: symbol
>              category: .:Base, a:ASCII, l:Latin, r:Roman
>              to input: type "C-x 8 RET 3e" or "C-x 8 RET GREATER-THAN SIGN"
>           buffer code: #x3E
>             file code: #x3E (encoded by coding system utf-8-unix)
>               display: by this font (glyph code):
>     ftcrhb:-SAJA-Cascadia Code-regular-italic-normal-*-17-*-*-*-m-0-iso10646-1 (#x3AC)
> 
> Character code properties: customize what to show
>   name: GREATER-THAN SIGN
>   general-category: Sm (Symbol, Math)
>   decomposition: (62) ('>')
> 
> There are text properties here:
>   face                 font-lock-string-face
>   fontified            t

If you want more information, I will keep both emacs window open for the 
next two hours.

[-- Attachment #2: screenshot1.png --]
[-- Type: image/png, Size: 15002 bytes --]

[-- Attachment #3: screenshot2.png --]
[-- Type: image/png, Size: 13496 bytes --]

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Visuwesh]

[ஞாயிறு அக்டோபர் 27, 2024] Eli Zaretskii wrote:

>> Cc: luangruo@yahoo.com, 73752@debbugs.gnu.org, xuan@xlk.me
>> Date: Sun, 27 Oct 2024 19:19:48 +0200
>> From: Eli Zaretskii <eliz@gnu.org>
>> 
>> It's quite clear from the image that the "misaligned" line uses a font
>> with a different slant/weight/height value.  If that is the reason, I
>> guess the problem is with composition caching, but why is that an
>> issue in real life?  Do real-life Lisp programs modify face font
>> attributes so frequently?

Real-life Lisp programs certainly do not change face font attributes so
often but I believe the script does it so to reproduce the issue
quickly.  In a regular Emacs session, it is enough for the same text to
be shown in different font sizes (as a consequence of using C-x C-+) and
font weights to eventually exhibit this misalignment IME.

> Or maybe I don't understand the original problem.  The bug report says
> "extra spaces", but eacg font has its own metrics of the SPC glyph, so
> highlighting a line with a given face can affect the metrics of the
> whitespace.  Why is this surprising?  Or what did I miss?

I do not understand the technical details but the width of the glyph
used to draw it is not the one that should be used for the underlying
font (weight, size, etc. included) which leads to this misalignment.

To make it more clear, let's say that =:= is shaped for a font X with a
specific weight, size, etc.  At a later point in time, the width of the
glyph corresponding to X is used to draw =:= with font Y of same family.
This leads to the observed misalignment AFAIU.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Date: Sun, 27 Oct 2024 16:32:34 -0400
> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
> From: Yixuan Chen <xuan@xlk.me>
> 
> On 10/27/24 16:07, Eli Zaretskii wrote:
> > To convince me that this is really happening (although I'm unable to
> > understand how it could, given how Emacs faces work), you will need to
> > show some code which generates such a situation in a reproducible
> > manner, and then show me by using "M-x describe-text-properties" and
> > "C-u C-x =" that indeed the same characters in the same face are shown
> > on different lines with different metrics.
> 
> OK, here you go.

Not exactly what I asked for, or understood how the problem manifests
itself...

> "screenshot1.png" shows the bugged display. Here's the result of 
> "describe-text-properties",
> > There are text properties here:
> >   face                 (face12 font-lock-string-face)
> >   fontified            t

But this is a completely different issue.  There's no indentation
here, right?  You are saying that in the "bad" display there's some
extra space between the ligature and the following quote, right?  Is
that extra space a real SPC glyph or is it just that the ligature is
considered "wider"?  What happens if you put the cursor on the ▷
ligature in the "bad" display -- does the block cursor then take up
all the space up to the next quote?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

[-- Attachment #1: Type: text/plain, Size: 2126 bytes --]



On 10/28/24 10:25, Eli Zaretskii wrote:
>> Date: Sun, 27 Oct 2024 16:32:34 -0400
>> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
>> From: Yixuan Chen <xuan@xlk.me>
>>
>> On 10/27/24 16:07, Eli Zaretskii wrote:
>>> To convince me that this is really happening (although I'm unable to
>>> understand how it could, given how Emacs faces work), you will need to
>>> show some code which generates such a situation in a reproducible
>>> manner, and then show me by using "M-x describe-text-properties" and
>>> "C-u C-x =" that indeed the same characters in the same face are shown
>>> on different lines with different metrics.
>>
>> OK, here you go.
> 
> Not exactly what I asked for, or understood how the problem manifests
> itself...
> 
>> "screenshot1.png" shows the bugged display. Here's the result of
>> "describe-text-properties",
>>> There are text properties here:
>>>    face                 (face12 font-lock-string-face)
>>>    fontified            t
> 
> But this is a completely different issue.  There's no indentation
> here, right? You are saying that in the "bad" display there's some
> extra space between the ligature and the following quote, right?

Maybe my use of words were terrible. The problem I want to report was 
never about indentation. It was always about the "extra space between 
the ligature and the following character" (and sometimes also some extra 
space between the ligature and the character, although not in this 
instance).

> Is
> that extra space a real SPC glyph or is it just that the ligature is
> considered "wider"?  What happens if you put the cursor on the ▷
> ligature in the "bad" display -- does the block cursor then take up
> all the space up to the next quote?

It's not a real SPC glyph. It's a single character ▷ (or ligature/glyph 
I should call it? the underlying text is "|||>"). I attached five 
screenshots, where the block cursor is at ", |, |, |, >, " respectively. 
Noticeably, the block cursor for the first | character (1_bar.png) is 
extra wide, and the block cursors for the following characters looks 
offset from where they are.

[-- Attachment #2: 0_quote.png --]
[-- Type: image/png, Size: 1816 bytes --]

[-- Attachment #3: 1_bar.png --]
[-- Type: image/png, Size: 1607 bytes --]

[-- Attachment #4: 2_bar.png --]
[-- Type: image/png, Size: 1597 bytes --]

[-- Attachment #5: 3_bar.png --]
[-- Type: image/png, Size: 1352 bytes --]

[-- Attachment #6: 4_gt.png --]
[-- Type: image/png, Size: 1454 bytes --]

[-- Attachment #7: 5_quote.png --]
[-- Type: image/png, Size: 1364 bytes --]

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

Also, this morning someone mentioned on the github issue I linked above 
stating that he/she encountered a similar problem, and switching from 
cairo to Xft fixed the problem. I don't know if it's related, but you 
might want to know.

I use wayland myself and xwayland doesn't support fractional scaling, so 
that's not a feasible solution for me.

On 10/28/24 10:44, Yixuan Chen wrote:
> 
> 
> On 10/28/24 10:25, Eli Zaretskii wrote:
>>> Date: Sun, 27 Oct 2024 16:32:34 -0400
>>> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
>>> From: Yixuan Chen <xuan@xlk.me>
>>>
>>> On 10/27/24 16:07, Eli Zaretskii wrote:
>>>> To convince me that this is really happening (although I'm unable to
>>>> understand how it could, given how Emacs faces work), you will need to
>>>> show some code which generates such a situation in a reproducible
>>>> manner, and then show me by using "M-x describe-text-properties" and
>>>> "C-u C-x =" that indeed the same characters in the same face are shown
>>>> on different lines with different metrics.
>>>
>>> OK, here you go.
>>
>> Not exactly what I asked for, or understood how the problem manifests
>> itself...
>>
>>> "screenshot1.png" shows the bugged display. Here's the result of
>>> "describe-text-properties",
>>>> There are text properties here:
>>>>    face                 (face12 font-lock-string-face)
>>>>    fontified            t
>>
>> But this is a completely different issue.  There's no indentation
>> here, right? You are saying that in the "bad" display there's some
>> extra space between the ligature and the following quote, right?
> 
> Maybe my use of words were terrible. The problem I want to report was 
> never about indentation. It was always about the "extra space between 
> the ligature and the following character" (and sometimes also some extra 
> space between the ligature and the character, although not in this 
> instance).
> 
>> Is
>> that extra space a real SPC glyph or is it just that the ligature is
>> considered "wider"?  What happens if you put the cursor on the ▷
>> ligature in the "bad" display -- does the block cursor then take up
>> all the space up to the next quote?
> 
> It's not a real SPC glyph. It's a single character ▷ (or ligature/glyph 
> I should call it? the underlying text is "|||>"). I attached five 
> screenshots, where the block cursor is at ", |, |, |, >, " respectively. 
> Noticeably, the block cursor for the first | character (1_bar.png) is 
> extra wide, and the block cursors for the following characters looks 
> offset from where they are.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Visuwesh <visuweshm@gmail.com>
> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
> Date: Mon, 28 Oct 2024 09:56:13 +0530
> 
> Real-life Lisp programs certainly do not change face font attributes so
> often but I believe the script does it so to reproduce the issue
> quickly.  In a regular Emacs session, it is enough for the same text to
> be shown in different font sizes (as a consequence of using C-x C-+) and
> font weights to eventually exhibit this misalignment IME.

You need to catch this situation in some reproducible recipe.  Because
up front I don't understand how is this possible: we cache each
composition with its font object, which includes the font size (and
also slant, weight, etc.), so a different variant of the same font
ought to match only cache entries that use the exact same font.  Or
maybe I don't understand well enough what
composition_gstring_put_cache does to compute the hash of a
glyph-string header (which is the hash key for a composition)?

> I do not understand the technical details but the width of the glyph
> used to draw it is not the one that should be used for the underlying
> font (weight, size, etc. included) which leads to this misalignment.

Which seems to indicate that we somehow use a different font's metric.

> To make it more clear, let's say that =:= is shaped for a font X with a
> specific weight, size, etc.  At a later point in time, the width of the
> glyph corresponding to X is used to draw =:= with font Y of same family.
> This leads to the observed misalignment AFAIU.

But how can this happen?  Without a reproducible recipe, which can be
reproduced without waiting for too long, it is very hard to
investigate this.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

[-- Attachment #1: Type: text/plain, Size: 990 bytes --]

> Date: Mon, 28 Oct 2024 10:44:23 -0400
> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
> From: Yixuan Chen <xuan@xlk.me>
> 
> > Is
> > that extra space a real SPC glyph or is it just that the ligature is
> > considered "wider"?  What happens if you put the cursor on the ▷
> > ligature in the "bad" display -- does the block cursor then take up
> > all the space up to the next quote?
> 
> It's not a real SPC glyph. It's a single character ▷ (or ligature/glyph 
> I should call it? the underlying text is "|||>"). I attached five 
> screenshots, where the block cursor is at ", |, |, |, >, " respectively. 
> Noticeably, the block cursor for the first | character (1_bar.png) is 
> extra wide, and the block cursors for the following characters looks 
> offset from where they are.

According to the screenshots, it actually looks like it's a real SPC
glyph or something.  What does "C-u C-x =" show when the block cursor
is shown as below?


[-- Attachment #2: 4_gt.png --]
[-- Type: image/png, Size: 1454 bytes --]

[-- Attachment #3: Type: text/plain, Size: 172 bytes --]


Actually, how about showing what "C-u C-x =" says in each of the 6
positions you show?  And then show what "C-u C-x =" says in the same 6
positions in the "good" display?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

[-- Attachment #1: Type: text/plain, Size: 2322 bytes --]

On 10/28/24 11:05, Eli Zaretskii wrote:
>> Date: Mon, 28 Oct 2024 10:44:23 -0400
>> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
>> From: Yixuan Chen <xuan@xlk.me>
>>
>>> Is
>>> that extra space a real SPC glyph or is it just that the ligature is
>>> considered "wider"?  What happens if you put the cursor on the ▷
>>> ligature in the "bad" display -- does the block cursor then take up
>>> all the space up to the next quote?
>>
>> It's not a real SPC glyph. It's a single character ▷ (or ligature/glyph
>> I should call it? the underlying text is "|||>"). I attached five
>> screenshots, where the block cursor is at ", |, |, |, >, " respectively.
>> Noticeably, the block cursor for the first | character (1_bar.png) is
>> extra wide, and the block cursors for the following characters looks
>> offset from where they are.
> 
> According to the screenshots, it actually looks like it's a real SPC
> glyph or something.  What does "C-u C-x =" show when the block cursor
> is shown as below?

It shows
>              position: 1277 of 3527 (36%), column: 34
>             character: > (displayed as >) (codepoint 62, #o76, #x3e)
>               charset: ascii (ASCII (ISO646 IRV))
> code point in charset: 0x3E
>                script: latin
>                syntax: _ 	which means: symbol
>              category: .:Base, a:ASCII, l:Latin, r:Roman
>              to input: type "C-x 8 RET 3e" or "C-x 8 RET GREATER-THAN SIGN"
>           buffer code: #x3E
>             file code: #x3E (encoded by coding system utf-8-unix)
>               display: by this font (glyph code):
>     ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x810)
> 
> Character code properties: customize what to show
>   name: GREATER-THAN SIGN
>   general-category: Sm (Symbol, Math)
>   decomposition: (62) ('>')
> 
> There are text properties here:
>   face                 (face12 font-lock-string-face)
>   fontified            t

> Actually, how about showing what "C-u C-x =" says in each of the 6
> positions you show?  And then show what "C-u C-x =" says in the same 6
> positions in the "good" display?

See "bad.txt" for the output of the bad emacs display, and "good.txt" 
for the output of good emacs display. I also attached the block cursor 
screenshots for good emacs display.

[-- Attachment #2: good.txt --]
[-- Type: text/plain, Size: 5447 bytes --]

0:

             position: 1273 of 3535 (36%), column: 30
            character: " (displayed as ") (codepoint 34, #o42, #x22)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x22
               script: latin
               syntax: " 	which means: string
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 22" or "C-x 8 RET QUOTATION MARK"
          buffer code: #x22
            file code: #x22 (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x779)

Character code properties: customize what to show
  name: QUOTATION MARK
  general-category: Po (Punctuation, Other)
  decomposition: (34) ('"')

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t

1:

             position: 1274 of 3535 (36%), column: 31
            character: | (displayed as |) (codepoint 124, #o174, #x7c)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x7C
               script: latin
               syntax: _ 	which means: symbol
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 7c" or "C-x 8 RET VERTICAL LINE"
          buffer code: #x7C
            file code: #x7C (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x9F6)

Character code properties: customize what to show
  name: VERTICAL LINE
  old-name: VERTICAL BAR
  general-category: Sm (Symbol, Math)
  decomposition: (124) ('|')

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t

2:

             position: 1275 of 3535 (36%), column: 32
            character: | (displayed as |) (codepoint 124, #o174, #x7c)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x7C
               script: latin
               syntax: _ 	which means: symbol
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 7c" or "C-x 8 RET VERTICAL LINE"
          buffer code: #x7C
            file code: #x7C (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x9F6)

Character code properties: customize what to show
  name: VERTICAL LINE
  old-name: VERTICAL BAR
  general-category: Sm (Symbol, Math)
  decomposition: (124) ('|')

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t

3:

             position: 1276 of 3535 (36%), column: 33
            character: | (displayed as |) (codepoint 124, #o174, #x7c)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x7C
               script: latin
               syntax: _ 	which means: symbol
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 7c" or "C-x 8 RET VERTICAL LINE"
          buffer code: #x7C
            file code: #x7C (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x9F6)

Character code properties: customize what to show
  name: VERTICAL LINE
  old-name: VERTICAL BAR
  general-category: Sm (Symbol, Math)
  decomposition: (124) ('|')

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t

4:

             position: 1277 of 3535 (36%), column: 34
            character: > (displayed as >) (codepoint 62, #o76, #x3e)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x3E
               script: latin
               syntax: _ 	which means: symbol
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 3e" or "C-x 8 RET GREATER-THAN SIGN"
          buffer code: #x3E
            file code: #x3E (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x810)

Character code properties: customize what to show
  name: GREATER-THAN SIGN
  general-category: Sm (Symbol, Math)
  decomposition: (62) ('>')

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t

5:

             position: 1278 of 3535 (36%), column: 35
            character: " (displayed as ") (codepoint 34, #o42, #x22)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x22
               script: latin
               syntax: " 	which means: string
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 22" or "C-x 8 RET QUOTATION MARK"
          buffer code: #x22
            file code: #x22 (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x779)

Character code properties: customize what to show
  name: QUOTATION MARK
  general-category: Po (Punctuation, Other)
  decomposition: (34) ('"')

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t

[-- Attachment #3: bad.txt --]
[-- Type: text/plain, Size: 5446 bytes --]

0:

             position: 1273 of 3527 (36%), column: 30
            character: " (displayed as ") (codepoint 34, #o42, #x22)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x22
               script: latin
               syntax: " 	which means: string
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 22" or "C-x 8 RET QUOTATION MARK"
          buffer code: #x22
            file code: #x22 (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x779)

Character code properties: customize what to show
  name: QUOTATION MARK
  general-category: Po (Punctuation, Other)
  decomposition: (34) ('"')

1:

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t

             position: 1274 of 3527 (36%), column: 31
            character: | (displayed as |) (codepoint 124, #o174, #x7c)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x7C
               script: latin
               syntax: _ 	which means: symbol
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 7c" or "C-x 8 RET VERTICAL LINE"
          buffer code: #x7C
            file code: #x7C (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x9F6)

Character code properties: customize what to show
  name: VERTICAL LINE
  old-name: VERTICAL BAR
  general-category: Sm (Symbol, Math)
  decomposition: (124) ('|')

2:

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t

             position: 1274 of 3527 (36%), column: 31
            character: | (displayed as |) (codepoint 124, #o174, #x7c)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x7C
               script: latin
               syntax: _ 	which means: symbol
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 7c" or "C-x 8 RET VERTICAL LINE"
          buffer code: #x7C
            file code: #x7C (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x9F6)

Character code properties: customize what to show
  name: VERTICAL LINE
  old-name: VERTICAL BAR
  general-category: Sm (Symbol, Math)
  decomposition: (124) ('|')

3:

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t
             position: 1276 of 3527 (36%), column: 33
            character: | (displayed as |) (codepoint 124, #o174, #x7c)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x7C
               script: latin
               syntax: _ 	which means: symbol
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 7c" or "C-x 8 RET VERTICAL LINE"
          buffer code: #x7C
            file code: #x7C (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x9F6)

Character code properties: customize what to show
  name: VERTICAL LINE
  old-name: VERTICAL BAR
  general-category: Sm (Symbol, Math)
  decomposition: (124) ('|')

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t

4:

             position: 1277 of 3527 (36%), column: 34
            character: > (displayed as >) (codepoint 62, #o76, #x3e)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x3E
               script: latin
               syntax: _ 	which means: symbol
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 3e" or "C-x 8 RET GREATER-THAN SIGN"
          buffer code: #x3E
            file code: #x3E (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x810)

Character code properties: customize what to show
  name: GREATER-THAN SIGN
  general-category: Sm (Symbol, Math)
  decomposition: (62) ('>')

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t

5:

             position: 1278 of 3527 (36%), column: 35
            character: " (displayed as ") (codepoint 34, #o42, #x22)
              charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x22
               script: latin
               syntax: " 	which means: string
             category: .:Base, a:ASCII, l:Latin, r:Roman
             to input: type "C-x 8 RET 22" or "C-x 8 RET QUOTATION MARK"
          buffer code: #x22
            file code: #x22 (encoded by coding system utf-8-unix)
              display: by this font (glyph code):
    ftcrhb:-SAJA-Cascadia Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x779)

Character code properties: customize what to show
  name: QUOTATION MARK
  general-category: Po (Punctuation, Other)
  decomposition: (34) ('"')

There are text properties here:
  face                 (face12 font-lock-string-face)
  fontified            t

[-- Attachment #4: 0.png --]
[-- Type: image/png, Size: 1630 bytes --]

[-- Attachment #5: 1.png --]
[-- Type: image/png, Size: 1625 bytes --]

[-- Attachment #6: 2.png --]
[-- Type: image/png, Size: 1314 bytes --]

[-- Attachment #7: 3.png --]
[-- Type: image/png, Size: 1934 bytes --]

[-- Attachment #8: 4.png --]
[-- Type: image/png, Size: 1788 bytes --]

[-- Attachment #9: 5.png --]
[-- Type: image/png, Size: 1458 bytes --]

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

On 10/28/24 10:59, Eli Zaretskii wrote:
>> From: Visuwesh <visuweshm@gmail.com>
>> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
>> Date: Mon, 28 Oct 2024 09:56:13 +0530
>>
>> Real-life Lisp programs certainly do not change face font attributes so
>> often but I believe the script does it so to reproduce the issue
>> quickly.  In a regular Emacs session, it is enough for the same text to
>> be shown in different font sizes (as a consequence of using C-x C-+) and
>> font weights to eventually exhibit this misalignment IME.
> 
> You need to catch this situation in some reproducible recipe.  Because
> up front I don't understand how is this possible: we cache each
> composition with its font object, which includes the font size (and
> also slant, weight, etc.), so a different variant of the same font
> ought to match only cache entries that use the exact same font.  Or
> maybe I don't understand well enough what
> composition_gstring_put_cache does to compute the hash of a
> glyph-string header (which is the hash key for a composition)?
> 
>> I do not understand the technical details but the width of the glyph
>> used to draw it is not the one that should be used for the underlying
>> font (weight, size, etc. included) which leads to this misalignment.
> 
> Which seems to indicate that we somehow use a different font's metric.
> 
>> To make it more clear, let's say that =:= is shaped for a font X with a
>> specific weight, size, etc.  At a later point in time, the width of the
>> glyph corresponding to X is used to draw =:= with font Y of same family.
>> This leads to the observed misalignment AFAIU.
> 
> But how can this happen?  Without a reproducible recipe, which can be
> reproduced without waiting for too long, it is very hard to
> investigate this.

Just my uninformed opinion. Considering,
- the fact that it's hard to reproduce the problem (the script I 
submitted is the best way to trigger the problem to my knowledge since 
it reproduces on multiple people's emacs consistently),
- the non-deterministic nature of this bug,
- changing fonts rapidly seems to help triggering the bug,
- my personal research bias,

This sounds like a concurrency problem, such as some kind of race 
condition leading to cache being over-written or else. But I have zero 
idea on how emacs works internally, so I can be completely wrong.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Visuwesh]

[திங்கள் அக்டோபர் 28, 2024] Eli Zaretskii wrote:

>> From: Visuwesh <visuweshm@gmail.com>
>> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
>> Date: Mon, 28 Oct 2024 09:56:13 +0530
>> 
>> Real-life Lisp programs certainly do not change face font attributes so
>> often but I believe the script does it so to reproduce the issue
>> quickly.  In a regular Emacs session, it is enough for the same text to
>> be shown in different font sizes (as a consequence of using C-x C-+) and
>> font weights to eventually exhibit this misalignment IME.
>
> You need to catch this situation in some reproducible recipe.  Because
> up front I don't understand how is this possible: we cache each
> composition with its font object, which includes the font size (and
> also slant, weight, etc.), so a different variant of the same font
> ought to match only cache entries that use the exact same font.  Or
> maybe I don't understand well enough what
> composition_gstring_put_cache does to compute the hash of a
> glyph-string header (which is the hash key for a composition)?

Is this hash dependent on the font driver?  IME and from what I see from
other issues in ligature.el's GitHub issue page, the problem seems to be
specific to Cairo builds which would also explain why you are not able
to reproduce this.

Also, do we clear the composition cache when all the GUI frames of an
Emacs daemon are deleted?  The misalignment goes away when I close all
the GUI frames of the daemon, and open a fresh new GUI frame.

>> I do not understand the technical details but the width of the glyph
>> used to draw it is not the one that should be used for the underlying
>> font (weight, size, etc. included) which leads to this misalignment.
>
> Which seems to indicate that we somehow use a different font's metric.

That is my impression, yes.

>> To make it more clear, let's say that =:= is shaped for a font X with a
>> specific weight, size, etc.  At a later point in time, the width of the
>> glyph corresponding to X is used to draw =:= with font Y of same family.
>> This leads to the observed misalignment AFAIU.
>
> But how can this happen?  Without a reproducible recipe, which can be
> reproduced without waiting for too long, it is very hard to
> investigate this.

I have no idea.  This is not easy to reproduce and seems to be heavily
dependent on the configuration.  Although I use a Cairo build (with
Lucid toolkit), the misalignment is not as severe as OP says and it
takes a lot more time to reproduce as well.  I could never come up with
a reproducer that takes less time.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Visuwesh <visuweshm@gmail.com>
> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
> Date: Mon, 28 Oct 2024 21:48:26 +0530
> 
> > You need to catch this situation in some reproducible recipe.  Because
> > up front I don't understand how is this possible: we cache each
> > composition with its font object, which includes the font size (and
> > also slant, weight, etc.), so a different variant of the same font
> > ought to match only cache entries that use the exact same font.  Or
> > maybe I don't understand well enough what
> > composition_gstring_put_cache does to compute the hash of a
> > glyph-string header (which is the hash key for a composition)?
> 
> Is this hash dependent on the font driver?

No.  Only the font used for the composed characters is recorded, not
the font backend which opened it.  But fonts are managed by the font
backend, so maybe there's some leakage by that way.

> Also, do we clear the composition cache when all the GUI frames of an
> Emacs daemon are deleted?

No, we don't, not directly.  But we clear the face cache, and that
removes the data of the fonts referred by the face cache from the
composition cache.  So it's quite possible the composition cache is
left empty when all the frames are deleted.

> The misalignment goes away when I close all
> the GUI frames of the daemon, and open a fresh new GUI frame.

Next time this happens, try one of these two:

  M-: (clear-face-cache t) RET
  M-: (clear-composition-cache) RET

and see if any of them corrects the problematic display.

Btw, how frequently do you use different frames, and how likely are
you to have different definitions for the same faces on different
frames in the same Emacs session at the same time?

> > But how can this happen?  Without a reproducible recipe, which can be
> > reproduced without waiting for too long, it is very hard to
> > investigate this.
> 
> I have no idea.  This is not easy to reproduce and seems to be heavily
> dependent on the configuration.  Although I use a Cairo build (with
> Lucid toolkit), the misalignment is not as severe as OP says and it
> takes a lot more time to reproduce as well.  I could never come up with
> a reproducer that takes less time.

The only way I see to investigate this is to wait for this to happen,
then attach GDB to Emacs and look at the problematic compositions in
the cache, comparing them to the corresponding compositions in a fresh
Emacs session.  I can tell what to look for with GDB, if that helps.
If we see that two compositions that should be identical differ by
their font objects, say, we'd at least have a lead and a starting
point for further debugging.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Date: Mon, 28 Oct 2024 11:20:29 -0400
> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
> From: Yixuan Chen <xuan@xlk.me>
> 
> > Actually, how about showing what "C-u C-x =" says in each of the 6
> > positions you show?  And then show what "C-u C-x =" says in the same 6
> > positions in the "good" display?
> 
> See "bad.txt" for the output of the bad emacs display, and "good.txt" 
> for the output of good emacs display. I also attached the block cursor 
> screenshots for good emacs display.

Thanks.  It's basically the same, as far as Emacs thinks, but do you
have any idea why EOB is different between these two cases (3535 vs
3527)?

Also, Emacs thinks that the second | character is at the same buffer
position and the same columns as the previous one -- any idea why?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Cc: luangruo@yahoo.com, 73752@debbugs.gnu.org, visuweshm@gmail.com
> Date: Mon, 28 Oct 2024 19:19:19 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> Also, Emacs thinks that the second | character is at the same buffer
> position and the same columns as the previous one -- any idea why?

That's in the "bad" output, I should say.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

On 10/28/24 13:19, Eli Zaretskii wrote:
>> Date: Mon, 28 Oct 2024 11:20:29 -0400
>> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
>> From: Yixuan Chen <xuan@xlk.me>
>>
>>> Actually, how about showing what "C-u C-x =" says in each of the 6
>>> positions you show?  And then show what "C-u C-x =" says in the same 6
>>> positions in the "good" display?
>>
>> See "bad.txt" for the output of the bad emacs display, and "good.txt"
>> for the output of good emacs display. I also attached the block cursor
>> screenshots for good emacs display.
> 
> Thanks.  It's basically the same, as far as Emacs thinks, but do you
> have any idea why EOB is different between these two cases (3535 vs
> 3527)?

Between the two versions (after bad.txt before good.txt), I made this 
change to the script: line 51 is modified from

     (run-at-time nil 1 #'random-faces)))
to
     ;; (run-at-time nil 1 #'random-faces)
     ))

so that I can manually set the face to face12. That's why EOB of 
"good.txt" is slightly larger.

> Also, Emacs thinks that the second | character is at the same buffer
> position and the same columns as the previous one -- any idea why?

I'm sorry, that's me being bad at mouse and copy-pasting. This is the 
correct output for the second | character in "bad.txt".

              position: 1275 of 3527 (36%), column: 32
             character: | (displayed as |) (codepoint 124, #o174, #x7c)
               charset: ascii (ASCII (ISO646 IRV))
code point in charset: 0x7C
                script: latin
                syntax: _ 	which means: symbol
              category: .:Base, a:ASCII, l:Latin, r:Roman
              to input: type "C-x 8 RET 7c" or "C-x 8 RET VERTICAL LINE"
           buffer code: #x7C
             file code: #x7C (encoded by coding system utf-8-unix)
               display: by this font (glyph code):
     ftcrhb:-SAJA-Cascadia 
Code-light-normal-normal-*-17-*-*-*-m-0-iso10646-1 (#x9F6)

Character code properties: customize what to show
   name: VERTICAL LINE
   old-name: VERTICAL BAR
   general-category: Sm (Symbol, Math)
   decomposition: (124) ('|')

There are text properties here:
   face                 (face12 font-lock-string-face)
   fontified            t

It seems like I simply pasted the output for the first bar character twice.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Date: Mon, 28 Oct 2024 13:28:14 -0400
> Cc: visuweshm@gmail.com, luangruo@yahoo.com, 73752@debbugs.gnu.org
> From: Yixuan Chen <xuan@xlk.me>
> 
> > Thanks.  It's basically the same, as far as Emacs thinks, but do you
> > have any idea why EOB is different between these two cases (3535 vs
> > 3527)?
> 
> Between the two versions (after bad.txt before good.txt), I made this 
> change to the script: line 51 is modified from
> 
>      (run-at-time nil 1 #'random-faces)))
> to
>      ;; (run-at-time nil 1 #'random-faces)
>      ))
> 
> so that I can manually set the face to face12. That's why EOB of 
> "good.txt" is slightly larger.
> 
> > Also, Emacs thinks that the second | character is at the same buffer
> > position and the same columns as the previous one -- any idea why?
> 
> I'm sorry, that's me being bad at mouse and copy-pasting. This is the 
> correct output for the second | character in "bad.txt".

OK, so Emacs really thinks the two buffers are identical, faces,
fonts, and all the rest.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Visuwesh]

[திங்கள் அக்டோபர் 28, 2024] Eli Zaretskii wrote:

>> From: Visuwesh <visuweshm@gmail.com>
>> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
>> Date: Mon, 28 Oct 2024 21:48:26 +0530
>> 
>> > You need to catch this situation in some reproducible recipe.  Because
>> > up front I don't understand how is this possible: we cache each
>> > composition with its font object, which includes the font size (and
>> > also slant, weight, etc.), so a different variant of the same font
>> > ought to match only cache entries that use the exact same font.  Or
>> > maybe I don't understand well enough what
>> > composition_gstring_put_cache does to compute the hash of a
>> > glyph-string header (which is the hash key for a composition)?
>> 
>> Is this hash dependent on the font driver?
>
> No.  Only the font used for the composed characters is recorded, not
> the font backend which opened it.  But fonts are managed by the font
> backend, so maybe there's some leakage by that way.

OK, thanks.  I wonder if we could compare the value returned by
font-info if something has gone wrong with the font object used to
compute the hash for the glyph?  

>> Also, do we clear the composition cache when all the GUI frames of an
>> Emacs daemon are deleted?
>
> No, we don't, not directly.  But we clear the face cache, and that
> removes the data of the fonts referred by the face cache from the
> composition cache.  So it's quite possible the composition cache is
> left empty when all the frames are deleted.

Thanks that would explain it but then...

>> The misalignment goes away when I close all
>> the GUI frames of the daemon, and open a fresh new GUI frame.
>
> Next time this happens, try one of these two:
>
>   M-: (clear-face-cache t) RET
>   M-: (clear-composition-cache) RET
>
> and see if any of them corrects the problematic display.

neither of them helped in the past.  I tried to if they help again but
failed to reproduce the problem today.

> Btw, how frequently do you use different frames, 

Quite often, I would say.  I usually have two frames but it can go
upwards of 5 to 6 if I have a mouse attached to my laptop.

> and how likely are you to have different definitions for the same
> faces on different frames in the same Emacs session at the same time?

I don't quite understand this question.  Are you asking if I have any
"frame-specific" face attributes i.e., non-nil FRAME argument in
set-face-attribute?  If so, no.

>> > But how can this happen?  Without a reproducible recipe, which can be
>> > reproduced without waiting for too long, it is very hard to
>> > investigate this.
>> 
>> I have no idea.  This is not easy to reproduce and seems to be heavily
>> dependent on the configuration.  Although I use a Cairo build (with
>> Lucid toolkit), the misalignment is not as severe as OP says and it
>> takes a lot more time to reproduce as well.  I could never come up with
>> a reproducer that takes less time.
>
> The only way I see to investigate this is to wait for this to happen,
> then attach GDB to Emacs and look at the problematic compositions in
> the cache, comparing them to the corresponding compositions in a fresh
> Emacs session.  I can tell what to look for with GDB, if that helps.

That would help.  But given how hard it is to reproduce this issue on my
end, I don't know when I can get back...

> If we see that two compositions that should be identical differ by
> their font objects, say, we'd at least have a lead and a starting
> point for further debugging.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Visuwesh <visuweshm@gmail.com>
> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
> Date: Tue, 29 Oct 2024 16:29:48 +0530
> 
> >> Is this hash dependent on the font driver?
> >
> > No.  Only the font used for the composed characters is recorded, not
> > the font backend which opened it.  But fonts are managed by the font
> > backend, so maybe there's some leakage by that way.
> 
> OK, thanks.  I wonder if we could compare the value returned by
> font-info if something has gone wrong with the font object used to
> compute the hash for the glyph?  

That would not be the first thing I'd look at.  According to the
screenshots, it is more likely that a wrong cache entry is used for a
composition, which uses the "wrong" font variant.  IOW, the font used
itself is fine, it just is not the font that's supposed to be used
with the composed characters in that place.

So I would first look at the font object stored in the header of the
cached composition.

> > Btw, how frequently do you use different frames, 
> 
> Quite often, I would say.  I usually have two frames but it can go
> upwards of 5 to 6 if I have a mouse attached to my laptop.
> 
> > and how likely are you to have different definitions for the same
> > faces on different frames in the same Emacs session at the same time?
> 
> I don't quite understand this question.  Are you asking if I have any
> "frame-specific" face attributes i.e., non-nil FRAME argument in
> set-face-attribute?

Yes.

> If so, no.

OK, so one more theory eats dust (we don't record the frame in the
composition cache).

> > The only way I see to investigate this is to wait for this to happen,
> > then attach GDB to Emacs and look at the problematic compositions in
> > the cache, comparing them to the corresponding compositions in a fresh
> > Emacs session.  I can tell what to look for with GDB, if that helps.
> 
> That would help.  But given how hard it is to reproduce this issue on my
> end, I don't know when I can get back...

It would not be useful for me to give instructions before you actually
hit the problem (because the code will change until then), so if you
want to try this, get back to me when you do reproduce the problem
(and then attach GDB and leave the Emacs session under GDB for any
investigations I'd ask you to do).

Thanks.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Visuwesh]

[-- Attachment #1: Type: text/plain, Size: 2787 bytes --]

[செவ்வாய் அக்டோபர் 29, 2024] Eli Zaretskii wrote:

>> From: Visuwesh <visuweshm@gmail.com>
>> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
>> Date: Tue, 29 Oct 2024 16:29:48 +0530
>> 
>> >> Is this hash dependent on the font driver?
>> >
>> > No.  Only the font used for the composed characters is recorded, not
>> > the font backend which opened it.  But fonts are managed by the font
>> > backend, so maybe there's some leakage by that way.
>> 
>> OK, thanks.  I wonder if we could compare the value returned by
>> font-info if something has gone wrong with the font object used to
>> compute the hash for the glyph?  
>
> That would not be the first thing I'd look at.  According to the
> screenshots, it is more likely that a wrong cache entry is used for a
> composition, which uses the "wrong" font variant.  IOW, the font used
> itself is fine, it just is not the font that's supposed to be used
> with the composed characters in that place.
>
> So I would first look at the font object stored in the header of the
> cached composition.
>
>> > Btw, how frequently do you use different frames, 
>> 
>> Quite often, I would say.  I usually have two frames but it can go
>> upwards of 5 to 6 if I have a mouse attached to my laptop.
>> 
>> > and how likely are you to have different definitions for the same
>> > faces on different frames in the same Emacs session at the same time?
>> 
>> I don't quite understand this question.  Are you asking if I have any
>> "frame-specific" face attributes i.e., non-nil FRAME argument in
>> set-face-attribute?
>
> Yes.
>
>> If so, no.
>
> OK, so one more theory eats dust (we don't record the frame in the
> composition cache).
>
>> > The only way I see to investigate this is to wait for this to happen,
>> > then attach GDB to Emacs and look at the problematic compositions in
>> > the cache, comparing them to the corresponding compositions in a fresh
>> > Emacs session.  I can tell what to look for with GDB, if that helps.
>> 
>> That would help.  But given how hard it is to reproduce this issue on my
>> end, I don't know when I can get back...
>
> It would not be useful for me to give instructions before you actually
> hit the problem (because the code will change until then), so if you
> want to try this, get back to me when you do reproduce the problem
> (and then attach GDB and leave the Emacs session under GDB for any
> investigations I'd ask you to do).

I seem to have run into the issue.  The attached images
"cascadia-code-bold-15-good" and "-bad.png" are the desired and
misaligned composite text of "-->" rendered in Cascadia Code bold 15
font.  The same text is composed fine with Cascadia Code bold 17.


[-- Attachment #2: cascadia-bold-15-good.png --]
[-- Type: image/png, Size: 636 bytes --]

[-- Attachment #3: cascadia-bold-15-bad.png --]
[-- Type: image/png, Size: 632 bytes --]

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Visuwesh]

[செவ்வாய் அக்டோபர் 29, 2024] Visuwesh wrote:

Just to make myself clear, the "good" image is from a fresh emacs -Q
session.  I still have the "bad" one open.

> [செவ்வாய் அக்டோபர் 29, 2024] Eli Zaretskii wrote:
>
>>> From: Visuwesh <visuweshm@gmail.com>
>>> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
>>> Date: Tue, 29 Oct 2024 16:29:48 +0530
>>> 
>>> >> Is this hash dependent on the font driver?
>>> >
>>> > No.  Only the font used for the composed characters is recorded, not
>>> > the font backend which opened it.  But fonts are managed by the font
>>> > backend, so maybe there's some leakage by that way.
>>> 
>>> OK, thanks.  I wonder if we could compare the value returned by
>>> font-info if something has gone wrong with the font object used to
>>> compute the hash for the glyph?  
>>
>> That would not be the first thing I'd look at.  According to the
>> screenshots, it is more likely that a wrong cache entry is used for a
>> composition, which uses the "wrong" font variant.  IOW, the font used
>> itself is fine, it just is not the font that's supposed to be used
>> with the composed characters in that place.
>>
>> So I would first look at the font object stored in the header of the
>> cached composition.
>>
>>> > Btw, how frequently do you use different frames, 
>>> 
>>> Quite often, I would say.  I usually have two frames but it can go
>>> upwards of 5 to 6 if I have a mouse attached to my laptop.
>>> 
>>> > and how likely are you to have different definitions for the same
>>> > faces on different frames in the same Emacs session at the same time?
>>> 
>>> I don't quite understand this question.  Are you asking if I have any
>>> "frame-specific" face attributes i.e., non-nil FRAME argument in
>>> set-face-attribute?
>>
>> Yes.
>>
>>> If so, no.
>>
>> OK, so one more theory eats dust (we don't record the frame in the
>> composition cache).
>>
>>> > The only way I see to investigate this is to wait for this to happen,
>>> > then attach GDB to Emacs and look at the problematic compositions in
>>> > the cache, comparing them to the corresponding compositions in a fresh
>>> > Emacs session.  I can tell what to look for with GDB, if that helps.
>>> 
>>> That would help.  But given how hard it is to reproduce this issue on my
>>> end, I don't know when I can get back...
>>
>> It would not be useful for me to give instructions before you actually
>> hit the problem (because the code will change until then), so if you
>> want to try this, get back to me when you do reproduce the problem
>> (and then attach GDB and leave the Emacs session under GDB for any
>> investigations I'd ask you to do).
>
> I seem to have run into the issue.  The attached images
> "cascadia-code-bold-15-good" and "-bad.png" are the desired and
> misaligned composite text of "-->" rendered in Cascadia Code bold 15
> font.  The same text is composed fine with Cascadia Code bold 17.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Visuwesh <visuweshm@gmail.com>
> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
> Date: Tue, 29 Oct 2024 19:24:26 +0530
> 
> I seem to have run into the issue.  The attached images
> "cascadia-code-bold-15-good" and "-bad.png" are the desired and
> misaligned composite text of "-->" rendered in Cascadia Code bold 15
> font.  The same text is composed fine with Cascadia Code bold 17.

I'm not sure this is the same issue we are talking about, but here are
the instructions anyway:

 . Move the cursor to where this ligature is displayed
 . Use "C-x =" to display the buffer position of the ligature
 . Attaching GDB to Emacs, then type these commands:

    (gdb) source /path/to/emacs/src/.gdbinit
    (gdb) thread 1
    (gdb) break set_cursor_from_row
    (gdb) continue

The breakpoint in set_cursor_from_row will break soon enough.  The
first time it breaks, just type "continue".  Second time it breaks,
type:

    (gdb) pgrow

This will show you the entire screen line (a.k.a. "glyph row") where
the cursor is displayed, which is also the screen line of the
ligature.  Here's how a similar line with ligatures looks here:

  (gdb) pgrow
  TEXT: 80 glyphs
    0    0: CHAR[ ] pos=857 blev=0,btyp=L w=9 a+d=17+4 MB
    1    9: CHAR[ ] pos=858 blev=0,btyp=L w=9 a+d=17+4 MB
    2   18: CHAR[ ] pos=859 blev=0,btyp=L w=9 a+d=17+4 MB
    3   27: CHAR[ ] pos=860 blev=0,btyp=L w=9 a+d=17+4 MB
    4   36: CHAR[ ] pos=861 blev=0,btyp=L w=9 a+d=17+4 MB
    5   45: CHAR[ ] pos=862 blev=0,btyp=L w=9 a+d=17+4 MB
    6   54: CHAR[ ] pos=863 blev=0,btyp=L w=9 a+d=17+4 MB
    7   63: CHAR[ ] pos=864 blev=0,btyp=L w=9 a+d=17+4 MB
    8   72: CHAR[ ] pos=865 blev=0,btyp=L w=9 a+d=17+4 MB
    9   81: CHAR[ ] pos=866 blev=0,btyp=L w=9 a+d=17+4 MB
   10   90: CHAR[ ] pos=867 blev=0,btyp=L w=9 a+d=17+4 MB
   11   99: CHAR[ ] pos=868 blev=0,btyp=L w=9 a+d=17+4 MB
   12  108: CHAR[ ] pos=869 blev=0,btyp=L w=9 a+d=17+4 MB
   13  117: CHAR[ ] pos=870 blev=0,btyp=L w=9 a+d=17+4 MB
   14  126: CHAR[ ] pos=871 blev=0,btyp=L w=9 a+d=17+4 MB
   15  135: CHAR[ ] pos=872 blev=0,btyp=L w=9 a+d=17+4 MB
   16  144: CHAR[ ] pos=873 blev=0,btyp=L w=9 a+d=17+4 MB
   17  153: CHAR[ ] pos=874 blev=0,btyp=L w=9 a+d=17+4 MB
   18  162: CHAR[ ] pos=875 blev=0,btyp=L w=9 a+d=17+4 MB
   19  171: CHAR[ ] pos=876 blev=0,btyp=L w=9 a+d=17+4 MB
   20  180: CHAR[ ] pos=877 blev=0,btyp=L w=9 a+d=17+4 MB
   21  189: CHAR[ ] pos=878 blev=0,btyp=L w=9 a+d=17+4 MB
   22  198: CHAR[ ] pos=879 blev=0,btyp=L w=9 a+d=17+4 MB
   23  207: CHAR[ ] pos=880 blev=0,btyp=L w=9 a+d=17+4 MB
   24  216: CHAR[ ] pos=881 blev=0,btyp=L w=9 a+d=17+4 MB
   25  225: CHAR[ ] pos=882 blev=0,btyp=L w=9 a+d=17+4 MB
   26  234: CHAR[ ] pos=883 blev=0,btyp=L w=9 a+d=17+4 MB
   27  243: CHAR[ ] pos=884 blev=0,btyp=L w=9 a+d=17+4 MB
   28  252: CHAR[ ] pos=885 blev=0,btyp=L w=9 a+d=17+4 MB
   29  261: CHAR[ ] pos=886 blev=0,btyp=L w=9 a+d=17+4 MB
   30  270: CHAR["] pos=887 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   31  279: COMP[69 (0..0)] pos=888 w=9 a+d=17+4 face=24 MB
   32  288: COMP[69 (1..1)] pos=889 w=9 a+d=17+4 face=24 MB
   33  297: COMP[69 (2..2)] pos=890 w=9 a+d=17+4 face=24 MB
   34  306: CHAR["] pos=891 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   35  315: CHAR[ ] pos=892 blev=0,btyp=L w=9 a+d=17+4 MB
   36  324: CHAR["] pos=893 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   37  333: COMP[70 (0..0)] pos=894 w=9 a+d=17+4 face=24 MB
   38  342: COMP[70 (1..1)] pos=895 w=9 a+d=17+4 face=24 MB
   39  351: CHAR["] pos=896 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   40  360: CHAR[ ] pos=897 blev=0,btyp=L w=9 a+d=17+4 MB
   41  369: CHAR["] pos=898 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   42  378: COMP[71 (0..0)] pos=899 w=9 a+d=17+4 face=24 MB
   43  387: COMP[71 (1..1)] pos=900 w=9 a+d=17+4 face=24 MB
   44  396: COMP[71 (2..2)] pos=901 w=9 a+d=17+4 face=24 MB
   45  405: CHAR["] pos=902 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   46  414: CHAR[ ] pos=903 blev=0,btyp=L w=9 a+d=17+4 MB
   47  423: CHAR["] pos=904 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   48  432: COMP[72 (0..0)] pos=905 w=9 a+d=17+4 face=24 MB
   49  441: COMP[72 (1..1)] pos=906 w=9 a+d=17+4 face=24 MB
   50  450: COMP[72 (2..2)] pos=907 w=9 a+d=17+4 face=24 MB
   51  459: CHAR["] pos=908 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   52  468: CHAR[ ] pos=909 blev=0,btyp=L w=9 a+d=17+4 MB
   53  477: CHAR["] pos=910 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   54  486: COMP[73 (0..0)] pos=911 w=9 a+d=17+4 face=24 MB
   55  495: COMP[73 (1..1)] pos=912 w=9 a+d=17+4 face=24 MB
   56  504: CHAR["] pos=913 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   57  513: CHAR[ ] pos=914 blev=0,btyp=L w=9 a+d=17+4 MB
   58  522: CHAR["] pos=915 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   59  531: COMP[74 (0..0)] pos=916 w=9 a+d=17+4 face=24 MB
   60  540: COMP[74 (1..1)] pos=917 w=9 a+d=17+4 face=24 MB
   61  549: COMP[74 (2..2)] pos=918 w=9 a+d=17+4 face=24 MB
   62  558: CHAR["] pos=919 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   63  567: CHAR[ ] pos=920 blev=0,btyp=L w=9 a+d=17+4 MB
   64  576: CHAR["] pos=921 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   65  585: COMP[75 (0..0)] pos=922 w=9 a+d=17+4 face=24 MB
   66  594: COMP[75 (1..1)] pos=923 w=9 a+d=17+4 face=24 MB
   67  603: COMP[75 (2..2)] pos=924 w=9 a+d=17+4 face=24 MB
   68  612: CHAR["] pos=925 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   69  621: CHAR[ ] pos=926 blev=0,btyp=L w=9 a+d=17+4 MB
   70  630: CHAR["] pos=927 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   71  639: COMP[76 (0..0)] pos=928 w=9 a+d=17+4 face=24 MB
   72  648: COMP[76 (1..1)] pos=929 w=9 a+d=17+4 face=24 MB
   73  657: COMP[76 (2..2)] pos=930 w=9 a+d=17+4 face=24 MB
   74  666: CHAR["] pos=931 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   75  675: CHAR[ ] pos=932 blev=0,btyp=L w=9 a+d=17+4 MB
   76  684: CHAR["] pos=933 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
   77  693: COMP[77 (0..0)] pos=934 w=9 a+d=17+4 face=24 MB
   78  702: COMP[77 (1..1)] pos=935 w=9 a+d=17+4 face=24 MB
   79  711: COMP[77 (2..2)] pos=936 w=9 a+d=17+4 face=24 MB

Each line here describes a glyph on display.  Where it says "CHAR[X]",
that's a character glyph of character X.  Where it says
"COMP[n (i..j)]", that's a composition whose cached ID is n, and i and
j are the characters in the composed sequence represented by this
glyph.

The "pos=NNNN" part is the buffer position from where each glyph came.

Find the cache ID of the composition which shows the problematic
ligature by its buffer position which you displayed at the beginning.
Let's assume that the ID of that composition is 69 (from the glyph
row shown above).  Then type:

   (gdb) pp composition_gstring_from_id(69)

This will show the composition cached at slot 116.  The structure of
the composition is described in the doc string of
composition-get-gstring.  Here's what I get here for the ligature
cached at slot 69:

    (gdb) pp composition_gstring_from_id(69)
    [[#<font-object "-outline-Cascadia Code-regular-normal-normal-mono-16-*-*-*-c-*-iso8859-1"> 45 45 62] 69 [0 0 45 1970 9 1 10 17 4 nil] [1 1 45 1969 9 0 10 17 4 nil] [2 2 62 2728 9 0 9 17 4 nil]]

It clearly shows the font used to display the 3 glyphs of this
ligature and the data of the 3 glyphs themselves.

The idea is then to compare what you get from the "bad" display with
what you get for the same ligature in a fresh Emacs session, where the
display should be good.

Let me know if you need more help or more detailed instructions or
have questions.

Thanks.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Visuwesh]

[செவ்வாய் அக்டோபர் 29, 2024] Eli Zaretskii wrote:

>> From: Visuwesh <visuweshm@gmail.com>
>> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
>> Date: Tue, 29 Oct 2024 19:24:26 +0530
>> 
>> I seem to have run into the issue.  The attached images
>> "cascadia-code-bold-15-good" and "-bad.png" are the desired and
>> misaligned composite text of "-->" rendered in Cascadia Code bold 15
>> font.  The same text is composed fine with Cascadia Code bold 17.
>
> I'm not sure this is the same issue we are talking about, but here are
> the instructions anyway:
>
>  . Move the cursor to where this ligature is displayed
>  . Use "C-x =" to display the buffer position of the ligature
>  . Attaching GDB to Emacs, then type these commands:
>
>     (gdb) source /path/to/emacs/src/.gdbinit
>     (gdb) thread 1
>     (gdb) break set_cursor_from_row
>     (gdb) continue
>
> The breakpoint in set_cursor_from_row will break soon enough.  The
> first time it breaks, just type "continue".  Second time it breaks,
> type:
>
>     (gdb) pgrow
>
> This will show you the entire screen line (a.k.a. "glyph row") where
> the cursor is displayed, which is also the screen line of the
> ligature.  Here's how a similar line with ligatures looks here:
>
>   (gdb) pgrow
>   TEXT: 80 glyphs
>     0    0: CHAR[ ] pos=857 blev=0,btyp=L w=9 a+d=17+4 MB
>     1    9: CHAR[ ] pos=858 blev=0,btyp=L w=9 a+d=17+4 MB
>     2   18: CHAR[ ] pos=859 blev=0,btyp=L w=9 a+d=17+4 MB
>     3   27: CHAR[ ] pos=860 blev=0,btyp=L w=9 a+d=17+4 MB
>     4   36: CHAR[ ] pos=861 blev=0,btyp=L w=9 a+d=17+4 MB
>     5   45: CHAR[ ] pos=862 blev=0,btyp=L w=9 a+d=17+4 MB
>     6   54: CHAR[ ] pos=863 blev=0,btyp=L w=9 a+d=17+4 MB
>     7   63: CHAR[ ] pos=864 blev=0,btyp=L w=9 a+d=17+4 MB
>     8   72: CHAR[ ] pos=865 blev=0,btyp=L w=9 a+d=17+4 MB
>     9   81: CHAR[ ] pos=866 blev=0,btyp=L w=9 a+d=17+4 MB
>    10   90: CHAR[ ] pos=867 blev=0,btyp=L w=9 a+d=17+4 MB
>    11   99: CHAR[ ] pos=868 blev=0,btyp=L w=9 a+d=17+4 MB
>    12  108: CHAR[ ] pos=869 blev=0,btyp=L w=9 a+d=17+4 MB
>    13  117: CHAR[ ] pos=870 blev=0,btyp=L w=9 a+d=17+4 MB
>    14  126: CHAR[ ] pos=871 blev=0,btyp=L w=9 a+d=17+4 MB
>    15  135: CHAR[ ] pos=872 blev=0,btyp=L w=9 a+d=17+4 MB
>    16  144: CHAR[ ] pos=873 blev=0,btyp=L w=9 a+d=17+4 MB
>    17  153: CHAR[ ] pos=874 blev=0,btyp=L w=9 a+d=17+4 MB
>    18  162: CHAR[ ] pos=875 blev=0,btyp=L w=9 a+d=17+4 MB
>    19  171: CHAR[ ] pos=876 blev=0,btyp=L w=9 a+d=17+4 MB
>    20  180: CHAR[ ] pos=877 blev=0,btyp=L w=9 a+d=17+4 MB
>    21  189: CHAR[ ] pos=878 blev=0,btyp=L w=9 a+d=17+4 MB
>    22  198: CHAR[ ] pos=879 blev=0,btyp=L w=9 a+d=17+4 MB
>    23  207: CHAR[ ] pos=880 blev=0,btyp=L w=9 a+d=17+4 MB
>    24  216: CHAR[ ] pos=881 blev=0,btyp=L w=9 a+d=17+4 MB
>    25  225: CHAR[ ] pos=882 blev=0,btyp=L w=9 a+d=17+4 MB
>    26  234: CHAR[ ] pos=883 blev=0,btyp=L w=9 a+d=17+4 MB
>    27  243: CHAR[ ] pos=884 blev=0,btyp=L w=9 a+d=17+4 MB
>    28  252: CHAR[ ] pos=885 blev=0,btyp=L w=9 a+d=17+4 MB
>    29  261: CHAR[ ] pos=886 blev=0,btyp=L w=9 a+d=17+4 MB
>    30  270: CHAR["] pos=887 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    31  279: COMP[69 (0..0)] pos=888 w=9 a+d=17+4 face=24 MB
>    32  288: COMP[69 (1..1)] pos=889 w=9 a+d=17+4 face=24 MB
>    33  297: COMP[69 (2..2)] pos=890 w=9 a+d=17+4 face=24 MB
>    34  306: CHAR["] pos=891 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    35  315: CHAR[ ] pos=892 blev=0,btyp=L w=9 a+d=17+4 MB
>    36  324: CHAR["] pos=893 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    37  333: COMP[70 (0..0)] pos=894 w=9 a+d=17+4 face=24 MB
>    38  342: COMP[70 (1..1)] pos=895 w=9 a+d=17+4 face=24 MB
>    39  351: CHAR["] pos=896 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    40  360: CHAR[ ] pos=897 blev=0,btyp=L w=9 a+d=17+4 MB
>    41  369: CHAR["] pos=898 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    42  378: COMP[71 (0..0)] pos=899 w=9 a+d=17+4 face=24 MB
>    43  387: COMP[71 (1..1)] pos=900 w=9 a+d=17+4 face=24 MB
>    44  396: COMP[71 (2..2)] pos=901 w=9 a+d=17+4 face=24 MB
>    45  405: CHAR["] pos=902 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    46  414: CHAR[ ] pos=903 blev=0,btyp=L w=9 a+d=17+4 MB
>    47  423: CHAR["] pos=904 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    48  432: COMP[72 (0..0)] pos=905 w=9 a+d=17+4 face=24 MB
>    49  441: COMP[72 (1..1)] pos=906 w=9 a+d=17+4 face=24 MB
>    50  450: COMP[72 (2..2)] pos=907 w=9 a+d=17+4 face=24 MB
>    51  459: CHAR["] pos=908 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    52  468: CHAR[ ] pos=909 blev=0,btyp=L w=9 a+d=17+4 MB
>    53  477: CHAR["] pos=910 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    54  486: COMP[73 (0..0)] pos=911 w=9 a+d=17+4 face=24 MB
>    55  495: COMP[73 (1..1)] pos=912 w=9 a+d=17+4 face=24 MB
>    56  504: CHAR["] pos=913 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    57  513: CHAR[ ] pos=914 blev=0,btyp=L w=9 a+d=17+4 MB
>    58  522: CHAR["] pos=915 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    59  531: COMP[74 (0..0)] pos=916 w=9 a+d=17+4 face=24 MB
>    60  540: COMP[74 (1..1)] pos=917 w=9 a+d=17+4 face=24 MB
>    61  549: COMP[74 (2..2)] pos=918 w=9 a+d=17+4 face=24 MB
>    62  558: CHAR["] pos=919 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    63  567: CHAR[ ] pos=920 blev=0,btyp=L w=9 a+d=17+4 MB
>    64  576: CHAR["] pos=921 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    65  585: COMP[75 (0..0)] pos=922 w=9 a+d=17+4 face=24 MB
>    66  594: COMP[75 (1..1)] pos=923 w=9 a+d=17+4 face=24 MB
>    67  603: COMP[75 (2..2)] pos=924 w=9 a+d=17+4 face=24 MB
>    68  612: CHAR["] pos=925 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    69  621: CHAR[ ] pos=926 blev=0,btyp=L w=9 a+d=17+4 MB
>    70  630: CHAR["] pos=927 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    71  639: COMP[76 (0..0)] pos=928 w=9 a+d=17+4 face=24 MB
>    72  648: COMP[76 (1..1)] pos=929 w=9 a+d=17+4 face=24 MB
>    73  657: COMP[76 (2..2)] pos=930 w=9 a+d=17+4 face=24 MB
>    74  666: CHAR["] pos=931 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    75  675: CHAR[ ] pos=932 blev=0,btyp=L w=9 a+d=17+4 MB
>    76  684: CHAR["] pos=933 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
>    77  693: COMP[77 (0..0)] pos=934 w=9 a+d=17+4 face=24 MB
>    78  702: COMP[77 (1..1)] pos=935 w=9 a+d=17+4 face=24 MB
>    79  711: COMP[77 (2..2)] pos=936 w=9 a+d=17+4 face=24 MB
>
> Each line here describes a glyph on display.  Where it says "CHAR[X]",
> that's a character glyph of character X.  Where it says
> "COMP[n (i..j)]", that's a composition whose cached ID is n, and i and
> j are the characters in the composed sequence represented by this
> glyph.
>
> The "pos=NNNN" part is the buffer position from where each glyph came.
>
> Find the cache ID of the composition which shows the problematic
> ligature by its buffer position which you displayed at the beginning.
> Let's assume that the ID of that composition is 69 (from the glyph
> row shown above).  Then type:
>
>    (gdb) pp composition_gstring_from_id(69)
>
> This will show the composition cached at slot 116.  The structure of
> the composition is described in the doc string of
> composition-get-gstring.  Here's what I get here for the ligature
> cached at slot 69:
>
>     (gdb) pp composition_gstring_from_id(69)
>     [[#<font-object "-outline-Cascadia Code-regular-normal-normal-mono-16-*-*-*-c-*-iso8859-1"> 45 45 62] 69 [0 0 45 1970 9 1 10 17 4 nil] [1 1 45 1969 9 0 10 17 4 nil] [2 2 62 2728 9 0 9 17 4 nil]]
>
> It clearly shows the font used to display the 3 glyphs of this
> ligature and the data of the 3 glyphs themselves.
>
> The idea is then to compare what you get from the "bad" display with
> what you get for the same ligature in a fresh Emacs session, where the
> display should be good.
>
> Let me know if you need more help or more detailed instructions or
> have questions.
>
> Thanks.

Thank you very much for the clear instructions.  I was testing this in a
fresh Emacs session.  And

    (gdb) pp composition_gstring_from_id(ID)

seems to show nothing?

    (gdb) c
    Continuing.

    Thread 1 "emacs" hit Breakpoint 3, set_cursor_from_row (w=0x55b8c994f338, row=0x55b8c9e20410, matrix=0x55b8c9960480, delta=0, delta_bytes=0, dy=0, dvpos=0) at xdisp.c:18217
    18217	  struct glyph *glyph = row->glyphs[TEXT_AREA];
    (gdb) pgrow
    TEXT: 4 glyphs
      0    0: COMP[16 (0..0)] pos=5 w=9 a+d=14+4 face=28 MB
      1    9: COMP[16 (1..1)] pos=6 w=9 a+d=14+4 face=28 MB
      2   18: COMP[16 (2..2)] pos=7 w=9 a+d=14+4 face=28 MB
      3   27: CHAR[ ] pos=0 blev=0,btyp=B w=9 a+d=14+4 MB
    (gdb) pp composition_gstring_from_id(16)
    (gdb) p composition_gstring_from_id(16)
    $1 = XIL(0x55b8cada607d)

Am I missing something?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Cc: luangruo@yahoo.com, 73752@debbugs.gnu.org, xuan@xlk.me
> Date: Tue, 29 Oct 2024 17:38:07 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
>    (gdb) pp composition_gstring_from_id(69)
> 
> This will show the composition cached at slot 116.
                                                ^^^
Sorry, a typo: that should have been 69, of course, not 116.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Visuwesh <visuweshm@gmail.com>
> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
> Date: Tue, 29 Oct 2024 22:16:01 +0530
> 
> [செவ்வாய் அக்டோபர் 29, 2024] Eli Zaretskii wrote:
> 
> >> From: Visuwesh <visuweshm@gmail.com>
> >> Cc: luangruo@yahoo.com,  73752@debbugs.gnu.org,  xuan@xlk.me
> >> Date: Tue, 29 Oct 2024 19:24:26 +0530
> >> 
> >> I seem to have run into the issue.  The attached images
> >> "cascadia-code-bold-15-good" and "-bad.png" are the desired and
> >> misaligned composite text of "-->" rendered in Cascadia Code bold 15
> >> font.  The same text is composed fine with Cascadia Code bold 17.
> >
> > I'm not sure this is the same issue we are talking about, but here are
> > the instructions anyway:
> >
> >  . Move the cursor to where this ligature is displayed
> >  . Use "C-x =" to display the buffer position of the ligature
> >  . Attaching GDB to Emacs, then type these commands:
> >
> >     (gdb) source /path/to/emacs/src/.gdbinit
> >     (gdb) thread 1
> >     (gdb) break set_cursor_from_row
> >     (gdb) continue
> >
> > The breakpoint in set_cursor_from_row will break soon enough.  The
> > first time it breaks, just type "continue".  Second time it breaks,
> > type:
> >
> >     (gdb) pgrow
> >
> > This will show you the entire screen line (a.k.a. "glyph row") where
> > the cursor is displayed, which is also the screen line of the
> > ligature.  Here's how a similar line with ligatures looks here:
> >
> >   (gdb) pgrow
> >   TEXT: 80 glyphs
> >     0    0: CHAR[ ] pos=857 blev=0,btyp=L w=9 a+d=17+4 MB
> >     1    9: CHAR[ ] pos=858 blev=0,btyp=L w=9 a+d=17+4 MB
> >     2   18: CHAR[ ] pos=859 blev=0,btyp=L w=9 a+d=17+4 MB
> >     3   27: CHAR[ ] pos=860 blev=0,btyp=L w=9 a+d=17+4 MB
> >     4   36: CHAR[ ] pos=861 blev=0,btyp=L w=9 a+d=17+4 MB
> >     5   45: CHAR[ ] pos=862 blev=0,btyp=L w=9 a+d=17+4 MB
> >     6   54: CHAR[ ] pos=863 blev=0,btyp=L w=9 a+d=17+4 MB
> >     7   63: CHAR[ ] pos=864 blev=0,btyp=L w=9 a+d=17+4 MB
> >     8   72: CHAR[ ] pos=865 blev=0,btyp=L w=9 a+d=17+4 MB
> >     9   81: CHAR[ ] pos=866 blev=0,btyp=L w=9 a+d=17+4 MB
> >    10   90: CHAR[ ] pos=867 blev=0,btyp=L w=9 a+d=17+4 MB
> >    11   99: CHAR[ ] pos=868 blev=0,btyp=L w=9 a+d=17+4 MB
> >    12  108: CHAR[ ] pos=869 blev=0,btyp=L w=9 a+d=17+4 MB
> >    13  117: CHAR[ ] pos=870 blev=0,btyp=L w=9 a+d=17+4 MB
> >    14  126: CHAR[ ] pos=871 blev=0,btyp=L w=9 a+d=17+4 MB
> >    15  135: CHAR[ ] pos=872 blev=0,btyp=L w=9 a+d=17+4 MB
> >    16  144: CHAR[ ] pos=873 blev=0,btyp=L w=9 a+d=17+4 MB
> >    17  153: CHAR[ ] pos=874 blev=0,btyp=L w=9 a+d=17+4 MB
> >    18  162: CHAR[ ] pos=875 blev=0,btyp=L w=9 a+d=17+4 MB
> >    19  171: CHAR[ ] pos=876 blev=0,btyp=L w=9 a+d=17+4 MB
> >    20  180: CHAR[ ] pos=877 blev=0,btyp=L w=9 a+d=17+4 MB
> >    21  189: CHAR[ ] pos=878 blev=0,btyp=L w=9 a+d=17+4 MB
> >    22  198: CHAR[ ] pos=879 blev=0,btyp=L w=9 a+d=17+4 MB
> >    23  207: CHAR[ ] pos=880 blev=0,btyp=L w=9 a+d=17+4 MB
> >    24  216: CHAR[ ] pos=881 blev=0,btyp=L w=9 a+d=17+4 MB
> >    25  225: CHAR[ ] pos=882 blev=0,btyp=L w=9 a+d=17+4 MB
> >    26  234: CHAR[ ] pos=883 blev=0,btyp=L w=9 a+d=17+4 MB
> >    27  243: CHAR[ ] pos=884 blev=0,btyp=L w=9 a+d=17+4 MB
> >    28  252: CHAR[ ] pos=885 blev=0,btyp=L w=9 a+d=17+4 MB
> >    29  261: CHAR[ ] pos=886 blev=0,btyp=L w=9 a+d=17+4 MB
> >    30  270: CHAR["] pos=887 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    31  279: COMP[69 (0..0)] pos=888 w=9 a+d=17+4 face=24 MB
> >    32  288: COMP[69 (1..1)] pos=889 w=9 a+d=17+4 face=24 MB
> >    33  297: COMP[69 (2..2)] pos=890 w=9 a+d=17+4 face=24 MB
> >    34  306: CHAR["] pos=891 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    35  315: CHAR[ ] pos=892 blev=0,btyp=L w=9 a+d=17+4 MB
> >    36  324: CHAR["] pos=893 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    37  333: COMP[70 (0..0)] pos=894 w=9 a+d=17+4 face=24 MB
> >    38  342: COMP[70 (1..1)] pos=895 w=9 a+d=17+4 face=24 MB
> >    39  351: CHAR["] pos=896 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    40  360: CHAR[ ] pos=897 blev=0,btyp=L w=9 a+d=17+4 MB
> >    41  369: CHAR["] pos=898 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    42  378: COMP[71 (0..0)] pos=899 w=9 a+d=17+4 face=24 MB
> >    43  387: COMP[71 (1..1)] pos=900 w=9 a+d=17+4 face=24 MB
> >    44  396: COMP[71 (2..2)] pos=901 w=9 a+d=17+4 face=24 MB
> >    45  405: CHAR["] pos=902 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    46  414: CHAR[ ] pos=903 blev=0,btyp=L w=9 a+d=17+4 MB
> >    47  423: CHAR["] pos=904 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    48  432: COMP[72 (0..0)] pos=905 w=9 a+d=17+4 face=24 MB
> >    49  441: COMP[72 (1..1)] pos=906 w=9 a+d=17+4 face=24 MB
> >    50  450: COMP[72 (2..2)] pos=907 w=9 a+d=17+4 face=24 MB
> >    51  459: CHAR["] pos=908 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    52  468: CHAR[ ] pos=909 blev=0,btyp=L w=9 a+d=17+4 MB
> >    53  477: CHAR["] pos=910 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    54  486: COMP[73 (0..0)] pos=911 w=9 a+d=17+4 face=24 MB
> >    55  495: COMP[73 (1..1)] pos=912 w=9 a+d=17+4 face=24 MB
> >    56  504: CHAR["] pos=913 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    57  513: CHAR[ ] pos=914 blev=0,btyp=L w=9 a+d=17+4 MB
> >    58  522: CHAR["] pos=915 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    59  531: COMP[74 (0..0)] pos=916 w=9 a+d=17+4 face=24 MB
> >    60  540: COMP[74 (1..1)] pos=917 w=9 a+d=17+4 face=24 MB
> >    61  549: COMP[74 (2..2)] pos=918 w=9 a+d=17+4 face=24 MB
> >    62  558: CHAR["] pos=919 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    63  567: CHAR[ ] pos=920 blev=0,btyp=L w=9 a+d=17+4 MB
> >    64  576: CHAR["] pos=921 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    65  585: COMP[75 (0..0)] pos=922 w=9 a+d=17+4 face=24 MB
> >    66  594: COMP[75 (1..1)] pos=923 w=9 a+d=17+4 face=24 MB
> >    67  603: COMP[75 (2..2)] pos=924 w=9 a+d=17+4 face=24 MB
> >    68  612: CHAR["] pos=925 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    69  621: CHAR[ ] pos=926 blev=0,btyp=L w=9 a+d=17+4 MB
> >    70  630: CHAR["] pos=927 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    71  639: COMP[76 (0..0)] pos=928 w=9 a+d=17+4 face=24 MB
> >    72  648: COMP[76 (1..1)] pos=929 w=9 a+d=17+4 face=24 MB
> >    73  657: COMP[76 (2..2)] pos=930 w=9 a+d=17+4 face=24 MB
> >    74  666: CHAR["] pos=931 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    75  675: CHAR[ ] pos=932 blev=0,btyp=L w=9 a+d=17+4 MB
> >    76  684: CHAR["] pos=933 blev=0,btyp=L w=9 a+d=17+4 face=23 MB
> >    77  693: COMP[77 (0..0)] pos=934 w=9 a+d=17+4 face=24 MB
> >    78  702: COMP[77 (1..1)] pos=935 w=9 a+d=17+4 face=24 MB
> >    79  711: COMP[77 (2..2)] pos=936 w=9 a+d=17+4 face=24 MB
> >
> > Each line here describes a glyph on display.  Where it says "CHAR[X]",
> > that's a character glyph of character X.  Where it says
> > "COMP[n (i..j)]", that's a composition whose cached ID is n, and i and
> > j are the characters in the composed sequence represented by this
> > glyph.
> >
> > The "pos=NNNN" part is the buffer position from where each glyph came.
> >
> > Find the cache ID of the composition which shows the problematic
> > ligature by its buffer position which you displayed at the beginning.
> > Let's assume that the ID of that composition is 69 (from the glyph
> > row shown above).  Then type:
> >
> >    (gdb) pp composition_gstring_from_id(69)
> >
> > This will show the composition cached at slot 116.  The structure of
> > the composition is described in the doc string of
> > composition-get-gstring.  Here's what I get here for the ligature
> > cached at slot 69:
> >
> >     (gdb) pp composition_gstring_from_id(69)
> >     [[#<font-object "-outline-Cascadia Code-regular-normal-normal-mono-16-*-*-*-c-*-iso8859-1"> 45 45 62] 69 [0 0 45 1970 9 1 10 17 4 nil] [1 1 45 1969 9 0 10 17 4 nil] [2 2 62 2728 9 0 9 17 4 nil]]
> >
> > It clearly shows the font used to display the 3 glyphs of this
> > ligature and the data of the 3 glyphs themselves.
> >
> > The idea is then to compare what you get from the "bad" display with
> > what you get for the same ligature in a fresh Emacs session, where the
> > display should be good.
> >
> > Let me know if you need more help or more detailed instructions or
> > have questions.
> >
> > Thanks.
> 
> Thank you very much for the clear instructions.  I was testing this in a
> fresh Emacs session.  And
> 
>     (gdb) pp composition_gstring_from_id(ID)
> 
> seems to show nothing?
> 
>     (gdb) c
>     Continuing.
> 
>     Thread 1 "emacs" hit Breakpoint 3, set_cursor_from_row (w=0x55b8c994f338, row=0x55b8c9e20410, matrix=0x55b8c9960480, delta=0, delta_bytes=0, dy=0, dvpos=0) at xdisp.c:18217
>     18217	  struct glyph *glyph = row->glyphs[TEXT_AREA];
>     (gdb) pgrow
>     TEXT: 4 glyphs
>       0    0: COMP[16 (0..0)] pos=5 w=9 a+d=14+4 face=28 MB
>       1    9: COMP[16 (1..1)] pos=6 w=9 a+d=14+4 face=28 MB
>       2   18: COMP[16 (2..2)] pos=7 w=9 a+d=14+4 face=28 MB
>       3   27: CHAR[ ] pos=0 blev=0,btyp=B w=9 a+d=14+4 MB
>     (gdb) pp composition_gstring_from_id(16)
>     (gdb) p composition_gstring_from_id(16)
>     $1 = XIL(0x55b8cada607d)
> 
> Am I missing something?

Maybe your Emacs is not built with enough debug info?

What does this produce:

  (gdb) p composition_gstring_from_id(16)
  $1 = XIL(0x55b8cada607d)
  (gdb) xpr

If this doesn't work, either, you will have to decypher
XIL(0x55b8cada607d) "by hand", using xvector etc...  I can give you
instructions for that as well.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Tim Ruffing]

I regularly experience yet another "variant" of this bug. Of course, I
can't be sure that is has the same cause but except for the exact
distortion in the rendering, the circumstances look pretty similar. 

I've seen the bug for a while, and I did some experiments in the past,
but then I got distracted and I didn't find the time to report it
properly. In fact, I see this quite often, and the xuan's test script
makes this very easy to reproduce for me. 

Here's all the meaningful information I could gather in the past. I had
already posted much of this in the mentioned GitHub issue, but let me
repeat here. Also, here a video that demonstrates most the following
items:
https://github.com/user-attachments/assets/56b15dfc-6467-4785-bf1f-f2cd05f33f04
This was recorded on an old Emacs version (should be 29.1, but I can't
vouch for it), but the bug has existed much longer, and I still see it
on my currently installed Emacs 31.0.50 (commit 174784).

 * When I experience that bug, it's not only that there's additional
   spacing at the end of the ligature, but I see this with additional
   spacing after every partial "character" of the ligature (e.g., the
   "===" ligature is rendered rather like "= = = "). For some
   ligatures, this means that individual components are also misaligned
   with respect to each other. For example, the "/" in the "!=" (not
   equal) ligature, which is supposed to strike through the "==" is
   vertically off. (You can see this exact problem in the right half of
   line 58 in the video.)
 * This can happen with all kinds of font changes. So the issue appears
   to be specific to a what I call I "realized face" with selected
   slope, weight, etc. (no idea what the proper Emacs term is).
   Ligatures may be broken in upright but not in italics, or ligatures
   may be broken in bold but not in normal. Once the face is broken, it
   stays broken. 
 * In the video, one can see that searching for "=" fixes the broken
   rendering of the "===" ligature in line 52. Contrary to the previous
   item, I believe this is *not* due to change in background color, but
   due to Emacs trying to highlight the individual "=" char. Still,
   it's interesting to see that this fixes the rendering. (In fact,
   changing the colors doesn't seem to influence the problem. I suspect
   that just the shapes are cached but not the colors?)
 * This one may be a nice for debugging: I use GNOME, and changing the
   font rendering settings triggers a "re-realization" of the fonts,
   and this will then fix the issue (again, see the video). One may be
   tempted to think that this should be a possibility to trigger the
   bug, but I could never observe this. It always fixed the problem for
   me, but maybe I was just "lucky".
 * There doesn't seem to be a way to reproduce this deterministically.
   I made multiple attempts to find a minimal reproducible config and
   fooled myself every time. I don't think this is an elisp config
   thing, I believe it can happen whenever there are ligatures (at
   least on affected systems). I've also tried multiple fonts (Iosevka,
   JetBrains Mono, Fira Code...). The video uses Iosevka.
 * Rebuilding the fontconfig cache with `fc-cache -f` or even `-r`
   doesn't do anything. `(clear-font-cache)` doesn't do anything (see
   video below).
 * Now that I saw this thread, I've also checked (clear-font-cache t)
   and (clear-composition-thread). These don't help.
 * If it's broken in one frame, then it's broken also in other frames. 
 * Notably, I also use the pgtk build on wayland. 
 * I have found this on the web, which seems yet another variant (?) of
   the bug:
   https://web.archive.org/web/20230226082659/https://www.reddit.com/r/emacs/comments/11c957r/ligatures_character_overlap_and_extreme_squashing/

I've just managed to reproduce this again, and I've attached gdb, but
I'm also stuck at pp doesn't displaying:
(gdb) pp composition_gstring_from_id(19)
(gdb) p composition_gstring_from_id(19)
$6 = (struct Lisp_X *) 0x58ad4d938225
(gdb) xpr
Lisp_Vectorlike
PVEC_NORMAL_VECTOR
$7 = (struct Lisp_Vector *) 0x58ad4d938220
{XIL(0x58ad4f32aabd), make_fixnum(19), XIL(0x58ad4d93f8e5),
XIL(0x58ad4d93f93d), XIL(0x58ad4d93f995)}

I still have the session open. Also happy to rebuild with more debug
info enabled, if you tell me how that works.

```
3.24.43, cairo version 1.18.2) of 2024-09-13 built on tonno.fritz.box
Repository revision: 04e8ad6489ebec121ace7ea6d582429a96af8f04
Repository branch: makepkg
System Description: Arch Linux

Configured using:
'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
--localstatedir=/var --mandir=/usr/share/man --with-gameuser=:games
--with-modules --without-m17n-flt --without-gconf
--with-native-compilation=yes --with-xinput2 --with-pgtk
--without-xaw3d --with-sound=no --with-tree-sitter --without-gpm
--without-compress-install
'--program-transform-name=s/\([ec]tags\)/\1.emacs/'
'CFLAGS=-march=native -O2 -pipe -fno-plt -fexceptions
-Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security
-fstack-clash-protection -fcf-protection -fno-omit-frame-pointer
-mno-omit-leaf-frame-pointer -g
-ffile-prefix-map=/home/tim/.cache/pikaur/build/emacs-
git/src=/usr/src/debug/emacs-git
-flto=auto' 'LDFLAGS=-Wl,-O1 -Wl,--sort-common -Wl,--as-needed
-Wl,-z,relro -Wl,-z,now -Wl,-z,pack-relative-relocs -flto=auto'
'CXXFLAGS=-march=native -O2 -pipe -fno-plt -fexceptions
-Wp,-D_FORTIFY_SOURCE=3 -Wformat -Werror=format-security
-fstack-clash-protection -fcf-protection -fno-omit-frame-pointer
-mno-omit-leaf-frame-pointer -Wp,-D_GLIBCXX_ASSERTIONS -g
-ffile-prefix-map=/home/tim/.cache/pikaur/build/emacs-
git/src=/usr/src/debug/emacs-git
-flto=auto''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG
LCMS2 LIBOTF LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY
PDUMPER PGTK PNG RSVG SECCOMP SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER WEBP XIM GTK3 ZLIB
```

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Visuwesh]

[செவ்வாய் அக்டோபர் 29, 2024] Eli Zaretskii wrote:

>> Thank you very much for the clear instructions.  I was testing this in a
>> fresh Emacs session.  And
>> 
>>     (gdb) pp composition_gstring_from_id(ID)
>> 
>> seems to show nothing?
>> 
>>     (gdb) c
>>     Continuing.
>> 
>>     Thread 1 "emacs" hit Breakpoint 3, set_cursor_from_row (w=0x55b8c994f338, row=0x55b8c9e20410, matrix=0x55b8c9960480, delta=0, delta_bytes=0, dy=0, dvpos=0) at xdisp.c:18217
>>     18217	  struct glyph *glyph = row->glyphs[TEXT_AREA];
>>     (gdb) pgrow
>>     TEXT: 4 glyphs
>>       0    0: COMP[16 (0..0)] pos=5 w=9 a+d=14+4 face=28 MB
>>       1    9: COMP[16 (1..1)] pos=6 w=9 a+d=14+4 face=28 MB
>>       2   18: COMP[16 (2..2)] pos=7 w=9 a+d=14+4 face=28 MB
>>       3   27: CHAR[ ] pos=0 blev=0,btyp=B w=9 a+d=14+4 MB
>>     (gdb) pp composition_gstring_from_id(16)
>>     (gdb) p composition_gstring_from_id(16)
>>     $1 = XIL(0x55b8cada607d)
>> 
>> Am I missing something?
>
> Maybe your Emacs is not built with enough debug info?

I built a debug version of Emacs specifically for this.

    ./configure --with-sound=alsa --with-x-toolkit=lucid --without-xaw3d \
                --without-gconf --without-libsystemd --with-cairo \
                --enable-checking='yes,glyphs' \
                --enable-check-lisp-object-type CFLAGS='-O0 -g3'

> What does this produce:
>
>   (gdb) p composition_gstring_from_id(16)
>   $1 = XIL(0x55b8cada607d)
>   (gdb) xpr
>
> If this doesn't work, either, you will have to decypher
> XIL(0x55b8cada607d) "by hand", using xvector etc...  I can give you
> instructions for that as well.

    (gdb) p composition_gstring_from_id(16)
    $3 = XIL(0x55b8cada607d)
    (gdb) xpr
    Lisp_Vectorlike
    PVEC_NORMAL_VECTOR
    $4 = (struct Lisp_Vector *) 0x55b8cada6078
    {XIL(0x55b8cada60ad), make_fixnum(16), XIL(0x55b8cada60d5), XIL(0x55b8cada612d), XIL(0x55b8cada6185)}

So I am guessing we do need to decypher it "by hand"?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Tim Ruffing <dev@real-or-random.org>
> Date: Wed, 30 Oct 2024 00:14:58 +0100
> 
> I've just managed to reproduce this again, and I've attached gdb, but
> I'm also stuck at pp doesn't displaying:
> (gdb) pp composition_gstring_from_id(19)
> (gdb) p composition_gstring_from_id(19)
> $6 = (struct Lisp_X *) 0x58ad4d938225
> (gdb) xpr
> Lisp_Vectorlike
> PVEC_NORMAL_VECTOR
> $7 = (struct Lisp_Vector *) 0x58ad4d938220
> {XIL(0x58ad4f32aabd), make_fixnum(19), XIL(0x58ad4d93f8e5),
> XIL(0x58ad4d93f93d), XIL(0x58ad4d93f995)}

Paul, any ideas why pp would fail to work?  I never say anything like
that, but maybe external-debugging-output doesn't work when GDB is
attached to a running Emacs?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Cc: 73752@debbugs.gnu.org
> Date: Wed, 30 Oct 2024 17:12:33 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > From: Tim Ruffing <dev@real-or-random.org>
> > Date: Wed, 30 Oct 2024 00:14:58 +0100
> > 
> > I've just managed to reproduce this again, and I've attached gdb, but
> > I'm also stuck at pp doesn't displaying:
> > (gdb) pp composition_gstring_from_id(19)
> > (gdb) p composition_gstring_from_id(19)
> > $6 = (struct Lisp_X *) 0x58ad4d938225
> > (gdb) xpr
> > Lisp_Vectorlike
> > PVEC_NORMAL_VECTOR
> > $7 = (struct Lisp_Vector *) 0x58ad4d938220
> > {XIL(0x58ad4f32aabd), make_fixnum(19), XIL(0x58ad4d93f8e5),
> > XIL(0x58ad4d93f93d), XIL(0x58ad4d93f995)}
> 
> Paul, any ideas why pp would fail to work?  I never say anything like
                                                      ^^^
"saw"

> that, but maybe external-debugging-output doesn't work when GDB is
> attached to a running Emacs?
> 
> 
> 
> 

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Andreas Schwab <schwab@suse.de>
> Cc: Tim Ruffing <dev@real-or-random.org>,  Paul Eggert <eggert@cs.ucla.edu>,
>   73752@debbugs.gnu.org
> Date: Wed, 30 Oct 2024 16:22:29 +0100
> 
> On Okt 30 2024, Eli Zaretskii wrote:
> 
> >> From: Tim Ruffing <dev@real-or-random.org>
> >> Date: Wed, 30 Oct 2024 00:14:58 +0100
> >> 
> >> I've just managed to reproduce this again, and I've attached gdb, but
> >> I'm also stuck at pp doesn't displaying:
> >> (gdb) pp composition_gstring_from_id(19)
> >> (gdb) p composition_gstring_from_id(19)
> >> $6 = (struct Lisp_X *) 0x58ad4d938225
> >> (gdb) xpr
> >> Lisp_Vectorlike
> >> PVEC_NORMAL_VECTOR
> >> $7 = (struct Lisp_Vector *) 0x58ad4d938220
> >> {XIL(0x58ad4f32aabd), make_fixnum(19), XIL(0x58ad4d93f8e5),
> >> XIL(0x58ad4d93f93d), XIL(0x58ad4d93f995)}
> >
> > Paul, any ideas why pp would fail to work?  I never say anything like
> > that, but maybe external-debugging-output doesn't work when GDB is
> > attached to a running Emacs?
> 
> pp is calling Emacs to print to its stderr.  If that is redirected
> somewhere else you won't see the output.

That's what I suspected, thanks.  So there's no way to use pp when GDB
is attached to an Emacs that was not started from a shell prompt?  Or
can the user do something to make that output show somewhere?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Tim Ruffing]

> > pp is calling Emacs to print to its stderr.  If that is redirected
> > somewhere else you won't see the output.
> 

Ah, this was the right hint. I'm using emacs in daemon mode, started
from systemd, so I can inspect stderr via journalctl.

------

Broken rendering for ligature "===" starting at pos 1290 (emacs happens
to be in daemon mode)

[...]
 39  480: COMP[19 (0..0)] pos=1290 w=20 a+d=20+6 face=39 MB
 40  500: COMP[19 (1..1)] pos=1291 w=20 a+d=20+6 face=39 MB
 41  520: COMP[19 (2..2)] pos=1292 w=20 a+d=20+6 face=39 MB
[...]

$ pp composition_gstring_from_id(19)
 
 [[#<font-object "-UKWN-Iosevka Term SS04-regular-normal-normal-*-20-*-*-*-d-0-iso10646-1"> 61 61 61] 19 [0 0 61 5852 10 1 11 11 -4 [0 0 20]] [1 1 61 5896 10 -1 11 11 -4 [0 0 20]] [2 2 61 5891 10 -1 9 11 -4 [0 0 20]]]

-------

Proper rendering (emacs happens to be not in daemon mode):

 39  390: COMP[13 (0..0)] pos=1290 w=10 a+d=20+6 face=39 MB
 40  400: COMP[13 (1..1)] pos=1291 w=10 a+d=20+6 face=39 MB
 41  410: COMP[13 (2..2)] pos=1292 w=10 a+d=20+6 face=39 MB

$ pp composition_gstring_from_id(13) 

 [[#<font-object "-UKWN-Iosevka Term SS04-regular-normal-normal-*-20-*-*-*-d-0-iso10646-1"> 61 61 61] 13 [0 0 61 5852 10 1 11 11 -4 nil] [1 1 61 5896 10 -1 11 11 -4 nil] [2 2 61 5891 10 -1 9 11 -4 nil]] 


Both sessions are still running. I Hope this helps. Let me know if need
more remote hands.


Best,
Tim 

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Tim Ruffing <dev@real-or-random.org>
> Cc: dev@real-or-random.org, eggert@cs.ucla.edu, 73752@debbugs.gnu.org
> Date: Wed, 30 Oct 2024 18:34:14 +0100
> 
> 
> > > pp is calling Emacs to print to its stderr.  If that is redirected
> > > somewhere else you won't see the output.
> > 
> 
> Ah, this was the right hint. I'm using emacs in daemon mode, started
> from systemd, so I can inspect stderr via journalctl.
> 
> ------
> 
> Broken rendering for ligature "===" starting at pos 1290 (emacs happens
> to be in daemon mode)
> 
> [...]
>  39  480: COMP[19 (0..0)] pos=1290 w=20 a+d=20+6 face=39 MB
>  40  500: COMP[19 (1..1)] pos=1291 w=20 a+d=20+6 face=39 MB
>  41  520: COMP[19 (2..2)] pos=1292 w=20 a+d=20+6 face=39 MB
> [...]
> 
> $ pp composition_gstring_from_id(19)
>  
>  [[#<font-object "-UKWN-Iosevka Term SS04-regular-normal-normal-*-20-*-*-*-d-0-iso10646-1"> 61 61 61] 19 [0 0 61 5852 10 1 11 11 -4 [0 0 20]] [1 1 61 5896 10 -1 11 11 -4 [0 0 20]] [2 2 61 5891 10 -1 9 11 -4 [0 0 20]]]
> 
> -------
> 
> Proper rendering (emacs happens to be not in daemon mode):
> 
>  39  390: COMP[13 (0..0)] pos=1290 w=10 a+d=20+6 face=39 MB
>  40  400: COMP[13 (1..1)] pos=1291 w=10 a+d=20+6 face=39 MB
>  41  410: COMP[13 (2..2)] pos=1292 w=10 a+d=20+6 face=39 MB
> 
> $ pp composition_gstring_from_id(13) 
> 
>  [[#<font-object "-UKWN-Iosevka Term SS04-regular-normal-normal-*-20-*-*-*-d-0-iso10646-1"> 61 61 61] 13 [0 0 61 5852 10 1 11 11 -4 nil] [1 1 61 5896 10 -1 11 11 -4 nil] [2 2 61 5891 10 -1 9 11 -4 nil]] 
> 
> 
> Both sessions are still running. I Hope this helps. Let me know if need
> more remote hands.

Did the "bad" display start from "good" at the beginning of a session?
Or did it start from "bad" to begin with?  If the former, the next
idea is to put a watchpoint on the cached composition in a session
with "good" display, and then do whatever it takes to make it "bad",
hoping that the watchpoint will break at some point and show us the
code which replaces nil with these [X-OFF Y-OFF WADJUST] vectors.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Tim Ruffing]

On Wed, 2024-10-30 at 19:46 +0200, Eli Zaretskii wrote:
> 
> Did the "bad" display start from "good" at the beginning of a
> session? Or did it start from "bad" to begin with?

I'm rather sure that it started bad from the beginning of the session
yesterday.

In general, I'm not so sure, unfortunately. (I had disabled ligatures
in the past months because this bug was too annoying.) Don't rely on
any of this: I can't recall that it ever flipped suddenly from good to
bad *when the affected ligatures was in the visible array of the
buffer*. But I'm not so sure if this has really never flipped from good
to bad during a session, when the ligature was not visible.

Perhaps the other reporters have a better memory. 

Tim

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Tim Ruffing <dev@real-or-random.org>
> Cc: 73752@debbugs.gnu.org, Visuwesh <visuweshm@gmail.com>, xuan@xlk.me
> Date: Wed, 30 Oct 2024 19:00:17 +0100
> 
> 
> 
> On Wed, 2024-10-30 at 19:46 +0200, Eli Zaretskii wrote:
> > 
> > Did the "bad" display start from "good" at the beginning of a
> > session? Or did it start from "bad" to begin with?
> 
> I'm rather sure that it started bad from the beginning of the session
> yesterday.
> 
> In general, I'm not so sure, unfortunately. (I had disabled ligatures
> in the past months because this bug was too annoying.) Don't rely on
> any of this: I can't recall that it ever flipped suddenly from good to
> bad *when the affected ligatures was in the visible array of the
> buffer*. But I'm not so sure if this has really never flipped from good
> to bad during a session, when the ligature was not visible.

Then what exactly is the difference between the session with "good"
display and the one with the "bad" display?  How come they show the
same ligature differently?  What did you do to make them have
different looks?

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Tim Ruffing]

On Wed, 2024-10-30 at 20:57 +0200, Eli Zaretskii wrote:
> 
> 
> Then what exactly is the difference between the session with "good"
> display and the one with the "bad" display?  How come they show the
> same ligature differently?  What did you do to make them have
> different looks?

Nothing (that I'm aware of.) This appears to be non-deterministic.
Sometimes it works, sometimes it doesn't. And Yixuan seems to say the
same when he says that this looks like race condition.

It will still be useful to figure out what sets the [X-OFF Y-OFF
WADJUST] vectors instead of nil. After digging into the code a bit, I
strongly suspect it's composition_gstring_adjust_zero_width.

I'm not sure if I have the time to verify this in the coming days, I'd
appreciate if some of the other affected users could give it a try. 

I'm not claiming that composition_gstring_adjust_zero_width is wrong,
but if my theory is right that it sets the vector if and only if the
rendering is bad, then we'll be a step further.

This function was introduced in
cd88f6de4be1f8eba1db038b371d769f584be53b because of bug#50951. AFAIU it
is supposed to correct a grapheme width of 0 pixels to 1 pixel, so that
won't explain why large spaces are added. But I skimmed bug#50951 and
there are some mentions that composition_gstring_adjust_zero_width is
supposed to suppress adding the "space width of the font", which is
what some people see here, so all of this sounds a bit related. Next
question would be why emacs thinks that the grapheme width was zero in
the first place.

Tim  

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

On 10/30/24 21:39, Tim Ruffing wrote:
> I'm not sure if I have the time to verify this in the coming days, I'd
> appreciate if some of the other affected users could give it a try.

I'm sorry I wasn't following this thread closely for the last few days. 
I'm currently busy with a submission due in two weeks so I don't have 
continuous time to investigate the problem using GDB before Nov. 14.

> I can't recall that it ever flipped suddenly from good to
> bad *when the affected ligatures was in the visible array of the
> buffer*. But I'm not so sure if this has really never flipped from good
> to bad during a session, when the ligature was not visible.

I'm pretty certain it doesn't happen to a line spontaneously. I usually 
encounter this behavior (with a probability) when
1. after a font changes, like what the script I submitted did.
2. after switching to a different buffer, even if both buffers contains 
ligatures and uses the same font/fontset setting (though maybe one of 
the buffers contains certain characters with bold/italic/etc font 
attributes, I don't recall it exactly).
3. (maybe) after scrolling down/up the buffer into a position where a 
line contains ligatures appears.

Best,
Yixuan

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Yixuan Chen]

Eli,

An idea on reproducibility: if I can build an container image (I don't 
know if it will work, but the idea is through wayland socket bind mount) 
or an VM image where emacs can consistently demonstrate the problem, 
would that be acceptable? That way maybe you can have a first hand 
experience with the problem. If so, what format do you accept?

On 10/30/24 22:36, Yixuan Chen wrote:
> On 10/30/24 21:39, Tim Ruffing wrote:
>> I'm not sure if I have the time to verify this in the coming days, I'd
>> appreciate if some of the other affected users could give it a try.
> 
> I'm sorry I wasn't following this thread closely for the last few days. 
> I'm currently busy with a submission due in two weeks so I don't have 
> continuous time to investigate the problem using GDB before Nov. 14.
> 
>> I can't recall that it ever flipped suddenly from good to
>> bad *when the affected ligatures was in the visible array of the
>> buffer*. But I'm not so sure if this has really never flipped from good
>> to bad during a session, when the ligature was not visible.
> 
> I'm pretty certain it doesn't happen to a line spontaneously. I usually 
> encounter this behavior (with a probability) when
> 1. after a font changes, like what the script I submitted did.
> 2. after switching to a different buffer, even if both buffers contains 
> ligatures and uses the same font/fontset setting (though maybe one of 
> the buffers contains certain characters with bold/italic/etc font 
> attributes, I don't recall it exactly).
> 3. (maybe) after scrolling down/up the buffer into a position where a 
> line contains ligatures appears.
> 
> Best,
> Yixuan

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> Date: Wed, 30 Oct 2024 22:46:30 -0400
> From: Yixuan Chen <xuan@xlk.me>
> Cc: 73752@debbugs.gnu.org, visuweshm@gmail.com
> 
> An idea on reproducibility: if I can build an container image (I don't 
> know if it will work, but the idea is through wayland socket bind mount) 
> or an VM image where emacs can consistently demonstrate the problem, 
> would that be acceptable?

I don't think so, no.  I have no system where I could run such an
image with debugging capabilities I need.

Let's try to follow up the information we already uncovered, it might
be that we have good leads already.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Visuwesh]

[-- Attachment #1: Type: text/plain, Size: 2615 bytes --]

[புதன் அக்டோபர் 30, 2024] Eli Zaretskii wrote:

>> From: Tim Ruffing <dev@real-or-random.org>
>> Cc: dev@real-or-random.org, eggert@cs.ucla.edu, 73752@debbugs.gnu.org
>> Date: Wed, 30 Oct 2024 18:34:14 +0100
>> 
>> 
>> > > pp is calling Emacs to print to its stderr.  If that is redirected
>> > > somewhere else you won't see the output.
>> > 
>> 
>> Ah, this was the right hint. I'm using emacs in daemon mode, started
>> from systemd, so I can inspect stderr via journalctl.
>> 
>> ------
>> 
>> Broken rendering for ligature "===" starting at pos 1290 (emacs happens
>> to be in daemon mode)
>> 
>> [...]
>>  39  480: COMP[19 (0..0)] pos=1290 w=20 a+d=20+6 face=39 MB
>>  40  500: COMP[19 (1..1)] pos=1291 w=20 a+d=20+6 face=39 MB
>>  41  520: COMP[19 (2..2)] pos=1292 w=20 a+d=20+6 face=39 MB
>> [...]
>> 
>> $ pp composition_gstring_from_id(19)
>>  
>>  [[#<font-object "-UKWN-Iosevka Term SS04-regular-normal-normal-*-20-*-*-*-d-0-iso10646-1"> 61 61 61] 19 [0 0 61 5852 10 1 11 11 -4 [0 0 20]] [1 1 61 5896 10 -1 11 11 -4 [0 0 20]] [2 2 61 5891 10 -1 9 11 -4 [0 0 20]]]
>> 
>> -------
>> 
>> Proper rendering (emacs happens to be not in daemon mode):
>> 
>>  39  390: COMP[13 (0..0)] pos=1290 w=10 a+d=20+6 face=39 MB
>>  40  400: COMP[13 (1..1)] pos=1291 w=10 a+d=20+6 face=39 MB
>>  41  410: COMP[13 (2..2)] pos=1292 w=10 a+d=20+6 face=39 MB
>> 
>> $ pp composition_gstring_from_id(13) 
>> 
>>  [[#<font-object "-UKWN-Iosevka Term SS04-regular-normal-normal-*-20-*-*-*-d-0-iso10646-1"> 61 61 61] 13 [0 0 61 5852 10 1 11 11 -4 nil] [1 1 61 5896 10 -1 11 11 -4 nil] [2 2 61 5891 10 -1 9 11 -4 nil]] 
>> 
>> 
>> Both sessions are still running. I Hope this helps. Let me know if need
>> more remote hands.

I observe very similar results:

Properly rendered (in a fresh Emacs session):

    [[#<font-object "-SAJA-Cascadia Code-bold-normal-normal-*-15-*-*-*-m-0-iso10646-1"> 45 45 62] 2 [0 0 45 1970 9 0 9 7 -4 nil] [1 1 45 1969 9 -1 10 7 -4 nil] [2 2 62 2728 9 -1 9 11 0 nil]]

Misaligned:

    [[#<font-object "-SAJA-Cascadia Code-bold-normal-normal-*-15-*-*-*-m-0-iso10646-1"> 45 45 62] 2231 [0 0 45 1970 9 0 9 7 -4 [0 0 10]] [1 1 45 1969 9 -1 10 7 -4 [0 0 10]] [2 2 62 2728 9 -1 9 11 0 [0 0 10]]]

In the misaligned session, cached entry for the same text ("-->") that
is rendered properly at a different font size:

    [[#<font-object "-SAJA-Cascadia Code-bold-normal-normal-*-17-*-*-*-m-0-iso10646-1"> 45 45 62] 2179 [0 0 45 1970 10 0 11 7 -4 nil] [1 1 45 1969 10 -1 11 7 -4 nil] [2 2 62 2728 10 -1 10 12 0 nil]]



[-- Attachment #2: Type: text/plain, Size: 687 bytes --]

> Did the "bad" display start from "good" at the beginning of a session?
> Or did it start from "bad" to begin with?  If the former, the next
> idea is to put a watchpoint on the cached composition in a session
> with "good" display, and then do whatever it takes to make it "bad",
> hoping that the watchpoint will break at some point and show us the
> code which replaces nil with these [X-OFF Y-OFF WADJUST] vectors.

I think it starts from "bad" to begin with.  But the former theory could
still apply, if you do not mind this vague answer, can you provide
instructions to set a watchpoint?  If the watchpoint never triggers, we
might be able to at least rule out the former theory.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Tim Ruffing <dev@real-or-random.org>
> Cc: 73752@debbugs.gnu.org, visuweshm@gmail.com, xuan@xlk.me
> Date: Thu, 31 Oct 2024 02:39:09 +0100
> 
> > Then what exactly is the difference between the session with "good"
> > display and the one with the "bad" display?  How come they show the
> > same ligature differently?  What did you do to make them have
> > different looks?
> 
> Nothing (that I'm aware of.) This appears to be non-deterministic.
> Sometimes it works, sometimes it doesn't. And Yixuan seems to say the
> same when he says that this looks like race condition.

If that's the case, we need to understand what causes such a "race
condition".  Emacs is single-threaded, so it's hard to imagine
something like that.

> It will still be useful to figure out what sets the [X-OFF Y-OFF
> WADJUST] vectors instead of nil. After digging into the code a bit, I
> strongly suspect it's composition_gstring_adjust_zero_width.
> 
> I'm not sure if I have the time to verify this in the coming days, I'd
> appreciate if some of the other affected users could give it a try. 
> 
> I'm not claiming that composition_gstring_adjust_zero_width is wrong,
> but if my theory is right that it sets the vector if and only if the
> rendering is bad, then we'll be a step further.
> 
> This function was introduced in
> cd88f6de4be1f8eba1db038b371d769f584be53b because of bug#50951. AFAIU it
> is supposed to correct a grapheme width of 0 pixels to 1 pixel, so that
> won't explain why large spaces are added. But I skimmed bug#50951 and
> there are some mentions that composition_gstring_adjust_zero_width is
> supposed to suppress adding the "space width of the font", which is
> what some people see here, so all of this sounds a bit related. Next
> question would be why emacs thinks that the grapheme width was zero in
> the first place.

The way forward would be to set a breakpoint inside the width == 0
condition in that function:

      if (NILP (glyph) || from != LGLYPH_FROM (glyph))
	{
	  eassert (i > 0);
	  Lisp_Object last = LGSTRING_GLYPH (gstring, i - 1);

	  if (width == 0)
	    {
	      if (NILP (LGLYPH_ADJUSTMENT (last)))  <<<<<<<<<<<<<<<<<<<<<<<<<
		LGLYPH_SET_ADJUSTMENT (last,
				       CALLN (Fvector,
					      make_fixnum (0), make_fixnum (0),
					      make_fixnum (LGLYPH_WIDTH (last)
							   + 1)));
	      else
		ASET (LGLYPH_ADJUSTMENT (last), 2,
		      make_fixnum (LGLYPH_WADJUST (last) + 1));
	    }

and see if it ever breaks in the scenario(s) where you see the
corrupted display.

In my testing, that breakpoint never breaks, which is consistent with
the fact that none of the glyphs we compose have zero width.  They
don't have zero width in your "broken rendering" case, either:

> $ pp composition_gstring_from_id(19)
>  
>  [[#<font-object "-UKWN-Iosevka Term SS04-regular-normal-normal-*-20-*-*-*-d-0-iso10646-1"> 61 61 61] 19 [0 0 61 5852 10 1 11 11 -4 [0 0 20]] [1 1 61 5896 10 -1 11 11 -4 [0 0 20]] [2 2 61 5891 10 -1 9 11 -4 [0 0 20]]]

(The width is the 5th component of the glyph vector, and they are all
positive there.)  Same in the example shown by Visuwesh:

> Misaligned:
> 
>     [[#<font-object "-SAJA-Cascadia Code-bold-normal-normal-*-15-*-*-*-m-0-iso10646-1"> 45 45 62] 2231 [0 0 45 1970 9 0 9 7 -4 [0 0 10]] [1 1 45 1969 9 -1 10 7 -4 [0 0 10]] [2 2 62 2728 9 -1 9 11 0 [0 0 10]]]

So there's something we are missing here.

Also note that in Visuwesh's case the WADJUST element corresponds to
the width of the previous glyph (9 + 1 = 10), but in your case it
doesn't (11 + 1 ≠ 20).  So this is another mystery.

There are a few calls to lglyph-set-adjustment in composite.el, which
could be responsible for this.  So if the breakpoint inside
composition_gstring_adjust_zero_width never breaks in your sessions,
either, the next step is to put a watchpoint on the cached composition
created by font-shape-gstring and wait for some code to modify the
adjustment part.

bug#73752: 29.4; Ligatures are randomly rendered with extra spaces

[Eli Zaretskii]

> From: Visuwesh <visuweshm@gmail.com>
> Cc: Tim Ruffing <dev@real-or-random.org>,  73752@debbugs.gnu.org,  xuan@xlk.me
> Date: Thu, 31 Oct 2024 13:42:06 +0530
> 
> > Did the "bad" display start from "good" at the beginning of a session?
> > Or did it start from "bad" to begin with?  If the former, the next
> > idea is to put a watchpoint on the cached composition in a session
> > with "good" display, and then do whatever it takes to make it "bad",
> > hoping that the watchpoint will break at some point and show us the
> > code which replaces nil with these [X-OFF Y-OFF WADJUST] vectors.
> 
> I think it starts from "bad" to begin with.  But the former theory could
> still apply, if you do not mind this vague answer, can you provide
> instructions to set a watchpoint?  If the watchpoint never triggers, we
> might be able to at least rule out the former theory.

Let's first see if a breakpoint inside
composition_gstring_adjust_zero_width ever breaks in your cases.  If
it doesn't we will try the watchpoint technique.