bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Tomohisa Kuranari]

This may not be a bug, but I have noticed different behavior compared to
Emacs 29.4.

I found that the behavior changed when the Android port was introduced, but
I haven't been able to pinpoint the exact cause.
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=c71a520d1da636a722cf87b46534ca3b5aafbc7b

## Steps to Reproduce
1. Start Emacs with emacs -Q.
2. Evaluate the following in the scratch buffer:

;; 29.4
(face-font 'default nil ?元)
"-*-Hiragino Sans-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"
(face-font 'default nil ?你)
"-*-Hiragino Sans-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"
(face-font 'default nil ?안)
"-*-Apple SD Gothic Neo-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"

;; 30.0.91
(face-font 'default nil ?元)
"-*-Arial Unicode MS-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"
(face-font 'default nil ?你)
"-*-Arial Unicode MS-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"
(face-font 'default nil ?안)
"-*-Arial Unicode MS-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"

## Workaround
Specify the font explicitly
```
(set-fontset-font nil 'japanese-jisx0208
                  (font-spec :family "Hiragino Sans"))
```


In GNU Emacs 30.0.91 (build 1, x86_64-apple-darwin23.1.0, NS
 appkit-2487.20 Version 14.1 (Build 23B74)) of 2024-09-14 built on
 kuranariMacBook-Pro.local
Windowing system distributor 'Apple', version 10.3.2487
System Description:  macOS 14.1

Configured using:
 'configure --disable-dependency-tracking --disable-silent-rules
 --enable-locallisppath=/usr/local/share/emacs/site-lisp
 --infodir=/usr/local/Cellar/emacs-plus@30/30.0.91/share/info/emacs
 --prefix=/usr/local/Cellar/emacs-plus@30/30.0.91 --with-xml2
 --with-gnutls --without-compress-install --without-dbus
 --without-imagemagick --with-modules --with-rsvg --with-webp --with-ns
 --disable-ns-self-contained 'CFLAGS=-O2 -DFD_SETSIZE=10000
 -DDARWIN_UNLIMITED_SELECT''

Configured features:
ACL GIF GLIB GMP GNUTLS JPEG LCMS2 LIBXML2 MODULES NOTIFY KQUEUE NS
PDUMPER PNG RSVG SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER
WEBP XIM ZLIB

Important settings:
  value of $LANG: ja_JP.UTF-8
  locale-coding-system: utf-8-unix

Major mode: ELisp/d

Minor modes in effect:
  which-key-mode: t
  pixel-scroll-precision-mode: t
  recentf-mode: t
  global-git-gutter-mode: t
  git-gutter-mode: t
  global-corfu-mode: t
  corfu-mode: t
  marginalia-mode: t
  vertico-mode: t
  delete-selection-mode: t
  global-auto-revert-mode: t
  override-global-mode: t
  puni-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(mailalias mailclient textsec uni-scripts idna-mapping uni-confusable
textsec-check qp ispell shadow sort mail-extr puni pulse color emacsbug
message yank-media puny dired dired-loaddefs rfc822 mml mml-sec epa
derived epg rfc6068 epg-config gnus-util mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils project consult vc-git
diff-mode track-changes vc-dispatcher which-key org-agenda org-element
org-persist xdg org-id org-element-ast inline avl-tree generator
org-refile org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro
org-src sh-script smie treesit executable ob-comint org-pcomplete
pcomplete comint ansi-osc ansi-color org-list org-footnote org-faces
org-entities time-date noutline outline org-version ob-emacs-lisp
ob-core ob-eval org-cycle org-table org-keys oc org-loaddefs find-func
cal-menu calendar cal-loaddefs ol rx org-fold org-fold-core org-compat
org-macs format-spec bookmark text-property-search time pixel-scroll
cua-base ring dashboard dashboard-widgets recentf tree-widget ffap
thingatpt git-gutter writeroom-mode visual-fill-column cape corfu
orderless marginalia vertico compat delsel cus-edit pp wid-edit
autorevert filenotify cus-load finder-inf edmacro kmacro
use-package-bind-key bind-key use-package-ensure cl-extra help-mode
use-package-core avy-autoloads cape-autoloads corfu-autoloads
dashboard-autoloads embark-consult-autoloads consult-autoloads
embark-autoloads git-gutter-autoloads marginalia-autoloads
orderless-autoloads org-contrib-autoloads org-modern-autoloads
puni-autoloads easy-mmode vertico-autoloads info
writeroom-mode-autoloads visual-fill-column-autoloads package browse-url
url url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs icons password-cache json
subr-x map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib
japan-util rmc iso-transl tooltip cconv eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads kqueue cocoa ns lcms2
multi-tty make-network-process emacs)

Memory information:
((conses 16 423228 149081) (symbols 48 27755 15)
 (strings 32 147709 4935) (string-bytes 1 3346886) (vectors 16 58280)
 (vector-slots 8 1414362 216992) (floats 8 413 1650)
 (intervals 56 3577 542) (buffers 992 20))

-- 
Tomohisa Kuranari
Email: tomohisa.kuranari@gmail.com

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Eli Zaretskii]

> From: Tomohisa Kuranari <tomohisa.kuranari@gmail.com>
> Date: Thu, 19 Sep 2024 23:50:58 +0900
> 
> This may not be a bug, but I have noticed different behavior compared to
> Emacs 29.4.
> 
> I found that the behavior changed when the Android port was introduced, but
> I haven't been able to pinpoint the exact cause.
> https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=c71a520d1da636a722cf87b46534ca3b5aafbc7b
> 
> ## Steps to Reproduce
> 1. Start Emacs with emacs -Q.
> 2. Evaluate the following in the scratch buffer:
> 
> ;; 29.4
> (face-font 'default nil ?元)
> "-*-Hiragino Sans-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"
> (face-font 'default nil ?你)
> "-*-Hiragino Sans-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"
> (face-font 'default nil ?안)
> "-*-Apple SD Gothic Neo-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"
> 
> ;; 30.0.91
> (face-font 'default nil ?元)
> "-*-Arial Unicode MS-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"
> (face-font 'default nil ?你)
> "-*-Arial Unicode MS-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"
> (face-font 'default nil ?안)
> "-*-Arial Unicode MS-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"
> 
> ## Workaround
> Specify the font explicitly
> ```
> (set-fontset-font nil 'japanese-jisx0208
>                   (font-spec :family "Hiragino Sans"))
> ```
> 
> 
> In GNU Emacs 30.0.91 (build 1, x86_64-apple-darwin23.1.0, NS
>  appkit-2487.20 Version 14.1 (Build 23B74)) of 2024-09-14 built on
>  kuranariMacBook-Pro.local

I cannot reproduce this, but I'm not on macOS.  Can someone reproduce
this on GNU/Linux and/or on macOS?

The only changes I see in fontset.el due to Android which could
explain this are supposed to be in effect only when Emacs is built for
Android.  I see quite some changes in macfont.m, but they seem to
affect font's weight and slant, so I'm not sure how that could be
relevant.

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Robert Pluim]

>>>>> On Thu, 19 Sep 2024 19:33:01 +0300, Eli Zaretskii <eliz@gnu.org> said:

    Eli> I cannot reproduce this, but I'm not on macOS.  Can someone reproduce
    Eli> this on GNU/Linux and/or on macOS?

On GNU/Linux I get the same result from
emacs-29 and emacs-30 ("-GOOG-Noto Sans CJK
KR-regular-normal-normal-*-16-*-*-*-*-0-iso10646-1""

On macOs it changes from "PingFang SC" to "Arial Unicode MS" when I go
from emacs-29 to emacs-30

    Eli> The only changes I see in fontset.el due to Android which could
    Eli> explain this are supposed to be in effect only when Emacs is built for
    Eli> Android.  I see quite some changes in macfont.m, but they seem to
    Eli> affect font's weight and slant, so I'm not sure how that could be
    Eli> relevant.

It doesnʼt appear to be either of those. Iʼd bisect, but right now I
donʼt have time for that.

Robert
-- 

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Tomohisa Kuranari <tomohisa.kuranari@gmail.com>,  Po Lu
>  <luangruo@yahoo.com>,  73363@debbugs.gnu.org
> Date: Thu, 19 Sep 2024 19:02:26 +0200
> 
> >>>>> On Thu, 19 Sep 2024 19:33:01 +0300, Eli Zaretskii <eliz@gnu.org> said:
> 
>     Eli> I cannot reproduce this, but I'm not on macOS.  Can someone reproduce
>     Eli> this on GNU/Linux and/or on macOS?
> 
> On GNU/Linux I get the same result from
> emacs-29 and emacs-30 ("-GOOG-Noto Sans CJK
> KR-regular-normal-normal-*-16-*-*-*-*-0-iso10646-1""
> 
> On macOs it changes from "PingFang SC" to "Arial Unicode MS" when I go
> from emacs-29 to emacs-30
> 
>     Eli> The only changes I see in fontset.el due to Android which could
>     Eli> explain this are supposed to be in effect only when Emacs is built for
>     Eli> Android.  I see quite some changes in macfont.m, but they seem to
>     Eli> affect font's weight and slant, so I'm not sure how that could be
>     Eli> relevant.
> 
> It doesnʼt appear to be either of those. Iʼd bisect, but right now I
> donʼt have time for that.

The OP already pointed to the offending commit: c71a520d1da.  The
problem is that it's a huge merge-commit (it lands Android support),
so it's not easy to see which part of it could cause this.  I see
there a non-trivial change in fontset.c, FWIW.

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Stephen Berman via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Thu, 19 Sep 2024 19:02:26 +0200 Robert Pluim <rpluim@gmail.com> wrote:

>>>>>> On Thu, 19 Sep 2024 19:33:01 +0300, Eli Zaretskii <eliz@gnu.org> said:
>
>     Eli> I cannot reproduce this, but I'm not on macOS.  Can someone reproduce
>     Eli> this on GNU/Linux and/or on macOS?
>
> On GNU/Linux I get the same result from
> emacs-29 and emacs-30 ("-GOOG-Noto Sans CJK
> KR-regular-normal-normal-*-16-*-*-*-*-0-iso10646-1""
>
> On macOs it changes from "PingFang SC" to "Arial Unicode MS" when I go
> from emacs-29 to emacs-30

On my GNU/Linux system:

emacs-29 -Q:
"-WenQ-WenQuanYi Zen Hei-medium-normal-normal-*-13-*-*-*-*-0-iso10646-1"

emacs-30 -Q and emacs-master -Q:
"-GOOG-Noto Sans JP-regular-normal-normal-*-13-*-*-*-*-0-iso10646-1"

The WenQuanYi Zen Hei font is in /usr/share/fonts and the Noto Sans JP
font is in ~/.local/share/fonts.

Steve Berman

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Thu, 19 Sep 2024 19:33:01 +0300, Eli Zaretskii <eliz@gnu.org>
> said:
>
>     Eli> I cannot reproduce this, but I'm not on macOS.  Can someone
>     Eli> reproduce
>     Eli> this on GNU/Linux and/or on macOS?
>
> On GNU/Linux I get the same result from
> emacs-29 and emacs-30 ("-GOOG-Noto Sans CJK
> KR-regular-normal-normal-*-16-*-*-*-*-0-iso10646-1""
>
> On macOs it changes from "PingFang SC" to "Arial Unicode MS" when I go
> from emacs-29 to emacs-30
>
>     Eli> The only changes I see in fontset.el due to Android which
>     Eli> could
>     Eli> explain this are supposed to be in effect only when Emacs is
>     Eli> built for
>     Eli> Android.  I see quite some changes in macfont.m, but they
>     Eli> seem to
>     Eli> affect font's weight and slant, so I'm not sure how that
>     Eli> could be
>     Eli> relevant.
>
> It doesnʼt appear to be either of those. Iʼd bisect, but right now I
> donʼt have time for that.
>
> Robert

Could it be 1727777a (a Git revision)?  Would the OP try reverting it?

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: Eli Zaretskii <eliz@gnu.org>,  Po Lu <luangruo@yahoo.com>,  Tomohisa
>  Kuranari <tomohisa.kuranari@gmail.com>,  73363@debbugs.gnu.org
> Date: Thu, 19 Sep 2024 22:10:34 +0200
> 
> On my GNU/Linux system:
> 
> emacs-29 -Q:
> "-WenQ-WenQuanYi Zen Hei-medium-normal-normal-*-13-*-*-*-*-0-iso10646-1"

For all 3 characters?  That is, Emacs selects the same font for han
and hangul characters?

> emacs-30 -Q and emacs-master -Q:
> "-GOOG-Noto Sans JP-regular-normal-normal-*-13-*-*-*-*-0-iso10646-1"
> 
> The WenQuanYi Zen Hei font is in /usr/share/fonts and the Noto Sans JP
> font is in ~/.local/share/fonts.

What happens in Emacs 30 if you prevent Emacs from using the font in
~/.local/share/fonts in some way?

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  Tomohisa Kuranari
>  <tomohisa.kuranari@gmail.com>,  73363@debbugs.gnu.org
> Date: Fri, 20 Sep 2024 07:51:13 +0800
> 
> Robert Pluim <rpluim@gmail.com> writes:
> 
> >>>>>> On Thu, 19 Sep 2024 19:33:01 +0300, Eli Zaretskii <eliz@gnu.org>
> > said:
> >
> >     Eli> I cannot reproduce this, but I'm not on macOS.  Can someone
> >     Eli> reproduce
> >     Eli> this on GNU/Linux and/or on macOS?
> >
> > On GNU/Linux I get the same result from
> > emacs-29 and emacs-30 ("-GOOG-Noto Sans CJK
> > KR-regular-normal-normal-*-16-*-*-*-*-0-iso10646-1""
> >
> > On macOs it changes from "PingFang SC" to "Arial Unicode MS" when I go
> > from emacs-29 to emacs-30
> >
> >     Eli> The only changes I see in fontset.el due to Android which
> >     Eli> could
> >     Eli> explain this are supposed to be in effect only when Emacs is
> >     Eli> built for
> >     Eli> Android.  I see quite some changes in macfont.m, but they
> >     Eli> seem to
> >     Eli> affect font's weight and slant, so I'm not sure how that
> >     Eli> could be
> >     Eli> relevant.
> >
> > It doesnʼt appear to be either of those. Iʼd bisect, but right now I
> > donʼt have time for that.
> >
> > Robert
> 
> Could it be 1727777a (a Git revision)?  Would the OP try reverting it?

That's the only font-related change I found there, yes.

Regardless of the effect of reverting it, I'd appreciate if Po Lu
could explain its rationale (and improve the comments with that),
since the log message and the comments don't say enough for me to
figure that out.  In particular, this part of the comment:

  TrueType fonts have contrived character map selection
  semantics which makes determining the repertory at font
  spec matching time unduly expensive.

What does this allude to (details and examples of such contrived
character map selection semantics, please), and where's the code where
spec matching becomes expensive without this change?

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Stephen Berman via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Fri, 20 Sep 2024 09:12:02 +0300 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  Po Lu <luangruo@yahoo.com>,  Tomohisa
>>  Kuranari <tomohisa.kuranari@gmail.com>,  73363@debbugs.gnu.org
>> Date: Thu, 19 Sep 2024 22:10:34 +0200
>> 
>> On my GNU/Linux system:
>> 
>> emacs-29 -Q:
>> "-WenQ-WenQuanYi Zen Hei-medium-normal-normal-*-13-*-*-*-*-0-iso10646-1"
>
> For all 3 characters?  That is, Emacs selects the same font for han
> and hangul characters?

No; in emacs-29:

(face-font 'default nil ?元)
"-WenQ-WenQuanYi Zen Hei-medium-normal-normal-*-13-*-*-*-*-0-iso10646-1"

(face-font 'default nil ?你)
"-WenQ-WenQuanYi Zen Hei-medium-normal-normal-*-13-*-*-*-*-0-iso10646-1"

(face-font 'default nil ?안)
"-GOOG-Noto Serif KR-regular-normal-normal-*-13-*-*-*-*-0-iso10646-1"

In emacs-30 and master:

(face-font 'default nil ?元)
"-GOOG-Noto Sans JP-regular-normal-normal-*-13-*-*-*-*-0-iso10646-1"

(face-font 'default nil ?你)
"-GOOG-Noto Sans JP-regular-normal-normal-*-13-*-*-*-*-0-iso10646-1"

(face-font 'default nil ?안)
"-GOOG-Noto Serif KR-regular-normal-normal-*-13-*-*-*-*-0-iso10646-1"

>> emacs-30 -Q and emacs-master -Q:
>> "-GOOG-Noto Sans JP-regular-normal-normal-*-13-*-*-*-*-0-iso10646-1"
>> 
>> The WenQuanYi Zen Hei font is in /usr/share/fonts and the Noto Sans JP
>> font is in ~/.local/share/fonts.
>
> What happens in Emacs 30 if you prevent Emacs from using the font in
> ~/.local/share/fonts in some way?

Then all three characters are displayed with WenQuanYi Zen Hei (not just
in emacs-30 but also in emacs-29 and master).

Steve Berman

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Robert Pluim]

>>>>> On Fri, 20 Sep 2024 09:27:40 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Po Lu <luangruo@yahoo.com>
    >> 
    >> Could it be 1727777a (a Git revision)?  Would the OP try reverting it?

    Eli> That's the only font-related change I found there, yes.

Reverting that gets me back "PingFang SC" on emacs-30 on macOS,
specifically:

"-*-PingFang SC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"

Robert
-- 

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Po Lu <luangruo@yahoo.com>,  tomohisa.kuranari@gmail.com,
>   73363@debbugs.gnu.org
> Date: Fri, 20 Sep 2024 10:44:19 +0200
> 
> >>>>> On Fri, 20 Sep 2024 09:27:40 +0300, Eli Zaretskii <eliz@gnu.org> said:
> 
>     >> From: Po Lu <luangruo@yahoo.com>
>     >> 
>     >> Could it be 1727777a (a Git revision)?  Would the OP try reverting it?
> 
>     Eli> That's the only font-related change I found there, yes.
> 
> Reverting that gets me back "PingFang SC" on emacs-30 on macOS,
> specifically:
> 
> "-*-PingFang SC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"

What do you mean by "gets back"?  The OP cited a different font for
Emacs 29.

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Robert Pluim]

>>>>> On Fri, 20 Sep 2024 13:29:21 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Robert Pluim <rpluim@gmail.com>
    >> Cc: Po Lu <luangruo@yahoo.com>,  tomohisa.kuranari@gmail.com,
    >> 73363@debbugs.gnu.org
    >> Date: Fri, 20 Sep 2024 10:44:19 +0200
    >> 
    >> >>>>> On Fri, 20 Sep 2024 09:27:40 +0300, Eli Zaretskii <eliz@gnu.org> said:
    >> 
    >> >> From: Po Lu <luangruo@yahoo.com>
    >> >> 
    >> >> Could it be 1727777a (a Git revision)?  Would the OP try reverting it?
    >> 
    Eli> That's the only font-related change I found there, yes.
    >> 
    >> Reverting that gets me back "PingFang SC" on emacs-30 on macOS,
    >> specifically:
    >> 
    >> "-*-PingFang SC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"

    Eli> What do you mean by "gets back"?  The OP cited a different font for
    Eli> Emacs 29.

I had PingFang with emacs-29. I do have Hiragino Sans installed, so
thatʼs another potential issue.

Robert
-- 

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> That's the only font-related change I found there, yes.
>
> Regardless of the effect of reverting it, I'd appreciate if Po Lu
> could explain its rationale (and improve the comments with that),
> since the log message and the comments don't say enough for me to
> figure that out.  In particular, this part of the comment:
>
>   TrueType fonts have contrived character map selection
>   semantics which makes determining the repertory at font
>   spec matching time unduly expensive.
>
> What does this allude to (details and examples of such contrived
> character map selection semantics, please), and where's the code where
> spec matching becomes expensive without this change?

Some of our font backends (sfntfont, at least, and perhaps macfont in
this case) will return different registries from that which was
specified in the font spec when enumerating font entities, in order to
enable font specs requesting an ASCII charset also to match fonts with
Unicode as the charset.  If cached, these font entities will be
repeatedly rejected in font_find_for_lface after being opened, with
obvious consequences for performance.

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Po Lu <luangruo@yahoo.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> That's the only font-related change I found there, yes.
>>
>> Regardless of the effect of reverting it, I'd appreciate if Po Lu
>> could explain its rationale (and improve the comments with that),
>> since the log message and the comments don't say enough for me to
>> figure that out.  In particular, this part of the comment:
>>
>>   TrueType fonts have contrived character map selection
>>   semantics which makes determining the repertory at font
>>   spec matching time unduly expensive.
>>
>> What does this allude to (details and examples of such contrived
>> character map selection semantics, please), and where's the code where
>> spec matching becomes expensive without this change?
>
> Some of our font backends (sfntfont, at least, and perhaps macfont in
> this case) will return different registries from that which was
> specified in the font spec when enumerating font entities, in order to
> enable font specs requesting an ASCII charset also to match fonts with
> Unicode as the charset.  If cached, these font entities will be
> repeatedly rejected in font_find_for_lface after being opened, with
> obvious consequences for performance.

I forgot to mention that sometimes no charset can be derived before the
font object is opened, and in these situations it is best not to cache
these font objects at all, if a registry should be specified in the
spec, for similar reasons.

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Robert Pluim]

>>>>> On Fri, 20 Sep 2024 20:08:10 +0800, Po Lu via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> said:
    >> Some of our font backends (sfntfont, at least, and perhaps macfont in
    >> this case) will return different registries from that which was
    >> specified in the font spec when enumerating font entities, in order to
    >> enable font specs requesting an ASCII charset also to match fonts with
    >> Unicode as the charset.  If cached, these font entities will be
    >> repeatedly rejected in font_find_for_lface after being opened, with
    >> obvious consequences for performance.

    Po Lu> I forgot to mention that sometimes no charset can be derived before the
    Po Lu> font object is opened, and in these situations it is best not to cache
    Po Lu> these font objects at all, if a registry should be specified in the
    Po Lu> spec, for similar reasons.

These are the fonts which are being rejected on macOS for me. And yet
PingFang SC seems to work ok.

strangeness detected: spec:’#<font-spec nil nil nil nil gb2312.1980-0 nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-PingFang SC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’
strangeness detected: spec:’#<font-spec nil nil nil nil jisx0208*-* nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-Hiragino Sans-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’
strangeness detected: spec:’#<font-spec nil nil nil nil jisx0212*-* nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-PingFang SC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’
strangeness detected: spec:’#<font-spec nil nil nil nil big5*-* nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-PingFang TC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’
strangeness detected: spec:’#<font-spec nil nil nil nil ksc5601.1987*-* nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-Apple SD Gothic Neo-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’
strangeness detected: spec:’#<font-spec nil nil nil nil cns11643.1992-1 nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-PingFang TC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’
strangeness detected: spec:’#<font-spec nil nil nil nil cns11643.1992-2 nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-PingFang SC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’
strangeness detected: spec:’#<font-spec nil nil nil nil cns11643.1992-3 nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-PingFang HK-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’
strangeness detected: spec:’#<font-spec nil nil nil nil gbk-0 nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-PingFang SC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’
strangeness detected: spec:’#<font-spec nil nil nil nil jisx0213.2000-1 nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-Hiragino Sans-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’
strangeness detected: spec:’#<font-spec nil nil nil nil jisx0213.2000-2 nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-PingFang SC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’
strangeness detected: spec:’#<font-spec nil nil nil nil jisx0213.2004-1 nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-PingFang HK-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’




Robert
-- 

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: luangruo@yahoo.com,  tomohisa.kuranari@gmail.com,  73363@debbugs.gnu.org
> Date: Fri, 20 Sep 2024 13:07:05 +0200
> 
> >>>>> On Fri, 20 Sep 2024 13:29:21 +0300, Eli Zaretskii <eliz@gnu.org> said:
> 
>     >> From: Robert Pluim <rpluim@gmail.com>
>     >> Cc: Po Lu <luangruo@yahoo.com>,  tomohisa.kuranari@gmail.com,
>     >> 73363@debbugs.gnu.org
>     >> Date: Fri, 20 Sep 2024 10:44:19 +0200
>     >> 
>     >> >>>>> On Fri, 20 Sep 2024 09:27:40 +0300, Eli Zaretskii <eliz@gnu.org> said:
>     >> 
>     >> >> From: Po Lu <luangruo@yahoo.com>
>     >> >> 
>     >> >> Could it be 1727777a (a Git revision)?  Would the OP try reverting it?
>     >> 
>     Eli> That's the only font-related change I found there, yes.
>     >> 
>     >> Reverting that gets me back "PingFang SC" on emacs-30 on macOS,
>     >> specifically:
>     >> 
>     >> "-*-PingFang SC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1"
> 
>     Eli> What do you mean by "gets back"?  The OP cited a different font for
>     Eli> Emacs 29.
> 
> I had PingFang with emacs-29. I do have Hiragino Sans installed, so
> thatʼs another potential issue.

Is PingFang supposed to be installed on GNU/Linux systems?  Or did you
install it your self (or maybe it's in some optional package people
usually don't install)?

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Robert Pluim]

>>>>> On Fri, 20 Sep 2024 16:24:41 +0300, Eli Zaretskii <eliz@gnu.org> said:
    >> I had PingFang with emacs-29. I do have Hiragino Sans installed, so
    >> thatʼs another potential issue.

    Eli> Is PingFang supposed to be installed on GNU/Linux systems?  Or did you
    Eli> install it your self (or maybe it's in some optional package people
    Eli> usually don't install)?

I have no idea, this is all on macOS.

Robert
-- 

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Po Lu <luangruo@yahoo.com>,  tomohisa.kuranari@gmail.com,
>   73363@debbugs.gnu.org
> Date: Fri, 20 Sep 2024 14:24:11 +0200
> 
> >>>>> On Fri, 20 Sep 2024 20:08:10 +0800, Po Lu via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org> said:
>     >> Some of our font backends (sfntfont, at least, and perhaps macfont in
>     >> this case) will return different registries from that which was
>     >> specified in the font spec when enumerating font entities, in order to
>     >> enable font specs requesting an ASCII charset also to match fonts with
>     >> Unicode as the charset.  If cached, these font entities will be
>     >> repeatedly rejected in font_find_for_lface after being opened, with
>     >> obvious consequences for performance.
> 
>     Po Lu> I forgot to mention that sometimes no charset can be derived before the
>     Po Lu> font object is opened, and in these situations it is best not to cache
>     Po Lu> these font objects at all, if a registry should be specified in the
>     Po Lu> spec, for similar reasons.
> 
> These are the fonts which are being rejected on macOS for me. And yet
> PingFang SC seems to work ok.
> 
> strangeness detected: spec:’#<font-spec nil nil nil nil gb2312.1980-0 nil nil nil nil nil nil nil nil>’ font:’#<font-object -*-PingFang SC-regular-normal-normal-*-12-*-*-*-p-0-iso10646-1>’

Po Lu, why does it make sense to reject iso10646-1 in these cases?  I
could perhaps understand why reject gb2312.1980-0 when we request
something like jisx0208*-*, but iso10646-1 is supposed to support all
the characters, so I don't understand the rationale for rejecting
that.  And the same for unicode-bmp and unicode-sip, I think (although
those are rarely used).

And what if the requested registry is ascii-0?  That one should match
any registry, IMO.  If not, why not?

More generally, what were the examples where you saw problems in
accepting registries different from the requested one?  Also, if those
problems are specific to certain platforms, how about making this test
only on those platforms?  Font selection machinery in Emacs is
delicate enough to try to "do no harm" first and foremost.

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: luangruo@yahoo.com,  tomohisa.kuranari@gmail.com,  73363@debbugs.gnu.org
> Date: Fri, 20 Sep 2024 15:42:54 +0200
> 
> >>>>> On Fri, 20 Sep 2024 16:24:41 +0300, Eli Zaretskii <eliz@gnu.org> said:
>     >> I had PingFang with emacs-29. I do have Hiragino Sans installed, so
>     >> thatʼs another potential issue.
> 
>     Eli> Is PingFang supposed to be installed on GNU/Linux systems?  Or did you
>     Eli> install it your self (or maybe it's in some optional package people
>     Eli> usually don't install)?
> 
> I have no idea, this is all on macOS.

Then I'm asking the same question about macOS.

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Robert Pluim]

>>>>> On Fri, 20 Sep 2024 16:53:34 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Robert Pluim <rpluim@gmail.com>
    >> Cc: luangruo@yahoo.com,  tomohisa.kuranari@gmail.com,  73363@debbugs.gnu.org
    >> Date: Fri, 20 Sep 2024 15:42:54 +0200
    >> 
    >> >>>>> On Fri, 20 Sep 2024 16:24:41 +0300, Eli Zaretskii <eliz@gnu.org> said:
    >> >> I had PingFang with emacs-29. I do have Hiragino Sans installed, so
    >> >> thatʼs another potential issue.
    >> 
    Eli> Is PingFang supposed to be installed on GNU/Linux systems?  Or did you
    Eli> install it your self (or maybe it's in some optional package people
    Eli> usually don't install)?
    >> 
    >> I have no idea, this is all on macOS.

    Eli> Then I'm asking the same question about macOS.

PingFang and Hiragino Sans are both part of the default system fonts,
on my macOS version 14.5 at least.

Robert
-- 

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> Po Lu, why does it make sense to reject iso10646-1 in these cases?  I
> could perhaps understand why reject gb2312.1980-0 when we request
> something like jisx0208*-*, but iso10646-1 is supposed to support all
> the characters, so I don't understand the rationale for rejecting
> that.  And the same for unicode-bmp and unicode-sip, I think (although
> those are rarely used).

Because such font objects that are cached are subsequently rejected,
once Emacs detects that their registries do not agree with the font
specs'.  I don't recall where this rejection is implemented,
unfortunately.

> And what if the requested registry is ascii-0?  That one should match
> any registry, IMO.  If not, why not?

See above.

> More generally, what were the examples where you saw problems in
> accepting registries different from the requested one?  Also, if those
> problems are specific to certain platforms, how about making this test
> only on those platforms?  Font selection machinery in Emacs is
> delicate enough to try to "do no harm" first and foremost.

I tested my original reproducer, and it appears that the problem has
resolved of itself, probably because the performance of the sfnt font
driver has since improved.  Therefore I will disable this change
elsewhere than Android on the release branch, and remove it entirely on
master.

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Robert Pluim <rpluim@gmail.com>,  tomohisa.kuranari@gmail.com,
>   73363@debbugs.gnu.org
> Date: Fri, 20 Sep 2024 22:24:34 +0800
> 
> > More generally, what were the examples where you saw problems in
> > accepting registries different from the requested one?  Also, if those
> > problems are specific to certain platforms, how about making this test
> > only on those platforms?  Font selection machinery in Emacs is
> > delicate enough to try to "do no harm" first and foremost.
> 
> I tested my original reproducer, and it appears that the problem has
> resolved of itself, probably because the performance of the sfnt font
> driver has since improved.  Therefore I will disable this change
> elsewhere than Android on the release branch, and remove it entirely on
> master.

Thanks, but why leave it enabled on systems other than Android on
master?  I'd say let's wait until we have some bug reports that could
be solved by enabling this logic.

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Tomohisa Kuranari]

Thank you for your help in resolving the issue.
I have confirmed that the same fonts as in the Emacs 29.4 are being
selected in the emacs-30 branch.

On Sat, Sep 21, 2024 at 12:42 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Po Lu <luangruo@yahoo.com>
> > Cc: Robert Pluim <rpluim@gmail.com>,  tomohisa.kuranari@gmail.com,
> >   73363@debbugs.gnu.org
> > Date: Fri, 20 Sep 2024 22:24:34 +0800
> >
> > > More generally, what were the examples where you saw problems in
> > > accepting registries different from the requested one?  Also, if those
> > > problems are specific to certain platforms, how about making this test
> > > only on those platforms?  Font selection machinery in Emacs is
> > > delicate enough to try to "do no harm" first and foremost.
> >
> > I tested my original reproducer, and it appears that the problem has
> > resolved of itself, probably because the performance of the sfnt font
> > driver has since improved.  Therefore I will disable this change
> > elsewhere than Android on the release branch, and remove it entirely on
> > master.
>
> Thanks, but why leave it enabled on systems other than Android on
> master?  I'd say let's wait until we have some bug reports that could
> be solved by enabling this logic.



-- 
Tomohisa Kuranari
Email: tomohisa.kuranari@gmail.com

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> Thanks, but why leave it enabled on systems other than Android on
> master?

No, I removed it completely on master.

bug#73363: 30.0.91; CJK Font Rendering Behavior Changed

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: rpluim@gmail.com,  tomohisa.kuranari@gmail.com,  73363@debbugs.gnu.org
> Date: Sat, 21 Sep 2024 10:31:21 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Thanks, but why leave it enabled on systems other than Android on
> > master?
> 
> No, I removed it completely on master.

Thanks, so I'm closing this bug.