bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Vladimir Panteleev]

On a GNU/Linux system running X.Org, if Emacs is set to use the X11
fixed font 7x13 (e.g. by specifying `--font 7x13' on the command line),
Emacs will use the incorrect font for Unicode (non-ASCII)
characters. Specifically:

- US-ASCII characters are rendered using the correct font.

- Cyrillic (e.g. letters of the Russian alphabet) and some other
  characters (e.g. '¢' (CENT SIGN)) are rendered with the 12x13 font
  instead of the indicated 7x13 font.

- Some characters, e.g. '·' (MIDDLE DOT) and '»' (RIGHT-POINTING DOUBLE
  ANGLE QUOTATION MARK) are rendered with an 8x13 font. This causes
  enabling whitespace-mode to misalign text.

Screenshots can be seen here:

http://emacs.stackexchange.com/questions/20278/unicode-in-x-bitmap-font-uses-a-different-font

I have found that evaluating the Sexpr:

(set-fontset-font "fontset-default" 'unicode "-misc-fixed-*-*-*--13-*-*-*-*-70-iso10646-1")

... causes characters with Unicode codepoints above 256 (incl. Cyrillic
characters) to be rendered with the correct font, but not those in the
codepoint range 128-255. Running set-fontset-font with '(128 . 255) as
the TARGET parameter does not seem to have any effect.

The problem can also be observed in Emacs version 24.5.  Other graphical
applications on the same system using this font do not exhibit this
behavior.

-----------------------------------------------------------------------

In GNU Emacs 25.1.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.20.9)
 of 2016-09-18 built on juergen
Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
System Description:	Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe
 -fstack-protector-strong' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
  value of $LC_TIME: en_DK.UTF-8
  value of $LANG: en_US.UTF8
  locale-coding-system: utf-8-unix

Major mode: Emacs-Lisp

Minor modes in effect:
  global-semanticdb-minor-mode: t
  global-semantic-idle-scheduler-mode: t
  semantic-mode: t
  which-key-mode: t
  flyspell-mode: t
  cua-mode: t
  magit-auto-revert-mode: t
  global-git-commit-mode: t
  global-diff-hl-mode: t
  diff-hl-mode: t
  diff-auto-refine-mode: t
  delete-selection-mode: t
  global-flycheck-mode: t
  flycheck-mode: t
  global-company-mode: t
  company-mode: t
  highlight-defined-mode: t
  recentf-mode: t
  highlight-symbol-mode: t
  helm-mode: t
  shell-dirtrack-mode: t
  async-bytecomp-package-mode: t
  show-paren-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
Mark set [4 times]
next-line: End of buffer [69 times]
Mark set [3 times]
previous-line: Beginning of buffer [59 times]
You should have a section marked ";;; Commentary:"

The first line should be of the form: ";;; package --- Summary"
Undo! [25 times]
Saving file /home/vladimir/.emacs.d/recentf...
Wrote /home/vladimir/.emacs.d/recentf

Load-path shadows:
/home/vladimir/.emacs.d/opt/bbcode-mode/bbcode-mode hides /home/vladimir/.emacs.d/elpa/bbcode-mode-20141103.1341/bbcode-mode
/home/vladimir/.emacs.d/opt/cil-mode/cil-mode hides /home/vladimir/.emacs.d/elpa/cil-mode-20160622.730/cil-mode
/home/vladimir/.emacs.d/opt/company-dcd/company-dcd hides /home/vladimir/.emacs.d/elpa/company-dcd-20160806.1852/company-dcd
/home/vladimir/.emacs.d/opt/Emacs-D-Mode/d-mode hides /home/vladimir/.emacs.d/elpa/d-mode-20160504.955/d-mode
/home/vladimir/.emacs.d/elpa/editorconfig-20160904.1508/editorconfig-core hides /home/vladimir/.emacs.d/elpa/editorconfig-core-20160212.106/editorconfig-core
/home/vladimir/.emacs.d/elpa/editorconfig-20160904.1508/editorconfig-core-handle hides /home/vladimir/.emacs.d/elpa/editorconfig-core-20160212.106/editorconfig-core-handle
/home/vladimir/.emacs.d/elpa/editorconfig-20160904.1508/editorconfig-fnmatch hides /home/vladimir/.emacs.d/elpa/editorconfig-fnmatch-20160212.106/editorconfig-fnmatch
/home/vladimir/.emacs.d/opt/flycheck/flycheck hides /home/vladimir/.emacs.d/elpa/flycheck-20160921.227/flycheck
/home/vladimir/.emacs.d/opt/flycheck/flycheck-ert hides /home/vladimir/.emacs.d/elpa/flycheck-20160921.227/flycheck-ert
/home/vladimir/.emacs.d/opt/flycheck/flycheck-buttercup hides /home/vladimir/.emacs.d/elpa/flycheck-20160921.227/flycheck-buttercup
/home/vladimir/.emacs.d/opt/pov-mode/pov-mode hides /home/vladimir/.emacs.d/elpa/pov-mode-20120825.716/pov-mode
/home/vladimir/.emacs.d/opt/pov-mode/pov-mode-pkg hides /home/vladimir/.emacs.d/elpa/pov-mode-20120825.716/pov-mode-pkg
/home/vladimir/.emacs.d/elpa/seq-2.16/seq hides /usr/share/emacs/25.1/lisp/emacs-lisp/seq

Features:
(shadow mail-extr emacsbug sendmail semantic/tag-write org org-macro
org-footnote org-pcomplete org-list org-faces org-entities org-version
ob-emacs-lisp ob ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys
ob-comint ob-core ob-eval org-compat org-macs org-loaddefs cal-menu
calendar cal-loaddefs semantic/edit semantic/db-typecache semantic/ia
semantic/senator semantic/tag-file semantic/db-file data-debug
cedet-files semantic/bovine/c semantic/decorate/include
semantic/decorate/mode semantic/decorate pulse hideif
semantic/bovine/c-by semantic/lex-spp semantic/bovine/gcc semantic/dep
semantic/bovine semantic/analyze/refs semantic/db-find semantic/db-ref
semantic/analyze semantic/sort semantic/scope semantic/analyze/fcn
semantic/db-mode semantic/db eieio-base semantic/idle semantic/format
ezimage semantic/tag-ls semantic/ctxt magit-extras linum gitignore-mode
conf-mode rect graphviz-dot-mode sort helm-command helm-elisp helm-eval
edebug helm-git-grep semantic/find helm-semantic helm-imenu imenu
semantic/util-modes semantic/util semantic semantic/tag semantic/lex
semantic/fw mode-local cedet pkgbuild-mode sh-script smie executable
d-mode derived cc-langs disp-table whitespace misearch multi-isearch
winner filecache ido copyright vc-git company-oddmuse company-keywords
company-etags etags xref project company-gtags company-dabbrev-code
company-dabbrev company-files company-capf company-cmake company-xcode
company-clang company-semantic company-eclim company-template
company-css company-nxml company-bbdb which-key my-stuff face-remap
my-keys smartparens-python python markdown-mode noutline outline
make-mode mediawiki url-cache mm-url gnus gnus-ems nnheader url-http tls
gnutls url-auth url-gw flyspell ispell move-text cua-base
my-persistent-selection my-prefs magit-blame magit-stash magit-bisect
magit-remote magit-commit magit-sequence magit magit-apply magit-wip
magit-log magit-diff smerge-mode magit-core magit-autorevert autorevert
filenotify magit-process magit-popup magit-mode magit-git crm
magit-section magit-utils git-commit log-edit message idna rfc822 mml
mml-sec epg mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047
rfc2045 ietf-drums mailabbrev mail-utils gmm-utils mailheader pcvs-util
add-log with-editor tramp-sh smart-tabs-mode cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs projectile
grep ibuf-ext ibuffer diff-hl vc-dir ewoc vc vc-dispatcher diff-mode
company-dcd ivy lv delsel popwin flycheck-dmd-dub f s flycheck find-func
subr-x json map yasnippet company highlight-defined cil-mode
smartparens-config smartparens dash recentf tree-widget wid-edit
highlight-symbol smart-mode-line-dark-theme smart-mode-line
rich-minority helm-mode helm-files rx image-dired tramp tramp-compat
tramp-loaddefs trampver ucs-normalize shell pcomplete format-spec
dired-x dired-aux ffap thingatpt helm-buffers helm-elscreen helm-tags
helm-bookmark helm-adaptive helm-info bookmark pp helm-locate helm-grep
helm-regexp helm-external helm-net browse-url xml url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf
url-util url-parse auth-source gnus-util mm-util help-fns mail-prsvr
password-cache url-vars mailcap helm-utils compile comint ansi-color
ring helm-help helm-types helm easy-mmode helm-source eieio-compat eieio
eieio-core helm-multi-match helm-lib dired helm-config helm-easymenu
async-bytecomp advice async saveplace my-recent my-packages my-server
server time-date deeper-blue-theme paren cus-start cus-load finder-inf
edmacro kmacro cl-seq cl-macs cl info package epg-config seq byte-opt gv
bytecomp byte-compile cl-extra help-mode easymenu cconv cl-loaddefs
pcase cl-lib mule-util tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame
cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese charscript
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 858192 556079)
 (symbols 48 60675 15)
 (miscs 40 3397 8168)
 (strings 32 187937 240917)
 (string-bytes 1 5200713)
 (vectors 16 104156)
 (vector-slots 8 2151017 320756)
 (floats 8 805 1255)
 (intervals 56 10754 21079)
 (buffers 976 71))

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Eli Zaretskii]

> From: Vladimir Panteleev <thecybershadow@gmail.com>
> Date: Wed, 28 Sep 2016 16:34:15 +0000
> 
> On a GNU/Linux system running X.Org, if Emacs is set to use the X11
> fixed font 7x13 (e.g. by specifying `--font 7x13' on the command line),
> Emacs will use the incorrect font for Unicode (non-ASCII)
> characters. Specifically:
> 
> - US-ASCII characters are rendered using the correct font.
> 
> - Cyrillic (e.g. letters of the Russian alphabet) and some other
>   characters (e.g. '¢' (CENT SIGN)) are rendered with the 12x13 font
>   instead of the indicated 7x13 font.
> 
> - Some characters, e.g. '·' (MIDDLE DOT) and '»' (RIGHT-POINTING DOUBLE
>   ANGLE QUOTATION MARK) are rendered with an 8x13 font. This causes
>   enabling whitespace-mode to misalign text.

Are you sure the 7x13 font installed on your system supports all those
characters?  Emacs won't switch to a different font if the default
font supports the character it needs to display.

Or could it be that some customizations, either yours or site-wide,
force Emacs to use other fonts for the above characters?

> I have found that evaluating the Sexpr:
> 
> (set-fontset-font "fontset-default" 'unicode "-misc-fixed-*-*-*--13-*-*-*-*-70-iso10646-1")
> 

Don't do that, this setting will cause all kinds of surprises.  To
customize the fonts, use specific character codepoint ranges, and only
specify font for a range if the font support most or all of that
range.

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Vladimir Panteleev]

On 2016-09-28 20:16, Eli Zaretskii wrote:
> Are you sure the 7x13 font installed on your system supports all those
> characters?  Emacs won't switch to a different font if the default
> font supports the character it needs to display.

As I mentioned in my bug report and as is visible in the screenshots,
the characters are indeed present in the font and are rendered correctly
in other applications. Only Emacs exhibits this problem.

> Or could it be that some customizations, either yours or site-wide,
> force Emacs to use other fonts for the above characters?

This problem is reproducible with "emacs -Q", without any
customizations, on all distributions I've tried.

Here is a complete recipe using a live CD, to remove any possibility of
user or site customization:

1. Get the Antergos live ISO (
https://antergos.com/download/antergos-live-iso/ ) and boot from it
(e.g. in a VM).
2. Choose "Start Antergos live".
3. Choose "Try It".
4. Install Emacs and urxvt (in a terminal, run "pacman -S emacs
rxvt-unicode").
5. Run "emacs -Q --font 7x13".
6. M-x view-hello-file
7. Observe that Cyrillic characters are rendered in a 12x13 font.
8. Run "urxvt -fn 7x13".
9. Copy some Cyrillic characters from Emacs and paste them in urxvt.
10. Observe that urxvt renders the Cyrillic characters correctly, in the
correct font.

This is not specific to Arch / Antegros: The problem is also
reproducible on Ubuntu and Fedora Workstation Live CDs, though you may
have to also install some font packages (xorg-x11-fonts-misc for
Fedora), and Emacs uses the 8x13 font for Cyrillic characters (so the
problem is less obvious). urxvt still uses the correct font there.

>> I have found that evaluating the Sexpr:
>>
>> (set-fontset-font "fontset-default" 'unicode "-misc-fixed-*-*-*--13-*-*-*-*-70-iso10646-1")
>>
> 
> Don't do that, this setting will cause all kinds of surprises.  To
> customize the fonts, use specific character codepoint ranges, and only
> specify font for a range if the font support most or all of that
> range.

Noted. However, immediately lacking a better concrete workaround, I
don't have a better option at my disposal.

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Eli Zaretskii]

Adding Handa-san to this discussion, in the hope that he could shed
some light on this issue.

> From: Vladimir Panteleev <thecybershadow@gmail.com>
> Cc: 24560@debbugs.gnu.org
> Date: Wed, 28 Sep 2016 21:58:20 +0000
> 
> On 2016-09-28 20:16, Eli Zaretskii wrote:
> > Are you sure the 7x13 font installed on your system supports all those
> > characters?  Emacs won't switch to a different font if the default
> > font supports the character it needs to display.
> 
> As I mentioned in my bug report and as is visible in the screenshots,
> the characters are indeed present in the font and are rendered correctly
> in other applications. Only Emacs exhibits this problem.
> 
> > Or could it be that some customizations, either yours or site-wide,
> > force Emacs to use other fonts for the above characters?
> 
> This problem is reproducible with "emacs -Q", without any
> customizations, on all distributions I've tried.

Then I guess there's some factor at work here that I don't quite
figure out.  Maybe the Emacs font selection algorithm doesn't like
that font for some reason?

If you start Emacs with 12x13 font or 8x13, does Emacs display more
characters than 7x13 before it switches to another font?

> Here is a complete recipe using a live CD, to remove any possibility of
> user or site customization:
> 
> 1. Get the Antergos live ISO (
> https://antergos.com/download/antergos-live-iso/ ) and boot from it
> (e.g. in a VM).
> 2. Choose "Start Antergos live".
> 3. Choose "Try It".
> 4. Install Emacs and urxvt (in a terminal, run "pacman -S emacs
> rxvt-unicode").
> 5. Run "emacs -Q --font 7x13".
> 6. M-x view-hello-file
> 7. Observe that Cyrillic characters are rendered in a 12x13 font.
> 8. Run "urxvt -fn 7x13".
> 9. Copy some Cyrillic characters from Emacs and paste them in urxvt.
> 10. Observe that urxvt renders the Cyrillic characters correctly, in the
> correct font.
> 
> This is not specific to Arch / Antegros: The problem is also
> reproducible on Ubuntu and Fedora Workstation Live CDs, though you may
> have to also install some font packages (xorg-x11-fonts-misc for
> Fedora), and Emacs uses the 8x13 font for Cyrillic characters (so the
> problem is less obvious). urxvt still uses the correct font there.
> 
> >> I have found that evaluating the Sexpr:
> >>
> >> (set-fontset-font "fontset-default" 'unicode "-misc-fixed-*-*-*--13-*-*-*-*-70-iso10646-1")
> >>
> > 
> > Don't do that, this setting will cause all kinds of surprises.  To
> > customize the fonts, use specific character codepoint ranges, and only
> > specify font for a range if the font support most or all of that
> > range.
> 
> Noted. However, immediately lacking a better concrete workaround, I
> don't have a better option at my disposal.

Hopefully, Handa-san will have some advice here.

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Vladimir Panteleev]

On 2016-09-29 15:30, Eli Zaretskii wrote:
> If you start Emacs with 12x13 font or 8x13, does Emacs display more
> characters than 7x13 before it switches to another font?

On my system, with `--font 8x13', I still see Cyrillic characters
rendered in 12x13.

Curiously, `--font 12x13' doesn't work (Emacs prints "Font ‘12x13’ is
not defined"). The only font matching that metric is
"-misc-fixed-medium-r-normal-ja-13-120-75-75-c-120-iso10646-1", I can
only guess that Emacs doesn't match it because its "adstyl" field is
"ja". With `--font "-*-*-*-*-*-*-13-*-*-*-*-120-*-*"', Emacs starts with
an evidently-fallback proportional font, not sure which one (possibly
Bitstream Vera Serif), however it does use the 12x13 bitmap font with
`--font "-misc-*-*-*-*-*-13-*-*-*-*-120-*-*"'.

Here are some screenshots:

https://dump.v.panteleev.md/a4234287449583e2b5a7152437ac5504/screenshot.png
https://dump.v.panteleev.md/61757322edb41551e1ffa7ccc3941719/screenshot.png
https://dump.v.panteleev.md/3b989295b09073b22dddd823d431bc64/screenshot.png

To answer your question: It is difficult to say since it's clear that
some characters only appear in some fonts, but at the very least,
Cyrillic characters are always rendered at 12x13 with the fonts I tried.

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Eli Zaretskii]

Ping!

> Date: Thu, 29 Sep 2016 18:30:37 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 24560@debbugs.gnu.org
> 
> Adding Handa-san to this discussion, in the hope that he could shed
> some light on this issue.
> 
> > From: Vladimir Panteleev <thecybershadow@gmail.com>
> > Cc: 24560@debbugs.gnu.org
> > Date: Wed, 28 Sep 2016 21:58:20 +0000
> > 
> > On 2016-09-28 20:16, Eli Zaretskii wrote:
> > > Are you sure the 7x13 font installed on your system supports all those
> > > characters?  Emacs won't switch to a different font if the default
> > > font supports the character it needs to display.
> > 
> > As I mentioned in my bug report and as is visible in the screenshots,
> > the characters are indeed present in the font and are rendered correctly
> > in other applications. Only Emacs exhibits this problem.
> > 
> > > Or could it be that some customizations, either yours or site-wide,
> > > force Emacs to use other fonts for the above characters?
> > 
> > This problem is reproducible with "emacs -Q", without any
> > customizations, on all distributions I've tried.
> 
> Then I guess there's some factor at work here that I don't quite
> figure out.  Maybe the Emacs font selection algorithm doesn't like
> that font for some reason?
> 
> If you start Emacs with 12x13 font or 8x13, does Emacs display more
> characters than 7x13 before it switches to another font?
> 
> > Here is a complete recipe using a live CD, to remove any possibility of
> > user or site customization:
> > 
> > 1. Get the Antergos live ISO (
> > https://antergos.com/download/antergos-live-iso/ ) and boot from it
> > (e.g. in a VM).
> > 2. Choose "Start Antergos live".
> > 3. Choose "Try It".
> > 4. Install Emacs and urxvt (in a terminal, run "pacman -S emacs
> > rxvt-unicode").
> > 5. Run "emacs -Q --font 7x13".
> > 6. M-x view-hello-file
> > 7. Observe that Cyrillic characters are rendered in a 12x13 font.
> > 8. Run "urxvt -fn 7x13".
> > 9. Copy some Cyrillic characters from Emacs and paste them in urxvt.
> > 10. Observe that urxvt renders the Cyrillic characters correctly, in the
> > correct font.
> > 
> > This is not specific to Arch / Antegros: The problem is also
> > reproducible on Ubuntu and Fedora Workstation Live CDs, though you may
> > have to also install some font packages (xorg-x11-fonts-misc for
> > Fedora), and Emacs uses the 8x13 font for Cyrillic characters (so the
> > problem is less obvious). urxvt still uses the correct font there.
> > 
> > >> I have found that evaluating the Sexpr:
> > >>
> > >> (set-fontset-font "fontset-default" 'unicode "-misc-fixed-*-*-*--13-*-*-*-*-70-iso10646-1")
> > >>
> > > 
> > > Don't do that, this setting will cause all kinds of surprises.  To
> > > customize the fonts, use specific character codepoint ranges, and only
> > > specify font for a range if the font support most or all of that
> > > range.
> > 
> > Noted. However, immediately lacking a better concrete workaround, I
> > don't have a better option at my disposal.
> 
> Hopefully, Handa-san will have some advice here.

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Eli Zaretskii]

Ping!  Ping!

Kenichi, could you please comment on this issue?

> Date: Sat, 08 Oct 2016 16:07:34 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 24560@debbugs.gnu.org, thecybershadow@gmail.com
> 
> Ping!
> 
> > Date: Thu, 29 Sep 2016 18:30:37 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: 24560@debbugs.gnu.org
> > 
> > Adding Handa-san to this discussion, in the hope that he could shed
> > some light on this issue.
> > 
> > > From: Vladimir Panteleev <thecybershadow@gmail.com>
> > > Cc: 24560@debbugs.gnu.org
> > > Date: Wed, 28 Sep 2016 21:58:20 +0000
> > > 
> > > On 2016-09-28 20:16, Eli Zaretskii wrote:
> > > > Are you sure the 7x13 font installed on your system supports all those
> > > > characters?  Emacs won't switch to a different font if the default
> > > > font supports the character it needs to display.
> > > 
> > > As I mentioned in my bug report and as is visible in the screenshots,
> > > the characters are indeed present in the font and are rendered correctly
> > > in other applications. Only Emacs exhibits this problem.
> > > 
> > > > Or could it be that some customizations, either yours or site-wide,
> > > > force Emacs to use other fonts for the above characters?
> > > 
> > > This problem is reproducible with "emacs -Q", without any
> > > customizations, on all distributions I've tried.
> > 
> > Then I guess there's some factor at work here that I don't quite
> > figure out.  Maybe the Emacs font selection algorithm doesn't like
> > that font for some reason?
> > 
> > If you start Emacs with 12x13 font or 8x13, does Emacs display more
> > characters than 7x13 before it switches to another font?
> > 
> > > Here is a complete recipe using a live CD, to remove any possibility of
> > > user or site customization:
> > > 
> > > 1. Get the Antergos live ISO (
> > > https://antergos.com/download/antergos-live-iso/ ) and boot from it
> > > (e.g. in a VM).
> > > 2. Choose "Start Antergos live".
> > > 3. Choose "Try It".
> > > 4. Install Emacs and urxvt (in a terminal, run "pacman -S emacs
> > > rxvt-unicode").
> > > 5. Run "emacs -Q --font 7x13".
> > > 6. M-x view-hello-file
> > > 7. Observe that Cyrillic characters are rendered in a 12x13 font.
> > > 8. Run "urxvt -fn 7x13".
> > > 9. Copy some Cyrillic characters from Emacs and paste them in urxvt.
> > > 10. Observe that urxvt renders the Cyrillic characters correctly, in the
> > > correct font.
> > > 
> > > This is not specific to Arch / Antegros: The problem is also
> > > reproducible on Ubuntu and Fedora Workstation Live CDs, though you may
> > > have to also install some font packages (xorg-x11-fonts-misc for
> > > Fedora), and Emacs uses the 8x13 font for Cyrillic characters (so the
> > > problem is less obvious). urxvt still uses the correct font there.
> > > 
> > > >> I have found that evaluating the Sexpr:
> > > >>
> > > >> (set-fontset-font "fontset-default" 'unicode "-misc-fixed-*-*-*--13-*-*-*-*-70-iso10646-1")
> > > >>
> > > > 
> > > > Don't do that, this setting will cause all kinds of surprises.  To
> > > > customize the fonts, use specific character codepoint ranges, and only
> > > > specify font for a range if the font support most or all of that
> > > > range.
> > > 
> > > Noted. However, immediately lacking a better concrete workaround, I
> > > don't have a better option at my disposal.
> > 
> > Hopefully, Handa-san will have some advice here.
> 
> 
> 
> 

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Eli Zaretskii]

Ping!  Ping!

> Date: Sat, 15 Oct 2016 17:10:41 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 24560@debbugs.gnu.org, thecybershadow@gmail.com
> 
> Ping!  Ping!
> 
> Kenichi, could you please comment on this issue?
> 
> > Date: Sat, 08 Oct 2016 16:07:34 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: 24560@debbugs.gnu.org, thecybershadow@gmail.com
> > 
> > Ping!
> > 
> > > Date: Thu, 29 Sep 2016 18:30:37 +0300
> > > From: Eli Zaretskii <eliz@gnu.org>
> > > Cc: 24560@debbugs.gnu.org
> > > 
> > > Adding Handa-san to this discussion, in the hope that he could shed
> > > some light on this issue.
> > > 
> > > > From: Vladimir Panteleev <thecybershadow@gmail.com>
> > > > Cc: 24560@debbugs.gnu.org
> > > > Date: Wed, 28 Sep 2016 21:58:20 +0000
> > > > 
> > > > On 2016-09-28 20:16, Eli Zaretskii wrote:
> > > > > Are you sure the 7x13 font installed on your system supports all those
> > > > > characters?  Emacs won't switch to a different font if the default
> > > > > font supports the character it needs to display.
> > > > 
> > > > As I mentioned in my bug report and as is visible in the screenshots,
> > > > the characters are indeed present in the font and are rendered correctly
> > > > in other applications. Only Emacs exhibits this problem.
> > > > 
> > > > > Or could it be that some customizations, either yours or site-wide,
> > > > > force Emacs to use other fonts for the above characters?
> > > > 
> > > > This problem is reproducible with "emacs -Q", without any
> > > > customizations, on all distributions I've tried.
> > > 
> > > Then I guess there's some factor at work here that I don't quite
> > > figure out.  Maybe the Emacs font selection algorithm doesn't like
> > > that font for some reason?
> > > 
> > > If you start Emacs with 12x13 font or 8x13, does Emacs display more
> > > characters than 7x13 before it switches to another font?
> > > 
> > > > Here is a complete recipe using a live CD, to remove any possibility of
> > > > user or site customization:
> > > > 
> > > > 1. Get the Antergos live ISO (
> > > > https://antergos.com/download/antergos-live-iso/ ) and boot from it
> > > > (e.g. in a VM).
> > > > 2. Choose "Start Antergos live".
> > > > 3. Choose "Try It".
> > > > 4. Install Emacs and urxvt (in a terminal, run "pacman -S emacs
> > > > rxvt-unicode").
> > > > 5. Run "emacs -Q --font 7x13".
> > > > 6. M-x view-hello-file
> > > > 7. Observe that Cyrillic characters are rendered in a 12x13 font.
> > > > 8. Run "urxvt -fn 7x13".
> > > > 9. Copy some Cyrillic characters from Emacs and paste them in urxvt.
> > > > 10. Observe that urxvt renders the Cyrillic characters correctly, in the
> > > > correct font.
> > > > 
> > > > This is not specific to Arch / Antegros: The problem is also
> > > > reproducible on Ubuntu and Fedora Workstation Live CDs, though you may
> > > > have to also install some font packages (xorg-x11-fonts-misc for
> > > > Fedora), and Emacs uses the 8x13 font for Cyrillic characters (so the
> > > > problem is less obvious). urxvt still uses the correct font there.
> > > > 
> > > > >> I have found that evaluating the Sexpr:
> > > > >>
> > > > >> (set-fontset-font "fontset-default" 'unicode "-misc-fixed-*-*-*--13-*-*-*-*-70-iso10646-1")
> > > > >>
> > > > > 
> > > > > Don't do that, this setting will cause all kinds of surprises.  To
> > > > > customize the fonts, use specific character codepoint ranges, and only
> > > > > specify font for a range if the font support most or all of that
> > > > > range.
> > > > 
> > > > Noted. However, immediately lacking a better concrete workaround, I
> > > > don't have a better option at my disposal.
> > > 
> > > Hopefully, Handa-san will have some advice here.
> > 
> > 
> > 
> > 
> 
> 
> 
> 

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Lars Ingebrigtsen]

Vladimir Panteleev <thecybershadow@gmail.com> writes:

> On a GNU/Linux system running X.Org, if Emacs is set to use the X11
> fixed font 7x13 (e.g. by specifying `--font 7x13' on the command line),
> Emacs will use the incorrect font for Unicode (non-ASCII)
> characters. Specifically:
>
> - US-ASCII characters are rendered using the correct font.
>
> - Cyrillic (e.g. letters of the Russian alphabet) and some other
>   characters (e.g. '¢' (CENT SIGN)) are rendered with the 12x13 font
>   instead of the indicated 7x13 font.

I tried this in Debian with Emacs 26 and 27, and inserted a CENT SIGN,
and it was displayed correctly:

    x:-misc-fixed-medium-r-normal--13-120-75-75-c-80-iso8859-1 (#xA2)

(which is the same as all the other characters).

Are you still seeing this problem in modern versions of Emacs?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Vladimir Panteleev]

[-- Attachment #1: Type: text/plain, Size: 272 bytes --]

On Sun, 17 Nov 2019 at 07:21, Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Are you still seeing this problem in modern versions of Emacs?
>

Yes:

https://dump.v.panteleev.md/35de394ba1a41d8512a94d938abb6458/2019-11-17%2013%3A07%3A35.mp4

Emacs 26.3 on Arch Linux x86_64.

[-- Attachment #2: Type: text/html, Size: 745 bytes --]

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Eli Zaretskii]

> From: Vladimir Panteleev <thecybershadow@gmail.com>
> Date: Sun, 17 Nov 2019 13:10:27 +0000
> Cc: 24560@debbugs.gnu.org
> 
> On Sun, 17 Nov 2019 at 07:21, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> 
>  Are you still seeing this problem in modern versions of Emacs?
> 
> Yes:

It sounds like something is wrong with the 7x13 font or with how Emacs
handles these fonts, because there are several bug reports that
basically say the same thing.

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Lars Ingebrigtsen]

[-- Attachment #1: Type: text/plain, Size: 769 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> It sounds like something is wrong with the 7x13 font or with how Emacs
> handles these fonts, because there are several bug reports that
> basically say the same thing.

I wonder why I can't reproduce it.  Hm.  The xft/harfbuzz thing
shouldn't interfere here, should it?  That's a bitfont?

I tried 

./configure --with-xft --without-harfbuzz

and the CENT SIGN was of the same size as the ASCII characters, and

./configure --without-xft --with-harfbuzz

...

Hang on, the latter gives me

  Does Emacs use HarfBuzz?                                no

but just ./configure gives me

  Does Emacs use HarfBuzz?                                yes

Sounds like a bug in the autoconfig stuff.

Anyway, I get the same result there:


[-- Attachment #2: screen6xpL9A.jpg --]
[-- Type: image/jpeg, Size: 904 bytes --]

[-- Attachment #3: Type: text/plain, Size: 341 bytes --]


One more detail.  When inserting CENT SIGN, I get this warning:

(emacs:29060): Gtk-CRITICAL **: 17:12:48.023: gtk_distribute_natural_allocation: assertion 'extra_space >= 0' failed

If I use a different font, I don't get that warning.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Vladimir Panteleev <thecybershadow@gmail.com>,  24560@debbugs.gnu.org
> Date: Sun, 17 Nov 2019 17:26:18 +0100
> 
> I wonder why I can't reproduce it.  Hm.  The xft/harfbuzz thing
> shouldn't interfere here, should it?  That's a bitfont?

AFAIK, HarfBuzz can use any font at all.  But I don't think it matters
here, because we don't call HarfBuzz functions for "simple" characters.

> ./configure --without-xft --with-harfbuzz
> 
> ...
> 
> Hang on, the latter gives me
> 
>   Does Emacs use HarfBuzz?                                no
> 
> but just ./configure gives me
> 
>   Does Emacs use HarfBuzz?                                yes
> 
> Sounds like a bug in the autoconfig stuff.

Yes.

> Anyway, I get the same result there:

Same as what?

> One more detail.  When inserting CENT SIGN, I get this warning:
> 
> (emacs:29060): Gtk-CRITICAL **: 17:12:48.023: gtk_distribute_natural_allocation: assertion 'extra_space >= 0' failed
> 
> If I use a different font, I don't get that warning.

If someone knows, or cn find out, what this means, please tell.

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> Anyway, I get the same result there:
>
> Same as what?

Same result with an xft Emacs as with a harfbuzz Emacs (i.e., I can't
reproduce the bug with the 7x13 font on my Debian "buster" system).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> Sounds like a bug in the autoconfig stuff.
>
> Yes.

The logic is a bit confused.

    if test "x${with_xft}" != "xno"; then
      EMACS_CHECK_MODULES([XFT], [xft >= 0.13.0], [], [HAVE_XFT=no])

[...]

    if test "$HAVE_XFT" != "yes"; then
       dnl For the "Does Emacs use" message at the end.
       HAVE_XFT=no
       HAVE_FREETYPE=no
    else
       dnl Strict linkers fail with
       dnl ftfont.o: undefined reference to symbol 'FT_New_Face'
       dnl if -lfreetype is not specified.
       dnl The following is needed to set FREETYPE_LIBS.
       EMACS_CHECK_MODULES([FREETYPE], [freetype2])

       test "$HAVE_FREETYPE" = "no" && AC_MSG_ERROR(libxft requires libfreetype)
    fi
  fi				  # $HAVE_CAIRO != yes

So if somebody says explicitly --without-xft, then HAVE_XFT won't be
"yes", and we set HAVE_FREETYPE to "no" without checking.  And harfbuzz
depends on HAVE_FREETYPE being "yes", so harfbuzz depends on xft being
available, too.

Is this correct behaviour?

In addition, I think the configure script should fail if you say
--with-harfbuzz and harfbuzz isn't available.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: thecybershadow@gmail.com,  24560@debbugs.gnu.org
> Date: Sun, 17 Nov 2019 18:58:26 +0100
> 
> So if somebody says explicitly --without-xft, then HAVE_XFT won't be
> "yes", and we set HAVE_FREETYPE to "no" without checking.  And harfbuzz
> depends on HAVE_FREETYPE being "yes", so harfbuzz depends on xft being
> available, too.
> 
> Is this correct behaviour?

Maybe I'm simply misremembering, and HarfBuzz does need xft.

> In addition, I think the configure script should fail if you say
> --with-harfbuzz and harfbuzz isn't available.

Probably.

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Lars Ingebrigtsen <larsi@gnus.org>
>> Cc: thecybershadow@gmail.com,  24560@debbugs.gnu.org
>> Date: Sun, 17 Nov 2019 18:58:26 +0100
>> 
>> So if somebody says explicitly --without-xft, then HAVE_XFT won't be
>> "yes", and we set HAVE_FREETYPE to "no" without checking.  And harfbuzz
>> depends on HAVE_FREETYPE being "yes", so harfbuzz depends on xft being
>> available, too.
>> 
>> Is this correct behaviour?
>
> Maybe I'm simply misremembering, and HarfBuzz does need xft.

I hacked the configure script to just say

diff --git a/configure.ac b/configure.ac
index 0976b665e6..d42f530407 100644
--- a/configure.ac
+++ b/configure.ac
@@ -3399,6 +3399,9 @@ AC_DEFUN
     fi
   fi				  # $HAVE_CAIRO != yes
 
+  EMACS_CHECK_MODULES([FREETYPE], [freetype2])
+  test "$HAVE_FREETYPE" = "no" && AC_MSG_ERROR(libxft requires libfreetype)
+
   HAVE_LIBOTF=no
   if test "${HAVE_FREETYPE}" = "yes"; then
     AC_DEFINE(HAVE_FREETYPE, 1,

to force

  Does Emacs use HarfBuzz?                                yes
  Does Emacs use -lm17n-flt?                              yes
  Does Emacs use -lotf?                                   yes
  Does Emacs use -lxft?                                   no

and compilation does indeed fail:

In file included from ftfont.c:46:
ftfont.h:77:3: error: unknown type name ‘XftFont’
   XftFont *xftfont;
   ^~~~~~~

>> In addition, I think the configure script should fail if you say
>> --with-harfbuzz and harfbuzz isn't available.
>
> Probably.

I've now added that, at least.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> One more detail.  When inserting CENT SIGN, I get this warning:
>> 
>> (emacs:29060): Gtk-CRITICAL **: 17:12:48.023:
>> gtk_distribute_natural_allocation: assertion 'extra_space >= 0'
>> failed
>> 
>> If I use a different font, I don't get that warning.
>
> If someone knows, or cn find out, what this means, please tell.

There's something about the same warning in the PROBLEMS file, although
I can't see the frame changing size when inserting the CENT SIGN
character.

So perhaps it's a separate issue?

----

*** Emacs built with GTK+ toolkit can unexpectedly widen frames

This resizing takes place when a frame is not wide enough to accommodate
its entire menu bar.  Typically, it occurs when switching buffers or
changing a buffer's major mode and the new mode adds entries to the menu
bar.  The frame is then widened by the window manager so that the menu
bar is fully shown.  Subsequently switching to another buffer or
changing the buffer's mode will not shrink the frame back to its
previous width.  The height of the frame remains unaltered.  Apparently,
the failure is also dependent on the chosen font.

The resizing is usually accompanied by console output like

Gtk-CRITICAL **: gtk_distribute_natural_allocation: assertion 'extra_space >= 0' failed

It's not clear whether the GTK version used has any impact on the
occurrence of the failure.  So far, the failure has been observed with
GTK+ versions 3.4.2, 3.14.5 and 3.18.7.  However, another 3.4.2 build
does not exhibit the bug.


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Robert Pluim]

>>>>> On Sun, 17 Nov 2019 18:58:26 +0100, Lars Ingebrigtsen <larsi@gnus.org> said:
    Lars> So if somebody says explicitly --without-xft, then HAVE_XFT won't be
    Lars> "yes", and we set HAVE_FREETYPE to "no" without checking.  And harfbuzz
    Lars> depends on HAVE_FREETYPE being "yes", so harfbuzz depends on xft being
    Lars> available, too.

Not quite. If you configure '--with-xft=no --with-cairo' you'll get
HarfBuzz and freetype.

Robert

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Not quite. If you configure '--with-xft=no --with-cairo' you'll get
> HarfBuzz and freetype.

Thanks.

I tried reproducing the 7x13 bug with that configuration, but I still
don't see the bug.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I tried reproducing the 7x13 bug with that configuration, but I still
> don't see the bug.

Vladimir, can you reproduce this issue with recent versions of Emacs
(and the OS you're using)?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24560: 25.1; With `--font 7x13', wrong font is used for Unicode characters

[Vladimir Panteleev]

Hi Lars,

Yes, the issue is still there.

If it helps, I posted a recipe for how to reproduce the issue from
scratch in a virtual machine earlier in this thread. In theory it
should allow anyone to reproduce the issue.

Thanks,
- Vladimir

On Sat, 23 Apr 2022 at 14:02, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>
> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
> > I tried reproducing the 7x13 bug with that configuration, but I still
> > don't see the bug.
>
> Vladimir, can you reproduce this issue with recent versions of Emacs
> (and the OS you're using)?
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no