bug#35079: 26.1; Font rendering - problem with bdf fonts

bug#35079: 26.1; Font rendering - problem with bdf fonts

[Max Xiong]

[-- Attachment #1: Type: text/plain, Size: 4496 bytes --]

When loading a bdf or a pcf font, specifically unifont in my case, the
width of characters is incorrect.

To reproduce:
Download the bdf.gz file from the unifont website at
http://unifoundry.com/unifont/index.html
Change the extension to .bdf
Install the font. In my case, I needed to copy it to
~/.local/share/fonts/misc/
Run fc-cache -fv in the terminal.
Start emacs.
From the menu bar, choose options -> select default font.
Search for and choose Unifont Sans Serif size 12 (size should go to 12
anyways).

The unexpected result is that each character is rendered with a full width,
as if there is a space after it. The cursor becomes square instead of a
half-width rectangle.

Additionally, note that this does NOT occur with the ttf version of
unifont. Nor does this bug occur when I use the bdf unifont inside the
gnome terminal.

In GNU Emacs 26.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2018-05-29 built on lcy01-amd64-029
Windowing system distributor 'The X.Org Foundation', version 11.0.12001000
System Description: Ubuntu 18.04.2 LTS

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --build=x86_64-linux-gnu --prefix=/usr
 '--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
 '--infodir=${prefix}/share/info' --sysconfdir=/etc --localstatedir=/var
 --disable-silent-rules '--libdir=${prefix}/lib/x86_64-linux-gnu'
 '--libexecdir=${prefix}/lib/x86_64-linux-gnu' --disable-maintainer-mode
 --disable-dependency-tracking --prefix=/usr --sharedstatedir=/var/lib
 --program-suffix=26 --with-modules --with-file-notification=inotify
 --with-mailutils --with-x=yes --with-x-toolkit=gtk3 --with-xwidgets
 --with-lcms2 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs26-pCvJBp/emacs26-26.1~1.git07f8f9b=.
-fstack-protector-strong
 -Wformat -Werror=format-security -no-pie' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro
 -no-pie''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 MODULES THREADS XWIDGETS LIBSYSTEMD LCMS2

Important settings:
  value of $LC_CTYPE: en_CA.UTF-8
  value of $LANG: en_CA.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils elec-pair time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting xwidget-internal
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 95503 6338)
 (symbols 48 20395 1)
 (miscs 40 47 118)
 (strings 32 28318 1256)
 (string-bytes 1 742574)
 (vectors 16 14677)
 (vector-slots 8 498364 9328)
 (floats 8 50 144)
 (intervals 56 253 0)
 (buffers 992 11))

[-- Attachment #2: Type: text/html, Size: 5886 bytes --]

bug#35079: 26.1; Font rendering - problem with bdf fonts

[Eli Zaretskii]

> From: Max Xiong <xsrvmy@gmail.com>
> Date: Tue, 2 Apr 2019 01:31:09 -0400
> 
> When loading a bdf or a pcf font, specifically unifont in my case, the width of characters is incorrect.

Does this happen only with Unifont, or with any BDF font?

> To reproduce:
> Download the bdf.gz file from the unifont website at http://unifoundry.com/unifont/index.html
> Change the extension to .bdf
> Install the font. In my case, I needed to copy it to ~/.local/share/fonts/misc/
> Run fc-cache -fv in the terminal.
> Start emacs.
> From the menu bar, choose options -> select default font.
> Search for and choose Unifont Sans Serif size 12 (size should go to 12 anyways).
> 
> The unexpected result is that each character is rendered with a full width, as if there is a space after it. The
> cursor becomes square instead of a half-width rectangle.

This could be the realization of the danger described in the
commentary to ftfont.c:get_adstyle_property, or some other issue with
BDF fonts.  I don't think they are used widely anymore, so maybe our
code accrued some bitrot.

> Additionally, note that this does NOT occur with the ttf version of unifont. Nor does this bug occur when I use
> the bdf unifont inside the gnome terminal.

CC'ing Handa-san, in the hope that he could look into this issue.

Thanks.

bug#35079: 26.1; Font rendering - problem with bdf fonts

[K.Handa]

In article <83sgv0foc3.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:

> > From: Max Xiong <xsrvmy@gmail.com>
> > Date: Tue, 2 Apr 2019 01:31:09 -0400
> > 
> > When loading a bdf or a pcf font, specifically unifont in my case, the width of characters is incorrect.

> Does this happen only with Unifont, or with any BDF font?

> > To reproduce:
> > Download the bdf.gz file from the unifont website at http://unifoundry.com/unifont/index.html
> > Change the extension to .bdf

There are two ways to use that font in Emacs.
(1) install that font as X bitmap font, and use it via 'x' font-backend.

(make-frame
 '((frame-parameter . (x xft))
   (font . "-gnu-unifont-medium-r-normal-sans-16-160-75-75-c-80-iso10646-1")))

The reason for setting frame-parameter is that the default is (xft x)
and thus that font is found via xft font-backend and via fontconfig.

This way, Emacs shows characters with correct width.

(2) install that font as you did, and use it via 'xft' font-backend.

For this, I found that fontconfig gives 'spacing' parameter of that font
as FC_CHARCELL.  I think the spacing of that font should be FC_DUAL.
Anyway, src/xftfont.c of Emacs does not know about FC_CHARCELL.  So,
emacs treats that font as a mono-space font.

I think you can modify the spacing paramter of that font to FC_DUAL by
writing some code in ~/.fonts.conf.

Or, please try to apply this patch to src/xftfont.c, and rebuild emacs.

diff --git a/src/xftfont.c b/src/xftfont.c
index 8a4516f7f9..bb15a83685 100644
--- a/src/xftfont.c
+++ b/src/xftfont.c
@@ -349,7 +349,7 @@ xftfont_open (struct frame *f, Lisp_Object entity, int pixel_size)
 
   if (spacing != FC_PROPORTIONAL
 #ifdef FC_DUAL
-      && spacing != FC_DUAL
+      && spacing != FC_DUAL && spacing != FC_CHARCELL
 #endif	/* FC_DUAL */
       )
     {

I don't know the exact meaning of FC_CHARCELL.  I could't find it on the
Internet.  Does anyone know what it means and why fontconfig uses it for
the gnu unifont?

---
K. Handa
handa@gnu.org

> > Install the font. In my case, I needed to copy it to ~/.local/share/fonts/misc/
> > Run fc-cache -fv in the terminal.
> > Start emacs.
> > From the menu bar, choose options -> select default font.
> > Search for and choose Unifont Sans Serif size 12 (size should go to 12 anyways).
> > 
> > The unexpected result is that each character is rendered with a full width, as if there is a space after it. The
> > cursor becomes square instead of a half-width rectangle.

> This could be the realization of the danger described in the
> commentary to ftfont.c:get_adstyle_property, or some other issue with
> BDF fonts.  I don't think they are used widely anymore, so maybe our
> code accrued some bitrot.

> > Additionally, note that this does NOT occur with the ttf version of unifont. Nor does this bug occur when I use
> > the bdf unifont inside the gnome terminal.

> CC'ing Handa-san, in the hope that he could look into this issue.

> Thanks.

bug#35079: 26.1; Font rendering - problem with bdf fonts

[Eli Zaretskii]

> From: handa.kenichi@aist.go.jp (K.Handa)
> Cc: xsrvmy@gmail.com, 35079@debbugs.gnu.org, handa@gnu.org
> Date: Sat, 13 Apr 2019 19:35:55 +0900
> 
> For this, I found that fontconfig gives 'spacing' parameter of that font
> as FC_CHARCELL.  I think the spacing of that font should be FC_DUAL.
> Anyway, src/xftfont.c of Emacs does not know about FC_CHARCELL.  So,
> emacs treats that font as a mono-space font.
> 
> I think you can modify the spacing paramter of that font to FC_DUAL by
> writing some code in ~/.fonts.conf.
> 
> Or, please try to apply this patch to src/xftfont.c, and rebuild emacs.

Should I install this change now?  Can it do any harm?

> I don't know the exact meaning of FC_CHARCELL.  I could't find it on the
> Internet.  Does anyone know what it means and why fontconfig uses it for
> the gnu unifont?

I found this in
https://github.com/freedesktop/xorg-libXft/blob/master/NEWS:

  Version 2.1.7

  Change FC_CHARCELL and FC_MONO interpretation.  FC_MONO no longer clips
  glyphs to charcell, you must specify FC_CHARCELL for that.

But I guess you saw this too?  It sounds like FC_CHARCELL is the same
as monospaced, but with clipping to character cell, whatever that may
mean.

Thanks.

bug#35079: 26.1; Font rendering - problem with bdf fonts

[handa]

In article <837eb8nddt.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:

> > For this, I found that fontconfig gives 'spacing' parameter of that font
> > as FC_CHARCELL.  I think the spacing of that font should be FC_DUAL.
> > Anyway, src/xftfont.c of Emacs does not know about FC_CHARCELL.  So,
> > emacs treats that font as a mono-space font.
> > 
> > I think you can modify the spacing paramter of that font to FC_DUAL by
> > writing some code in ~/.fonts.conf.
> > 
> > Or, please try to apply this patch to src/xftfont.c, and rebuild emacs.

> Should I install this change now?  Can it do any harm?

I can't tell because I don't know the exact meaning of FC_CHARCELL.

> I found this in
> https://github.com/freedesktop/xorg-libXft/blob/master/NEWS:

>   Version 2.1.7

>   Change FC_CHARCELL and FC_MONO interpretation.  FC_MONO no longer clips
>   glyphs to charcell, you must specify FC_CHARCELL for that.

> But I guess you saw this too?  It sounds like FC_CHARCELL is the same
> as monospaced, but with clipping to character cell, whatever that may
> mean.

I noticed that page, but I did not understand what it means.  If
spacing of FC_CHARCELL is the same as FC_MONO, why is it used for
Unifont which is surely not mono-spaced?

---
K. Handa
handa@gnu.org

bug#35079: 26.1; Font rendering - problem with bdf fonts

[Lars Ingebrigtsen]

handa.kenichi@aist.go.jp (K.Handa) writes:

> Or, please try to apply this patch to src/xftfont.c, and rebuild emacs.
>
> diff --git a/src/xftfont.c b/src/xftfont.c
> index 8a4516f7f9..bb15a83685 100644
> --- a/src/xftfont.c
> +++ b/src/xftfont.c
> @@ -349,7 +349,7 @@ xftfont_open (struct frame *f, Lisp_Object entity, int pixel_size)
>
>    if (spacing != FC_PROPORTIONAL
>  #ifdef FC_DUAL
> -      && spacing != FC_DUAL
> +      && spacing != FC_DUAL && spacing != FC_CHARCELL
>  #endif	/* FC_DUAL */
>        )
>      {
>
> I don't know the exact meaning of FC_CHARCELL.  I could't find it on the
> Internet.  Does anyone know what it means and why fontconfig uses it for
> the gnu unifont?

It seems like many people are wondering what it means, but

https://gitlab.freedesktop.org/fontconfig/fontconfig/issues/176#note_221223

says

"FC_DUAL should be an alias to FC_CHARCELL and use FC_CHARCELL in this
case too perhaps"

So it seems like the solution here is possibly correct (although perhaps
there should be an #ifdef FC_CHARCELL there, too).

It seems like it shouldn't hurt, at least.

Opinions?  If nobody objects, then perhaps the following could be
installed:

diff --git a/src/xftfont.c b/src/xftfont.c
index e003580817..fa06d96736 100644
--- a/src/xftfont.c
+++ b/src/xftfont.c
@@ -211,6 +211,9 @@ xftfont_open (struct frame *f, Lisp_Object entity, int pixel_size)
 #ifdef FC_DUAL
       && spacing != FC_DUAL
 #endif	/* FC_DUAL */
+#ifdef FC_CHARCELL
+      && spacing != FC_CHARCELL
+#endif	/* FC_CHARCELL */
       )
     {
       font->min_width = font->max_width = font->average_width


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#35079: 26.1; Font rendering - problem with bdf fonts

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  handa@gnu.org,  xsrvmy@gmail.com,
>   35079@debbugs.gnu.org
> Date: Sun, 17 Nov 2019 10:01:16 +0100
> 
> "FC_DUAL should be an alias to FC_CHARCELL and use FC_CHARCELL in this
> case too perhaps"
> 
> So it seems like the solution here is possibly correct (although perhaps
> there should be an #ifdef FC_CHARCELL there, too).
> 
> It seems like it shouldn't hurt, at least.
> 
> Opinions?  If nobody objects, then perhaps the following could be
> installed:

Please install this, and thanks.

bug#35079: 26.1; Font rendering - problem with bdf fonts

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Please install this, and thanks.

OK; done.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no