bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Vincent Lefevre]

[-- Attachment #1: Type: text/plain, Size: 222 bytes --]


On the following text file:

underline
‾‾‾‾‾‾‾‾‾

(where ‾ is U+203E OVERLINE), the command

  emacs -Q -fn "Fantasque Sans Mono" text

shows an incorrect alignment (see attached screenshot).

[-- Attachment #2: screenshot --]
[-- Type: image/png, Size: 1577 bytes --]

[-- Attachment #3: Type: text/plain, Size: 3100 bytes --]


There is no such issue with xterm.

Tested under Debian/unstable with
  fonts-fantasque-sans 1.7.2~alpha.3~dfsg-2

Debian's package GNU Emacs 26.3 has the same issue.


In GNU Emacs 27.0.91 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20)
 of 2020-05-14 built on zira
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: Debian GNU/Linux bullseye/sid

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --prefix=/usr/local/emacs-27.0.91'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD PDUMPER
LCMS2 GMP

Important settings:
  value of $LC_COLLATE: POSIX
  value of $LC_CTYPE: C.UTF-8
  value of $LC_TIME: en_DK
  value of $LANG: POSIX
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search seq
byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils time-date subr-x
cl-loaddefs cl-lib vc-dispatcher vc-svn tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 48117 9155)
 (symbols 48 6158 1)
 (strings 32 16333 1771)
 (string-bytes 1 543005)
 (vectors 16 11261)
 (vector-slots 8 144865 9168)
 (floats 8 20 40)
 (intervals 56 195 0)
 (buffers 1000 12))

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Robert Pluim]

>>>>> On Thu, 14 May 2020 21:41:13 +0200, Vincent Lefevre <vincent@vinc17.net> said:

    Vincent> On the following text file:

    Vincent> underline
    Vincent> ‾‾‾‾‾‾‾‾‾

    Vincent> (where ‾ is U+203E OVERLINE), the command

    Vincent>   emacs -Q -fn "Fantasque Sans Mono" text

    Vincent> shows an incorrect alignment (see attached screenshot).

Two questions:

    - is Fantasque being used for the U+203E (check with C-u C-x =)
    - does this same problem happen if you build emacs 27 with Cairo
    enabled? (you might need to install some cairo development
    packages)

Robert

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Vincent Lefevre]

On 2020-05-15 11:57:16 +0200, Robert Pluim wrote:
> >>>>> On Thu, 14 May 2020 21:41:13 +0200, Vincent Lefevre <vincent@vinc17.net> said:
> 
>     Vincent> On the following text file:
> 
>     Vincent> underline
>     Vincent> ‾‾‾‾‾‾‾‾‾
> 
>     Vincent> (where ‾ is U+203E OVERLINE), the command
> 
>     Vincent>   emacs -Q -fn "Fantasque Sans Mono" text
> 
>     Vincent> shows an incorrect alignment (see attached screenshot).
> 
> Two questions:
> 
>     - is Fantasque being used for the U+203E (check with C-u C-x =)

No:

    xfthb:-PfEd-Linux Libertine Display O-normal-normal-normal-*-22-*-*-*-*-0-iso10646-1 (#x619)

So I suppose that XTerm does a better replacement.

>     - does this same problem happen if you build emacs 27 with Cairo
>     enabled? (you might need to install some cairo development
>     packages)

Same problem.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Eli Zaretskii]

> Date: Fri, 15 May 2020 12:28:48 +0200
> From: Vincent Lefevre <vincent@vinc17.net>
> Cc: 41278@debbugs.gnu.org
> 
> >     - is Fantasque being used for the U+203E (check with C-u C-x =)
> 
> No:
> 
>     xfthb:-PfEd-Linux Libertine Display O-normal-normal-normal-*-22-*-*-*-*-0-iso10646-1 (#x619)
> 
> So I suppose that XTerm does a better replacement.

Does xterm support variable-pitch fonts?  If not, you will always see
alignment in xterm, but not always in Emacs.

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Robert Pluim]

>>>>> On Fri, 15 May 2020 14:25:03 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> Date: Fri, 15 May 2020 12:28:48 +0200
    >> From: Vincent Lefevre <vincent@vinc17.net>
    >> Cc: 41278@debbugs.gnu.org
    >> 
    >> >     - is Fantasque being used for the U+203E (check with C-u C-x =)
    >> 
    >> No:
    >> 
    >> xfthb:-PfEd-Linux Libertine Display O-normal-normal-normal-*-22-*-*-*-*-0-iso10646-1 (#x619)
    >> 
    >> So I suppose that XTerm does a better replacement.

    Eli> Does xterm support variable-pitch fonts?  If not, you will always see
    Eli> alignment in xterm, but not always in Emacs.

Plus Iʼm not sure Fantasque has a glyph for U+203E anyway.

Robert

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Vincent Lefevre <vincent@vinc17.net>,  41278@debbugs.gnu.org
> Date: Fri, 15 May 2020 13:52:07 +0200
> 
> Plus Iʼm not sure Fantasque has a glyph for U+203E anyway.

I'm sure it doesn't.  I checked that and saw there's no glyph.

This is not a bug, but the expected behavior.

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Vincent Lefevre]

On 2020-05-15 14:25:03 +0300, Eli Zaretskii wrote:
> Does xterm support variable-pitch fonts?

Yes, but AFAIK, it systematically regards them as monospaced fonts,
i.e. the cell dimensions are always the same (and double-width
characters take 2 cells).

> If not, you will always see alignment in xterm, but not always in
> Emacs.

I think that if the main font is a monospaced font, Emacs should
honor this choice in font replacements, keeping the cell dimensions
of the main font.

Now, this would solve the alignment problem, but if Emacs just does
that using the current font replacement, there will still be a space
between the U+203E OVERLINE characters. This issue does not occur
with xterm.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Vincent Lefevre]

On 2020-05-15 13:57:39 +0200, Vincent Lefevre wrote:
> On 2020-05-15 14:25:03 +0300, Eli Zaretskii wrote:
> > If not, you will always see alignment in xterm, but not always in
> > Emacs.
> 
> I think that if the main font is a monospaced font, Emacs should
> honor this choice in font replacements, keeping the cell dimensions
> of the main font.
> 
> Now, this would solve the alignment problem, but if Emacs just does
> that using the current font replacement, there will still be a space
> between the U+203E OVERLINE characters. This issue does not occur
> with xterm.

BTW, Noto Mono is also affected by font replacements that don't match
the cell width of the main font, both for U+203E OVERLINE and for
box drawing characters.

Some tests can be done with Markus Kuhn's demo file at
  https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt

In particular:

Box drawing alignment tests:                                          █
                                                                      ▉
  ╔══╦══╗  ┌──┬──┐  ╭──┬──╮  ╭──┬──╮  ┏━━┳━━┓  ┎┒┏┑   ╷  ╻ ┏┯┓ ┌┰┐    ▊ ╱╲╱╲╳╳╳
  ║┌─╨─┐║  │╔═╧═╗│  │╒═╪═╕│  │╓─╁─╖│  ┃┌─╂─┐┃  ┗╃╄┙  ╶┼╴╺╋╸┠┼┨ ┝╋┥    ▋ ╲╱╲╱╳╳╳
  ║│╲ ╱│║  │║   ║│  ││ │ ││  │║ ┃ ║│  ┃│ ╿ │┃  ┍╅╆┓   ╵  ╹ ┗┷┛ └┸┘    ▌ ╱╲╱╲╳╳╳
  ╠╡ ╳ ╞╣  ├╢   ╟┤  ├┼─┼─┼┤  ├╫─╂─╫┤  ┣┿╾┼╼┿┫  ┕┛┖┚     ┌┄┄┐ ╎ ┏┅┅┓ ┋ ▍ ╲╱╲╱╳╳╳
  ║│╱ ╲│║  │║   ║│  ││ │ ││  │║ ┃ ║│  ┃│ ╽ │┃  ░░▒▒▓▓██ ┊  ┆ ╎ ╏  ┇ ┋ ▎
  ║└─╥─┘║  │╚═╤═╝│  │╘═╪═╛│  │╙─╀─╜│  ┃└─╂─┘┃  ░░▒▒▓▓██ ┊  ┆ ╎ ╏  ┇ ┋ ▏
  ╚══╩══╝  └──┴──┘  ╰──┴──╯  ╰──┴──╯  ┗━━┻━━┛  ▗▄▖▛▀▜   └╌╌┘ ╎ ┗╍╍┛ ┋  ▁▂▃▄▅▆▇█
                                               ▝▀▘▙▄▟

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Eli Zaretskii]

> Date: Fri, 15 May 2020 14:24:23 +0200
> From: Vincent Lefevre <vincent@vinc17.net>
> Cc: rpluim@gmail.com, 41278@debbugs.gnu.org
> 
> BTW, Noto Mono is also affected by font replacements that don't match
> the cell width of the main font, both for U+203E OVERLINE and for
> box drawing characters.

Yes, you can easily have unaligned display if you use inappropriate
fonts.

This is not a bug in Emacs.

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Stefan Kangas]

tags 41278 + notabug
close 41278
thanks

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Fri, 15 May 2020 14:24:23 +0200
>> From: Vincent Lefevre <vincent@vinc17.net>
>> Cc: rpluim@gmail.com, 41278@debbugs.gnu.org
>>
>> BTW, Noto Mono is also affected by font replacements that don't match
>> the cell width of the main font, both for U+203E OVERLINE and for
>> box drawing characters.
>
> Yes, you can easily have unaligned display if you use inappropriate
> fonts.
>
> This is not a bug in Emacs.

From reading this thread, the solution here is to use a font which
supports the features you want to use.

I'm therefore closing this bug report.

Best regards,
Stefan Kangas

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Eli Zaretskii]

> Date: Fri, 15 May 2020 13:57:39 +0200
> From: Vincent Lefevre <vincent@vinc17.net>
> Cc: rpluim@gmail.com, 41278@debbugs.gnu.org
> 
> On 2020-05-15 14:25:03 +0300, Eli Zaretskii wrote:
> > Does xterm support variable-pitch fonts?
> 
> Yes, but AFAIK, it systematically regards them as monospaced fonts,
> i.e. the cell dimensions are always the same (and double-width
> characters take 2 cells).

IOW, xterm doesn't support variable-pitch fonts.

> > If not, you will always see alignment in xterm, but not always in
> > Emacs.
> 
> I think that if the main font is a monospaced font, Emacs should
> honor this choice in font replacements, keeping the cell dimensions
> of the main font.

I don't see how Emacs can second-guess what the user wants in this
case.  Mixing different fonts is at the heart of the Emacs display
engine, and having different fonts have different dimensions is one of
the basic features of that.  Users will not appreciate if we will
override the font metrics based on such arbitrary considerations.  You
may want that (then again, this is just one use case, and you could
find out that in other situations even you will want something
different), but other users will not necessarily want the same.

The solution to your problem (assuming you really need this overline
character and cannot use another) is to either find a suitable font
that supports that character as use it as the default, or configure
your fontset so that some other font, which has the same width as the
default one, is used for symbols instead of Linux Libertine.

> Now, this would solve the alignment problem, but if Emacs just does
> that using the current font replacement, there will still be a space
> between the U+203E OVERLINE characters. This issue does not occur
> with xterm.

Maybe xterm uses a different font, in which case you can find what
that font is and configure Emacs to use it.

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Vincent Lefevre]

On 2020-05-15 15:55:14 +0300, Eli Zaretskii wrote:
> > I think that if the main font is a monospaced font, Emacs should
> > honor this choice in font replacements, keeping the cell dimensions
> > of the main font.
> 
> I don't see how Emacs can second-guess what the user wants in this
> case.

Well, at least for the box drawing characters and other alignment
related characters such as U+203E OVERLINE, it can, because the
purpose of such characters is to work by taking alignment into
account, assuming a monospaced font. So, as long as the user uses
a (single) monospaced font for text, such characters should work
as expected. I don't see why a user would want something different.

BTW, such characters can also be handled directly by the application
or library (e.g., this is the case with GNOME Terminal via the VTE
library), which might be one of the reasons why some fonts do not
provide them.

Emacs even takes different box drawing characters in different fonts!
(U+2500 from Tinos, U+256D from Noto Serif CJK TC).

Note that font substitution done by gucharmap seems correct: it
usually chooses DejaVu Sans Mono for box drawing characters (except
for Noto Mono, where it surprisingly chooses DejaVu Sans, but there
does not seem to be any difference anyway). This is actually done
by fontconfig itself. Since it does a better job than Emacs, why
doesn't Emacs let it do the substitution?

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Vincent Lefevre]

On 2020-05-15 12:28:48 +0200, Vincent Lefevre wrote:
> On 2020-05-15 11:57:16 +0200, Robert Pluim wrote:
> >     - is Fantasque being used for the U+203E (check with C-u C-x =)
> 
> No:
> 
>     xfthb:-PfEd-Linux Libertine Display O-normal-normal-normal-*-22-*-*-*-*-0-iso10646-1 (#x619)
> 
> So I suppose that XTerm does a better replacement.

FYI, I can see with "xterm -report-fonts", concerning the font
replacement:

Loaded XftFonts(fallback[fNorm])
                first char:    32
                last char:     120831
                missing-chars: 117478
                present-chars: 3322
        DejaVu Sans Mono-10
        familylang=en
        style=Book
        stylelang=en
        fullname=DejaVu Sans Mono
        fullnamelang=en
        slant=0
        weight=80
        width=100
        pixelsize=18.3333
        spacing=100
        foundry=PfEd
        antialias=True
        hintstyle=1
        hinting=False
        verticallayout=False
        autohint=False
        globaladvance=True
        file=/usr/share/fonts/truetype/dejavu/DejaVuSansMono.ttf
[...]

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

bug#41278: 27.0.91; incorrect U+203E OVERLINE alignment with Fantasque Sans Mono

[Eli Zaretskii]

> From: Vincent Lefevre <vincent@vinc17.net>
> Date: Thu, 14 May 2020 21:41:13 +0200
> 
> On the following text file:
> 
> underline
> ‾‾‾‾‾‾‾‾‾
> 
> (where ‾ is U+203E OVERLINE), the command
> 
>   emacs -Q -fn "Fantasque Sans Mono" text
> 
> shows an incorrect alignment (see attached screenshot).

AFAICT, that font doesn't have a glyph for U+203E, so Emacs is
probably using some other font, which has a different width.  (I
looked in version 1.8.0 of the font, not 1.7.2.)