bug#57976: 29.0.50; Complex emoji have extra spacing

bug#57976: 29.0.50; Complex emoji have extra spacing

[Protesilaos Stavrou]

[-- Attachment #1: Type: text/plain, Size: 1674 bytes --]

Dear maintainers,

With a recent build of Emacs and harfbuzz version 5.2.0 all the complex
emoji have extra spacing after them.  This space is present everywhere
the affected emoji is displayed.  One example is the interface of the
command 'emoji-search'.

Steps to reproduce the attached screenshot on 'emacs -Q':

- Type 'C-x 8 e s'
- Type "face" and then TAB twice to bring up the Completions' buffer.

All the best,
Protesilaos (or simply "Prot")

* * *

In GNU Emacs 29.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version
 3.24.34, cairo version 1.17.6) of 2022-09-21 built on kronos
Repository revision: 1231a601ebe1fd9fe454c504dbeb9267440242e7
Repository branch: makepkg
System Description: Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --mandir=/usr/share/man --with-gameuser=:games
 --without-libotf --without-m17n-flt --without-gconf
 --with-native-compilation --with-pgtk --with-sound=no --without-gpm
 --without-compress-install
 '--program-transform-name=s/\([ec]tags\)/\1.emacs/'
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions
 -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security
 -fstack-clash-protection -fcf-protection'
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK
PNG RSVG SECCOMP SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS WEBP XIM GTK3
ZLIB

Important settings:
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

-- 
Protesilaos Stavrou
https://protesilaos.com

[-- Attachment #2: Screenshot from 2022-09-21 13-08-26.png --]
[-- Type: image/png, Size: 123088 bytes --]

bug#57976: 29.0.50; Complex emoji have extra spacing

[Lars Ingebrigtsen]

[-- Attachment #1: Type: text/plain, Size: 1195 bytes --]

Protesilaos Stavrou <info@protesilaos.com> writes:

> With a recent build of Emacs and harfbuzz version 5.2.0 all the complex
> emoji have extra spacing after them.  This space is present everywhere
> the affected emoji is displayed.  One example is the interface of the
> command 'emoji-search'.
>
> Steps to reproduce the attached screenshot on 'emacs -Q':
>
> - Type 'C-x 8 e s'
> - Type "face" and then TAB twice to bring up the Completions' buffer.

[...]

> In GNU Emacs 29.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version
>  3.24.34, cairo version 1.17.6) of 2022-09-21 built on kronos
> Repository revision: 1231a601ebe1fd9fe454c504dbeb9267440242e7
> Repository branch: makepkg
> System Description: Arch Linux

I don't see this with:

In GNU Emacs 29.0.50 (build 61, x86_64-pc-linux-gnu, GTK+ Version
 3.24.33, cairo version 1.16.0) of 2022-09-16 built on joga
Repository revision: 7d39453fd64e355526291b0ca5672e838de5fb58
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101003
System Description: Ubuntu 22.04.1 LTS

This is with an earlier version of harfbuzz, though.

libharfbuzz-dev/jammy-updates,jammy-security,now 2.7.4-1ubuntu3.1 


[-- Attachment #2: Type: image/png, Size: 98205 bytes --]

bug#57976: 29.0.50; Complex emoji have extra spacing

[Protesilaos Stavrou]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Wed, 21 Sep 2022 12:30:44 +0200
>
> Protesilaos Stavrou <info@protesilaos.com> writes:
>
>> With a recent build of Emacs and harfbuzz version 5.2.0 all the complex
>> emoji have extra spacing after them.  This space is present everywhere
>> the affected emoji is displayed.  One example is the interface of the
>> command 'emoji-search'.
>>
>> Steps to reproduce the attached screenshot on 'emacs -Q':
>>
>> - Type 'C-x 8 e s'
>> - Type "face" and then TAB twice to bring up the Completions' buffer.
>
> [...]
>
>> In GNU Emacs 29.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version
>>  3.24.34, cairo version 1.17.6) of 2022-09-21 built on kronos
>> Repository revision: 1231a601ebe1fd9fe454c504dbeb9267440242e7
>> Repository branch: makepkg
>> System Description: Arch Linux
>
> I don't see this with:
>
> In GNU Emacs 29.0.50 (build 61, x86_64-pc-linux-gnu, GTK+ Version
>  3.24.33, cairo version 1.16.0) of 2022-09-16 built on joga
> Repository revision: 7d39453fd64e355526291b0ca5672e838de5fb58
> Repository branch: master
> Windowing system distributor 'The X.Org Foundation', version 11.0.12101003
> System Description: Ubuntu 22.04.1 LTS
>
> This is with an earlier version of harfbuzz, though.
>
> libharfbuzz-dev/jammy-updates,jammy-security,now 2.7.4-1ubuntu3.1 

I just downgraded to harfbuzz 5.1.0 and restarted Emacs.  The spacing is
fine now.  I suppose this means we should close this bug and report it
to the harfbuzz maintainers.  Though I thought I would bring it to your
attention.

-- 
Protesilaos Stavrou
https://protesilaos.com

bug#57976: 29.0.50; Complex emoji have extra spacing

[Lars Ingebrigtsen]

Protesilaos Stavrou <info@protesilaos.com> writes:

> I just downgraded to harfbuzz 5.1.0 and restarted Emacs.  The spacing is
> fine now.  I suppose this means we should close this bug and report it
> to the harfbuzz maintainers.  Though I thought I would bring it to your
> attention.

Yes, if you could bring it to their attention, it would be nice, but I'd
rather keep this bug report open until there's an answer -- perhaps
there's something we have to adjust in the code on our side with newer
harfbuzz versions?

If this is not an actual bug in harfbuzz, perhaps the harfbuzz
developers have some advice here...

bug#57976: 29.0.50; Complex emoji have extra spacing

[Protesilaos Stavrou]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Wed, 21 Sep 2022 12:53:14 +0200
>
> Protesilaos Stavrou <info@protesilaos.com> writes:
>
>> I just downgraded to harfbuzz 5.1.0 and restarted Emacs.  The spacing is
>> fine now.  I suppose this means we should close this bug and report it
>> to the harfbuzz maintainers.  Though I thought I would bring it to your
>> attention.
>
> Yes, if you could bring it to their attention, it would be nice, but I'd
> rather keep this bug report open until there's an answer -- perhaps
> there's something we have to adjust in the code on our side with newer
> harfbuzz versions?
>
> If this is not an actual bug in harfbuzz, perhaps the harfbuzz
> developers have some advice here...

I have no reproducible recipe with harfbuzz 5.2.0 beside Emacs.  I wrote
the M-x emoji-list buffer to a file and visited it with a generic text
editor: the display looks fine.

I will try some other programs to see if they are affected.

-- 
Protesilaos Stavrou
https://protesilaos.com

bug#57976: 29.0.50; Complex emoji have extra spacing

[Lars Ingebrigtsen]

Protesilaos Stavrou <info@protesilaos.com> writes:

> I have no reproducible recipe with harfbuzz 5.2.0 beside Emacs.  I wrote
> the M-x emoji-list buffer to a file and visited it with a generic text
> editor: the display looks fine.

If you take one of the emojis that displayed correctly and one that was
wrong and then say

(string-pixel-width "😶‍🌫️")

on both -- what does that return?  From the display, it kinda looked
like Emacs thought that some of the glyphs were really wide.

bug#57976: 29.0.50; Complex emoji have extra spacing

[Protesilaos Stavrou]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Wed, 21 Sep 2022 15:35:02 +0200
>
> Protesilaos Stavrou <info@protesilaos.com> writes:
>
>> I have no reproducible recipe with harfbuzz 5.2.0 beside Emacs.  I wrote
>> the M-x emoji-list buffer to a file and visited it with a generic text
>> editor: the display looks fine.
>
> If you take one of the emojis that displayed correctly and one that was
> wrong and then say
>
> (string-pixel-width "😶‍🌫️")
>
> on both -- what does that return?  From the display, it kinda looked
> like Emacs thought that some of the glyphs were really wide.

I get these:

    (string-pixel-width "😶‍🌫️")
    ;; => 136

    (string-pixel-width "🤣")
    ;; => 16

-- 
Protesilaos Stavrou
https://protesilaos.com

bug#57976: 29.0.50; Complex emoji have extra spacing

[Lars Ingebrigtsen]

Protesilaos Stavrou <info@protesilaos.com> writes:

> I get these:
>
>     (string-pixel-width "😶‍🌫️")
>     ;; => 136
>
>     (string-pixel-width "🤣")
>     ;; => 16

Wow.  So Emacs gets the dimensions of 😶‍🌫️ really, really wrong with that
harfbuzz version.  😶‍🌫️ is composed of

FACE WITHOUT MOUTH
ZERO WIDTH JOINER
FOG
VARIATION SELECTOR-16

so even if it counted all four of those separately, we'd just get to
(* 16 4) = 64, not 136...

bug#57976: 29.0.50; Complex emoji have extra spacing

[YAMAMOTO Mitsuharu]

On Wed, 21 Sep 2022 22:46:32 +0900,
Protesilaos Stavrou wrote:
> 
> > From: Lars Ingebrigtsen <larsi@gnus.org>
> > Date: Wed, 21 Sep 2022 15:35:02 +0200
> >
> > Protesilaos Stavrou <info@protesilaos.com> writes:
> >
> >> I have no reproducible recipe with harfbuzz 5.2.0 beside Emacs.  I wrote
> >> the M-x emoji-list buffer to a file and visited it with a generic text
> >> editor: the display looks fine.
> >
> > If you take one of the emojis that displayed correctly and one that was
> > wrong and then say
> >
> > (string-pixel-width "😶‍🌫️")
> >
> > on both -- what does that return?  From the display, it kinda looked
> > like Emacs thought that some of the glyphs were really wide.
> 
> I get these:
> 
>     (string-pixel-width "😶‍🌫️")
>     ;; => 136
> 
>     (string-pixel-width "🤣")
>     ;; => 16

Hmm, the fix for Bug#57066 needs to be undone for HarfBuzz 5.2.0.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

diff --git a/src/ftcrfont.c b/src/ftcrfont.c
index e089f9dea8..a02ff99870 100644
--- a/src/ftcrfont.c
+++ b/src/ftcrfont.c
@@ -679,8 +679,12 @@ ftcrhbfont_begin_hb_font (struct font *font, double *position_unit)
   hb_font_t *hb_font = fthbfont_begin_hb_font (font, position_unit);
   /* HarfBuzz 5 correctly scales bitmap-only fonts without position
      unit adjustment.
-     (https://github.com/harfbuzz/harfbuzz/issues/489) */
-  if (!hb_version_atleast (5, 0, 0)
+     (https://github.com/harfbuzz/harfbuzz/issues/489)
+
+     Update: HarfBuzz 5.2.0 no longer does this for an hb_font_t font
+     object created from a given FT_Face.
+     (https://github.com/harfbuzz/harfbuzz/issues/3788) */
+  if ((hb_version_atleast (5, 2, 0) || !hb_version_atleast (5, 0, 0))
       && ftcrfont_info->bitmap_position_unit)
     *position_unit = ftcrfont_info->bitmap_position_unit;
 

bug#57976: 29.0.50; Complex emoji have extra spacing

[Eli Zaretskii]

> Cc: Lars Ingebrigtsen <larsi@gnus.org>, 57976@debbugs.gnu.org
> Date: Thu, 22 Sep 2022 11:00:15 +0900
> From: YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>
> 
> > I get these:
> > 
> >     (string-pixel-width "😶‍🌫️")
> >     ;; => 136
> > 
> >     (string-pixel-width "🤣")
> >     ;; => 16
> 
> Hmm, the fix for Bug#57066 needs to be undone for HarfBuzz 5.2.0.

Thanks, please install this on the emacs-28 branch.

bug#57976: 29.0.50; Complex emoji have extra spacing

[YAMAMOTO Mitsuharu]

On Thu, 22 Sep 2022 16:08:23 +0900,
Eli Zaretskii wrote:
> 
> > Cc: Lars Ingebrigtsen <larsi@gnus.org>, 57976@debbugs.gnu.org
> > Date: Thu, 22 Sep 2022 11:00:15 +0900
> > From: YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>
> > 
> > > I get these:
> > > 
> > >     (string-pixel-width "😶‍🌫️")
> > >     ;; => 136
> > > 
> > >     (string-pixel-width "🤣")
> > >     ;; => 16
> > 
> > Hmm, the fix for Bug#57066 needs to be undone for HarfBuzz 5.2.0.
> 
> Thanks, please install this on the emacs-28 branch.

Done.  Closing.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp