bug#59606: flags take up so many columns

bug#59606: flags take up so many columns

[Dan Jacobson]

Why do the flags,
🇨🇦; 🇺🇸;👁;🔭;
seem to take up so many columns?
emacs-version "28.2"

bug#59606: flags take up so many columns

[Werner LEMBERG]

[-- Attachment #1: Type: Text/Plain, Size: 318 bytes --]


> Why do the flags,
> 🇨🇦; 🇺🇸;👁;🔭;
> seem to take up so many columns?
> emacs-version "28.2"

Do they?  Attached is what I see on GNU Emacs 29.0.50 (build 1,
x86_64-pc-linux-gnu, GTK+ Version 3.24.34, cairo version 1.16.0) of
2022-10-27.

It thus seems to be a font issue.


    Werner

[-- Attachment #2: flags.png --]
[-- Type: Image/Png, Size: 8957 bytes --]

bug#59606: flags take up so many columns

[Kévin Le Gouguec]

Dan Jacobson <jidanni@jidanni.org> writes:

> Why do the flags,
> 🇨🇦; 🇺🇸;👁;🔭;
> seem to take up so many columns?

It would really help us understand the issue if you could give more
details about what you are seeing.  FWIW, I see almost the same results
as Werner, except for "👁", which is not displayed with a color font
(which seems right AFAIU, since your message did not include variation
selector 16, and admin/unidata/emoji-data.txt shows that U+1F441 has
Emoji_Presentation=No).

My understanding (though I don't have a unicode.org reference handy) is
that emoji ought to be presented with "fullwidth", i.e. take twice as
much space as "halfwidth" characters that compose Latin text.

Emacs's string-width function seems to report the "correct" results
here:

(string-width "🇨🇦") ; 2
(string-width "🇺🇸") ; 2
(string-width "👁") ; 1
(string-width "🔭") ; 2

Though FWIW U+1F441 is misaligned on my system, since it is displayed
with Symbola instead of DejaVu Sans Mono (my default font family).

> emacs-version "28.2"

bug#59606: flags take up so many columns

[Dan Jacobson]

OK, sorry. It must be the noto fonts.
'"No Tofu" font families with large Unicode coverage'.
I suppose I will have to take it up with them.


             position: 1026 of 1267 (81%), column: 15
            character: 🇨 (displayed as 🇨) (codepoint 127464, #o370750, #x1f1e8)
              charset: unicode (Unicode (ISO10646))
code point in charset: 0x1F1E8
               script: emoji
               syntax: w 	which means: word
             category: .:Base, L:Strong L2R
             to input: type "C-x 8 RET 1f1e8" or "C-x 8 RET REGIONAL INDICATOR SYMBOL LETTER C"
          buffer code: #xF0 #x9F #x87 #xA8
            file code: #xF0 #x9F #x87 #xA8 (encoded by coding system utf-8-unix)
              display: composed to form "🇨🇦" (see below)
              display: composed to form "FFBBBBBBBBBBBBBBB" (see below)(I see a two column flag, followed by many blanks)

Composed with the following character(s) "🇦" using this font:
  ftcrhb:-GOOG-Noto Color Emoji-normal-normal-normal-*-20-*-*-*-m-0-iso10646-1
by these glyphs:
  [0 1 127464 1561 25 0 25 19 5 [0 0 136]]
with these character(s):
  🇦 (#x1f1e6) REGIONAL INDICATOR SYMBOL LETTER A

Character code properties: customize what to show
  name: REGIONAL INDICATOR SYMBOL LETTER C
  general-category: So (Symbol, Other)
  decomposition: (127464) ('🇨')

bug#59606: flags take up so many columns

[Kévin Le Gouguec]

[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]

Dan Jacobson <jidanni@jidanni.org> writes:

> OK, sorry. It must be the noto fonts.
> '"No Tofu" font families with large Unicode coverage'.

Yes, as the describe-char output you posted confirms, Emacs defaults to
Noto Color Emoji (when available) to display emoji (specifically those
that either have Emoji_Presentation=Yes, or those that are followed by
VS-16.  Compare ⚠ with ⚠️).

So nothing surprising there, AFAIU.

> I suppose I will have to take it up with them.

Unless I missed something, you still haven't described what you are
seeing precisely (specifically, how many columns each emoji takes on
your setup), so it is not clear what you wish to "take up" to the Noto
developers.

(FWIW, I don't know what Emacs aims to do when faced with mixed fonts.
AFAICT it does not try to re-scale glyphs to make them match their
string-width, e.g. here Noto Color Emoji characters take up slightly
more than 2 "columns"; Symbola characters like 👁 are somewhere in
between:

  (dolist (char '("c" "⚠" "👁" "⚠️" "👁️"))
    (let ((font (font-at (1- (length char)) nil char)))
      (insert (format "%s: %d\n" char (string-pixel-width char)))))

  c: 9
  ⚠: 9
  👁: 15
  ⚠️: 19
  👁️: 19

Not saying Emacs should do anything more; just wondering what behaviour
users should expect there)


[-- Attachment #2: Screenshot_20221129_084242.png --]
[-- Type: image/png, Size: 126092 bytes --]

bug#59606: flags take up so many columns

[Dan Jacobson]

[-- Attachment #1: Type: text/plain, Size: 347 bytes --]

KLG> Unless I missed something, you still haven't described what you are
KLG> seeing precisely (specifically, how many columns each emoji takes on
KLG> your setup), so it is not clear what you wish to "take up" to the Noto
KLG> developers.

Actually I mentioned it with a string of BBBB for blanks.

And indeed I was seeing around 19-9=10 blanks.

[-- Attachment #2: 20221129T043731.jpg --]
[-- Type: image/jpeg, Size: 31780 bytes --]

[-- Attachment #3: Type: text/plain, Size: 105 bytes --]


In fact it would be great if you could report the exact problem to the
parties involved for me. Thanks.

bug#59606: flags take up so many columns

[Kévin Le Gouguec]

Dan Jacobson <jidanni@jidanni.org> writes:

> KLG> Unless I missed something, you still haven't described what you are
> KLG> seeing precisely (specifically, how many columns each emoji takes on
> KLG> your setup), so it is not clear what you wish to "take up" to the Noto
> KLG> developers.
>
> Actually I mentioned it with a string of BBBB for blanks.

(Apologies; my eyes glazed over the describe-char output)

> And indeed I was seeing around 19-9=10 blanks.

(Not sure I follows your logic; are you implying that the difference in
pixel size is converted in an equivalent number of columns?)

> In fact it would be great if you could report the exact problem to the
> parties involved for me. Thanks.

No other participant in this thread has witnessed the symptoms you have,
so we need more information about your setup.  Your OP says:

> emacs-version 28.2

And your font backend seems to be ftcrhb.  Given that, could you…

* tell us what versions of Cairo and Harfbuzz you are using?

* try to compile the tip of the emacs-28 branch?  There's a bugfix for
  Harfbuzz in the emacs-28.2..emacs-28 range, and the corresponding
  report (bug#57976) mentions extra spacing for emoji (the OP's
  screenshot actually shows symptoms similar to yours, AFAICT).

* try to compile the emacs-29 branch?

* (confirm that you see this from emacs -Q?  I don't remember you saying
  so, but it wouldn't be the first thing I've missed)

bug#59606: flags take up so many columns

[Dan Jacobson]

Sorry. My computer is now packed up until after Christmas. As far as 19 
- 9, I was just saying I saw the letter c had the number nine next to it 
in your photo and there was zero erroneous columns, and then something 
else had the number 19 next to it and then there was 10 erroneous 
columns, so I was experiencing the same symptoms. As far as emacs -Q I 
only used a little q I forgot about the big Q. Anyways I was just using 
Debian sid...

bug#59606: flags take up so many columns

[Kévin Le Gouguec]

# Boldly closing & merging as this looks like 57976 to me.  See below.
close 59606
merge 57976 59606
thanks

Dan Jacobson <jidanni@jidanni.org> writes:

>    Anyways I was just using Debian sid...

… which has both Harfbuzz 5.2.0 and Emacs 28.2, which is known to be a
problematic configuration.  This really looks like bug#57976, especially
with your screenshot; a fix was pushed to the emacs-28 branch, but I'm
guessing there won't be another 28.x release, so AFAICT your options are

* if you build Emacs yourself, use either the tip of the emacs-28
  branch, emacs-29, or master;

* if you use Emacs as distributed in Debian sid, report the problem to
  the distro, so they apply the fix[1] to their tree.

On the Emacs side, IIUC, there is nothing more the the project can do,
so closing.


[1] 2022-09-23 "Fix shaping with bitmap-only fonts on HarfBuzz 5.2.0
(Bug#57976)" (60ac12d21f).