bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Lars Ingebrigtsen]

If I start "emacs -Q" and load a file with this character:

🎉

Emacs shows that as a black-and-white symbol using the Symbola font.

If I then say

(set-fontset-font t 'symbol "Noto Color Emoji")
(set-fontset-font t 'symbol "Symbola" nil 'append)

I get a colourful party popper.

It'd be nice if this worked out of the box.  What do other programs do?
Have a huge list of fonts they prefer over other fonts?


In GNU Emacs 28.0.50 (build 49, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, cairo version 1.16.0)
 of 2020-10-14 built on xo
Repository revision: 1ee5a4cb1afe5a55d78e35cab2ad8651196c8967
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: Debian GNU/Linux bullseye/sid


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Thu, 15 Oct 2020 20:10:44 +0200
> 
> 
> If I start "emacs -Q" and load a file with this character:
> 
> 🎉
> 
> Emacs shows that as a black-and-white symbol using the Symbola font.
> 
> If I then say
> 
> (set-fontset-font t 'symbol "Noto Color Emoji")
> (set-fontset-font t 'symbol "Symbola" nil 'append)
> 
> I get a colourful party popper.
> 
> It'd be nice if this worked out of the box.

AFAIR, Robert Pluim worked on better support for Emoji display; that
needs a few more tweaks and importing one more file from the UCD, to
support Emoji sequences.  I hope this will be ready some time soon.
Robert?

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Robert Pluim]

>>>>> On Thu, 15 Oct 2020 21:52:19 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Lars Ingebrigtsen <larsi@gnus.org>
    >> Date: Thu, 15 Oct 2020 20:10:44 +0200
    >> 
    >> 
    >> If I start "emacs -Q" and load a file with this character:
    >> 
    >> 🎉
    >> 
    >> Emacs shows that as a black-and-white symbol using the Symbola font.
    >> 
    >> If I then say
    >> 
    >> (set-fontset-font t 'symbol "Noto Color Emoji")
    >> (set-fontset-font t 'symbol "Symbola" nil 'append)
    >> 
    >> I get a colourful party popper.
    >> 
    >> It'd be nice if this worked out of the box.

    Eli> AFAIR, Robert Pluim worked on better support for Emoji display; that
    Eli> needs a few more tweaks and importing one more file from the UCD, to
    Eli> support Emoji sequences.  I hope this will be ready some time soon.
    Eli> Robert?

Thereʼs three parts to that:

1. Deciding what fonts we should use for emoji. We can get a lot of
   good results by adding appropriate 'set-fontset-font' calls to
   emacs, independently of any other work (I also wonder if we should
   change the default of use-default-font-for-symbols, given the
   proliferation of both emoji and fonts dedicated to displaying them)

2. Extract the emoji sequences from the Unicode data files and assign
   them to a new 'emoji' script. Be prepared to choose your favourite
   bikeshed colour.

3. Add the appropriate entries to composition-function-table so that
   the various multi-codepoint emoji sequences have a chance to be
   rendered correctly, whether by HarfBuzz or the Apple thing whose
   name I forget.

For [1], I think "Noto Color Emoji" and "Apple Color Emoji" would be a
good start.

For [2], I think that was finished, except that Eli then updated Emacs
to the next version of Unicode, and I haven't adjusted yet.

[3] is done for the cases that composition can currently handle. I
recall that there are sequences that require composition to do
lookback, which I think YAMAMOTO-san has patches for, but they've not
been submitted to Emacs (a quick perusal of my notes says this is for
keycap emojis, so I guess thatʼs a minor nit).

Oh, and there was a complication with enabling this by default on
macOS that I donʼt fully understand yet. Iʼll reread the history.

Iʼll see if I can clean this up this weekend and throw it on a branch.

Robert
-- 

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>,  44020@debbugs.gnu.org
> Date: Fri, 16 Oct 2020 10:19:12 +0200
> 
> 1. Deciding what fonts we should use for emoji. We can get a lot of
>    good results by adding appropriate 'set-fontset-font' calls to
>    emacs, independently of any other work (I also wonder if we should
>    change the default of use-default-font-for-symbols, given the
>    proliferation of both emoji and fonts dedicated to displaying them)
> 
> 2. Extract the emoji sequences from the Unicode data files and assign
>    them to a new 'emoji' script. Be prepared to choose your favourite
>    bikeshed colour.
> 
> 3. Add the appropriate entries to composition-function-table so that
>    the various multi-codepoint emoji sequences have a chance to be
>    rendered correctly, whether by HarfBuzz or the Apple thing whose
>    name I forget.
> 
> For [1], I think "Noto Color Emoji" and "Apple Color Emoji" would be a
> good start.

I agree, but [1] depends on [2], because set-fontset-font needs to
know the script.

Also, Noto Color Emoji has a free license, so it's okay for it to be
in fontset.el, but it is not clear to me whether Apple Color Emoji is
distributed under a free license.

> For [2], I think that was finished, except that Eli then updated Emacs
> to the next version of Unicode, and I haven't adjusted yet.

"Finished", in the sense that it is in Emacs already?

> Iʼll see if I can clean this up this weekend and throw it on a branch.

Thanks.

FWIW, I think this is good enough to land on master directly, I see no
need for a feature branch.

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Robert Pluim]

>>>>> On Fri, 16 Oct 2020 13:26:42 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Robert Pluim <rpluim@gmail.com>
    >> Cc: Lars Ingebrigtsen <larsi@gnus.org>,  44020@debbugs.gnu.org
    >> Date: Fri, 16 Oct 2020 10:19:12 +0200
    >> 
    >> 1. Deciding what fonts we should use for emoji. We can get a lot of
    >> good results by adding appropriate 'set-fontset-font' calls to
    >> emacs, independently of any other work (I also wonder if we should
    >> change the default of use-default-font-for-symbols, given the
    >> proliferation of both emoji and fonts dedicated to displaying them)
    >> 
    >> 2. Extract the emoji sequences from the Unicode data files and assign
    >> them to a new 'emoji' script. Be prepared to choose your favourite
    >> bikeshed colour.
    >> 
    >> 3. Add the appropriate entries to composition-function-table so that
    >> the various multi-codepoint emoji sequences have a chance to be
    >> rendered correctly, whether by HarfBuzz or the Apple thing whose
    >> name I forget.
    >> 
    >> For [1], I think "Noto Color Emoji" and "Apple Color Emoji" would be a
    >> good start.

    Eli> I agree, but [1] depends on [2], because set-fontset-font needs to
    Eli> know the script.

set-fontset-font with a TARGET covering the emoji codepoints in
question, I meant. Youʼre right itʼs probably cleaner to do both at
the same time.

    Eli> Also, Noto Color Emoji has a free license, so it's okay for it to be
    Eli> in fontset.el, but it is not clear to me whether Apple Color Emoji is
    Eli> distributed under a free license.

Itʼs marked 'Copyright Apple 2011-2016' in the macOS font
selector. And Apple appear to be both sensitive and litigious about
it, so probably best not to promote it.

    >> For [2], I think that was finished, except that Eli then updated Emacs
    >> to the next version of Unicode, and I haven't adjusted yet.

    Eli> "Finished", in the sense that it is in Emacs already?

No, finished as in "it works for me on my mac and GNU/Linux boxes" 🙂

As I said, it needs adjusting to the latest Unicode update.

    >> Iʼll see if I can clean this up this weekend and throw it on a branch.

    Eli> Thanks.

    Eli> FWIW, I think this is good enough to land on master directly, I see no
    Eli> need for a feature branch.

OK.

Robert
-- 

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: larsi@gnus.org,  44020@debbugs.gnu.org
> Date: Fri, 16 Oct 2020 12:38:18 +0200
> 
>     >> 1. Deciding what fonts we should use for emoji. We can get a lot of
>     >> good results by adding appropriate 'set-fontset-font' calls to
>     >> emacs, independently of any other work (I also wonder if we should
>     >> change the default of use-default-font-for-symbols, given the
>     >> proliferation of both emoji and fonts dedicated to displaying them)
>     >> 
>     >> 2. Extract the emoji sequences from the Unicode data files and assign
>     >> them to a new 'emoji' script. Be prepared to choose your favourite
>     >> bikeshed colour.
>     >> 
>     >> 3. Add the appropriate entries to composition-function-table so that
>     >> the various multi-codepoint emoji sequences have a chance to be
>     >> rendered correctly, whether by HarfBuzz or the Apple thing whose
>     >> name I forget.
>     >> 
>     >> For [1], I think "Noto Color Emoji" and "Apple Color Emoji" would be a
>     >> good start.
> 
>     Eli> I agree, but [1] depends on [2], because set-fontset-font needs to
>     Eli> know the script.
> 
> set-fontset-font with a TARGET covering the emoji codepoints in
> question, I meant. Youʼre right itʼs probably cleaner to do both at
> the same time.

Yes, because using the codepoints will mean more maintenance in the
future, as more Emoji blocks are added (and they will be, quite
certainly).

> 
>     Eli> Also, Noto Color Emoji has a free license, so it's okay for it to be
>     Eli> in fontset.el, but it is not clear to me whether Apple Color Emoji is
>     Eli> distributed under a free license.
> 
> Itʼs marked 'Copyright Apple 2011-2016' in the macOS font
> selector. And Apple appear to be both sensitive and litigious about
> it, so probably best not to promote it.

Right.  People will have to do that on their own, if they don't want
to install a free font.

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Alan Third]

On Fri, Oct 16, 2020 at 12:38:18PM +0200, Robert Pluim wrote:
> >>>>> On Fri, 16 Oct 2020 13:26:42 +0300, Eli Zaretskii <eliz@gnu.org> said:
> 
>     Eli> Also, Noto Color Emoji has a free license, so it's okay for it to be
>     Eli> in fontset.el, but it is not clear to me whether Apple Color Emoji is
>     Eli> distributed under a free license.
> 
> Itʼs marked 'Copyright Apple 2011-2016' in the macOS font
> selector. And Apple appear to be both sensitive and litigious about
> it, so probably best not to promote it.

On the other hand it comes preinstalled, it only works on Apple
devices, and Noto Color Emoji doesn't work on Apple devices. I'm not
sure we'd really be "promoting" it.

IIRC Windows comes with Segoe UI Emoji preinstalled too.
-- 
Alan Third

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Lars Ingebrigtsen]

Alan Third <alan@idiocy.org> writes:

> On the other hand it comes preinstalled, it only works on Apple
> devices, and Noto Color Emoji doesn't work on Apple devices. I'm not
> sure we'd really be "promoting" it.

On the third hand, since it's preinstalled, Emacs doesn't need to
mention it, because it's always there, and Emacs won't have problems
displaying the characters it offers.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Robert Pluim <rpluim@gmail.com>,  Eli Zaretskii <eliz@gnu.org>,
>   44020@debbugs.gnu.org
> Date: Sat, 17 Oct 2020 08:38:10 +0200
> 
> Alan Third <alan@idiocy.org> writes:
> 
> > On the other hand it comes preinstalled, it only works on Apple
> > devices, and Noto Color Emoji doesn't work on Apple devices. I'm not
> > sure we'd really be "promoting" it.
> 
> On the third hand, since it's preinstalled, Emacs doesn't need to
> mention it, because it's always there, and Emacs won't have problems
> displaying the characters it offers.

Are you sure?  What exactly makes sure Emacs won't have any problems
in those cases?  For starters, does Emacs really know that a given
character belongs to "Emoji"?  And if it does, does it know how to
filter fonts that support Emoji?

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> On the third hand, since it's preinstalled, Emacs doesn't need to
>> mention it, because it's always there, and Emacs won't have problems
>> displaying the characters it offers.
>
> Are you sure?  What exactly makes sure Emacs won't have any problems
> in those cases?  For starters, does Emacs really know that a given
> character belongs to "Emoji"?  And if it does, does it know how to
> filter fonts that support Emoji?

I thought that Emacs looped through all the fonts on the system and
checked whether the font had coverage for the character we wanted to
display?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: alan@idiocy.org,  rpluim@gmail.com,  44020@debbugs.gnu.org
> Date: Sun, 18 Oct 2020 10:09:09 +0200
> 
> > Are you sure?  What exactly makes sure Emacs won't have any problems
> > in those cases?  For starters, does Emacs really know that a given
> > character belongs to "Emoji"?  And if it does, does it know how to
> > filter fonts that support Emoji?
> 
> I thought that Emacs looped through all the fonts on the system and
> checked whether the font had coverage for the character we wanted to
> display?

It does -- for some value of "check".  Opening a font and trying to
find a glyph for a character is expensive, so Emacs tries to avoid
that as much as possible.  Instead, it uses less expensive "checks"
that don't require actually opening the font.  The details are
somewhat complicated (and I'm not sure I have a sufficiently clear
picture of them, nor that we have someone on board who does), and the
are to some extent font-backend dependent, but they have to do with
the character's script, the script's characters in
script-representative-chars, and system APIs for finding fonts that
support those.  And the first problem with Emoji characters in this
context is that their script is 'symbol', which doesn't allow us to
assign an Emoji font to them as a script.

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> It does -- for some value of "check".  Opening a font and trying to
> find a glyph for a character is expensive, so Emacs tries to avoid
> that as much as possible.  Instead, it uses less expensive "checks"
> that don't require actually opening the font.  The details are
> somewhat complicated (and I'm not sure I have a sufficiently clear
> picture of them, nor that we have someone on board who does), and the
> are to some extent font-backend dependent, but they have to do with
> the character's script, the script's characters in
> script-representative-chars, and system APIs for finding fonts that
> support those.  And the first problem with Emoji characters in this
> context is that their script is 'symbol', which doesn't allow us to
> assign an Emoji font to them as a script.

Ah, I see.  Thank you for the explanation.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Alan Third]

On Sat, Oct 17, 2020 at 08:38:10AM +0200, Lars Ingebrigtsen wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > On the other hand it comes preinstalled, it only works on Apple
> > devices, and Noto Color Emoji doesn't work on Apple devices. I'm not
> > sure we'd really be "promoting" it.
> 
> On the third hand, since it's preinstalled, Emacs doesn't need to
> mention it, because it's always there, and Emacs won't have problems
> displaying the characters it offers.

Even once I remove this code:

modified   src/macfont.m
@@ -2414,11 +2414,6 @@ So we use CTFontDescriptorCreateMatchingFontDescriptor (no
                   != (spacing >= FONT_SPACING_MONO)))
             continue;
 
-          /* Don't use a color bitmap font unless its family is
-             explicitly specified.  */
-          if ((sym_traits & kCTFontTraitColorGlyphs) && NILP (family))
-            continue;
-
           if (j > 0
               && !macfont_supports_charset_and_languages_p (desc, charset,
                                                             chars, languages))

Emacs almost never finds Apple Color Emoji without help. I've no idea
why.

For some reason it finds the font for BLACK DOWN-POINTING DOUBLE
TRIANGLE, but not SMILING FACE WITH SMILING EYES unless the former is
already in the document.

Should this Just Work?

BTW, there is code in ns-win.el that plays with the default fontset on
Macs which may be breaking our rules.
-- 
Alan Third

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Alan Third]

On Sat, Oct 17, 2020 at 04:27:58PM +0100, Alan Third wrote:
> 
> Emacs almost never finds Apple Color Emoji without help. I've no idea
> why.

Sorry, I've just realised I'm going round in circles as this is
exactly what Robert's working on. Ignore me, please. :)

-- 
Alan Third

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> Date: Fri, 16 Oct 2020 20:30:41 +0100
> From: Alan Third <alan@idiocy.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, 44020@debbugs.gnu.org, larsi@gnus.org
> 
> On Fri, Oct 16, 2020 at 12:38:18PM +0200, Robert Pluim wrote:
> > >>>>> On Fri, 16 Oct 2020 13:26:42 +0300, Eli Zaretskii <eliz@gnu.org> said:
> > 
> >     Eli> Also, Noto Color Emoji has a free license, so it's okay for it to be
> >     Eli> in fontset.el, but it is not clear to me whether Apple Color Emoji is
> >     Eli> distributed under a free license.
> > 
> > Itʼs marked 'Copyright Apple 2011-2016' in the macOS font
> > selector. And Apple appear to be both sensitive and litigious about
> > it, so probably best not to promote it.
> 
> On the other hand it comes preinstalled, it only works on Apple
> devices, and Noto Color Emoji doesn't work on Apple devices. I'm not
> sure we'd really be "promoting" it.
> 
> IIRC Windows comes with Segoe UI Emoji preinstalled too.

Both are true, but AFAIK our policy until now was not to mention such
fonts in fontset.el.

Emoji is just one example, btw.  At least for MS-Windows, the latest
versions come with many good fonts that cover most of the scripts.  So
if the restriction to avoid mentioning proprietary fonts in fontset.el
could be lifted, we could produce a very capable and complete fontset
for MS-Windows.  I wouldn't be surprised if the same situation didn't
exist on macOS.  Not sure how things are on GNU/Linux, and what are
the differences between the distros in this respect.

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Robert Pluim]

>>>>> On Sat, 17 Oct 2020 09:44:44 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> Date: Fri, 16 Oct 2020 20:30:41 +0100
    >> From: Alan Third <alan@idiocy.org>
    >> Cc: Eli Zaretskii <eliz@gnu.org>, 44020@debbugs.gnu.org, larsi@gnus.org
    >> 
    >> On Fri, Oct 16, 2020 at 12:38:18PM +0200, Robert Pluim wrote:
    >> > >>>>> On Fri, 16 Oct 2020 13:26:42 +0300, Eli Zaretskii <eliz@gnu.org> said:
    >> > 
    >> >     Eli> Also, Noto Color Emoji has a free license, so it's okay for it to be
    >> >     Eli> in fontset.el, but it is not clear to me whether Apple Color Emoji is
    >> >     Eli> distributed under a free license.
    >> > 
    >> > Itʼs marked 'Copyright Apple 2011-2016' in the macOS font
    >> > selector. And Apple appear to be both sensitive and litigious about
    >> > it, so probably best not to promote it.
    >> 
    >> On the other hand it comes preinstalled, it only works on Apple
    >> devices, and Noto Color Emoji doesn't work on Apple devices. I'm not
    >> sure we'd really be "promoting" it.
    >> 
    >> IIRC Windows comes with Segoe UI Emoji preinstalled too.

    Eli> Both are true, but AFAIK our policy until now was not to mention such
    Eli> fonts in fontset.el.

    Eli> Emoji is just one example, btw.  At least for MS-Windows, the latest
    Eli> versions come with many good fonts that cover most of the scripts.  So
    Eli> if the restriction to avoid mentioning proprietary fonts in fontset.el
    Eli> could be lifted, we could produce a very capable and complete fontset
    Eli> for MS-Windows.  I wouldn't be surprised if the same situation didn't
    Eli> exist on macOS.  Not sure how things are on GNU/Linux, and what are
    Eli> the differences between the distros in this respect.

Another thing that could be useful on GNU/Linux is to take advantage
of the fact that most of them go out of their way to define
pseudo-families for particular uses, such as 'emoji' or 'math', which
then automatically adjust to the fonts installed on the system:

$ fc-match 'emoji'
NotoColorEmoji.ttf: "Noto Color Emoji" "Regular"
:~

If I were to uninstall Noto Color Emoji, the same command would now
return "Emoji One"

Robert
-- 

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Iʼll see if I can clean this up this weekend and throw it on a branch.

Looking forward to it.  :-)  And I agree with Eli; doing it on the trunk
would be fine.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Robert Pluim]

>>>>> On Fri, 16 Oct 2020 16:49:43 +0200, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Robert Pluim <rpluim@gmail.com> writes:
    >> Iʼll see if I can clean this up this weekend and throw it on a branch.

    Lars> Looking forward to it.  :-)  And I agree with Eli; doing it on the trunk
    Lars> would be fine.

So revisiting this, it turns out I have *two* features sitting in my workspace:

1. splitting out the emoji into a separate script and setting the right
  fontset
2. adding composition rules for the zwj emoji described in
  emoji-zwj-sequences.txt

[2] has some complications regarding the default display of the
variation-selector, so Iʼll leave that aside for now, and continue on
updating [1] to Unicode 13.0

Robert
-- 

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: 44020@debbugs.gnu.org,  Eli Zaretskii <eliz@gnu.org>
> Date: Mon, 19 Oct 2020 12:56:33 +0200
> 
> So revisiting this, it turns out I have *two* features sitting in my workspace:
> 
> 1. splitting out the emoji into a separate script and setting the right
>   fontset
> 2. adding composition rules for the zwj emoji described in
>   emoji-zwj-sequences.txt
> 
> [2] has some complications regarding the default display of the
> variation-selector, so Iʼll leave that aside for now, and continue on
> updating [1] to Unicode 13.0

Thanks.

Can you summarize the problems with [2]?  I thought we had all of that
figured out back when we discussed this.

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Robert Pluim]

[-- Attachment #1: Type: text/plain, Size: 1636 bytes --]

>>>>> On Mon, 19 Oct 2020 17:55:55 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Robert Pluim <rpluim@gmail.com>
    >> Cc: 44020@debbugs.gnu.org,  Eli Zaretskii <eliz@gnu.org>
    >> Date: Mon, 19 Oct 2020 12:56:33 +0200
    >> 
    >> So revisiting this, it turns out I have *two* features sitting in my workspace:
    >> 
    >> 1. splitting out the emoji into a separate script and setting the right
    >> fontset
    >> 2. adding composition rules for the zwj emoji described in
    >> emoji-zwj-sequences.txt
    >> 
    >> [2] has some complications regarding the default display of the
    >> variation-selector, so Iʼll leave that aside for now, and continue on
    >> updating [1] to Unicode 13.0

    Eli> Thanks.

    Eli> Can you summarize the problems with [2]?  I thought we had all of that
    Eli> figured out back when we discussed this.

Perhaps we did: I have no clear memory of what we decided, so Iʼll
have to go and re-read it.

Iʼm attaching a gzipped patch for emoji support (it contains the
entire emoji-data.txt from Unicode 13.0 🙂), which works for me on
macOS when I do

(set-fontset-font t 'emoji '("Apple Color Emoji . "iso1064601") nil
'prepend)

It *should* work as-is on GNU/Linux, but for some reason Emacs is
completely refusing to use any Emoji fonts for me there. Itʼs possible
Iʼve messed up my system somehow, since emacs-27 has the same issue,
but Iʼm seeing some very suspicious results from
ftcrfont_glyph_extents (the font average width is always 0), so perhaps
we need to adjust our font code.

Testing appreciated.

Robert
-- 

[-- Attachment #2: 0001-emoji-using-blocks.txt.patch.gz --]
[-- Type: application/octet-stream, Size: 26713 bytes --]

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: 44020@debbugs.gnu.org,  larsi@gnus.org
> Date: Mon, 19 Oct 2020 20:34:41 +0200
> 
> (set-fontset-font t 'emoji '("Apple Color Emoji . "iso1064601") nil
> 'prepend)
> 
> It *should* work as-is on GNU/Linux, but for some reason Emacs is
> completely refusing to use any Emoji fonts for me there. Itʼs possible
> Iʼve messed up my system somehow, since emacs-27 has the same issue,
> but Iʼm seeing some very suspicious results from
> ftcrfont_glyph_extents (the font average width is always 0), so perhaps
> we need to adjust our font code.

Didn't someone say they can see color emoji on Cairo?

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Robert Pluim]

>>>>> On Mon, 19 Oct 2020 21:52:13 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Robert Pluim <rpluim@gmail.com>
    >> Cc: 44020@debbugs.gnu.org,  larsi@gnus.org
    >> Date: Mon, 19 Oct 2020 20:34:41 +0200
    >> 
    >> (set-fontset-font t 'emoji '("Apple Color Emoji . "iso1064601") nil
    >> 'prepend)
    >> 
    >> It *should* work as-is on GNU/Linux, but for some reason Emacs is
    >> completely refusing to use any Emoji fonts for me there. Itʼs possible
    >> Iʼve messed up my system somehow, since emacs-27 has the same issue,
    >> but Iʼm seeing some very suspicious results from
    >> ftcrfont_glyph_extents (the font average width is always 0), so perhaps
    >> we need to adjust our font code.

    Eli> Didn't someone say they can see color emoji on Cairo?

Lars said he could, and itʼs in the NEWS file, so it worked at some
point. I canʼt get either Noto Color Emoji nor Emoji One to work on
GNU/Linux, because the following code in font.c bugs out

  /* We always open a font of manageable size; i.e non-zero average
     width and height.  */
  for (psize = pixel_size; ; psize++)
    {
      font_object = driver_list->driver->open_font (f, entity, psize);
      if (NILP (font_object))
	return Qnil;
      font = XFONT_OBJECT (font_object);
      if (font->average_width > 0 && font->height > 0)
	break;
      /* Avoid an infinite loop.  */
      if (psize > pixel_size + 15)
	return Qnil; <====== here
    }

and thatʼs because font->average_width is always 0.

Robert
-- 

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Stephen Berman]

On Mon, 19 Oct 2020 21:52:13 +0300 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Robert Pluim <rpluim@gmail.com>
>> Cc: 44020@debbugs.gnu.org,  larsi@gnus.org
>> Date: Mon, 19 Oct 2020 20:34:41 +0200
>> 
>> (set-fontset-font t 'emoji '("Apple Color Emoji . "iso1064601") nil
>> 'prepend)
>> 
>> It *should* work as-is on GNU/Linux, but for some reason Emacs is
>> completely refusing to use any Emoji fonts for me there. Itʼs possible
>> Iʼve messed up my system somehow, since emacs-27 has the same issue,
>> but Iʼm seeing some very suspicious results from
>> ftcrfont_glyph_extents (the font average width is always 0), so perhaps
>> we need to adjust our font code.
>
> Didn't someone say they can see color emoji on Cairo?

Yes, Lars noted (and I confirm) that the following two lines suffice:

(set-fontset-font t 'symbol "Noto Color Emoji")
(set-fontset-font t 'symbol "Symbola" nil 'append)

Steve Berman

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: Robert Pluim <rpluim@gmail.com>,  44020@debbugs.gnu.org,  larsi@gnus.org
> Date: Mon, 19 Oct 2020 21:03:49 +0200
> 
> > Didn't someone say they can see color emoji on Cairo?
> 
> Yes, Lars noted (and I confirm) that the following two lines suffice:
> 
> (set-fontset-font t 'symbol "Noto Color Emoji")
> (set-fontset-font t 'symbol "Symbola" nil 'append)

Are the Emoji displayed by Noto Color Emoji or by Symbola?

And what version of Noto Color Emoji do you have?

And finally, could you step through the code shown by Robert and tell
whether the Noto Color Emoji's average_width you get is also zero or
non-zero?

Thanks.

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Stephen Berman]

On Mon, 19 Oct 2020 22:46:50 +0300 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Cc: Robert Pluim <rpluim@gmail.com>,  44020@debbugs.gnu.org,  larsi@gnus.org
>> Date: Mon, 19 Oct 2020 21:03:49 +0200
>>
>> > Didn't someone say they can see color emoji on Cairo?
>>
>> Yes, Lars noted (and I confirm) that the following two lines suffice:
>>
>> (set-fontset-font t 'symbol "Noto Color Emoji")
>> (set-fontset-font t 'symbol "Symbola" nil 'append)
>
> Are the Emoji displayed by Noto Color Emoji or by Symbola?

The majority of characters in emoji-data.txt are displayed by Noto Color
Emoji, but a number of them are displayed by Symbola and some are
displayed as tofu.

> And what version of Noto Color Emoji do you have?

v2018-04-24-pistol-update

> And finally, could you step through the code shown by Robert and tell
> whether the Noto Color Emoji's average_width you get is also zero or
> non-zero?

I started emacs -Q in gdb, evaluated the above two set-fontset-font
sexps, put a breakpoint on font_open_entity, and typed `M-x 8 RET
GRINNING FACE RET' and gdb took control, then I stepped through the
function until gdb output `if (font->average_width > 0 && font->height >
0)' and then I entered `p font->average_width' and gdb output $1 = 16.
Did I do this right and does that answer your question?  If not, please
advise.

Steve Berman

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: rpluim@gmail.com,  44020@debbugs.gnu.org,  larsi@gnus.org
> Date: Tue, 20 Oct 2020 00:00:35 +0200
> 
> > And what version of Noto Color Emoji do you have?
> 
> v2018-04-24-pistol-update
> 
> > And finally, could you step through the code shown by Robert and tell
> > whether the Noto Color Emoji's average_width you get is also zero or
> > non-zero?
> 
> I started emacs -Q in gdb, evaluated the above two set-fontset-font
> sexps, put a breakpoint on font_open_entity, and typed `M-x 8 RET
> GRINNING FACE RET' and gdb took control, then I stepped through the
> function until gdb output `if (font->average_width > 0 && font->height >
> 0)' and then I entered `p font->average_width' and gdb output $1 = 16.
> Did I do this right and does that answer your question?  If not, please
> advise.

Yes, that's it.  I wonder why Robert gets zero, perhaps a different
version of the font?

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Robert Pluim]

>>>>> On Tue, 20 Oct 2020 05:35:39 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Stephen Berman <stephen.berman@gmx.net>
    >> Cc: rpluim@gmail.com,  44020@debbugs.gnu.org,  larsi@gnus.org
    >> Date: Tue, 20 Oct 2020 00:00:35 +0200
    >> 
    >> > And what version of Noto Color Emoji do you have?
    >> 
    >> v2018-04-24-pistol-update
    >> 
    >> > And finally, could you step through the code shown by Robert and tell
    >> > whether the Noto Color Emoji's average_width you get is also zero or
    >> > non-zero?
    >> 
    >> I started emacs -Q in gdb, evaluated the above two set-fontset-font
    >> sexps, put a breakpoint on font_open_entity, and typed `M-x 8 RET
    >> GRINNING FACE RET' and gdb took control, then I stepped through the
    >> function until gdb output `if (font->average_width > 0 && font->height >
    >> 0)' and then I entered `p font->average_width' and gdb output $1 = 16.
    >> Did I do this right and does that answer your question?  If not, please
    >> advise.

    Eli> Yes, that's it.  I wonder why Robert gets zero, perhaps a different
    Eli> version of the font?

it seems Iʼm not alone in having problems with Noto Color Emoji on
Ubuntu: <https://github.com/googlefonts/noto-emoji/issues/36>

Debian time, I think (which has been on my list for a while in any
case).

Robert
-- 

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Robert Pluim]

>>>>> On Tue, 20 Oct 2020 09:33:49 +0200, Robert Pluim <rpluim@gmail.com> said:

>>>>> On Tue, 20 Oct 2020 05:35:39 +0300, Eli Zaretskii <eliz@gnu.org> said:

    Eli> Yes, that's it.  I wonder why Robert gets zero, perhaps a different
    Eli> version of the font?

    Robert> it seems Iʼm not alone in having problems with Noto Color Emoji on
    Robert> Ubuntu: <https://github.com/googlefonts/noto-emoji/issues/36>

    Robert> Debian time, I think (which has been on my list for a while in any
    Robert> case).

So it was failing on Debian as well, until I deleted my
~/.config/fontconfig directory, which ended up fixing it on Ubuntu as
well.

Robert
-- 

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: 44020@debbugs.gnu.org,  larsi@gnus.org,  Stephen Berman
>  <stephen.berman@gmx.net>
> Date: Tue, 20 Oct 2020 10:58:31 +0200
> 
> So it was failing on Debian as well, until I deleted my
> ~/.config/fontconfig directory, which ended up fixing it on Ubuntu as
> well.

This Fontconfig cache thingy appears in too many trouble reports.
Sounds like something we should ask users to do up front whenever they
report some weird font-related issue.

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Robert Pluim]

>>>>> On Tue, 20 Oct 2020 17:28:14 +0300, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Robert Pluim <rpluim@gmail.com>
    >> Cc: 44020@debbugs.gnu.org,  larsi@gnus.org,  Stephen Berman
    >> <stephen.berman@gmx.net>
    >> Date: Tue, 20 Oct 2020 10:58:31 +0200
    >> 
    >> So it was failing on Debian as well, until I deleted my
    >> ~/.config/fontconfig directory, which ended up fixing it on Ubuntu as
    >> well.

    Eli> This Fontconfig cache thingy appears in too many trouble reports.
    Eli> Sounds like something we should ask users to do up front whenever they
    Eli> report some weird font-related issue.

To be clear: this is not the fontconfig cache, itʼs the user-specific
directory for configuring fontconfig. I had one because I was playing
around with hinting styles at one point, and thatʼs exactly what can
cause Noto Color Emoji to fail (donʼt ask me why).

But yes, we should ask people to move both the fontconfig cache and
the fontconfig user directory out of the way (or ask them to run
FC_DEBUG=1024 emacs -Q, which will show us which fontconfig config
files are being loaded).

Robert
-- 

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Stefan Kangas]

Robert Pluim <rpluim@gmail.com> writes:

> It *should* work as-is on GNU/Linux, but for some reason Emacs is
> completely refusing to use any Emoji fonts for me there. Itʼs possible
> Iʼve messed up my system somehow, since emacs-27 has the same issue,
> but Iʼm seeing some very suspicious results from
> ftcrfont_glyph_extents (the font average width is always 0), so perhaps
> we need to adjust our font code.
>
> Testing appreciated.

I applied your patch running on Debian GNU/Linux (bullseye/testing), and
was able to display the "🎉" character from Lars' original message just
fine here.

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Robert Pluim]

>>>>> On Mon, 19 Oct 2020 19:03:22 +0000, Stefan Kangas <stefankangas@gmail.com> said:

    Stefan> Robert Pluim <rpluim@gmail.com> writes:
    >> It *should* work as-is on GNU/Linux, but for some reason Emacs is
    >> completely refusing to use any Emoji fonts for me there. Itʼs possible
    >> Iʼve messed up my system somehow, since emacs-27 has the same issue,
    >> but Iʼm seeing some very suspicious results from
    >> ftcrfont_glyph_extents (the font average width is always 0), so perhaps
    >> we need to adjust our font code.
    >> 
    >> Testing appreciated.

    Stefan> I applied your patch running on Debian GNU/Linux (bullseye/testing), and
    Stefan> was able to display the "🎉" character from Lars' original message just
    Stefan> fine here.

Thanks. I guess that means my local installation is screwed up
somehow [1]. Time to download a new one 😊

Robert

Footnotes:
[1]  Itʼs gone through a dist-upgrade at least 3 times, thatʼs bound
     to leave some scars.

-- 

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Mon, 19 Oct 2020 19:03:22 +0000
> Cc: 44020@debbugs.gnu.org, larsi@gnus.org
> 
> I applied your patch running on Debian GNU/Linux (bullseye/testing), and
> was able to display the "🎉" character from Lars' original message just
> fine here.

Please see if you can answer the questions I asked Stephen.

Thanks.

bug#44020: 28.0.50; Prefer selection of colour emoji fonts?

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> If I start "emacs -Q" and load a file with this character:
>
> 🎉

[...]

> It'd be nice if this worked out of the box.

This does work out of the box now, so I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no