bug#63731: [PATCH] Support Emoji Variation Sequence 16 (FE0F) where appropriate

[Eli Zaretskii <eliz@gnu.org>]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: 63731@debbugs.gnu.org,  steven@stebalien.com
> Date: Mon, 29 May 2023 12:44:58 +0200
> 
> In all these cases, consider the sequence U+1F44D U+FE0F
> 
> - emacs-29:
> 
>     Displays as colour emoji, followed by an empty box
> 
> - emacs-29 with the following change in composite.el:
> 
>       (set-char-table-range
>        composition-function-table
>        #xFE0F
>        `([,(purecopy "\\c.\ufe0f") 1 compose-gstring-for-graphic]))
> 
>     Displays as colour emoji. Much rejoicing. If I follow my own
>     advice, and customize `glyphless-char-display-control' to show
>     hex-boxes for variation selectors, you then see that in actual
>     fact, we are still displaying the FE0F, but since it uses
>     thin-space by default, it wasnʼt obvious. Much sadness.
> 
>     C-u C-x =:
> 
>                   display: composed to form "👍️" (see below)

This is not what I see.  I didn't use the above set-char-table-range
expression literally, but instead started "emacs -Q", and then
evaluated in *scratch*:

      (set-char-table-range
       composition-function-table
       #xFE0F
       '(["\\c.\ufe0f" 1 compose-gstring-for-graphic]))

After that, the sequence U+1F44D U+FE0F displays as a single glyph,
and there's no thin space after it.  What am I missing?  Is this
somehow specific to ftcrhb font driver or something?

> Now I notice (via emoji-variation-sequences.txt), that this is only
> happening for the following codepoints.
> 
>    U+1F408
>    U+1F415
>    U+1F426
>    U+1F446
>    U+1F447
>    U+1F448
>    U+1F449
>    U+1F44D
>    U+1F44E
> 
> And if I look in lisp/international/emoji-zwj.el, I find:
> 
> (#x1F44D .
> ,(eval-when-compile (regexp-opt
> '(
> "\N{U+1F44D}\N{U+1F3FB}"
> "\N{U+1F44D}\N{U+1F3FC}"
> "\N{U+1F44D}\N{U+1F3FD}"
> "\N{U+1F44D}\N{U+1F3FE}"
> "\N{U+1F44D}\N{U+1F3FF}"
> ))))
> 
> If I add
> 
> "\N{U+1F44D}\N{U+FE0F}"
> 
> to that, and undo the composite.el change, then everything is
> fine. Hurrah! This means that the
> 
> `([,(purecopy "\\c.\\c^+") 1 compose-gstring-for-graphic]
> 	       [nil 0 compose-gstring-for-graphic])
> 
> is not doing the right thing for this case.

You are saying that the entry in composition-function-table for
U+1F44D (and other similar characters) is used in preference to the
entry for U+FE0F that follows it, even though there's no U+1F3FB
etc. after it to "steal" the composition?  Did you try stepping
through composite.c to see whether and why this is the case?

> I can change the emoji-zwj.awk script to add CHAR+FE0F for all emoji,
> unless someone knows how to fix composition to do the right thing
> here.

I think we need first to understand the issue at hand better.  There's
more here than meets the eye, I think.