Re: Entering emojis

[Eli Zaretskii <eliz@gnu.org>]

> Date: Fri, 29 Oct 2021 10:32:03 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: mattiase@acm.org, emacs-devel@gnu.org, schwab@linux-m68k.org, 
>     stefankangas@gmail.com, raman@google.com
> 
> >> In this case, ISTM that the problem is not the font, but the shaping 
> >> engine.  If Harfbuzz does not know how handle the joiners and segment 
> >> delimiters, it should behave as they did not exist, and use the font 
> >> ligatures (if the font does have ligatures)
> >
> > AFAICT, this is what happens here.
> 
> No, because Harfbuzz displays the fictitious joiner and deelimiter glyphs, 
> and does not try to use the ligatures that the font provides.

Because the font and/or HarfBuzz don't support the formatting
controls.

If you have evidence that the font does support the formatting
controls, but HarfBuzz doesn't, please show it.  I think that's not
the case, because the same or similar display problems happen with
LibreOffice when using the format controls.

When a sequence of codepoints is not recognized as a composable one,
what we get as result is either a separate glyph for each codepoint,
or maybe composed glyphs for some sub-sequence.  That is normal and
expected when a sequence is not recognized.

> >> instead of displaying the fictitious glyph at that codepoint (at the 
> >> codepoint of the joiner or delimiter).
> >
> > I don't think I understand what fictitious glyph you allude to here. The 
> > joiners were displayed as thin spaces by the Emacs 
> > glyphless-char-display feature, because HarfBuzz+font didn't compose the 
> > sequence, and returned the separate font glyphs for each codepoint in 
> > the sequence.  IOW, the composition failed, and therefore Emacs 
> > displayed each of these characters as it's supposed to do.
> >
> 
> A picture is worth a thousand words.  I attach four files:
> 
> In 1.png you see what Harfbuzz displays with the previous HELLO entry. 
> The three glyphs with a thick rectangle above and a crossed rectangle 
> below are a fictitious glyph in the Aegyptus font for the codepoint 
> hieroglyph vertical joiner, and the opening and closing parentheses are 
> fictitious glyphs in the Aegyptus font for the codepoints hieroglyph begin 
> and end segment.

Why do you call them "fictitious"?  If those are the glyphs returned
by the font, then that's what the font designers want us to display

> In 2.png you see what I would expect Harfbuzz to do with the previous 
> HELLO entry, if it knows that it cannot handle the joiner and segment 
> delimiters or if it detects that the font does not provide enough 
> information to handle them appropriately, and if the font has no 
> ligatures: displaying the hieroglyphs one after the other.  That's what I 
> would expect to see with the Noto Hieroglyph font for example.
> 
> In 3.png you see what I would expect Harfbuzz to do with the previous 
> HELLO entry, if it knows that it cannot handle the joiner and segment 
> delimiters or if it detects that the font does not provide enough 
> information to handle them appropriately, and if the font does have 
> ligatures: displaying the hieroglyphs with the ligatures provided by the 
> font.  That's what I would expect to see with the Aegyptus font.
> 
> In 4.png you see what I would expect Harfbuzz to do with the previous 
> HELLO entry, if it knows that it can handle the joiner and segment 
> delimiters and if it detects that the font does provide enough information 
> to handle them appropriately.

Your expectations are incorrect.  The job of producing the correct
glyphs for a sequence involves both the font and the shaping engine.
The shaping engine in general should not produce any glyphs that the
font didn't return, and AFAIU has no means to "detect that the font
doesn't provide enough information" or "doesn't have ligatures".  The
shaping engine is supposed to trust the font that it knows what it's
doing.  The role of the shaping engine is to collect information about
the context of the character sequence (language, script,
directionality, etc.), and communicate that information to the font so
that the font could select the appropriate glyphs.

> > No, the joiners are supposed to tell the shaping engine and the font 
> > that we want the ligatures and not separate font glyphs.
> 
> Unless I misunderstand something, a text without joiners and delimiters 
> would thus be displayed as 2.png, even if the underlying font provides 
> ligatures with which it could be displayed as 3.png.  And ZWNJ would be 
> ignored.  Which makes, as I said, the task of those who want to edit 
> egyptian texts much harder, and unnecessarily so.

It is irrelevant what you and me think about whether this makes
sense.  We try to follow the Unicode Standard where we don't have a
reason to deviate from it.  If and when the fonts will implement what
Unicode says, we should also do what Unicode says, regardless of our
private opinions about the convenience of writing this script.