bug#20140: 24.4; M17n shaper output rejected

[Eli Zaretskii <eliz@gnu.org>]

> Date: Sun, 13 Feb 2022 21:11:52 +0000
> From: Richard Wordingham <richard.wordingham@ntlworld.com>
> Cc: larsi@gnus.org, 20140@debbugs.gnu.org
> 
> > Btw, the _only_ reason Handa-san and now myself were able to implement
> > something like the forward/backward-char-intrusive commands is that we
> > DO control which parts of text are composed and which aren't.  If we
> > were to follow HarfBuzz developers' advice, and were to hand all the
> > text to HarfBuzz for shaping, we would need the HarfBuzz cooperation
> > to implement such features in the editor.
> 
> You mean the more sophisticated mechanisms which position the cursor
> intelligently.  Those two commands you named work by completely
> ignoring the composition mechanism.

Yes.  And the reason we can ignore compositions in certain portions
of the text is that we have control on what is passed to HarfBuzz.

> Correct me if I am wrong, but for Arabic, is not Emacs restricted to
> typewriter-like fonts?

No, that's not true.  I'm not aware of any such limitation; AFAIK
Arabic shaping works correctly in Emacs, certainly with HarfBuzz and
Emacs 27 or later.

Or maybe I misunderstand what you mean by "typewriter-like" fonts?
Can you give an example of a non-typewriter-like font for Arabic that
I can find on MS-Windows and try?

> There would be a similar problem with the use of Tai Khuen or other
> tunnelling fonts for Northern Thai if you used the current mechanism
> for advancing character by character.  Tunnelling fonts write parts of
> one cluster under the next.  The Tai Khuen fonts I've seen do this by
> relying on characteristics of Tai Khuen spelling.  The rules don't hold
> for Northern Thai, and consequently the subscript portions of
> successive orthographic syllables can overwrite one another.  A
> sophisticated font could check for clashes, but that needs the
> orthographic syllables to be passed to the shaper together.

I'm not sure I understand.  Does HarfBuzz know about these advancement
features?  We rely on HarfBuzz to give us back as many grapheme
clusters as it sees fit for a given chunk of text, and we expect each
grapheme cluster to include glyphs with relative offsets as needed by
the script and the font.

IOW, this job is delegated to the shaping engine, such as HarfBuzz;
Emacs just takes the glyphs and offsets HarfBuzz gives us and blindly
obeys them.