bug#20140: 24.4; M17n shaper output rejected

[Eli Zaretskii <eliz@gnu.org>]

> Date: Sun, 6 Feb 2022 22:09:58 +0000
> From: Richard Wordingham <richard.wordingham@ntlworld.com>
> Cc: larsi@gnus.org, 20140@debbugs.gnu.org, Kenichi Handa <handa@gnu.org>
> 
> > > Sad to see that Khaled Hosny's suggestion not to use composition
> > > rules seems not to have been taken.  
> > 
> > You mean, to pass all the text via HarfBuzz instead?  That makes the
> > Emacs redisplay painfully slow, and would require a complete redesign
> > of how we render text to be bearable.  So as long as such a redesign
> > is not available, we cannot use that advice.
> 
> Except for Malayalam!  (Subexpression XX* in indian.el at the moment.)

(That was changed lately.  But it is a tangent.)

> > > They're complicated by the facts that the 'regular expressions' are
> > > not interpreted as regular expressions and they are not interpreted
> > > as closed under canonical equivalence.  I therefore calculate the
> > > regular expression.  
> > 
> > I'm not sure I understand the issue: what you do seems to be very
> > similar to what we do for the Indic scripts in indian.el, so what kind
> > of complications are you talking about here?
> 
> Well, those rules themselves are a bit odd.  Why are you composing
> single clusters?  Why are you breaking clusters where Microsoft
> imitators are likely to insert dotted circles?

I'm not sure this is what I asked.  I asked why you think this way of
defining patterns for composition rules is in any way exceptional.  It
seems pretty much boilerplate to me.

> The best sources are the regular expressions in the proposals, but they
> missed out the combination of tone mark and final consonant signs.

Can you be more specific about those proposals?  Any specific
pointers?

Also, does this mean there's currently no widely accepted agreement
regarding Tai Tham shaping?  What do native readers of that script
expect?

> What do you mean by 'shaping'?

Whatever is needed to produce correct display from a sequence of
codepoints in a given script.

> > At least for the dotted circle case, Emacs has a general composition
> > rule; see compose-gstring-for-dotted-circle and the corresponding rule
> > in composite.c.  So I'm not sure we need anything specific to Tai Tham
> > there.
> 
> Does the 3-character Khmer sequence "◌្ក" <U+25CC, U+17D2, U+1780> work
> in Version 28?  It doesn't in Version 26.3.  It should look like a
> dotted circle with the lower part of ក្ក below it.  In Version 26.3, I
> don't even get the consonant U+1780 subscripted!

No, it doesn't produce what you want (though the 2nd and the 3rd
characters do combine), but that's not surprising: the general rules
for U+25CC that we have cover only a single combining mark after it:

  (aset composition-function-table #x25CC
	`([,(purecopy ".\\c^") 0 compose-gstring-for-dotted-circle]))

So a sequence of more than one character after U+25CC needs an
explicit rule to work.  What is the rule in this case?  (And what does
Khmer have to do with the question I asked, which is about Tai Tham?)

> With HarfBuzz, if you don't compose U+25CC with the following mark, you
> are very likely to get two dotted circles - are you deliberately
> deleting one?

No.  And I don't get 2 dotted circles with the above in Emacs 28 with
HarfBuzz.

Anyway, Khmer is a separate issue.