Re: how reliable is rendering of complex scripts?

[Werner LEMBERG <wl@gnu.org>]

>> I wonder how reliable emacs displays complex scripts like
>> Devanagari or Arabic.
>
> AFAIK, no one has ever performed a study about this, let alone
> repeated it when the relevant standards changed.

This essentially means that Emacs developers wait for users to report
bad renderings, right?

>> For example, the maintainers of the HarfBuzz library did extensive
>> comparisons of the rendering results with the MS engine to iron out
>> zillions of small buglets in OpenType handling.
>
> At least on MS-Windows, Emacs uses the MS engine directly, so some
> of similar buglets should not affect us on Windows.

Well, this makes Emacs on MS-Windows really superior to other
platforms in this area, which is less than ideal...  I mean `superior'
in the sense that the rendering results on MS-Windows are well tested
and can be trusted in general, something that is missing otherwise.

>> AFAIK, Emacs relies on the m17n libraries, at least on GNU/Linux (no
>> idea about other environments), controlling the OpenType handling
>> (partially?) with Lisp code
>
> That is correct.  And while the shaping engines, like libm17n-flt
> and Uniscribe, are beyond the scope of Emacs maintenance, the
> supporting Lisp and C code is on our table.  However, we currently
> lack a maintainer in that area (have been lacking for a long time),
> so I guess we are not up to speed with the latest developments.  I'm
> talking first and foremost about the definitions of
> character-composition patterns, which tell Emacs which sequences of
> characters should be rendered as a single grapheme cluster.  There's
> a lot to do in this area for various languages.

Given that HarfBuzz is very mature today, and that it has been
extensively tested against Windows rendering results, and that it also
contains a large corpus of test cases for complex ligatures together
with a simple test TTY program (`hb-shape'), I suggest that someone
(probably Ken'ichi) writes a similar test program for libm17n so that
diffing would be possible.

  https://github.com/behdad/harfbuzz/tree/master/test/shaping
  https://github.com/behdad/harfbuzz/tree/master/util

>> are there test suites to compare the results?
>
> There's a test suite for bidirectional display, but it only tests
> the reordering of characters for display, not the shaping.  There's
> nothing else, AFAIK.  If you, or someone else, can work on adding
> one, that'd be great.

In case someone is working on this issue, asking the HarfBuzz
developer for assistance might be a good thing.  I guess that they
have even larger corpora that could be probably provided for testing
purposes.


    Werner