bug#11860: 24.1; Arabic - Harakat (diacritics, short vowels) don't appear

[Eli Zaretskii <eliz@gnu.org>]

> From: Kenichi Handa <handa@gnu.org>
> Cc: 11860@debbugs.gnu.org, smias@yandex.ru, jasonr@gnu.org
> Date: Sun, 19 Aug 2012 22:20:27 +0900
> 
> >   If fLogicalOrder is set to FALSE, the function
> >   generates right-to-left items in reverse order so that ScriptTextOut
> >   does not have to reverse them before calling ExtTextOut.
> 
> Doesn't it mean that, if fLogicalOrder is TRUE, ScriptPlace
> generates xadvance and left/right bearing while expecting
> that the glyphs are re-ordered before actually rendered?

It could mean that.  But it's still only a guess, as the documentation
is unclear.

> > You can find some details here:
> 
> >    http://msdn.microsoft.com/en-us/library/windows/desktop/dd317792%28v=vs.85%29.aspx
> 
> > I hope this will allow you to understand the meaning of the above
> > code, by looking at how the results are used in the calls to
> > LGLYPH_SET_* macros right below the above snippet.
> 
> Thank you for the pointer.

Here are 3 more:

  http://maxradi.us/documents/uniscribe/
  http://www.catch22.net/tuts/uniscribe-mysteries
  http://www.catch22.net/tuts/more-uniscribe-mysteries

> I have questions in the section
> "Display Text Using Uniscribe" in that page.  The step 2
> says:
> 
>     1. Extract an array of bidirectional embedding levels,
>        one per range. The embedding level is given by
>        (SCRIPT_ITEM) si.(SCRIPT_ANALYSIS) a. (SCRIPT_STATE)
>        s.uBidiLevel.
> 
> From what and how to extract that array?

From items[i].a.s.uBidiLevel.  I showed an example in an earlier
message, where you can see that uBidiLevel is 1 (i.e. RTL).

We don't use this information because Emacs reorders characters
itself, it doesn't need the UAX#9 implementation contained in
Uniscribe.

>     2. Pass this array to ScriptLayout to generate a map of
>        visual positions to logical positions.
> 
> There's no place in Emacs that calls ScriptLayout.  Isn't it
> a problem?

I don't think so, at least not directly.  ScriptLayout actually draws
the shaped glyphs on the screen.  Emacs doesn't use it because it
draws the glyphs by itself, using the information in the glyph-strings
generated from the data returned by the shaping engine.  Or am I
missing something?

However, the ScriptLayout issue affects us indirectly because most
(all?) other applications do use ScriptLayout, where Emacs draws
glyphs by itself.  That is why one of the references above explicitly
says:

  pGoffset  [...] The application generally doesn’t have to pay
      attention to these offsets at all. They are generated by
      ScriptPlace and used by ScriptTextOut, and all the application
      needs to do is keep track of the values in the meantime.

The problem is, Emacs does use "all these offsets" and other stuff,
and so we are being hit by their insufficient documentation.