Re: Handling invisible text in bidirectional display

[Eli Zaretskii <eliz@gnu.org>]

> From: Richard Stallman <rms@gnu.org>
> CC: dak@gnu.org, emacs-bidi@gnu.org, emacs-devel@gnu.org
> Date: Sun, 17 Jan 2010 11:05:03 -0500
> 
> Your description of the interaction of invisible with bidi seems right
> to me, but I was surprised by your response to this
> 
>     > I think a reasonable model is to display the text as if the invisible
>     > characters are not there.
> 
>     I think the resulting change of the visual order will surprise the
>     users.  It also complicates implementation, which for me is an
>     important downside.
> 
> because I don't see a differnce between "display the text as if the
> invisible characters are not there" and what you described (in the
> no-ellipsis case).  What you described
> 
>     One consequence is that a run of invisible characters can now be split
>     into several non-contiguous runs.  For example, this text:
> 
>       abcABCxyz
> 
>     with c and A covered by an invisible property will be displayed as
> 
>       abCBxyz
> 
> seems to agree entirely with that description.  Is there something I
> have misunderstood?

It's just that the example I showed is a simple one, where the
distinction doesn't show.  However, in general, the bidirectional
reordering can cause simple insertion to have non-local effect on the
visual order.  This is because so-called weak characters (digits,
plus, minus, punctuation, etc.) and neutral characters (blanks, tabs,
etc.) change their directionality depending on the surrounding text.

Here's an example.  Suppose the buffer contains this text:

  ABCD(4+5)*3

(As usual, capital letters are strong right-to-left characters.)  This
text will be displayed as

  3*(5+4)DCBA

By contrast, the same text with `+' removed will display as

  3*(45)DCBA

As you see, 4 and 5 switched their visual order.  This means that if
we first remove the invisible characters and then reorder, the 4 and 5
will appear in a different order when `+' is invisible than when it is
visible.  With my suggestion, we _first_ reorder and _then_ hide
invisible characters, so the display with `+' invisible will be

  3*(54)DCBA

which I think is less surprising.