Re: bidi-display-reordering is now non-nil by default

[Kenichi Handa <handa@m17n.org>]

In article <E1QoqI3-0004zu-Ij@fencepost.gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:

> Yes, but we are not talking about the internal handling.  We are
> talking about display, which is an external, user-visible part of the
> issue.  The Unicode Bidirectional Algorithm is a specification for
> converting a stream of text into an array of character glyphs on the
> screen.  It is not a wire protocol.  Nor is it specific to text
> external to Emacs: after all, the internal storage of text in Emacs,
> as in many other applications, is just a linear byte stream.

> Let's go back to the issue at hand: the directional control
> characters.  A quote from UAX#9:

>   [...] there are circumstances where an implicit bidirectional
>   ordering is not sufficient to produce comprehensible text. To deal
>   with these cases, a minimal set of directional formatting codes is
>   defined to control the ordering of characters when rendered. This
>   allows exact control of the display ordering for legible interchange
>   and ensures that plain text used for simple items like filenames or
>   labels can always be correctly ordered for display.

>   The directional formatting codes are used only to influence the
>   display ordering of text. [...]

UAX#9's algorithm is for a paragraph, and as for paragraph,
it says:

  The algorithm reorders text only within a paragraph;
  characters in one paragraph have no effect on characters
  in a different paragraph. Paragraphs are divided by the
  Paragraph Separator or appropriate Newline Function (for
  guidelines on the handling of CR, LF, and CRLF, see
  Section 4.4, Directionality, and Section 5.8, Newline
  Guidelines of [Unicode]). Paragraphs may also be
  determined by higher-level protocols: for example, the
  text in two different cells of a table will be in
  different paragraphs.

The case of summary line belongs to the last sentence above.
In a summary line, we surely have multiple cells, and
UAX#9's algorithm should be applied to each cell
independently.

So what we need is a way to tell the display engine about
cells themselves or about cell-separators (or artificial
paragraph separators).

To display something like summarly line, we always use
separating character(s) (tab, space, |, ...).  How about
putting the text property `paragraph-separator' to them, and
making handle_stop (in xdisp.c) to pay attention to that
property too.

---
Kenichi Handa
handa@m17n.org