Re: bidi-display-reordering is now non-nil by default

[Michael Welsh Duggan <md5i@md5i.com>]

Eli Zaretskii <eliz@gnu.org> writes:

> You are wrong.  LRM, RLM, etc. exist _precisely_ for these purposes:
> to arrange bidirectional text for display as the display designer
> wishes, where the implicit reordering mandated by the UBA does not
> give satisfactory results.  These _explicit_ directional control
> characters exist for when the _implicit_ reordering fails.  It's the
> functional equivalent of the corresponding HTML directives -- you
> won't say that HTML is "suboptimal" when it dictates to the browser
> how to render bidirectional text, would you?

[...]

> This issue was discussed at length long ago, when the basic design of
> bidi support for Emacs was on the table.  Text properties were
> suggested almost immediately, then rejected after prolonged debates,
> because they simply aren't up to this job, and because the Unicode
> Standard already tells us how to DTRT, we just need to heed.

I have no issue with using directional control characters in the buffer
to handle alignment issues, etc.  I do have the following questions,
though, and am looking for suggestions for how they should be
implemented.

Scenario: Say we are desirous of updating XML mode to protect the
neutral characters that make up tags so they don't end up in visually
confusing locations.  This can be done by adding proper directional
codes in the right locations.  However, we would need to solve the
following two questions:

1) We need to strip these directional characters (but not other ones in
the data sections of the XML) when saving the files.

2) We don't really want cursor movement to stop on each of these
directional characters.  They are there to make display look correct,
but are not part of the data stream that the user is editing.

3) When doing search, people are not going to want to have to add the
directional codes to their search strings in order to, say, search for a
tag next to some text.  These formatting characters need to be
"invisible" with respect to isearch and friends.

4) Not as important, but in this case we would want the formatting
characters we add in order to make markup display appropriately to be
non-visible, whereas formatting characters in the user text, we might
want displayed using thin-space or as a boxed character.

How should these problems be solved?  Or what feature(s) should be added
to allow this to be implemented?

-- 
Michael Welsh Duggan
(md5i@md5i.com)