Bidi TODO (was: Handling invisible text in bidirectional display)

[Eli Zaretskii <eliz@gnu.org>]

At Richard's request, I'm posting the todo list composed out of notes
I keep while working on the bidirectional support in Emacs.  What is
already done and underwent some minimal testing is marked by "(done)".

Please note that this list is by no means complete, not even close.
I'm discovering new issues as I go along.

-------------------------------------------------------------------

* Basic display engine infrastructure

** Basic bidirectional display

*** Bidirectional iteration through display elements

  (The names below in this section are subroutines in xdisp.c)

**** next_element_from_buffer (done)

**** next_element_from_string/c_string

**** next_element_from_display_vector

  Need to decide whether to reorder the original characters of the
  ones that come out of display tables.  Maybe have an option to do
  both.

**** next_element_from_image

**** next_element_from_stretch

*** Display of text without properties (done)

*** Line truncation and continuation (done)

*** Scrolling (done)

*** Cursor positioning (done)

*** Text insertion and deletion (done)

*** Left-to-right and right-to-left paragraphs

**** Paragraph direction variable (done)

**** Dynamic determination of paragraph direction (done)

**** Support for paragraph-start and paragraph-separate that don't
begin at line beginning

*** Horizontal cursor motion (done)

*** Vertical cursor motion

  Needs a fixed current-column/move-to-column

**** Fix visual-line-mode for bidirectional text and mixed R2L/L2R lines

  Also needs a fixed current-column/move-to-column

*** Text properties and overlays

**** Properties that determine faces (done)

  This includes the `fontified' property.

**** Invisible text (in progress)

**** `display' property

***** Property value is a string

***** Property value is an image

***** Other property values

*** Selective display and ellipsis

*** Composed characters

** Terminal-specific back-ends

*** TTY back-end

  Largely done, with the single possible exception of composite
  characters (see term.c:append_composite_glyph)

*** Support reversed (R2L) glyph rows in xdisp.c:append_*

  This is a prerequisite for the GUI back-ends (below).

*** X11 back-end

*** MS-Windows back-end

*** NS back-end

* New display features specific for bidi

** Mode to show/hide LRE, RLO, PDF, and other formatting characters

** Mode to show bidirectional level of each character

** Mirrored continuation and arrow glyphs in R2L glyph rows

* Display-related subroutines

** Support bidirectional text in current-column, move-to-column, etc.

*** Many packages use these primitives: fix them for bidi

  This might need a separate solution for each one of the packages,
  depending on how it uses these primitives and for what purpose.

* Design issues

** Load bidirectional properties of characters from uni-bidi.el

  (Currently, a char-table of properties is created in C at compile
  time, independently of Unicode properties we have in uni-*.el.)

*** When to load? at dump time? upon first use?

*** Load everything at once? only for the charset being used?

** Load mirrored properties from Unicode DB

  (Currently, only ASCII characters with mirrored properties are
  supported.)

*** Same issues as with bidirectional properties above

* Applications

** UI issues in typing bidi text

  See these URLs:

    http://www-01.ibm.com/software/globalization/topics/bidiui/index.jsp
    "better tutorial on mixed directionality" in comp.windows.x.intrinsics
    http://osdir.com/ml/region.israel.ivrix.discuss/2003-05/msg00031.html
    https://bugzilla.gnome.org/show_bug.cgi?id=70451
    https://bugzilla.gnome.org/show_bug.cgi?id=136529
    https://bugzilla.gnome.org/show_bug.cgi?id=116626

** Reverse mirrored characters in input methods for bidi languages

  (Note: Some localized keyboards do that themselves.)

** Add a convenient way of typing formatting characters (RLO, LRE, PDF)

** Support in programming modes for bidi reordering in strings and comments

  Maybe we need special text properties?