bug#11700: 24.1.50; Bad interaction between BiDi and org-tables

[Eli Zaretskii <eliz@gnu.org>]

> From: Nicolas Goaziou <mail@nicolasgoaziou.fr>
> Date: Fri, 08 Dec 2017 18:08:57 +0100
> Cc: dov.grobgeld@gmail.com, 11700@debbugs.gnu.org
> 
> For tests, we use `org-test-with-temp-text' macro, e.g.,
> 
>   (org-test-with-temp-text "| a | b |\n| c | d |"
>     ... do something in that buffer ...)

You didn't say that this macro is only available in the Org's Git
repository...

Anyway, since it's a very simple macro, I just used its guts below.

I found both methods doing well, so I'm going to show both, and let
you decide which one is better.  The below provides a demonstration of
each method by displaying a buffer with a table whose columns include
both L2R and R2L text, such that the table columns are still laid out
left to right, unlike when one just types the characters in the cells.

Method 1: wrap each string with (invisible) bidi formatting control
characters which isolate each string from the surrounding text.

(defun bidi-isolate-string (str)
  (concat (propertize (string ?\x2068) 'invisible t)
	  str
	  (propertize (string ?\x2069) 'invisible t)))

(with-current-buffer (get-buffer-create "bidi-org-table1")
  (org-mode)
  (insert (concat "| "
		(bidi-isolate-string "abcd")
		" | "
		(bidi-isolate-string "efgh")
		" |\n| "
		(bidi-isolate-string "אבגד")
		" | "
		(bidi-isolate-string "הוזח")
		" |"))
  (pop-to-buffer "bidi-org-table"))

This has a minor issue: it fails to conceal the bidi control
characters on display, although I used the 'invisible' property for
that purpose.  I'm guessing that Org takes control of the 'invisible'
properties, in which case perhaps this method should use some other
property, if possible.  If it is not practical to conceal the bidi
controls on display, the following method is preferable.

Method 2: give the spaces around the cell text the display property
which makes the spaces serve as segment separators for the purposes of
the bidirectional reordering.

(defun bidi-separator-space ()
  (propertize " " 'display '(space :width 1)))

(with-current-buffer (get-buffer-create "bidi-org-table2")
  (org-mode)
  (insert (concat "|"
		  (bidi-separator-space)
		  "abcd"
		  (bidi-separator-space)
		  "|"
		  (bidi-separator-space)
		  "efgh"
		  (bidi-separator-space)
		  "|\n|"
		  (bidi-separator-space)
		  "אבגד"
		  (bidi-separator-space)
		  "|"
		  (bidi-separator-space)
		  "הוזח"
		  (bidi-separator-space)
		  "|"))
  (pop-to-buffer "bidi-org-table2"))

Let me know if I can help you further, or if you have additional
questions.