bug#44076: 27.1.50; Weird display effect with RTL text in lists

[Eli Zaretskii <eliz@gnu.org>]

On October 19, 2020 10:39:29 AM GMT+03:00, Joost Kremers <joostkremers@fastmail.fm> wrote:
> 
> In an IELM buffer:
> 
> ```
> *** Welcome to IELM ***  Type (describe-mode) for help.
> ELISP> (setq mylist nil)
> nil
> 
> ELISP> (push "2nd-place clitics" mylist)
> ("2nd-place clitics")
> 
> ELISP> (push "صباح الخير" mylist)
> ("صباح الخير" "2nd-place clitics")
> 
> ELISP> (cl-first mylist) 
> "صباح الخير"
> 
> ELISP> (cl-second mylist)
> "2nd-place clitics"
> ```
> 
> The contents of the list is correct, but it is displayed incorrectly,
> as shown.
> This is not dependent on IELM, it also happens in the `*scratch*`
> buffer or in
> the echo area when evaluating `mylist`. The problem does not occur if
> the second
> element in the list does not start with a number.
> 
> I used Arabic in the example, but Hebrew shows the same problem. The
> font is
> also not the issue: the above interaction is from `emacs -Q`, which
> uses a
> different font for Arabic (and Hebrew) on my system than when I use my
> full
> config. The problem occurs in either case.

The display is correct according to the rules of the UBA, the Unicode Bidirectilnal Algorithm, which mandates how mixed RTL and LTR text should be reordered for display.  As you see, the result can be confusing when structured text is rendered as if it were a single plain-text string.

Emacs has capabilities to do better, so if we are willing to invest efforts in this direction, I can suggest a couple of possible solutions, which will have to do with using Unicode control characters around rhe textual representation of Lisp data.