bug#32516: 26.1; Reshape Arabic Characters in the Terminal

[Eli Zaretskii <eliz@gnu.org>]

> From: Naheel Azawy <naheelazawy@gmail.com>
> Date: Fri, 24 Aug 2018 00:06:19 +0300
> 
> The problem occurs in the terminal mode with Arabic characters. Emacs
> handles the bidirectional text correctly in terms of direction but it
> doesn't reshape the characters depending on its location. More details
> can be seen with screenshots on my question in Emacs stack exchange:
> https://emacs.stackexchange.com/questions/44144/arabic-in-terminal-mode
> I have also written a little minor mode as workaround to this problem:
> https://github.com/Naheel-Azawy/emacs-arabic-reshaper
> but an actual solution is expected to only render the characters while
> they're displayed in the screen and not reshape the entire file as I'm
> doing with my minor mode.

Does it help to turn off auto-composition-mode?

If that doesn't help, then I'm afraid you will have to look for a
terminal emulator that supports character composition and Arabic
shaping, but can let you turn off its bidi reordering.  (PuTTY can do
that, but it's for logging from Windows into Unix systems, so AFAIK
will not run natively on Posix hosts.)

You see, Emacs supports Arabic shaping by using a shaping engine
(a.k.a. "font backend") that works with OTF/TTF fonts, which know what
glyphs are needed to represent a correctly-shaped Arabic letter or
ligature composed from certain codepoints.  But this only works on GUI
frames, because on text-mode frames, Emacs just writes the codes of
the characters to the terminal driver, it cannot specify font glyphs
to display those characters.

As you have discovered, disabling bidi-display-reordering is not the
solution, because there's much more to bidi editing support in Emacs
than just reordering characters for display.  That is why we need
bidi-aware display engine in Emacs in the first place, and cannot rely
on external solutions like what exists in Pango etc., or in terminal
emulators, for that matter.

If BiCon can let you disable its reordering, and still perform
shaping, then that's your solution.  If it doesn't, then maybe asking
its developers to provide such a feature, or coding it yourself, is
the way forward for you and other Arabic-speaking users of text-mode
frames in Emacs.

Bottom line: this isn't a bug in Emacs, this is how the code was
designed and supposed to work on TTY frames.

Of course, if someone knows a way to communicate character shaping to
a terminal driver, we could think of implementing that in Emacs.  But
does such a protocol exist?

Thanks.