From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Naheel Azawy Newsgroups: gmane.emacs.bugs Subject: bug#32516: 26.1; Reshape Arabic Characters in the Terminal Date: Sat, 25 Aug 2018 01:25:22 +0300 Message-ID: References: <83o9ds9t24.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="00000000000025326a057435dc15" X-Trace: blaine.gmane.org 1535152231 3033 195.159.176.226 (24 Aug 2018 23:10:31 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 24 Aug 2018 23:10:31 +0000 (UTC) Cc: 32516@debbugs.gnu.org To: eliz@gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Aug 25 01:10:27 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ftLDm-0000Tp-Qy for geb-bug-gnu-emacs@m.gmane.org; Sat, 25 Aug 2018 01:10:26 +0200 Original-Received: from localhost ([::1]:43841 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ftKXx-0002rt-4W for geb-bug-gnu-emacs@m.gmane.org; Fri, 24 Aug 2018 18:27:13 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:57452) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ftKXq-0002rn-3j for bug-gnu-emacs@gnu.org; Fri, 24 Aug 2018 18:27:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ftKXm-0000CO-Tv for bug-gnu-emacs@gnu.org; Fri, 24 Aug 2018 18:27:06 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:55040) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ftKXm-0000CE-Pf for bug-gnu-emacs@gnu.org; Fri, 24 Aug 2018 18:27:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ftKXm-0005qX-GK for bug-gnu-emacs@gnu.org; Fri, 24 Aug 2018 18:27:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Naheel Azawy Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 24 Aug 2018 22:27:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 32516 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 32516-submit@debbugs.gnu.org id=B32516.153514956722408 (code B ref 32516); Fri, 24 Aug 2018 22:27:02 +0000 Original-Received: (at 32516) by debbugs.gnu.org; 24 Aug 2018 22:26:07 +0000 Original-Received: from localhost ([127.0.0.1]:60058 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ftKWt-0005pL-2n for submit@debbugs.gnu.org; Fri, 24 Aug 2018 18:26:07 -0400 Original-Received: from mail-lf1-f50.google.com ([209.85.167.50]:41848) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ftKWr-0005op-C7 for 32516@debbugs.gnu.org; Fri, 24 Aug 2018 18:26:05 -0400 Original-Received: by mail-lf1-f50.google.com with SMTP id l26-v6so7784931lfc.8 for <32516@debbugs.gnu.org>; Fri, 24 Aug 2018 15:26:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=VC2BwsWEePSdf7qibEmnSkSUeNP3ZC7J7s0DlS1UV9o=; b=NXEyRRmQ6Dhml2EEwtJlrvi76rkT5evfUKY9CD50gk1TNNIdjXF/UHsrCgUWm30twS t0tPn9DVnk69g7iPwmPl9G+CceMHT5fwVb7ym1Bb0VpElP+i+Hw3HPaum80KSNiQe4p0 Zibq9s1eOO2BZnh4GE6WIipNvU9dRifz11qyrF30HsAUh08a+Nk94a9WEDSmgwA2wcCc mdLejNSYAoHl4iXA+LlJEtlk6u3XgirmsqNnzo3vhp19QggQidnoFZWmq2syMKflWmJn so6TCUw1tVNSBuGc8cCKqDeIdxdFMrp0fdi3YiM04DBrsuod0PcUDlQDTpG4Z7fkEXyc QWJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=VC2BwsWEePSdf7qibEmnSkSUeNP3ZC7J7s0DlS1UV9o=; b=bAV5ByKAjB5tH68kZqr5dPSxfHTSFdqelqNWBvODj/sP22fUOxuAjl1bs394pwDl9v X9GDOZ5qWDU8A56o/KjQKKgKq3o2Ydauh3hcEsW9WiJzu0LWXEw2lUWEfebiEnevy+x6 bx2OGmfRIHpYPsNs57n4FwLjKmyy8l8KGbpgYcDeAipKoa6MDT6Zov/CjygkvCaGmzAh IlPNabh6koahb2gsHarH7h3+dfxGGz8WfXfKiQFLroRgSU9OVOxLZVginkuHsxvq0C/G 0bF9akV7op2TVJD6/qDRbjQp3BeUqRm4+tXNRQ0S7rg+ptTDngp2Yu6yj4z3CuTiFnr3 T+fg== X-Gm-Message-State: APzg51AHJQRHFkbnfnWy9r3RDC9Q2On7ZNV+ZWYAhxx87A8cD9uRKT9x PfoAV5Mt5xqDnw2IHkmgvR8ToF2Tk6CXLb1/5m8= X-Google-Smtp-Source: ANB0VdbpK8H2lDs9j3mKqRr4lgIsAZAi6tNkm0A3R2JLP8I/iSO3wL1Th9oCZbENB2pHJpDFMzYYNbhL6EktyDN80r4= X-Received: by 2002:a19:f83:: with SMTP id 3-v6mr2417438lfp.131.1535149559066; Fri, 24 Aug 2018 15:25:59 -0700 (PDT) In-Reply-To: <83o9ds9t24.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:149731 Archived-At: --00000000000025326a057435dc15 Content-Type: text/plain; charset="UTF-8" Thanks for you quick reply! Nope, auto-composition-mode didn't help. But https://github.com/behdad/bicon/pull/20 did! Now `bicon.bin --reshape-only emacsclient -nw $@` is pretty ok. It still has some issues but that should be fixed in bicon. Thanks, On Fri, Aug 24, 2018 at 9:29 AM Eli Zaretskii wrote: > > From: Naheel Azawy > > Date: Fri, 24 Aug 2018 00:06:19 +0300 > > > > The problem occurs in the terminal mode with Arabic characters. Emacs > > handles the bidirectional text correctly in terms of direction but it > > doesn't reshape the characters depending on its location. More details > > can be seen with screenshots on my question in Emacs stack exchange: > > https://emacs.stackexchange.com/questions/44144/arabic-in-terminal-mode > > I have also written a little minor mode as workaround to this problem: > > https://github.com/Naheel-Azawy/emacs-arabic-reshaper > > but an actual solution is expected to only render the characters while > > they're displayed in the screen and not reshape the entire file as I'm > > doing with my minor mode. > > Does it help to turn off auto-composition-mode? > > If that doesn't help, then I'm afraid you will have to look for a > terminal emulator that supports character composition and Arabic > shaping, but can let you turn off its bidi reordering. (PuTTY can do > that, but it's for logging from Windows into Unix systems, so AFAIK > will not run natively on Posix hosts.) > > You see, Emacs supports Arabic shaping by using a shaping engine > (a.k.a. "font backend") that works with OTF/TTF fonts, which know what > glyphs are needed to represent a correctly-shaped Arabic letter or > ligature composed from certain codepoints. But this only works on GUI > frames, because on text-mode frames, Emacs just writes the codes of > the characters to the terminal driver, it cannot specify font glyphs > to display those characters. > > As you have discovered, disabling bidi-display-reordering is not the > solution, because there's much more to bidi editing support in Emacs > than just reordering characters for display. That is why we need > bidi-aware display engine in Emacs in the first place, and cannot rely > on external solutions like what exists in Pango etc., or in terminal > emulators, for that matter. > > If BiCon can let you disable its reordering, and still perform > shaping, then that's your solution. If it doesn't, then maybe asking > its developers to provide such a feature, or coding it yourself, is > the way forward for you and other Arabic-speaking users of text-mode > frames in Emacs. > > Bottom line: this isn't a bug in Emacs, this is how the code was > designed and supposed to work on TTY frames. > > Of course, if someone knows a way to communicate character shaping to > a terminal driver, we could think of implementing that in Emacs. But > does such a protocol exist? > > Thanks. > --00000000000025326a057435dc15 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks for you quick rep= ly!
Nope, auto-composition-mode didn't help.
Now `bicon.bin --reshape-only emacsclient = -nw $@` is pretty ok.
It still has some issues but that should be= fixed in bicon.

Thanks,

On Fri, Aug 24, 2018 at 9= :29 AM Eli Zaretskii <eliz@gnu.org&g= t; wrote:
> From: Naheel Azawy &= lt;naheelazawy@g= mail.com>
> Date: Fri, 24 Aug 2018 00:06:19 +0300
>
> The problem occurs in the terminal mode with Arabic characters. Emacs<= br> > handles the bidirectional text correctly in terms of direction but it<= br> > doesn't reshape the characters depending on its location. More det= ails
> can be seen with screenshots on my question in Emacs stack exchange: > https://emacs.stackexcha= nge.com/questions/44144/arabic-in-terminal-mode
> I have also written a little minor mode as workaround to this problem:=
> https://github.com/Naheel-Azawy/emacs-ara= bic-reshaper
> but an actual solution is expected to only render the characters while=
> they're displayed in the screen and not reshape the entire file as= I'm
> doing with my minor mode.

Does it help to turn off auto-composition-mode?

If that doesn't help, then I'm afraid you will have to look for a terminal emulator that supports character composition and Arabic
shaping, but can let you turn off its bidi reordering.=C2=A0 (PuTTY can do<= br> that, but it's for logging from Windows into Unix systems, so AFAIK
will not run natively on Posix hosts.)

You see, Emacs supports Arabic shaping by using a shaping engine
(a.k.a. "font backend") that works with OTF/TTF fonts, which know= what
glyphs are needed to represent a correctly-shaped Arabic letter or
ligature composed from certain codepoints.=C2=A0 But this only works on GUI=
frames, because on text-mode frames, Emacs just writes the codes of
the characters to the terminal driver, it cannot specify font glyphs
to display those characters.

As you have discovered, disabling bidi-display-reordering is not the
solution, because there's much more to bidi editing support in Emacs than just reordering characters for display.=C2=A0 That is why we need
bidi-aware display engine in Emacs in the first place, and cannot rely
on external solutions like what exists in Pango etc., or in terminal
emulators, for that matter.

If BiCon can let you disable its reordering, and still perform
shaping, then that's your solution.=C2=A0 If it doesn't, then maybe= asking
its developers to provide such a feature, or coding it yourself, is
the way forward for you and other Arabic-speaking users of text-mode
frames in Emacs.

Bottom line: this isn't a bug in Emacs, this is how the code was
designed and supposed to work on TTY frames.

Of course, if someone knows a way to communicate character shaping to
a terminal driver, we could think of implementing that in Emacs.=C2=A0 But<= br> does such a protocol exist?

Thanks.
--00000000000025326a057435dc15--