From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: hossein valizadeh Newsgroups: gmane.emacs.bugs Subject: bug#41005: problem with rendering Persian text in Emacs 27 Date: Fri, 5 Jun 2020 17:02:55 +0430 Message-ID: References: <831rmwc9ke.fsf@gnu.org> <35A46479-A62C-42FF-995B-B295FE3408C0@gnu.org> <08A9D65F-0C9C-4EE2-B3B9-2AA25BFFAD54@gnu.org> <878sh35j6f.fsf@gmail.com> <83y2p3as6c.fsf@gnu.org> <87pnae4nhx.fsf@gmail.com> <83lfl2av8z.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000005cecf105a7556d52" Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="62829"; mail-complaints-to="usenet@ciao.gmane.io" Cc: pipcet@gmail.com, 41005@debbugs.gnu.org, Nicholas Drozd To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Jun 05 14:31:11 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jhBV9-000GC8-5g for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 05 Jun 2020 14:31:11 +0200 Original-Received: from localhost ([::1]:40894 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jhBV8-0006oX-6y for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 05 Jun 2020 08:31:10 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:32774) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jhBV0-0006nF-KE for bug-gnu-emacs@gnu.org; Fri, 05 Jun 2020 08:31:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:36951) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jhBV0-0000i5-BZ for bug-gnu-emacs@gnu.org; Fri, 05 Jun 2020 08:31:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jhBV0-0002EN-6v for bug-gnu-emacs@gnu.org; Fri, 05 Jun 2020 08:31:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: hossein valizadeh Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 05 Jun 2020 12:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41005 X-GNU-PR-Package: emacs Original-Received: via spool by 41005-submit@debbugs.gnu.org id=B41005.15913602418547 (code B ref 41005); Fri, 05 Jun 2020 12:31:02 +0000 Original-Received: (at 41005) by debbugs.gnu.org; 5 Jun 2020 12:30:41 +0000 Original-Received: from localhost ([127.0.0.1]:48497 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhBUe-0002Dm-Co for submit@debbugs.gnu.org; Fri, 05 Jun 2020 08:30:40 -0400 Original-Received: from mail-lf1-f52.google.com ([209.85.167.52]:40435) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jhBUc-0002DZ-Nq for 41005@debbugs.gnu.org; Fri, 05 Jun 2020 08:30:39 -0400 Original-Received: by mail-lf1-f52.google.com with SMTP id h188so5681024lfd.7 for <41005@debbugs.gnu.org>; Fri, 05 Jun 2020 05:30:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=81uwVr4B7lymtPVB08wvxJaP6/uvVWlqQoK0PF6eMVE=; b=Cv2whqVYy6vGG8vq7pB6Z98g+tIdsWPZEzb/1CgW35l1a/A5qCodPzUNRCmcjylu9a pqy3Fl98r+zW/yUMQVM76RNIdMAB3uy9KNF/9MgueBMQMhZ1lVltmbCtUW5Feo8qssPQ 00hYwyXMxss3xnfXqEqzttoX3VSCia3MhUriU/mDVTz7e1DpNfYXaEaDB+IX1h+KAYDG lchCrMAjw1E+X3RGmUFR6z8oa83HQc2qCm7RKT1pbQn+LCwSO5sHMqPXWXMGbKC7na0y N0LSMU7Myhbnjq2VouDwTwojrXxY5uenxZyTSC1hmKnnOK0EfQZfWU8IoN7pF09LjNW2 mbvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=81uwVr4B7lymtPVB08wvxJaP6/uvVWlqQoK0PF6eMVE=; b=sQvv5uq0ttbBltXGeX8E7OhpLpZwnN4FzPVvL4Uz2Wc5pCd5hgiVT3FRAn8nPIXb/B HYFj0stuIIBVdeKjlKMr62bMEPoSGZTn+phaZxbOYTZgYIBYIv02wc3BIIpoagnu0YlB dmvj0WV0yZTURDkzL9GQ9Bi1sO1nP7VjsMhdZEuwT4obi421aP0HLGE2g4fEYJDso/Ee Rt33zJFsqfD0BRzK40dXtu44ejg1ikDpRT6TGRcNXSN/y2Yi7MfwVCz2crGleqDtu9cp 3vv+JiM34f60FTT+i83rER+Fvv7RKz65pszGC2Y53zLpxeo7+GH8523JyQ+T5qVnWWLI ay5g== X-Gm-Message-State: AOAM532UKYwgZe7NpcN/LVpmLwom+H3b6Xj6X83Ix/waNTrf4kYU3QRt yyhx5PMUIS2uwlyhMA+bWF2Y7ijq1zktOsRSCK4= X-Google-Smtp-Source: ABdhPJy8r5IWPZXOBlHHGquOFCoZxYzRqvaNDllxZiRQl+WRivKuzKoRQL3UOg0N0tcc+p0GjCXUZ590fV2c1iYIhlY= X-Received: by 2002:a19:c1c3:: with SMTP id r186mr5124084lff.49.1591360232417; Fri, 05 Jun 2020 05:30:32 -0700 (PDT) In-Reply-To: <83lfl2av8z.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:181554 Archived-At: --0000000000005cecf105a7556d52 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable > Please tell which patch was that, and in what Emacs version you tried > it. (Please understand that you are generally talking to people some > of whom don't read Arabic or Persian, so the more details you supply > the less misunderstanding and confusion will follow, and the faster > this problem will be solved.) I'm sorry, my English is so bad, at least in writing. That's why it's hard for me to give a full explanation. My Emacs version: 27.0.91 I applied this patch: diff --git a/src/hbfont.c b/src/hbfont.c index 576c5fe..4b3f64e 100644 --- a/src/hbfont.c +++ b/src/hbfont.c @@ -26,6 +26,7 @@ #include "composite.h" #include "font.h" #include "dispextern.h" +#include "buffer.h" #ifdef HAVE_NTGUI @@ -438,7 +439,11 @@ hbfont_shape (Lisp_Object lgstring, Lisp_Object direction) /* If the caller didn't provide a meaningful DIRECTION, let HarfBuzz guess it. */ - if (!NILP (direction)) + if (!NILP (direction) + /* If they bind bidi-display-reordering to nil, the DIRECTION + they provide is meaningless, and we should let HarfBuzz guess + the real direction. */ + && !NILP (BVAR (current_buffer, bidi_display_reordering))) { hb_direction_t dir =3D HB_DIRECTION_LTR; if (EQ (direction, QL2R)) ---------------------------------------------------------------------------= ----- > I cannot play this on my system, I see a bunch of ads (or what looks > like ads), and the name of a .mkv file. video file reuploaded (.mp4 and .ogg): https://srv-file9.gofile.io/download/cz7P41/out.mp4 https://srv-file4.gofile.io/download/Mwv8k4/out.ogg ---------------------------------------------------------------------------= ----- This patch solved the problems of eww, newsticker, (and possibly some other major modes) in displaying Persian/Arabic words. However, if one of the auto-fill-mode or column-number-mode is enabled, there is still the same problem in files that use Persian or Arabic characters. Especially when you want to go back a few characters in a line and add something to that line. ---------------------------------------------------------------------------= ----- > Please tell the codepoint of each character you type to reproduce > the problem and each Emacs command. For example, if you type in the following sentence: =D8=A7=DB=8C=D9=86 =D9=86=D8=A7=D9=85 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =D9= =85=DB=8C=E2=80=8C=D9=85=D8=A7=D9=86=D8=AF codepoint: \u0627\u06cc\u0646 \u0646\u0627\u0645 \u0627\u0633\u062a \u06a9\u0647 \u0645\u06cc\u200c\u0645\u0627\u0646\u062f Then go back a few characters in the same line and type words randomly. You will see that the letters in some words are displayed separately. I type a few words at random, after the word =D8=A7=DB=8C=D9=86 and before the word = =D9=86=D8=A7=D9=85 :=E2=80=8C =D8=A7=DB=8C=D9=86 =D9=81=D8=B1=D8=A7=D9=85=D9=88=D8=B4=DB=8C =D8=B1=D8=A7 = =D8=A8=D9=87 =D9=87=D9=85=D9=87 =D8=A7=DB=8C=D9=86=DA=A9=D9=87 =D9=81=D8=B1= =D9=85=D8=AA =D9=85=D8=B1=D8=A7=D8=AA=D8=A8 =D8=A7=D9=81=D8=AA=D8=AA=D8=A7= =D8=AD =DA=AF=D8=B1=D8=A7=D9=85=DB=8C =DA=AF=D8=B1=D8=A7=DB=8C=D8=B4 =D8=B3= =D8=B1=D8=A7=D8=B3=DB=8C=D9=85=D9=87 =D9=86=D8=A7=D9=85 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =D9=85=DB=8C=E2=80=8C=D9=85=D8=A7=D9=86=D8=AF codepoint: \u0627\u06cc\u0646 \u0641\u0631\u0627\u0645\u0648\u0634\u06cc \u0631\u0627 \u0628\u0647 \u0647\u0645\u0647 \u0627\u06cc\u0646\u06a9\u0647 \u0641\u0631\u0645\u062a \u0645\u0631\u0627\u062a\u0628 \u0627\u0641\u062a\u062a\u0627\u062d \u06af\u0631\u0627\u0645\u06cc \u06af\u0631\u0627\u06cc\u0634 \u0633\u0631\u0627\u0633\u06cc\u0645\u0647 \u0646\u0627\u0645 \u0627\u0633\u062a \u06a9\u0647 \u0645\u06cc\u200c\u0645\u0627\u0646\u062f This line should look like this: http://s12.picofile.com/file/8399190550/correct.png But if one of the auto-fill-mode or column-number-mode is enabled. it will be displayed this way: http://s13.picofile.com/file/8399190584/malformed.png ---------------------------------------------------------------------------= ----- > Please show a full recipe for reproducing this, starting from > "emacs -Q", and describing every step to reproduce the result. All steps are displayed in the video file. > I'd also greatly appreciate if you specifically point out at which > parts of the display to look at what differences to pay attention > to. This is needed to fully understand the problem and analyze its > root cause(s), given that not all of us can read the Arabic script. The letters that should normally be connected will be displayed separately: http://s12.picofile.com/file/8399222618/latest_screenshot.png ---------------------------------------------------------------------------= ----- --0000000000005cecf105a7556d52 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
> Please tell which patch was that, and in what Emacs vers= ion you tried
> it. =C2=A0(Please understand that you are generally t= alking to people some
> of whom don't read Arabic or Persian, so = the more details you supply
> the less misunderstanding and confusion= will follow, and the faster
> this problem will be solved.)

I= 'm sorry, my English is so bad, at least in writing. That's why it&= #39;s
hard for me to give a full explanation.
My Emacs version: =C2= =A027.0.91
I applied this patch:

diff --git a/src/hbfont.c b/src/= hbfont.c
index 576c5fe..4b3f64e 100644
--- a/src/hbfont.c
+++ b/sr= c/hbfont.c
@@ -26,6 +26,7 @@
=C2=A0#include "composite.h"=C2=A0#include "font.h"
=C2=A0#include "dispextern.h&qu= ot;
+#include "buffer.h"

=C2=A0#ifdef HAVE_NTGUI
@@ -438,7 +439,11 @@ hbfont_shape (Lisp_Object lgstring, Lisp_Object direc= tion)

=C2=A0 =C2=A0/* If the caller didn't provide a meaningful = DIRECTION, let HarfBuzz
=C2=A0 =C2=A0 =C2=A0 guess it. */
- =C2=A0if = (!NILP (direction))
+ =C2=A0if (!NILP (direction)
+ =C2=A0 =C2=A0 =C2= =A0/* If they bind bidi-display-reordering to nil, the DIRECTION
+ =C2= =A0 =C2=A0 =C2=A0 =C2=A0they provide is meaningless, and we should let Harf= Buzz guess
+ =C2=A0 =C2=A0 =C2=A0 =C2=A0the real direction. =C2=A0*/
= + =C2=A0 =C2=A0 =C2=A0&& !NILP (BVAR (current_buffer, bidi_display_= reordering)))
=C2=A0 =C2=A0 =C2=A0{
=C2=A0 =C2=A0 =C2=A0 =C2=A0hb_dir= ection_t dir =3D HB_DIRECTION_LTR;
=C2=A0 =C2=A0 =C2=A0 =C2=A0if (EQ (di= rection, QL2R))

----------------------------------------------------= ----------------------------
> I cannot play this on my system, I see= a bunch of ads (or what looks
> like ads), and the name of a .mkv fi= le.

video file reuploaded (.mp4 and .ogg):
https://srv-file9.gofile.io/downl= oad/cz7P41/out.mp4
https://srv-file4.gofile.io/download/Mwv8k4/out.ogg
-= ---------------------------------------------------------------------------= ----

This patch solved the problems of eww, newsticker, (and possibl= y some
other major modes) in displaying Persian/Arabic words. However, i= f one
of the auto-fill-mode or column-number-mode is enabled, there is s= till
the same problem in files that use Persian or Arabic characters.Especially when you want to go back a few characters in a line and add
= something to that line.

--------------------------------------------= ------------------------------------
> Please tell the codepoint of e= ach character you type to reproduce
> the problem and each Emacs comm= and.

For example, if you type in the following sentence:
=D8=A7= =DB=8C=D9=86 =D9=86=D8=A7=D9=85 =D8=A7=D8=B3=D8=AA =DA=A9=D9=87 =D9=85=DB= =8C=E2=80=8C=D9=85=D8=A7=D9=86=D8=AF

codepoint:
\u0627\u06cc\u064= 6 \u0646\u0627\u0645 \u0627\u0633\u062a \u06a9\u0647 \u0645\u06cc\u200c\u06= 45\u0627\u0646\u062f


Then go back a few characters in the same l= ine and type words randomly. You will see that the letters in some words ar= e displayed separately. I type a few words at random, after the word =D8=A7= =DB=8C=D9=86 and before the word =D9=86=D8=A7=D9=85 :=E2=80=8C

=D8= =A7=DB=8C=D9=86 =D9=81=D8=B1=D8=A7=D9=85=D9=88=D8=B4=DB=8C =D8=B1=D8=A7 =D8= =A8=D9=87 =D9=87=D9=85=D9=87 =D8=A7=DB=8C=D9=86=DA=A9=D9=87 =D9=81=D8=B1=D9= =85=D8=AA =D9=85=D8=B1=D8=A7=D8=AA=D8=A8 =D8=A7=D9=81=D8=AA=D8=AA=D8=A7=D8= =AD =DA=AF=D8=B1=D8=A7=D9=85=DB=8C =DA=AF=D8=B1=D8=A7=DB=8C=D8=B4 =D8=B3=D8= =B1=D8=A7=D8=B3=DB=8C=D9=85=D9=87 =D9=86=D8=A7=D9=85 =D8=A7=D8=B3=D8=AA =DA= =A9=D9=87 =D9=85=DB=8C=E2=80=8C=D9=85=D8=A7=D9=86=D8=AF

codepoint:\u0627\u06cc\u0646 \u0641\u0631\u0627\u0645\u0648\u0634\u06cc \u0631\u062= 7 \u0628\u0647 \u0647\u0645\u0647 \u0627\u06cc\u0646\u06a9\u0647 \u0641\u06= 31\u0645\u062a \u0645\u0631\u0627\u062a\u0628 \u0627\u0641\u062a\u062a\u062= 7\u062d \u06af\u0631\u0627\u0645\u06cc \u06af\u0631\u0627\u06cc\u0634 \u063= 3\u0631\u0627\u0633\u06cc\u0645\u0647 \u0646\u0627\u0645 \u0627\u0633\u062a= \u06a9\u0647 \u0645\u06cc\u200c\u0645\u0627\u0646\u062f

This line s= hould look like this:
http://s12.picofile.com/file/8399190550/correct.png
But if one of the auto-fill-mode or column-number-mode is enabled. it wil= l be displayed this way:
http://s13.picofile.com/file/8399190584/malformed.png

------------------------------------------------------------------= --------------
> Please show a full recipe for reproducing this, star= ting from
> "emacs -Q", and describing every step to reprod= uce the result.

All steps are displayed in the video file.

&g= t; I'd also greatly appreciate if you specifically point out at which> parts of the display to look at what differences to pay attention> to. This is needed to fully understand the problem and analyze its> root cause(s), given that not all of us can read the Arabic script.
The letters that should normally be connected will be displayed separ= ately:
http://s12.picofile.com/file/8399222618/latest_screenshot.png<= br>------------------------------------------------------------------------= --------
--0000000000005cecf105a7556d52--