From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#41005: problem with rendering Persian text in Emacs 27 Date: Thu, 04 Jun 2020 16:15:07 +0300 Message-ID: <83y2p3as6c.fsf@gnu.org> References: <831rmwc9ke.fsf@gnu.org> <35A46479-A62C-42FF-995B-B295FE3408C0@gnu.org> <08A9D65F-0C9C-4EE2-B3B9-2AA25BFFAD54@gnu.org> <878sh35j6f.fsf@gmail.com> Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="15102"; mail-complaints-to="usenet@ciao.gmane.io" Cc: valizadeh.ho@gmail.com, 41005@debbugs.gnu.org, nicholasdrozd@gmail.com To: Pip Cet Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Jun 04 15:16:10 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jgpj7-0003m2-EP for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 04 Jun 2020 15:16:09 +0200 Original-Received: from localhost ([::1]:51034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgpj6-0007Nn-0P for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 04 Jun 2020 09:16:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:58526) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgpj0-0007NU-15 for bug-gnu-emacs@gnu.org; Thu, 04 Jun 2020 09:16:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:33917) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgpiz-0001oa-Nh for bug-gnu-emacs@gnu.org; Thu, 04 Jun 2020 09:16:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jgpiz-0002xm-Jj for bug-gnu-emacs@gnu.org; Thu, 04 Jun 2020 09:16:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 04 Jun 2020 13:16:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41005 X-GNU-PR-Package: emacs Original-Received: via spool by 41005-submit@debbugs.gnu.org id=B41005.159127652811342 (code B ref 41005); Thu, 04 Jun 2020 13:16:01 +0000 Original-Received: (at 41005) by debbugs.gnu.org; 4 Jun 2020 13:15:28 +0000 Original-Received: from localhost ([127.0.0.1]:45461 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jgpiS-0002wr-ER for submit@debbugs.gnu.org; Thu, 04 Jun 2020 09:15:28 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:45624) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jgpiQ-0002wd-QY for 41005@debbugs.gnu.org; Thu, 04 Jun 2020 09:15:27 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:60663) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgpiL-0001gR-JN; Thu, 04 Jun 2020 09:15:21 -0400 Original-Received: from [176.228.60.248] (port=2888 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jgpiK-0000m0-V0; Thu, 04 Jun 2020 09:15:21 -0400 In-Reply-To: <878sh35j6f.fsf@gmail.com> (message from Pip Cet on Thu, 04 Jun 2020 08:28:24 +0000) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:181505 Archived-At: > From: Pip Cet > Cc: Eli Zaretskii , 41005@debbugs.gnu.org, > nicholasdrozd@gmail.com > Date: Thu, 04 Jun 2020 08:28:24 +0000 > > What happens is that font-shape-gstring is called with direction == L2R, > mis-shapes the text, then caches that version in the composition gstring > cache. The cache doesn't distinguish directions, and it's never cleared, > so this "infects" other buffers, but only if they're opened afterwards, > and only for the same words. > > shr appears to force bidi-display-reordering off. Removing that let > binding from shr-insert-document avoids the problem. Right, thanks. When I added that binding of bidi-display-reordering, we didn't yet have HarfBuzz, and the other shapers' Arabic shaping wasn't affected by the local text direction like HarfBuzz is. > We should consider adding direction to our gstrings, on master. While > we're there, let's also add script, language, and harfbuzz features to > the gstrings so we know we've captured the precise harfbuzz parameters? On emacs-27, I can fix this by a simpler band-aid below. On master, I think we should indeed add direction to the cached gstrings, as there could be other much more subtle situations where looking at bidi-display-reordering is not enough, and we could then still cache a composition with the wrong direction. Patches welcome. As for script and language, for now adding them would just waste memory, since we don't yet have any meaningful support for buffer-local, let-alone paragraph-local, scripts and languages. When we added HarfBuzz support, I considered adding some functionality to determine language and/or script from the codepoints, but decided that it made little sense, since HarfBuzz already does exactly that in hb_buffer_guess_segment_properties. So I think we should add to the FIXME in hbfont.c the fact that when we do support those internally in Emacs, we should add these attributes to cached gstrings, but not yet actually add them for now. Here's the patch I propose for emacs-27: diff --git a/src/hbfont.c b/src/hbfont.c index 576c5fe..4b3f64e 100644 --- a/src/hbfont.c +++ b/src/hbfont.c @@ -26,6 +26,7 @@ #include "composite.h" #include "font.h" #include "dispextern.h" +#include "buffer.h" #ifdef HAVE_NTGUI @@ -438,7 +439,11 @@ hbfont_shape (Lisp_Object lgstring, Lisp_Object direction) /* If the caller didn't provide a meaningful DIRECTION, let HarfBuzz guess it. */ - if (!NILP (direction)) + if (!NILP (direction) + /* If they bind bidi-display-reordering to nil, the DIRECTION + they provide is meaningless, and we should let HarfBuzz guess + the real direction. */ + && !NILP (BVAR (current_buffer, bidi_display_reordering))) { hb_direction_t dir = HB_DIRECTION_LTR; if (EQ (direction, QL2R))