From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#35721: 27.0.50; Strange Arabic shaping behavior Date: Tue, 14 May 2019 18:10:19 +0300 Message-ID: <83mujp9in8.fsf@gnu.org> References: <87h89ycpnw.fsf@tcd.ie> Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="100644"; mail-complaints-to="usenet@blaine.gmane.org" Cc: 35721@debbugs.gnu.org To: "Basil L. Contovounesios" , Behdad Esfahbod , Kenichi Handa Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Tue May 14 17:11:15 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hQZ5H-000Q4E-0i for geb-bug-gnu-emacs@m.gmane.org; Tue, 14 May 2019 17:11:15 +0200 Original-Received: from localhost ([127.0.0.1]:49648 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQZ5G-0003tI-0g for geb-bug-gnu-emacs@m.gmane.org; Tue, 14 May 2019 11:11:14 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:50116) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQZ55-0003rG-Tg for bug-gnu-emacs@gnu.org; Tue, 14 May 2019 11:11:05 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQZ54-0002nA-D5 for bug-gnu-emacs@gnu.org; Tue, 14 May 2019 11:11:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:36069) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hQZ54-0002mw-9m for bug-gnu-emacs@gnu.org; Tue, 14 May 2019 11:11:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hQZ54-0003Iq-1m for bug-gnu-emacs@gnu.org; Tue, 14 May 2019 11:11:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 14 May 2019 15:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35721 X-GNU-PR-Package: emacs Original-Received: via spool by 35721-submit@debbugs.gnu.org id=B35721.155784664212667 (code B ref 35721); Tue, 14 May 2019 15:11:02 +0000 Original-Received: (at 35721) by debbugs.gnu.org; 14 May 2019 15:10:42 +0000 Original-Received: from localhost ([127.0.0.1]:49613 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hQZ4j-0003IE-U1 for submit@debbugs.gnu.org; Tue, 14 May 2019 11:10:42 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:35603) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hQZ4i-0003I1-4K for 35721@debbugs.gnu.org; Tue, 14 May 2019 11:10:40 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:37247) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQZ4b-0002Gq-UD; Tue, 14 May 2019 11:10:34 -0400 Original-Received: from [176.228.60.248] (port=2313 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1hQZ4b-0005ux-AO; Tue, 14 May 2019 11:10:33 -0400 In-reply-to: <87h89ycpnw.fsf@tcd.ie> (contovob@tcd.ie) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:159276 Archived-At: > From: "Basil L. Contovounesios" > Date: Mon, 13 May 2019 23:09:06 +0100 > > I see the following on the master, harfbuzz, and emacs-26 branches > (precise versions follow my signature), but I'm not sure how much of > this is expected or due to e.g. my font. > > 0. emacs -Q > 1. C-x 8 RET 0634 RET > > The "tail" of the sheen is truncated by the fringe: After looking at the code and thinking about this, I think this is a feature (as strange as it may sound); see below for why I think so. And yes, it definitely depends on the font, in this case DejaVu Sans Mono. I don't see this with any other fixed-pitch font I have. I'm not 100% sure I'm right here, so I CC Behdad and Kenichi in the hope that they will comment on this. Behdad, I see a very similar issue with hb-view, when it renders this character using DejaVu Sans Mono, so it isn't just an Emacs issue (and seeing what hb-view produces actually made me think my opinion is correct about this). > 3. SPC > > The "tail" of the sheen becomes visible, but falls outside of the box > cursor: Yes, this particular font's glyph for sheen has a negative value of left bearing. Which AFAIU means it extends beyond the box dimensions to the left. > 4. C-x 8 RET 0643 RET > > The kaf is correctly shaped in its initial form: > > 5. C-SPC > > The kaf changes to its isolated form: This is different problem, related to how we redraw portions of the buffer inside the region (more generally, those which have colors different from the default face). The problem is that we only pass to the shaping engine stretches of text that have the same face. The basic reason for that is that a different face can use a different font, and we can only handle character composition for characters supported by the same font. Another fundamental reason is that the display engine processes text in chunks that have the same face. So when the active region, or some other Emacs feature, paints portions of text in some non-default face, we redraw the display, and pass to the shaping engine only the portion that has that different face. If that portion is a single character, you will see that it loses its correct shape and is rendered in its isolated form. And if the colors change between two characters that need to be shaped together, the shaping will break. You can easily see this effect if you display HELLO, and then shift-select portions of the Arabic greeting (or any other script that is a heavy user of character compositions). To fix this, we need some mechanism that will pass larger chunks of text to the shaper in these cases, which will need some changes in how the display engine iterates through buffer/string text when it prepares them for display: we currently stop at every change of face. Patches to fix this are most welcome. > I occasionally see this happen even without typing anything, as if by a > timer, but I'm not sure how to reproduce it. I think, without being > 100% certain, that it's only happened while using the 'arabic' input > method. Maybe, but given my description above, I'm not surprised, because it's enough that Emacs decides, for some reason, to redraw just that one character. Now to the original problem. Let me turn the table and ask you: what did you expect to happen instead? This is a fixed-pitch font, so how can Emacs display a character that extends to the left from its box, at the left-most window coordinate? It has no choice but consider its extension be off-screen, as if the window was hscrolled. The "normal" case for this character is to be part of R2L text, which begins at the right window margin, and flows to the left. In that case, the extension will overlap the character cell of the next (in the logical order) character. So I think we have no bug here, we behave as expected. If not, I'm sure Behdad and Kenichi will correct me.