From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: "Basil L. Contovounesios" Newsgroups: gmane.emacs.bugs Subject: bug#35721: 27.0.50; Strange Arabic shaping behavior Date: Thu, 16 May 2019 21:47:04 +0100 Message-ID: <87bm02dt4n.fsf@tcd.ie> References: <87h89ycpnw.fsf@tcd.ie> <83mujp9in8.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="121313"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: Behdad Esfahbod , 35721@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Thu May 16 22:48:21 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hRNIZ-000VRf-4S for geb-bug-gnu-emacs@m.gmane.org; Thu, 16 May 2019 22:48:19 +0200 Original-Received: from localhost ([127.0.0.1]:35912 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRNIY-0003NS-70 for geb-bug-gnu-emacs@m.gmane.org; Thu, 16 May 2019 16:48:18 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:42050) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRNIR-0003NC-RI for bug-gnu-emacs@gnu.org; Thu, 16 May 2019 16:48:13 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hRNIP-0006Px-Tr for bug-gnu-emacs@gnu.org; Thu, 16 May 2019 16:48:11 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:43049) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hRNII-0006Gr-DC for bug-gnu-emacs@gnu.org; Thu, 16 May 2019 16:48:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hRNII-0003hk-8S for bug-gnu-emacs@gnu.org; Thu, 16 May 2019 16:48:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: "Basil L. Contovounesios" Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 16 May 2019 20:48:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35721 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: notabug Original-Received: via spool by 35721-submit@debbugs.gnu.org id=B35721.155803964014183 (code B ref 35721); Thu, 16 May 2019 20:48:02 +0000 Original-Received: (at 35721) by debbugs.gnu.org; 16 May 2019 20:47:20 +0000 Original-Received: from localhost ([127.0.0.1]:56591 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hRNHc-0003gg-DV for submit@debbugs.gnu.org; Thu, 16 May 2019 16:47:20 -0400 Original-Received: from mail-ed1-f65.google.com ([209.85.208.65]:41033) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hRNHa-0003gU-Jz for 35721@debbugs.gnu.org; Thu, 16 May 2019 16:47:19 -0400 Original-Received: by mail-ed1-f65.google.com with SMTP id m4so7100563edd.8 for <35721@debbugs.gnu.org>; Thu, 16 May 2019 13:47:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tcd-ie.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=xiJ3/HSHm64VG9EJS9W9ptTMPDK0qIjTKCz4OvvxpN8=; b=0TslgGrqtMPa6ACWyB6zxuFABVTqAuStAOLwmE5AXQyorcH+cF2Msn0WuTC1/rjbdL qsXGff1trdIfLjUtQ9KuQk8VagcutUcl6wqjgWH6l2Gy+eE9Om3FUZVoeegY0+jjNyur eFabVX2cSwuW1U0YVzN0Ho4Ub0wrlEhBjJ2L2/s0Z2Xtc+BqRU/8KM/kces0D3f/av5s Hy0HRwcvaH1YLdYANj1T7VOJzyyU++WmN/jjiy3sKqTiLqGZVlB+xsv8ptJ4AINxefF+ JRM6DB1xUzL/LtngZ1eUEZFG6lZ9jy+u/t6F0ux7N2m5OapULznIZPktDNCBtUvmEfO8 B46A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=xiJ3/HSHm64VG9EJS9W9ptTMPDK0qIjTKCz4OvvxpN8=; b=D1tP4QnTGPgwpjR1x1u1GkPLOKcmnc9SI8a4vAto4qWISgrBUEsQ0kft+M+IvTxOZq NwcbamG2SGvjf34GCOO+2vtUUZcgoDqoNfc9syHVyjZhwFP2xI+HVaF8/XvgJAaZ4zJi G5UW4eRZgdDxgKsfhH+SFkx6chSqhej+eBEkdjtCZzEhjXOo6im+EqPq7Ekw2s30y2Yb z2rOOR4v0TRb1YJFZfJu5Fj3522iHzrBIIgEaouirU48ChVxWx0cWcJh6ncYKvjFPVlm flJNcqp34VTzjW2GZ+69Naz8erOnEwdGOcxbly0Az45x+iKOpRp3hXt7HQp8LxoM8CjQ +t5A== X-Gm-Message-State: APjAAAVRiLlePsAiTIL13fn2rGK9o0FC5i0QvmrB0rqNtiRU8/FV1mum +dkZwhMCCj4MIxfhJVyz0Bx4jQ== X-Google-Smtp-Source: APXvYqyVa3eYNM+3OeZWer/mSeY4BgSSCEEtFnK/INmSFCCeP04RZZO4rlTiURQ6N/YK3Tl3XN9O6g== X-Received: by 2002:a17:906:50e:: with SMTP id j14mr40553677eja.248.1558039632497; Thu, 16 May 2019 13:47:12 -0700 (PDT) Original-Received: from localhost ([2a02:8084:20e2:c380:6fa:38d6:1fce:ddb3]) by smtp.gmail.com with ESMTPSA id bq13sm1232269ejb.63.2019.05.16.13.47.10 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 16 May 2019 13:47:11 -0700 (PDT) In-Reply-To: <83mujp9in8.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 14 May 2019 18:10:19 +0300") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:159415 Archived-At: Eli Zaretskii writes: >> From: "Basil L. Contovounesios" >> Date: Mon, 13 May 2019 23:09:06 +0100 >> >> I see the following on the master, harfbuzz, and emacs-26 branches >> (precise versions follow my signature), but I'm not sure how much of >> this is expected or due to e.g. my font. >> >> 0. emacs -Q >> 1. C-x 8 RET 0634 RET >> >> The "tail" of the sheen is truncated by the fringe: > > After looking at the code and thinking about this, I think this is a > feature (as strange as it may sound); see below for why I think so. > And yes, it definitely depends on the font, in this case DejaVu Sans > Mono. I don't see this with any other fixed-pitch font I have. > > I'm not 100% sure I'm right here, so I CC Behdad and Kenichi in the > hope that they will comment on this. Behdad, I see a very similar > issue with hb-view, when it renders this character using DejaVu Sans > Mono, so it isn't just an Emacs issue (and seeing what hb-view > produces actually made me think my opinion is correct about this). > >> 3. SPC >> >> The "tail" of the sheen becomes visible, but falls outside of the box >> cursor: > > Yes, this particular font's glyph for sheen has a negative value of > left bearing. Which AFAIU means it extends beyond the box dimensions > to the left. OK. >> 4. C-x 8 RET 0643 RET >> >> The kaf is correctly shaped in its initial form: >> >> 5. C-SPC >> >> The kaf changes to its isolated form: > > This is different problem, related to how we redraw portions of the > buffer inside the region (more generally, those which have colors > different from the default face). > > The problem is that we only pass to the shaping engine stretches of > text that have the same face. The basic reason for that is that a > different face can use a different font, and we can only handle > character composition for characters supported by the same font. > Another fundamental reason is that the display engine processes text > in chunks that have the same face. So when the active region, or some > other Emacs feature, paints portions of text in some non-default face, > we redraw the display, and pass to the shaping engine only the portion > that has that different face. If that portion is a single character, > you will see that it loses its correct shape and is rendered in its > isolated form. And if the colors change between two characters that > need to be shaped together, the shaping will break. > > You can easily see this effect if you display HELLO, and then > shift-select portions of the Arabic greeting (or any other script that > is a heavy user of character compositions). > > To fix this, we need some mechanism that will pass larger chunks of > text to the shaper in these cases, which will need some changes in how > the display engine iterates through buffer/string text when it > prepares them for display: we currently stop at every change of face. Makes sense, thanks for explaining. > Patches to fix this are most welcome. I don't think I need to tell you not to hold your breath. Maybe one day... >> I occasionally see this happen even without typing anything, as if by a >> timer, but I'm not sure how to reproduce it. I think, without being >> 100% certain, that it's only happened while using the 'arabic' input >> method. > > Maybe, but given my description above, I'm not surprised, because it's > enough that Emacs decides, for some reason, to redraw just that one > character. Your description above explains the case where the mark is activated in the middle of a composition, but I don't think it explains why inserting characters further down the buffer would affect compositions on previous lines, as in steps 9 and 10 in the OP, where no face change is involved. > Now to the original problem. Let me turn the table and ask you: what > did you expect to happen instead? This is a fixed-pitch font, so how > can Emacs display a character that extends to the left from its box, > at the left-most window coordinate? It has no choice but consider its > extension be off-screen, as if the window was hscrolled. > > The "normal" case for this character is to be part of R2L text, which > begins at the right window margin, and flows to the left. In that > case, the extension will overlap the character cell of the next (in > the logical order) character. Right. I only noticed the truncation while writing up the rest of the report, but it seemed relevant enough to mention. Following your explanation, the current truncating behaviour seems reasonable to me. > So I think we have no bug here, we behave as expected. If not, I'm > sure Behdad and Kenichi will correct me. Thanks, -- Basil