From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Behdad Esfahbod Newsgroups: gmane.emacs.bugs Subject: bug#35721: 27.0.50; Strange Arabic shaping behavior Date: Wed, 15 May 2019 16:02:19 -0700 Message-ID: References: <87h89ycpnw.fsf@tcd.ie> <83mujp9in8.fsf@gnu.org> <83d0kkao99.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000e53da80588f5245e" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="174886"; mail-complaints-to="usenet@blaine.gmane.org" Cc: contovob@tcd.ie, Khaled Hosny , 35721@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Thu May 16 01:03:35 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hR2vu-000jLD-V4 for geb-bug-gnu-emacs@m.gmane.org; Thu, 16 May 2019 01:03:35 +0200 Original-Received: from localhost ([127.0.0.1]:44955 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hR2vu-0000i0-22 for geb-bug-gnu-emacs@m.gmane.org; Wed, 15 May 2019 19:03:34 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:58610) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hR2vX-0000d7-Q9 for bug-gnu-emacs@gnu.org; Wed, 15 May 2019 19:03:17 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hR2vU-0005lp-4B for bug-gnu-emacs@gnu.org; Wed, 15 May 2019 19:03:11 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:40073) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hR2vP-0005ht-2R for bug-gnu-emacs@gnu.org; Wed, 15 May 2019 19:03:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hR2vO-0004mS-Gf for bug-gnu-emacs@gnu.org; Wed, 15 May 2019 19:03:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Behdad Esfahbod Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 15 May 2019 23:03:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35721 X-GNU-PR-Package: emacs Original-Received: via spool by 35721-submit@debbugs.gnu.org id=B35721.155796135918322 (code B ref 35721); Wed, 15 May 2019 23:03:02 +0000 Original-Received: (at 35721) by debbugs.gnu.org; 15 May 2019 23:02:39 +0000 Original-Received: from localhost ([127.0.0.1]:53614 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hR2v0-0004lR-Or for submit@debbugs.gnu.org; Wed, 15 May 2019 19:02:39 -0400 Original-Received: from mail-it1-f172.google.com ([209.85.166.172]:37192) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hR2uy-0004l4-NF for 35721@debbugs.gnu.org; Wed, 15 May 2019 19:02:37 -0400 Original-Received: by mail-it1-f172.google.com with SMTP id m140so2984851itg.2 for <35721@debbugs.gnu.org>; Wed, 15 May 2019 16:02:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=PavM4iY8hRn8HVzXyzxvVBYxskyMT3J15+7SrTMJ8IA=; b=JcZ8FNWWjH7P1uUo2A32qw6k0a3SHz/KzpvMYqvvq80yyMjOa36onSXFKsPWqeAZxI nyzbPNlB3q9QJV5t2zFQOkWLYHnGxbZtga6RN4PEJm9KZef40V/3MwLNICrrMzaq1R7V M6WJKcfe9/2nvA7WM3lDu4nwSL8cswMth/JuYDLOg0xTf4gJsaUJ4j8UGpjETcbWLTgi xLe/4ccMEcG6+pDajb+YzBVhDnNs4rC+ahiL0mdHtd/A/AqEZ3iY9/jhXy0jXztIRNlC 86cjVYJ0ZFi313IYTBbcpTJWf8O8cI/eecKE04jKR/bBZgQtDMGHfIpsB8mFRtBeSKhy J1nA== X-Gm-Message-State: APjAAAXnwhIlL9Ru0sC23mY3m9v/ozLAWlgntj/QoZJ+cNHtHC606uVl 4H4fjUDn7YAnvo98hQu05N7zSqCu+EeYgYQyGhY= X-Google-Smtp-Source: APXvYqziRPbvODDSx0f9flY71xK5JzdVcDZNrpHupYaZpae+gz3OvirzvqsxXpWsZLs/m9hamRPf4oB82jHU7NoNoeE= X-Received: by 2002:a24:1c8f:: with SMTP id c137mr1073936itc.165.1557961350919; Wed, 15 May 2019 16:02:30 -0700 (PDT) In-Reply-To: <83d0kkao99.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:159368 Archived-At: --000000000000e53da80588f5245e Content-Type: text/plain; charset="UTF-8" Hi Eli, Pretty much things you said: if font is not mono-spaced, there's nothing we can do. Also, if you don't pass neighboring context text to HarfBuzz, again, nothing we can do. On Tue, May 14, 2019 at 11:24 AM Eli Zaretskii wrote: > Adding Khaled. > > The original message with images can be viewed here: > > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35721#5 > > > Date: Tue, 14 May 2019 18:10:19 +0300 > > From: Eli Zaretskii > > Cc: 35721@debbugs.gnu.org > > > > > From: "Basil L. Contovounesios" > > > Date: Mon, 13 May 2019 23:09:06 +0100 > > > > > > I see the following on the master, harfbuzz, and emacs-26 branches > > > (precise versions follow my signature), but I'm not sure how much of > > > this is expected or due to e.g. my font. > > > > > > 0. emacs -Q > > > 1. C-x 8 RET 0634 RET > > > > > > The "tail" of the sheen is truncated by the fringe: > > > > After looking at the code and thinking about this, I think this is a > > feature (as strange as it may sound); see below for why I think so. > > And yes, it definitely depends on the font, in this case DejaVu Sans > > Mono. I don't see this with any other fixed-pitch font I have. > > > > I'm not 100% sure I'm right here, so I CC Behdad and Kenichi in the > > hope that they will comment on this. Behdad, I see a very similar > > issue with hb-view, when it renders this character using DejaVu Sans > > Mono, so it isn't just an Emacs issue (and seeing what hb-view > > produces actually made me think my opinion is correct about this). > > > > > 3. SPC > > > > > > The "tail" of the sheen becomes visible, but falls outside of the box > > > cursor: > > > > Yes, this particular font's glyph for sheen has a negative value of > > left bearing. Which AFAIU means it extends beyond the box dimensions > > to the left. > > > > > 4. C-x 8 RET 0643 RET > > > > > > The kaf is correctly shaped in its initial form: > > > > > > 5. C-SPC > > > > > > The kaf changes to its isolated form: > > > > This is different problem, related to how we redraw portions of the > > buffer inside the region (more generally, those which have colors > > different from the default face). > > > > The problem is that we only pass to the shaping engine stretches of > > text that have the same face. The basic reason for that is that a > > different face can use a different font, and we can only handle > > character composition for characters supported by the same font. > > Another fundamental reason is that the display engine processes text > > in chunks that have the same face. So when the active region, or some > > other Emacs feature, paints portions of text in some non-default face, > > we redraw the display, and pass to the shaping engine only the portion > > that has that different face. If that portion is a single character, > > you will see that it loses its correct shape and is rendered in its > > isolated form. And if the colors change between two characters that > > need to be shaped together, the shaping will break. > > > > You can easily see this effect if you display HELLO, and then > > shift-select portions of the Arabic greeting (or any other script that > > is a heavy user of character compositions). > > > > To fix this, we need some mechanism that will pass larger chunks of > > text to the shaper in these cases, which will need some changes in how > > the display engine iterates through buffer/string text when it > > prepares them for display: we currently stop at every change of face. > > > > Patches to fix this are most welcome. > > > > > I occasionally see this happen even without typing anything, as if by a > > > timer, but I'm not sure how to reproduce it. I think, without being > > > 100% certain, that it's only happened while using the 'arabic' input > > > method. > > > > Maybe, but given my description above, I'm not surprised, because it's > > enough that Emacs decides, for some reason, to redraw just that one > > character. > > > > Now to the original problem. Let me turn the table and ask you: what > > did you expect to happen instead? This is a fixed-pitch font, so how > > can Emacs display a character that extends to the left from its box, > > at the left-most window coordinate? It has no choice but consider its > > extension be off-screen, as if the window was hscrolled. > > > > The "normal" case for this character is to be part of R2L text, which > > begins at the right window margin, and flows to the left. In that > > case, the extension will overlap the character cell of the next (in > > the logical order) character. > > > > So I think we have no bug here, we behave as expected. If not, I'm > > sure Behdad and Kenichi will correct me. > -- behdad http://behdad.org/ --000000000000e53da80588f5245e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Eli,

Pretty much things you said: if= font is not mono-spaced, there's nothing we can do.=C2=A0 Also, if you= don't pass neighboring context text to HarfBuzz, again, nothing we can= do.

On Tue, May 14, 2019 at 11:24 AM Eli Zaretskii <eliz@gnu.org> wrote:
Adding Khaled.

The original message with images can be viewed here:

=C2=A0 https://debbugs.gnu.org/cgi/bugreport.= cgi?bug=3D35721#5

> Date: Tue, 14 May 2019 18:10:19 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 35721@d= ebbugs.gnu.org
>
> > From: "Basil L. Contovounesios" <contovob@tcd.ie>
> > Date: Mon, 13 May 2019 23:09:06 +0100
> >
> > I see the following on the master, harfbuzz, and emacs-26 branche= s
> > (precise versions follow my signature), but I'm not sure how = much of
> > this is expected or due to e.g. my font.
> >
> > 0. emacs -Q
> > 1. C-x 8 RET 0634 RET
> >
> > The "tail" of the sheen is truncated by the fringe:
>
> After looking at the code and thinking about this, I think this is a > feature (as strange as it may sound); see below for why I think so. > And yes, it definitely depends on the font, in this case DejaVu Sans > Mono.=C2=A0 I don't see this with any other fixed-pitch font I hav= e.
>
> I'm not 100% sure I'm right here, so I CC Behdad and Kenichi i= n the
> hope that they will comment on this.=C2=A0 Behdad, I see a very simila= r
> issue with hb-view, when it renders this character using DejaVu Sans > Mono, so it isn't just an Emacs issue (and seeing what hb-view
> produces actually made me think my opinion is correct about this).
>
> > 3. SPC
> >
> > The "tail" of the sheen becomes visible, but falls outs= ide of the box
> > cursor:
>
> Yes, this particular font's glyph for sheen has a negative value o= f
> left bearing.=C2=A0 Which AFAIU means it extends beyond the box dimens= ions
> to the left.
>
> > 4. C-x 8 RET 0643 RET
> >
> > The kaf is correctly shaped in its initial form:
> >
> > 5. C-SPC
> >
> > The kaf changes to its isolated form:
>
> This is different problem, related to how we redraw portions of the > buffer inside the region (more generally, those which have colors
> different from the default face).
>
> The problem is that we only pass to the shaping engine stretches of > text that have the same face.=C2=A0 The basic reason for that is that = a
> different face can use a different font, and we can only handle
> character composition for characters supported by the same font.
> Another fundamental reason is that the display engine processes text > in chunks that have the same face.=C2=A0 So when the active region, or= some
> other Emacs feature, paints portions of text in some non-default face,=
> we redraw the display, and pass to the shaping engine only the portion=
> that has that different face.=C2=A0 If that portion is a single charac= ter,
> you will see that it loses its correct shape and is rendered in its > isolated form.=C2=A0 And if the colors change between two characters t= hat
> need to be shaped together, the shaping will break.
>
> You can easily see this effect if you display HELLO, and then
> shift-select portions of the Arabic greeting (or any other script that=
> is a heavy user of character compositions).
>
> To fix this, we need some mechanism that will pass larger chunks of > text to the shaper in these cases, which will need some changes in how=
> the display engine iterates through buffer/string text when it
> prepares them for display: we currently stop at every change of face.<= br> >
> Patches to fix this are most welcome.
>
> > I occasionally see this happen even without typing anything, as i= f by a
> > timer, but I'm not sure how to reproduce it.=C2=A0 I think, w= ithout being
> > 100% certain, that it's only happened while using the 'ar= abic' input
> > method.
>
> Maybe, but given my description above, I'm not surprised, because = it's
> enough that Emacs decides, for some reason, to redraw just that one > character.
>
> Now to the original problem.=C2=A0 Let me turn the table and ask you: = what
> did you expect to happen instead?=C2=A0 This is a fixed-pitch font, so= how
> can Emacs display a character that extends to the left from its box, > at the left-most window coordinate?=C2=A0 It has no choice but conside= r its
> extension be off-screen, as if the window was hscrolled.
>
> The "normal" case for this character is to be part of R2L te= xt, which
> begins at the right window margin, and flows to the left.=C2=A0 In tha= t
> case, the extension will overlap the character cell of the next (in > the logical order) character.
>
> So I think we have no bug here, we behave as expected.=C2=A0 If not, I= 'm
> sure Behdad and Kenichi will correct me.


--
behdad
http://behdad.org/
--000000000000e53da80588f5245e--