bug#37683: Eww doesn't render Arabic script correctly

bug#37683: Eww doesn't render Arabic script correctly

[Nicholas Drozd]

Here is the Persian Wikipedia page for Emacs:
https://fa.wikipedia.org/wiki/%D8%A7%DB%8C%D9%85%DA%A9%D8%B3

When I open it in Eww, the cursive Perso-Arabic characters are not
strung together properly. For example, the article mentions the
"editor war", جنگ ویرایشگرها. In Eww, this is rendered as جنگ
ویرایشگرها with wrong forms (initial vs middle vs final vs isolated).
`D` runs `eww-toggle-paragraph-direction`, but that doesn't have any
effect on the problem. I think this might be a cursive issue rather
than a directionality issue, but I could be wrong.

(As I prepare to send this, I notice that Gmail won't render that
phrase incorrectly, so the issue might not be communicated clearly
here.)

bug#37683: Eww doesn't render Arabic script correctly

[Eli Zaretskii]

> From: Nicholas Drozd <nicholasdrozd@gmail.com>
> Date: Wed, 9 Oct 2019 12:16:16 -0500
> 
> Here is the Persian Wikipedia page for Emacs:
> https://fa.wikipedia.org/wiki/%D8%A7%DB%8C%D9%85%DA%A9%D8%B3
> 
> When I open it in Eww, the cursive Perso-Arabic characters are not
> strung together properly. For example, the article mentions the
> "editor war", جنگ ویرایشگرها. In Eww, this is rendered as جنگ
> ویرایشگرها with wrong forms (initial vs middle vs final vs isolated).
> `D` runs `eww-toggle-paragraph-direction`, but that doesn't have any
> effect on the problem. I think this might be a cursive issue rather
> than a directionality issue, but I could be wrong.
> 
> (As I prepare to send this, I notice that Gmail won't render that
> phrase incorrectly, so the issue might not be communicated clearly
> here.)

Thank you for your report.

First, you cannot effectively report text-shaping (and in general,
rendering) problems by pasting the problematic text into an email
message, because most Emacs maintainers use Emacs to read email, so
any problems you report will be invisible to them, especially if you
show two similar segments of text that are supposed to be rendered
differently.  The only reliable way of reporting such rendering
problems is by including screenshots of the wrong and the correct
rendering (the latter from some other application or from a different
Emacs version, if that's applicable).

And second, you didn't provide any details about the Emacs version you
used and how it was built (i.e. the features it can support) -- this
information is collected by "M-x report-emacs-bug" when you invoke
it.  My first suspicion is that you are using Emacs 26 or older and/or
an outdated version of the libm17n library.  Emacs 27 uses HarfBuzz by
default (if available on the system), and I cannot imagine that
HarfBuzz doesn't render Farsi text correctly.  (If you send
screenshots, I can verify that on my system, as I don't read Farsi, so
I need an image of correct display to compare to.)

bug#37683: Eww doesn't render Arabic script correctly

[Nicholas Drozd]

[-- Attachment #1: Type: text/plain, Size: 3986 bytes --]

I discovered that the problem only occurs when text is NOT scaled --
if the text is made larger or smaller by any amount (with
`text-scale-increase` etc), the cursive characters are joined
correctly.

A few screenshots are attached (hopefully their names are self-explanatory).

----------------------------------------

Bug report info:

In GNU Emacs 27.0.50 (build 6, x86_64-pc-linux-gnu, GTK+ Version
3.22.30) of 2019-10-09 built on ...
Repository revision: 6fa1558ca56c20226821e440a70129ece7b808b6
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.2 LTS

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM THREADS PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

----------------------------------------

Not sure if this is the right command, but:

$ dpkg -l | grep "\(harf\)\|\(libm17n\)"
ii  gir1.2-harfbuz 1.7.2-1ubunt amd64        OpenType text shaping engine (GOb
ii  libharfbuzz-de 1.7.2-1ubunt amd64        Development files for OpenType te
ii  libharfbuzz-go 1.7.2-1ubunt amd64        OpenType text shaping engine ICU
ii  libharfbuzz-ic 1.7.2-1ubunt amd64        OpenType text shaping engine ICU
ii  libharfbuzz0b: 1.7.2-1ubunt amd64        OpenType text shaping engine (sha
ii  libm17n-0:amd6 1.7.0-3build amd64        multilingual text processing libr
ii  libm17n-dev:am 1.7.0-3build amd64        multilingual text processing libr

----------------------------------------

On Thu, Oct 10, 2019 at 1:37 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Nicholas Drozd <nicholasdrozd@gmail.com>
> > Date: Wed, 9 Oct 2019 12:16:16 -0500
> >
> > Here is the Persian Wikipedia page for Emacs:
> > https://fa.wikipedia.org/wiki/%D8%A7%DB%8C%D9%85%DA%A9%D8%B3
> >
> > When I open it in Eww, the cursive Perso-Arabic characters are not
> > strung together properly. For example, the article mentions the
> > "editor war", جنگ ویرایشگرها. In Eww, this is rendered as جنگ
> > ویرایشگرها with wrong forms (initial vs middle vs final vs isolated).
> > `D` runs `eww-toggle-paragraph-direction`, but that doesn't have any
> > effect on the problem. I think this might be a cursive issue rather
> > than a directionality issue, but I could be wrong.
> >
> > (As I prepare to send this, I notice that Gmail won't render that
> > phrase incorrectly, so the issue might not be communicated clearly
> > here.)
>
> Thank you for your report.
>
> First, you cannot effectively report text-shaping (and in general,
> rendering) problems by pasting the problematic text into an email
> message, because most Emacs maintainers use Emacs to read email, so
> any problems you report will be invisible to them, especially if you
> show two similar segments of text that are supposed to be rendered
> differently.  The only reliable way of reporting such rendering
> problems is by including screenshots of the wrong and the correct
> rendering (the latter from some other application or from a different
> Emacs version, if that's applicable).
>
> And second, you didn't provide any details about the Emacs version you
> used and how it was built (i.e. the features it can support) -- this
> information is collected by "M-x report-emacs-bug" when you invoke
> it.  My first suspicion is that you are using Emacs 26 or older and/or
> an outdated version of the libm17n library.  Emacs 27 uses HarfBuzz by
> default (if available on the system), and I cannot imagine that
> HarfBuzz doesn't render Farsi text correctly.  (If you send
> screenshots, I can verify that on my system, as I don't read Farsi, so
> I need an image of correct display to compare to.)

[-- Attachment #2: firefox-good.png --]
[-- Type: image/png, Size: 198016 bytes --]

[-- Attachment #3: eww-big-good.png --]
[-- Type: image/png, Size: 143545 bytes --]

[-- Attachment #4: eww-small-good.png --]
[-- Type: image/png, Size: 127575 bytes --]

[-- Attachment #5: eww-default-bad.png --]
[-- Type: image/png, Size: 211209 bytes --]

bug#37683: Eww doesn't render Arabic script correctly

[Eli Zaretskii]

> From: Nicholas Drozd <nicholasdrozd@gmail.com>
> Date: Thu, 10 Oct 2019 11:55:33 -0500
> Cc: 37683@debbugs.gnu.org
> 
> I discovered that the problem only occurs when text is NOT scaled --
> if the text is made larger or smaller by any amount (with
> `text-scale-increase` etc), the cursive characters are joined
> correctly.

Then I suggest to try a different font, because on my system I see the
same correct display even without the text scaling.  Or maybe you
should upgrade your HarfBuzz (1.7.2 is quite old).

bug#37683: Eww doesn't render Arabic script correctly

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> I discovered that the problem only occurs when text is NOT scaled --
>> if the text is made larger or smaller by any amount (with
>> `text-scale-increase` etc), the cursive characters are joined
>> correctly.
>
> Then I suggest to try a different font, because on my system I see the
> same correct display even without the text scaling.  Or maybe you
> should upgrade your HarfBuzz (1.7.2 is quite old).

There doesn't seem to be anything further to do here, so I'm closing
this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no