bug#35811: 27.0.50; Arabic character (de)compositions affected by edits elsewhere in buffer

bug#35811: 27.0.50; Arabic character (de)compositions affected by edits elsewhere in buffer

[Basil L. Contovounesios]

[-- Attachment #1: Type: text/plain, Size: 1438 bytes --]

This report is a followup to bug#35721[1] focussing only on the
alternating composition of Arabic characters when editing seemingly
unrelated parts of the buffer.

[1]: https://debbugs.gnu.org/35721

Observe:

0. emacs -Q
1. C-u C-\ arabic RET
2. a ; C-a C-u C-x =

--8<---------------cut here---------------start------------->8---
             position: 146 of 147 (99%), column: 0
            character: ش‎ (displayed as ش‎) (codepoint 1588, #o3064, #x634)
              charset: unicode (Unicode (ISO10646))
code point in charset: 0x0634
               script: arabic
               syntax: w 	which means: word
             category: .:Base, R:Right-to-left (strong), b:Arabic
             to input: type "a" with arabic input method
          buffer code: #xD8 #xB4
            file code: #xD8 #xB4 (encoded by coding system utf-8-unix)
              display: composed to form "ش" (see below)

Composed using this font:
  xft:-PfEd-DejaVu Sans Mono-normal-normal-normal-*-15-*-*-*-m-0-iso10646-1
by these glyphs:
  [0 0 0 3186 9 -1 9 9 1 nil]

Character code properties: customize what to show
  name: ARABIC LETTER SHEEN
  general-category: Lo (Letter, Other)
  decomposition: (1588) ('ش')

There are text properties here:
  fontified            nil
--8<---------------cut here---------------end--------------->8---

3. C-e RET

The sheen is correctly shaped in its initial form:


[-- Attachment #2: 01.png --]
[-- Type: image/png, Size: 1082 bytes --]

[-- Attachment #3: Type: text/plain, Size: 73 bytes --]


3. a ; RET

The first sheen unexpectedly changes to its isolated form:


[-- Attachment #4: 02.png --]
[-- Type: image/png, Size: 1661 bytes --]

[-- Attachment #5: Type: text/plain, Size: 53 bytes --]


4. a

The first sheen reverts to its initial form:


[-- Attachment #6: 03.png --]
[-- Type: image/png, Size: 1539 bytes --]

[-- Attachment #7: Type: text/plain, Size: 4763 bytes --]


5. ; RET 

Now the second line of Arabic is decomposed.

6. C-p C-p C-a C-u C-x =

Now the second line of Arabic is composed again.

--8<---------------cut here---------------start------------->8---
             position: 149 of 154 (96%), column: 0
            character: ش‎ (displayed as ش‎) (codepoint 1588, #o3064, #x634)
              charset: unicode (Unicode (ISO10646))
code point in charset: 0x0634
               script: arabic
               syntax: w 	which means: word
             category: .:Base, R:Right-to-left (strong), b:Arabic
             to input: type "a" with arabic input method
          buffer code: #xD8 #xB4
            file code: #xD8 #xB4 (encoded by coding system utf-8-unix)
              display: composed to form "ش" (see below)

Composed using this font:
  xft:-PfEd-DejaVu Sans Mono-normal-normal-normal-*-15-*-*-*-m-0-iso10646-1
by these glyphs:
  [0 0 0 3186 9 -1 9 9 1 nil]

Character code properties: customize what to show
  name: ARABIC LETTER SHEEN
  general-category: Lo (Letter, Other)
  decomposition: (1588) ('ش')

There are text properties here:
  fontified            t
--8<---------------cut here---------------end--------------->8---

Notice fontified is t now.  I don't think this matters much (because
there doesn't seem to be a correlation between character decompositions
and the value of this property), but could font-lock or some other major
mode feature have something to do with this issue?

0. emacs -Q
1. DEL                [optional, forces L2R paragraph direction]
2. M-x text-mode RET  [fundamental-mode also works]
3. C-u C-\ arabic RET
4. a ; RET a ; RET

Sure enough, the letters never decompose.

Note that, in the lisp-interaction-mode examples, the characters on
previous lines decompose not only when inserting repetitions of "a ;
RET", but also when deleting these insertions with repetitions of DEL.

Details of the three Emacs versions I can reproduce this on (master,
harfbuzz, emacs-26) follow my signature.

Thanks,

-- 
Basil

In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2019-05-20 built on thunk
Repository revision: afdc20d73c8588e5a744ecf7bffaf4401a557d20
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: Debian GNU/Linux 10 (buster)

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O2 -march=native' --config-cache
 --prefix=/home/blc/.local --with-mailutils --with-x-toolkit=lucid
 --with-modules --with-file-notification=yes --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON
PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: en_IE.UTF-8
  locale-coding-system: utf-8-unix


In GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2019-05-13 built on thunk
Repository revision: 5d7dafacf4afc888511649f6fc24c28210cd0dfc
Repository branch: harfbuzz
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: Debian GNU/Linux 10 (buster)

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O0 -g3 -ggdb -gdwarf-4'
 --config-cache --prefix=/home/blc/.local --program-suffix=-harfbuzz
 --enable-checking=yes,glyphs --enable-check-lisp-object-type
 --with-mailutils --with-x-toolkit=lucid --with-modules
 --with-file-notification=yes --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS
GLIB NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ
M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES
THREADS LIBSYSTEMD JSON PDUMPER LCMS2 GMP


In GNU Emacs 26.2.50 (build 5, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2019-05-20 built on thunk
Repository revision: 122ba1689046c53535b4d6c5142cfd81752808d0
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description:	Debian GNU/Linux 10 (buster)

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O0 -g3 -ggdb -gdwarf-4'
 --config-cache --prefix=/home/blc/.local --program-suffix=26
 --enable-checking=yes,glyphs --enable-check-lisp-object-type
 --with-mailutils --with-x-toolkit=lucid --with-modules
 --with-file-notification=yes --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS
GLIB NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT
ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS LIBSYSTEMD
LCMS2

bug#35811: 27.0.50; Arabic character (de)compositions affected by edits elsewhere in buffer

[Eli Zaretskii]

> From: "Basil L. Contovounesios" <contovob@tcd.ie>
> Date: Mon, 20 May 2019 20:03:42 +0100
> 
> 3. C-e RET
> 
> The sheen is correctly shaped in its initial form:
> 
> 3. a ; RET
> 
> The first sheen unexpectedly changes to its isolated form:

I haven't yet found a solution, but I did find a significant clue: in
an Emacs configured with --enable-checking=glyphs, set
inhibit-try-window-id non-nil, and the problem goes away.  So the root
cause is likely in try_window_id or the functions it calls.  Stay
tuned.

bug#35811: 27.0.50; Arabic character (de)compositions affected by edits elsewhere in buffer

[Eli Zaretskii]

> Date: Wed, 05 Jun 2019 18:23:11 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 35811@debbugs.gnu.org
> 
> I haven't yet found a solution, but I did find a significant clue: in
> an Emacs configured with --enable-checking=glyphs, set
> inhibit-try-window-id non-nil, and the problem goes away.  So the root
> cause is likely in try_window_id or the functions it calls.  Stay
> tuned.

Should be fixed now.

bug#35811: 27.0.50; Arabic character (de)compositions affected by edits elsewhere in buffer

[Basil L. Contovounesios]

tags 35811 fixed
close 35811 27.1
quit

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Wed, 05 Jun 2019 18:23:11 +0300
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: 35811@debbugs.gnu.org
>> 
>> I haven't yet found a solution, but I did find a significant clue: in
>> an Emacs configured with --enable-checking=glyphs, set
>> inhibit-try-window-id non-nil, and the problem goes away.  So the root
>> cause is likely in try_window_id or the functions it calls.  Stay
>> tuned.
>
> Should be fixed now.

It is indeed, thank you!

-- 
Basil