bug#17874: 24.3.92; Incorrectly isolated ع in علی

bug#17874: 24.3.92; Incorrectly isolated ع in علی

[penthief]

The following should display differently, as in 24.3.1.
A.  ع ل ی - these are the "isolated" types.
B.  علی  - this is a final, medial and initial type.
C.  عل ی‎ - the isolated ی should be joined when the space is removed. 
This is the bug demonstrated.

0. Start emacs: $ emacs -Q
1. Output علی with a bold face in the appropriate font:
(progn
   (setq buffer-face-mode-face '(:family  "DejaVu Sans Mono" 
:alternate-families  "Monaco" :height 100))
   (buffer-face-mode)
   (buffer-face-set 'bold)
   (newline)
   (insert " علی"))

  علی

2. Change to a regular face:
(buffer-face-set 'default)
  علی

What happens is ی (ARABIC LETTER FARSI YEH) switches from the final to 
the isolated type.
This is incorrect since there should be no difference in the 
representation when eval-ing the following:

(buffer-face-toggle 'bold)

The output of describe-char on the emboldened and regular differs on the 
following:

Regular (incorrect):

Composed using this font:
   
xft:-unknown-KacstDecorative-normal-normal-normal-*-11-*-*-*-*-0-iso10646-1
by these glyphs:
   [0 0 0 107 5 0 6 5 2 nil]

Bold (correct):

Composed using this font:
   xft:-unknown-KacstOne-bold-normal-normal-*-11-*-*-*-*-0-iso10646-1
by these glyphs:
   [2 2 0 200 7 0 8 2 4 nil]

Please note that this bug does not exist on 24.3.1, and I apologise if 
it is a problem with my compilation settings.

Regards,
Matt


In GNU Emacs 24.3.92.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.10.8)
  of 2014-06-28 on zz
Repository revision: 117323 
schwab@linux-m68k.org-20140628072401-95qhsfaufm9fxrj8
Windowing system distributor `The X.Org Foundation', version 
11.0.11501000
System Description:	Ubuntu 14.04 LTS

Configured using:
  `configure --with-dbus --with-libotf --with-m17n-flt'

Important settings:
   value of $LC_CTYPE: en_US.UTF-8
   value of $LANG: en_US.UTF-8
   value of $XMODIFIERS: @im=ibus
   locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
   tooltip-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   line-number-mode: t
   transient-mark-mode: t

Recent input:
M-x r e p o <tab> - e - b <tab> <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list...

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
help-fns mail-prsvr mail-utils help-mode easymenu time-date tooltip
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
gfilenotify dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty emacs)

Memory information:
((conses 16 72971 7323)
  (symbols 48 17511 0)
  (miscs 40 61 164)
  (strings 32 9378 3890)
  (string-bytes 1 256561)
  (vectors 16 9668)
  (vector-slots 8 389112 14907)
  (floats 8 63 385)
  (intervals 56 182 0)
  (buffers 960 12)
  (heap 1024 46452 877))

bug#17874: 24.3.92; Incorrectly isolated ع in علی

[Eli Zaretskii]

> Date: Sun, 29 Jun 2014 13:30:36 +0100
> From: penthief <penthief@SDF.ORG>
> 
> The following should display differently, as in 24.3.1.
> A.  ع ل ی - these are the "isolated" types.
> B.  علی  - this is a final, medial and initial type.
> C.  عل ی‎ - the isolated ی should be joined when the space is removed. 
> This is the bug demonstrated.
> 
> 0. Start emacs: $ emacs -Q
> 1. Output علی with a bold face in the appropriate font:
> (progn
>    (setq buffer-face-mode-face '(:family  "DejaVu Sans Mono" 
> :alternate-families  "Monaco" :height 100))
>    (buffer-face-mode)
>    (buffer-face-set 'bold)
>    (newline)
>    (insert " علی"))
> 
>   علی
> 
> 2. Change to a regular face:
> (buffer-face-set 'default)
>   علی
> 
> What happens is ی (ARABIC LETTER FARSI YEH) switches from the final to 
> the isolated type.

I cannot reproduce this on MS-Windows.  It works for me correctly both
in Emacs 24.3 and in the emacs-24 branch as of r117323.  Could be
libotf-specific.  Are you sure nothing changed in libotf and/or
m17n-flt libraries between the time you built v24.3 and now?

bug#17874: 24.3.92; Incorrectly isolated ع in علی

[penthief]

On 2014-06-29 19:32, Eli Zaretskii wrote:
>> Date: Sun, 29 Jun 2014 13:30:36 +0100
>> From: penthief <penthief@SDF.ORG>
>> 
>> The following should display differently, as in 24.3.1.
>> A.  ع ل ی - these are the "isolated" types.
>> B.  علی  - this is a final, medial and initial type.
>> C.  عل ی‎ - the isolated ی should be joined when the space is removed.
>> This is the bug demonstrated.
>> 
>> 0. Start emacs: $ emacs -Q
>> 1. Output علی with a bold face in the appropriate font:
>> (progn
>>    (setq buffer-face-mode-face '(:family  "DejaVu Sans Mono"
>> :alternate-families  "Monaco" :height 100))
>>    (buffer-face-mode)
>>    (buffer-face-set 'bold)
>>    (newline)
>>    (insert " علی"))
>> 
>>   علی
>> 
>> 2. Change to a regular face:
>> (buffer-face-set 'default)
>>   علی
>> 
>> What happens is ی (ARABIC LETTER FARSI YEH) switches from the final to
>> the isolated type.
> 
> I cannot reproduce this on MS-Windows.  It works for me correctly both
> in Emacs 24.3 and in the emacs-24 branch as of r117323.  Could be
> libotf-specific.  Are you sure nothing changed in libotf and/or
> m17n-flt libraries between the time you built v24.3 and now?

After building 24.3 rather than relying on the binary provided by my 
distro I now see the issue in both versions, so I think you are right to 
say the problem is with one of the dependencies, rather than with Emacs.

Perhaps I can find a way to reproduce an issue on libotf / m17n-flt 
separately and raise it in the appropriate place.

Thanks,
Matt

bug#17874: 24.3.1; Incorrectly isolated ی in علی

[Mat Smiglarski]

[-- Attachment #1.1: Type: text/plain, Size: 1132 bytes --]

 

I wonder whether this bug should be reopened?

I have corrected the subject which was incorrect and specified the
version as 24.3.1 because this is not a regression.

The bug does not occur with bold text, nor with a dotted yeh (ARABIC
LETTER YEH); only the dotless version (ARABIC LETTER FARSI YEH). 
Eli has already reported that he does not see this problem on Windows.
The behaviour is also seen when emacs is configured using
"--without-m17n". Configuring using "--without-libotf" results in all
glyphs being displayed as isolated.

Should this be reopened? Screenshot attached for evidence.

Thanks,
Mat

On 2014-08-12 06:47, help-debbugs@gnu.org wrote:
> This is an automatic notification regarding your bug report
> #17874: 24.3.92; Incorrectly isolated ع in علی,
> which was filed against the emacs package.
> 
> Thank you for your report, which has now been closed.
> You can view the full report at
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17874
> 
> If you require further information, please followup to 17874@debbugs.gnu.org.
> 
> debbugs.gnu.org maintainers
> (administrator, GNU bugs database)
 

[-- Attachment #1.2: Type: text/html, Size: 1412 bytes --]

[-- Attachment #2: incorrectly-isolated-arabic-farsi-yeh.png --]
[-- Type: image/png, Size: 40793 bytes --]

bug#17874: 24.3.1; Incorrectly isolated ی in علی

[Eli Zaretskii]

> Date: Sun, 17 Aug 2014 11:40:10 +0100
> From: Mat Smiglarski <penthief@SDF.ORG>
> 
> I wonder whether this bug should be reopened?

I'm not sure.  All the signs point towards shaping libraries (libotf
etc.) that are not maintained by the Emacs team, or maybe to specific
fonts.  If that is true, I don't see what we can do with this problem.

Can you try upgrading or downgrading to different versions of libotf
and/or libm17n, or use different fonts?

> The behaviour is also seen when emacs is configured using
> "--without-m17n". Configuring using "--without-libotf" results in all
> glyphs being displayed as isolated.

This is expected.  The INSTALL file says:

  * Complex Text Layout support libraries

  On GNU and Unix systems, Emacs needs the optional libraries "m17n-db",
  "libm17n-flt", "libotf" to correctly display such complex scripts as
  Indic and Khmer, and also for scripts that require Arabic shaping
  support (Arabic and Farsi).

So you cannot have Arabic shaping without those 3 libraries.