bug#12352: 24.2.50; Failure to compose characters in mode-line input method indicator

bug#12352: 24.2.50; Failure to compose characters in mode-line input method indicator

[Eli Zaretskii]

This bug report will be sent to the Bug-GNU-Emacs mailing list
and the GNU bug tracker at debbugs.gnu.org.  Please check that
the From: line contains a valid email address.  After a delay of up
to one day, you should receive an acknowledgment at that address.

Please write in English if possible, as the Emacs maintainers
usually do not have translators for other languages.

Please describe exactly what actions triggered the bug, and
the precise symptoms of the bug.  If you can, give a recipe
starting from `emacs -Q':

 emacs -Q
 C-u C-\ hebrew-full RET

Then look at the input method's indicator at the left edge of the mode
line.  It should be displayed as a single composed character with 2
diacriticals.  Instead, on Windows XP, it is displayed as 3 separate
characters, one after the other.  (On Windows 7 Emacs displays them
correctly, but that's because the font driver "composes" the
diacriticals for us.)

For some reason, Emacs does not compose the first and the second
characters (AYIN and HIRIQ) when we display them on the mode line.
But they _are_ composed when they are displayed in a buffer, just
visit leim/quail/hebrew.el and search for "hebrew-full".

In the debugger, I clearly see the composition in the glyph row
created for buffer display:

  Breakpoint 5, display_line (it=0x82ca00) at xdisp.c:19258
  19258     struct glyph_row *row = it->glyph_row;
  (gdb) n
  19261     void *wrap_data = NULL;
  (gdb) pgrow
  TEXT: 54 glyphs
    0    0: CHAR[ ] pos=7864 blev=0,btyp=L w=8 a+d=12+4 MB
    1    8: CHAR["] pos=7865 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
    2   16: CHAR[h] pos=7866 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
    3   24: CHAR[e] pos=7867 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
    4   32: CHAR[b] pos=7868 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
    5   40: CHAR[r] pos=7869 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
    6   48: CHAR[e] pos=7870 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
    7   56: CHAR[w] pos=7871 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
    8   64: CHAR[-] pos=7872 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
    9   72: CHAR[f] pos=7873 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   10   80: CHAR[u] pos=7874 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   11   88: CHAR[l] pos=7875 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   12   96: CHAR[l] pos=7876 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   13  104: CHAR["] pos=7877 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   14  112: CHAR[ ] pos=7878 blev=0,btyp=L w=8 a+d=12+4 MB
   15  120: CHAR["] pos=7879 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   16  128: CHAR[H] pos=7880 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   17  136: CHAR[e] pos=7881 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   18  144: CHAR[b] pos=7882 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   19  152: CHAR[r] pos=7883 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   20  160: CHAR[e] pos=7884 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   21  168: CHAR[w] pos=7885 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   22  176: CHAR["] pos=7886 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   23  184: CHAR[ ] pos=7887 blev=0,btyp=L w=8 a+d=12+4 MB
   24  192: CHAR["] pos=7888 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   25  200: CHAR[0x5ab] pos=7891 blev=1,btyp=R w=5 a+d=14+4 face=24 MB
   26  205: COMP[8 (0..1)] pos=7889 w=8 a+d=12+4 face=25 MB slice=0,0,1,0
   27  213: CHAR["] pos=7892 blev=0,btyp=L w=8 a+d=12+4 face=21 MB
   28  221: CHAR[ ] pos=7893 blev=0,btyp=L w=8 a+d=12+4 MB

but I see a strange single-character "composition" when the same 3
characters are displayed from a Lisp string on the mode line:

  Breakpoint 6, display_string (string=0x0, lisp_string=55651665,
      face_string=54827034, face_string_pos=0, start=0, it=0x82c9e0,
      field_width=0, precision=-1, max_x=0, multibyte=1) at xdisp.c:21887
  21887     int hpos_at_start = it->hpos;
  (gdb) pp lisp_string
  #("עִ֫" 0 3 (mouse-face mode-line-highlight local-map (keymap (mode-line keymap
  ...
  (gdb) finish
  Run till exit from #0  display_string (string=0x0, lisp_string=55651665,
      face_string=54827034, face_string_pos=0, start=0, it=0x82c9e0,
      field_width=0, precision=-1, max_x=0, multibyte=1) at xdisp.c:21887
  0x011988d5 in display_mode_element (it=0x82c9e0, depth=8, field_width=0,
      precision=-1, elt=55651665, props=56509982, risky=1) at xdisp.c:20543
  20543                   n += display_string (NULL, elt, Qnil, 0, 0, it,
  Value returned is $8 = 3
  (gdb) pgrowx it->glyph_row
  TEXT: 4 glyphs
    0    0: CHAR[ ] str=35e7fb1[0] blev=0,btyp=L w=9 a+d=12+4 face=1 [
    1    9: CHAR[0x5ab] str=3512d51[2] blev=1,btyp=R w=5 a+d=14+4 face=15 MB
    2   14: COMP[0 (0..0)] str=3512d51[1] w=8 a+d=12+4 face=16 MB
    3   22: CHAR[0x5e2] str=3512d51[0] blev=1,btyp=R w=8 a+d=12+4 face=16 MB

Why isn't Emacs composing these characters when they are displayed
from a Lisp string?

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
d:/gnu/bzr/emacs/trunk/etc/DEBUG.


In GNU Emacs 24.2.50.1 (i386-mingw-nt5.1.2600)
 of 2012-09-04 on HOME-C4E4A596F7
Bzr revision: 109879 eggert@cs.ucla.edu-20120904182904-vhuzro412a8v1ue5
Windowing system distributor `Microsoft Corp.', version 5.1.2600
Configured using:
 `configure --with-gcc (3.4) --no-opt --enable-checking --cflags
 -Id:/usr/include/libxml2 -DGLYPH_DEBUG=1'

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-u C-\ h e b r <tab> - f <tab> <return> M-x r e d 
r <tab> <return> C-x C-f <M-backspace> l e i <tab> 
q u a <tab> h e b <tab> <return> C-s f u l l <down> 
<down> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> <mouse-1> 
<mouse-1> <mouse-1> <mouse-1> <mouse-1> <up> <up> <mouse-1> 
M-x <up> <return> C-x b <return> M-x <up> <return> 
M-x r e p o r t - e m <tab> <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Loading quail/hebrew...done
Loading vc-bzr...done
Mark saved where search started

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils misearch multi-isearch vc-bzr mule-util cus-start
cus-load quail help-mode easymenu time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 disp-table ls-lisp w32-win w32-vars
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process multi-tty emacs)

bug#12352: 24.2.50; Failure to compose characters in mode-line input method indicator

[Eli Zaretskii]

> From: Kenichi Handa <handa@gnu.org>
> Cc: handa@gnu.org
> Date: Sat, 22 Sep 2012 13:04:03 +0900
> 
> In article <876279dvv7.fsf@gnu.org>, Kenichi Handa <handa@gnu.org> writes:
> 
> > I think I tracked down the problem to the function
> > composition_reseat_it which calls autocmp_chars.  This bug
> > happens when a Hebrew (or any other R2L script) syllable has
> > N characters and a font selected for the last character of
> > the syllable is different from a font selected for the first
> > character.  But, autocmp_chars was not coded to work on such
> > a situation.  I'm now trying to find a solution.
> 
> I've just installed a fix to the trunk.  Could you please
> try it?

Thanks, it works now the same in both buffer and mode-line display.  I
think you can close this bug now.

The last character, HEBREW ACCENT OLE, is still not composed with the
first two, both in the buffer display and on the mode line.  But I
understand that this is because that character is taken from a
different font, and we don't currently support compositions for
characters that come from different fonts, is that right?  If I use
the "Keter YG" font for Hebrew, then all 3 of the characters are
composed.

bug#12352: 24.2.50; Failure to compose characters in mode-line input method indicator

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Kenichi Handa <handa@gnu.org>
>> Cc: handa@gnu.org
>> Date: Sat, 22 Sep 2012 13:04:03 +0900
>> 
>> In article <876279dvv7.fsf@gnu.org>, Kenichi Handa <handa@gnu.org> writes:
>> 
>> > I think I tracked down the problem to the function
>> > composition_reseat_it which calls autocmp_chars.  This bug
>> > happens when a Hebrew (or any other R2L script) syllable has
>> > N characters and a font selected for the last character of
>> > the syllable is different from a font selected for the first
>> > character.  But, autocmp_chars was not coded to work on such
>> > a situation.  I'm now trying to find a solution.
>> 
>> I've just installed a fix to the trunk.  Could you please
>> try it?
>
> Thanks, it works now the same in both buffer and mode-line display.  I
> think you can close this bug now.
>
> The last character, HEBREW ACCENT OLE, is still not composed with the
> first two, both in the buffer display and on the mode line.  But I
> understand that this is because that character is taken from a
> different font, and we don't currently support compositions for
> characters that come from different fonts, is that right?  If I use
> the "Keter YG" font for Hebrew, then all 3 of the characters are
> composed.

Hi Eli,

Above you suggest to close the bug, but you also report a different
issue.  Can this bug be closed, or should it stay open?

TIA.

Best regards,
Stefan Kangas

bug#12352: 24.2.50; Failure to compose characters in mode-line input method indicator

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Cc: Kenichi Handa <handa@gnu.org>,  12352@debbugs.gnu.org
> Date: Thu, 31 Oct 2019 00:08:55 +0100
> 
> Above you suggest to close the bug, but you also report a different
> issue.  Can this bug be closed, or should it stay open?

It should be closed (done); the issue I mention is a general
(mis)feature of character compositions in Emacs, and works as
designed.

Thanks.