Incorrect glyph info in describe-char

Incorrect glyph info in describe-char

[Anand Tamariya]

[-- Attachment #1.1: Type: text/plain, Size: 1295 bytes --]

describe-char for shin (character at point in shin.png) provides glyph info
as belonging to DejaVuSans. However, the actual glyph in DejaVuSans is
different as shown in ftview.png.

             position: 4 of 11 (27%), column: 4
            character: ש‎ (displayed as ש‎) (codepoint 1513, #o2751, #x5e9)
              charset: unicode (Unicode (ISO10646))
code point in charset: 0x05E9
               script: hebrew
               syntax: w which means: word
             category: .:Base, R:Right-to-left (strong)
             to input: type "C-x 8 RET 5e9" or "C-x 8 RET HEBREW LETTER
SHIN"
          buffer code: #xD7 #xA9
            file code: not encodable by coding system iso-latin-1-unix
              display: composed to form "שָׁ" (see below)

Composed with the following character(s) "ָׁ" using this font:
  *ftcrhb:-PfEd-DejaVu Sans-bold-oblique-normal-*-45-*-*-*-*-0-iso10646-1*
by these glyphs:
  [0 2 1464 1305 0 4 21 0 10 nil]
  [0 2 1473 1314 0 36 42 33 -28 nil]
  [0 2 1513 1344 34 4 39 25 0 nil]

Character code properties: customize what to show
  name: HEBREW LETTER SHIN
  general-category: Lo (Letter, Other)
  decomposition: (1513) ('ש')

There are text properties here:
  face                 (:weight bold :slant oblique)

[-- Attachment #1.2: Type: text/html, Size: 1538 bytes --]

[-- Attachment #2: shin.png --]
[-- Type: image/png, Size: 2509 bytes --]

[-- Attachment #3: ftview.png --]
[-- Type: image/png, Size: 11181 bytes --]

Re: Incorrect glyph info in describe-char

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Thu, 22 Jul 2021 12:32:12 +0530
> 
> describe-char for shin (character at point in shin.png) provides glyph info as belonging to DejaVuSans.
> However, the actual glyph in DejaVuSans is different as shown in ftview.png.

Please show the entire describe-char display in Emacs as an image, and
also the buffer text where you invoked that command as an image from
an Emacs window.

Your shin.png image shows a font that isn't DejaVu Sans, so I'm
curious why Emacs says it is that font.

Re: Incorrect glyph info in describe-char

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 345 bytes --]

> Please show the entire describe-char display in Emacs as an image, and
> also the buffer text where you invoked that command as an image from
> an Emacs window.
>
> Your shin.png image shows a font that isn't DejaVu Sans, so I'm
> curious why Emacs says it is that font.
>
I've uploaded a complete frame image here https://imgur.com/fh8Vtr0 .

[-- Attachment #2: Type: text/html, Size: 1735 bytes --]

Re: Incorrect glyph info in describe-char

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Thu, 22 Jul 2021 13:33:20 +0530
> Cc: emacs-devel@gnu.org
> 
>  Please show the entire describe-char display in Emacs as an image, and
>  also the buffer text where you invoked that command as an image from
>  an Emacs window.
> 
>  Your shin.png image shows a font that isn't DejaVu Sans, so I'm
>  curious why Emacs says it is that font.
> 
> I've uploaded a complete frame image here https://imgur.com/fh8Vtr0 .

Thanks, but I see nothing when I go there.

Could you please post the images here?  TIA.

Re: Incorrect glyph info in describe-char

[Anand Tamariya]

[-- Attachment #1.1: Type: text/plain, Size: 345 bytes --]

> Could you please post the images here?  TIA.
>
See attached.

Additional info: I applied text properties (it's in the describe-char
output, but easy to overlook) using
(put-text-property (point) (1+ (point)) 'face '(:weight bold :slant
oblique))

Apparently, this is what is causing the issue. Even though the glyph is
neither bold nor slant.

[-- Attachment #1.2: Type: text/html, Size: 645 bytes --]

[-- Attachment #2: shin1.png --]
[-- Type: image/png, Size: 122444 bytes --]

Re: Incorrect glyph info in describe-char

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Thu, 22 Jul 2021 14:42:42 +0530
> Cc: emacs-devel@gnu.org
> 
>  Could you please post the images here?  TIA.
> 
> See attached.

Thanks.

> Additional info: I applied text properties (it's in the describe-char output, but easy to overlook) using 
> (put-text-property (point) (1+ (point)) 'face '(:weight bold :slant oblique))

You did that only for a single character SHIN?  But the image shows
that the entire string is displayed by a font that is not DejaVu Sans
Mono.  And at least on my system, DejaVu Sans Mono doesn't have glyphs
for the Hebrew block, so when I tell Emacs to use DejaVu Sans Mono as
the default font, the Hebrew letters are displayed using a different
font.

If you don't apply the text property, do you see the Hebrew text
displayed by DejaVu Sans Mono, or does Emacs use a different font for
it?  And what does describe-char say about that if you don't apply the
text property?

> Apparently, this is what is causing the issue.

It could be, but I don't see how this could happen, by just looking at
the code involved in this.  I'm probably missing something.

Re: Incorrect glyph info in describe-char

[Eli Zaretskii]

> Date: Thu, 22 Jul 2021 13:06:17 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > Apparently, this is what is causing the issue.
> 
> It could be, but I don't see how this could happen, by just looking at
> the code involved in this.  I'm probably missing something.

Btw, is this in "emacs -Q"?  If "emacs -Q" doesn't exhibit this
problem, perhaps you have some font-related customizations which do?

Re: Incorrect glyph info in describe-char

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 1134 bytes --]

> You did that only for a single character SHIN?  But the image shows
> that the entire string is displayed by a font that is not DejaVu Sans
> Mono.  And at least on my system, DejaVu Sans Mono doesn't have glyphs
> for the Hebrew block, so when I tell Emacs to use DejaVu Sans Mono as
> the default font, the Hebrew letters are displayed using a different
> font.
>
> If you don't apply the text property, do you see the Hebrew text
> displayed by DejaVu Sans Mono, or does Emacs use a different font for
> it?  And what does describe-char say about that if you don't apply the
> text property?
>
> > Apparently, this is what is causing the issue.
>
> It could be, but I don't see how this could happen, by just looking at
> the code involved in this.  I'm probably missing something.
>
It is "emacs -Q". My default font is DejaVuSans Mono. Hence H is displayed
using the same. The Hebrew text is using Linux Libertine Display O. For
printing, I needed text with font variation. So I applied the text property
for weight and slant - though it didn't change the appearance. After that,
desc-char starts reporting as mentioned in OP.

[-- Attachment #2: Type: text/html, Size: 1427 bytes --]

Re: Incorrect glyph info in describe-char

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Fri, 23 Jul 2021 10:33:06 +0530
> Cc: emacs-devel@gnu.org
> 
>  You did that only for a single character SHIN?  But the image shows
>  that the entire string is displayed by a font that is not DejaVu Sans
>  Mono.  And at least on my system, DejaVu Sans Mono doesn't have glyphs
>  for the Hebrew block, so when I tell Emacs to use DejaVu Sans Mono as
>  the default font, the Hebrew letters are displayed using a different
>  font.
> 
>  If you don't apply the text property, do you see the Hebrew text
>  displayed by DejaVu Sans Mono, or does Emacs use a different font for
>  it?  And what does describe-char say about that if you don't apply the
>  text property?
> 
>  > Apparently, this is what is causing the issue.
> 
>  It could be, but I don't see how this could happen, by just looking at
>  the code involved in this.  I'm probably missing something.
> 
> It is "emacs -Q". My default font is DejaVuSans Mono. Hence H is displayed using the same. The Hebrew
> text is using Linux Libertine Display O. For printing, I needed text with font variation. So I applied the text
> property for weight and slant - though it didn't change the appearance. After that, desc-char starts reporting
> as mentioned in OP. 

Thanks, but this answers only part of my questions.  Please also
answer the following:

  . did you put the text property only on the single character SHIN or
    on the whole Hebrew text?
  . what does describe-char report for the SHIN character before you
    put the text property?

TIA

Re: Incorrect glyph info in describe-char

[Eli Zaretskii]

> Date: Fri, 23 Jul 2021 08:52:23 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> Thanks, but this answers only part of my questions.  Please also
> answer the following:
> 
>   . did you put the text property only on the single character SHIN or
>     on the whole Hebrew text?
>   . what does describe-char report for the SHIN character before you
>     put the text property?

And one new question:

  . Your original report shows Hebrew text with diacritics (so-called
    "Hebrew points").  Specifically, the letter SHIN is followed by
    two such diacritics: U+05C1 and U+05B8.  Emacs composes those on
    display into a single grapheme cluster.  If you use the Hebrew
    text without those diacritics, i.e. just "שלום", does the problem
    happen in that case as well, i.e. do you still see the incorrect
    font name after adding the text property?

Re: Incorrect glyph info in describe-char

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 970 bytes --]

>   . did you put the text property only on the single character SHIN or
>     on the whole Hebrew text?
>
 Only on single character using (put-text-property (point) (1+ (point))
'face '(:weight bold :slant oblique)) while the point was under that
character.

  . what does describe-char report for the SHIN character before you
>     put the text property?
>
Linux Libertine Display O


> . Your original report shows Hebrew text with diacritics (so-called
>     "Hebrew points").  Specifically, the letter SHIN is followed by
>     two such diacritics: U+05C1 and U+05B8.  Emacs composes those on
>     display into a single grapheme cluster.  If you use the Hebrew
>     text without those diacritics, i.e. just "שלום", does the problem
>     happen in that case as well, i.e. do you still see the incorrect
>     font name after adding the text property?
>
 This atleast applies the weight correctly but reported font is DejaVu
Sans-bold.

[-- Attachment #2: Type: text/html, Size: 1549 bytes --]

Re: Incorrect glyph info in describe-char

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Fri, 23 Jul 2021 12:11:17 +0530
> Cc: emacs-devel@gnu.org
> 
>    . what does describe-char report for the SHIN character before you
>      put the text property?
> 
> Linux Libertine Display O
> 
>  . Your original report shows Hebrew text with diacritics (so-called
>      "Hebrew points").  Specifically, the letter SHIN is followed by
>      two such diacritics: U+05C1 and U+05B8.  Emacs composes those on
>      display into a single grapheme cluster.  If you use the Hebrew
>      text without those diacritics, i.e. just "שלום", does the problem
>      happen in that case as well, i.e. do you still see the incorrect
>      font name after adding the text property?
> 
>  This atleast applies the weight correctly but reported font is DejaVu Sans-bold.

OK, thanks.  And in the last case, the text us indeed displayed using
DejaVu Sans, i.e. the glyphs look similar to what ftview shows?

And what does the below display in the original case, i.e. with Hebrew
points in the text:

  M-: (find-composition (point) nil nil t) RET

when point is on the SHIN character?

Re: Incorrect glyph info in describe-char

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 424 bytes --]

>
> And what does the below display in the original case, i.e. with Hebrew
> points in the text:
>
>   M-: (find-composition (point) nil nil t) RET
>
> when point is on the SHIN character?
>
(10 13 [[#<font-object "-PfEd-Linux Libertine Display
O-normal-normal-normal-*-78-*-*-*-*-0-iso10646-1"> 1513 1473 1464] 47 [0 2
1464 967 0 6 20 -5 17 [8 0 0]] [0 2 1473 976 0 24 32 52 -45 [9 0 0]] [0 2
1513 1005 48 2 46 44 0 nil]])

[-- Attachment #2: Type: text/html, Size: 660 bytes --]

Re: Incorrect glyph info in describe-char

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Fri, 23 Jul 2021 13:06:57 +0530
> Cc: emacs-devel@gnu.org
> 
>  And what does the below display in the original case, i.e. with Hebrew
>  points in the text:
> 
>    M-: (find-composition (point) nil nil t) RET
> 
>  when point is on the SHIN character?
> 
> (10 13 [[#<font-object "-PfEd-Linux Libertine Display O-normal-normal-normal-*-78-*-*-*-*-0-iso10646-1">
> 1513 1473 1464] 47 [0 2 1464 967 0 6 20 -5 17 [8 0 0]] [0 2 1473 976 0 24 32 52 -45 [9 0 0]] [0 2 1513 1005
> 48 2 46 44 0 nil]]) 

That's the correct font, which is unexpected: the above
find-composition call is from describe-char, so if that returns the
correct font, how come describe-char winds up reporting an incorrect
font?

Could you perhaps step in Edebug through describe-char and see what
happens to the value returned by find-composition in your case, before
the font is reported?  The call to find-composition is near the
beginning of describe-char, here:

  (interactive "d")
  (unless (buffer-live-p buffer) (setq buffer (current-buffer)))
  (let ((src-buf (current-buffer)))
    (set-buffer buffer)
    (if (>= pos (point-max))
        (error "No character follows specified position"))
    (let* ((char (char-after pos))
           (eight-bit-p (and (not enable-multibyte-characters) (>= char 128)))
           (charset (if eight-bit-p 'eight-bit
                      (or (get-text-property pos 'charset)
                          (char-charset char))))
           (composition (find-composition pos nil nil t))  <<<<<<<<<<<<<<<
           (component-chars nil)

Thanks.

Re: Incorrect glyph info in describe-char

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 1154 bytes --]

On Fri, Jul 23, 2021 at 1:15 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Anand Tamariya <atamariya@gmail.com>
> > Date: Fri, 23 Jul 2021 13:06:57 +0530
> > Cc: emacs-devel@gnu.org
> >
> >  And what does the below display in the original case, i.e. with Hebrew
> >  points in the text:
> >
> >    M-: (find-composition (point) nil nil t) RET
> >
> >  when point is on the SHIN character?
> >
> > (10 13 [[#<font-object "-PfEd-Linux Libertine Display
> O-normal-normal-normal-*-78-*-*-*-*-0-iso10646-1">
> > 1513 1473 1464] 47 [0 2 1464 967 0 6 20 -5 17 [8 0 0]] [0 2 1473 976 0
> 24 32 52 -45 [9 0 0]] [0 2 1513 1005
> > 48 2 46 44 0 nil]])
>
> That's the correct font, which is unexpected: the above
> find-composition call is from describe-char, so if that returns the
> correct font, how come describe-char winds up reporting an incorrect
> font?


My bad. I mixed up the test. After applying the text property on SHIN, the
result is:
(10 13 [[#<font-object "-PfEd-DejaVu
Sans-bold-oblique-normal-*-78-*-*-*-*-0-iso10646-1"> 1513 1473 1464] 63 [0
2 1464 1305 0 8 35 0 17 nil] [0 2 1473 1314 0 63 73 56 -48 nil] [0 2 1513
1344 58 6 67 43 0 nil]])

[-- Attachment #2: Type: text/html, Size: 1745 bytes --]

Re: Incorrect glyph info in describe-char

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Fri, 23 Jul 2021 13:45:23 +0530
> Cc: emacs-devel@gnu.org
> 
> My bad. I mixed up the test. After applying the text property on SHIN, the result is:
> (10 13 [[#<font-object "-PfEd-DejaVu Sans-bold-oblique-normal-*-78-*-*-*-*-0-iso10646-1"> 1513 1473
> 1464] 63 [0 2 1464 1305 0 8 35 0 17 nil] [0 2 1473 1314 0 63 73 56 -48 nil] [0 2 1513 1344 58 6 67 43 0 nil]]) 

Ah.  So the problem is somewhere inside find_automatic_composition or
the functions it calls.  If you can step through that code with GDB,
please do (I'll provide instructions if you need them).  Failing that,
someone who can reproduce the problem would need to see what happens
in there; I cannot reproduce the issue with the fonts I have here.