emacs hebrew script bidi nikud rendering bugs

emacs hebrew script bidi nikud rendering bugs

[Mark David]

I want to report and maybe fix bugs in rendering Hebrew script, particularly with diacritics (nikud). It only partially works in environments I'm using (Ubunutu, Emacs 25.x).  (Hebrew without diacritics, used for everyday Hebrew language writing, is evidently functioning well.)  Is there a mailing list best suited for that?  Is there a separate development team for that kind of thing?  Where would I report, say, inability to correctly render a sequence of Hebrew base character and diacritic like this



Unicode Character 'HEBREW LETTER ALEF' (U+05D0)

Unicode Character 'HEBREW POINT QAMATS' (U+05B8)


(with a given font, window system, OS, "multlingual environment" setting, etc.)?

Thanks,

Mark

Re: emacs hebrew script bidi nikud rendering bugs

[Eli Zaretskii]

> Date: Fri, 06 Nov 2015 11:36:08 -0800
> From: Mark David <markhd@fastmail.fm>
> 
> I want to report and maybe fix bugs in rendering Hebrew script, particularly with diacritics (nikud). It only partially works in environments I'm using (Ubunutu, Emacs 25.x).  (Hebrew without diacritics, used for everyday Hebrew language writing, is evidently functioning well.)  Is there a mailing list best suited for that?  Is there a separate development team for that kind of thing?  Where would I report, say, inability to correctly render a sequence of Hebrew base character and diacritic like this
> 
> 
> 
> Unicode Character 'HEBREW LETTER ALEF' (U+05D0)
> 
> Unicode Character 'HEBREW POINT QAMATS' (U+05B8)
> 
> 
> (with a given font, window system, OS, "multlingual environment" setting, etc.)?

Please use "M-x report-emacs-bug RET", and include with your report
all the details, and preferably a reproducible recipe starting from
"emacs -Q" that demonstrates the problem on your machine.

Thanks.

Re: emacs hebrew script bidi nikud rendering bugs

[Uwe Brauer]

>>> "Mark" == Mark David <markhd@fastmail.fm> writes:

    > I want to report and maybe fix bugs in rendering Hebrew script,
    > particularly with diacritics (nikud). It only partially works in
    > environments I'm using (Ubunutu, Emacs 25.x). (Hebrew without
    > diacritics, used for everyday Hebrew language writing, is evidently
    > functioning well.) Is there a mailing list best suited for that? Is
    > there a separate development team for that kind of thing? Where would
    > I report, say, inability to correctly render a sequence of Hebrew base
    > character and diacritic like this

    > Unicode Character 'HEBREW LETTER ALEF' (U+05D0)

    > Unicode Character 'HEBREW POINT QAMATS' (U+05B8)

I am not sure what you mean by rendered correctly.

Let A=aleph
    ^=QAMATS

Then for example OpenOffice displays this correctly as

   A
   ^
while Emacs 25 (and 24.5) displays it as

   A
  ^ 

That is what you mean? (Not sure whether to write hebrew+niqqud on the
list) 

Remark: using xelatex the hebrew text with niqquds get compiled and the
pdf displays the niqqud correctly.


Uwe Brauer 

Re: emacs hebrew script bidi nikud rendering bugs

[Eli Zaretskii]

> From: Uwe Brauer <oub@mat.ucm.es>
> Date: Sat, 07 Nov 2015 09:41:46 +0000
> 
> >>> "Mark" == Mark David <markhd@fastmail.fm> writes:
> 
>     > I want to report and maybe fix bugs in rendering Hebrew script,
>     > particularly with diacritics (nikud). It only partially works in
>     > environments I'm using (Ubunutu, Emacs 25.x). (Hebrew without
>     > diacritics, used for everyday Hebrew language writing, is evidently
>     > functioning well.) Is there a mailing list best suited for that? Is
>     > there a separate development team for that kind of thing? Where would
>     > I report, say, inability to correctly render a sequence of Hebrew base
>     > character and diacritic like this
> 
>     > Unicode Character 'HEBREW LETTER ALEF' (U+05D0)
> 
>     > Unicode Character 'HEBREW POINT QAMATS' (U+05B8)
> 
> I am not sure what you mean by rendered correctly.
> 
> Let A=aleph
>     ^=QAMATS
> 
> Then for example OpenOffice displays this correctly as
> 
>    A
>    ^
> while Emacs 25 (and 24.5) displays it as
> 
>    A
>   ^ 

Not here: I see A, as expected.
                ^

Re: emacs hebrew script bidi nikud rendering bugs

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 407 bytes --]



   > Not here: I see A, as expected.
   >                 ^

You are right for 25, at least partially. I my default font is 
 '(default ((t (:family "DejaVu Sans Mono" :foundry "unknown" :slant
 normal :weight normal :height 143 :width normal)))) 

And I don't see *all* niqqud correctly. I attach a screenshot with the
misplaced niqqud of a latex file and a pdf file where they are displayed
correctly.


[-- Attachment #2: niqqud.png --]
[-- Type: image/png, Size: 14507 bytes --]

[-- Attachment #3: testheb.pdf --]
[-- Type: application/pdf, Size: 17688 bytes --]

Re: emacs hebrew script bidi nikud rendering bugs

[Eli Zaretskii]

> From: Uwe Brauer <oub@mat.ucm.es>
> Date: Sat, 07 Nov 2015 17:27:57 +0000
> 
>    > Not here: I see A, as expected.
>    >                 ^
> 
> You are right for 25, at least partially. I my default font is 
>  '(default ((t (:family "DejaVu Sans Mono" :foundry "unknown" :slant
>  normal :weight normal :height 143 :width normal)))) 

The default font is not what matters.  You should find out which
font(s) is/are used to display the problematic character combinations.
Emacs cannot compose characters that are displayed by different fonts.
Use "C-u C-x =" to show the font for each character.

Re: emacs hebrew script bidi nikud rendering bugs

[Uwe Brauer]

    > The default font is not what matters.  You should find out which
    > font(s) is/are used to display the problematic character combinations.
    > Emacs cannot compose characters that are displayed by different fonts.
    > Use "C-u C-x =" to show the font for each character.

Out of curiosity, are all niqqud displayed correctly in your setting?


    -  First of all the ones which work
       for example aleph+qamats: I obtain



,----
| 
|                     position: 431 of 708 (61%), column: 12
|             character: א‎ (displayed as א‎) (codepoint 1488, #o2720, #x5d0)
|     preferred charset: unicode (Unicode (ISO10646))
| code point in charset: 0x05D0
|                script: hebrew
|                syntax: w 	which means: word
|              category: .:Base, R:Right-to-left (strong)
|              to input: type "C-x 8 RET 5d0" or "C-x 8 RET HEBREW LETTER ALEF"
|           buffer code: #xD7 #x90
|             file code: #xD7 #x90 (encoded by coding system utf-8-unix)
|               display: composed to form "אָ" (see below)
| 
| Composed with the following character(s) "ָ" using this font:
|   xft:-unknown-DejaVu Sans-normal-normal-normal-*-19-*-*-*-*-0-iso10646-1
| by these glyphs:
|   [0 1 64303 4765 13 2 11 10 3 nil]
| 
| Character code properties: customize what to show
|   name: HEBREW LETTER ALEF
|   general-category: Lo (Letter, Other)
|   decomposition: (1488) ('א')
| 
| There are text properties here:
|   fontified            t
`----


    -  now to aleph+qubuts where qubuts is not displayed directly under
       the aleph. Cursor on the aleph

,----
| 
|              position: 464 of 708 (65%), column: 12
|             character: א‎ (displayed as א‎) (codepoint 1488, #o2720, #x5d0)
|     preferred charset: unicode (Unicode (ISO10646))
| code point in charset: 0x05D0
|                script: hebrew
|                syntax: w 	which means: word
|              category: .:Base, R:Right-to-left (strong)
|              to input: type "C-x 8 RET 5d0" or "C-x 8 RET HEBREW LETTER ALEF"
|           buffer code: #xD7 #x90
|             file code: #xD7 #x90 (encoded by coding system utf-8-unix)
|               display: composed to form "אֻ" (see below)
| 
| Composed with the following character(s) "ֻ" using this font:
|   xft:-unknown-DejaVu Sans-normal-normal-normal-*-19-*-*-*-*-0-iso10646-1
| by these glyphs:
|   [0 1 1488 1316 13 2 11 10 0 nil]
|   [0 1 1467 1305 0 2 11 -2 5 nil]
| 
| Character code properties: customize what to show
|   name: HEBREW LETTER ALEF
|   general-category: Lo (Letter, Other)
|   decomposition: (1488) ('א')
| 
| There are text properties here:
|   fontified            t
| 
| [back]
`----

I fail to see the difference.

Re: emacs hebrew script bidi nikud rendering bugs

[Eli Zaretskii]

> From: Uwe Brauer <oub@mat.ucm.es>
> Date: Sat, 07 Nov 2015 18:32:05 +0000
> 
> 
> 
>     > The default font is not what matters.  You should find out which
>     > font(s) is/are used to display the problematic character combinations.
>     > Emacs cannot compose characters that are displayed by different fonts.
>     > Use "C-u C-x =" to show the font for each character.
> 
> Out of curiosity, are all niqqud displayed correctly in your setting?

Yes.

> I fail to see the difference.

Right, it seems to be a problem with the shaping engine.  Try
upgrading libotf, libm17n-flt, and m17n-db.  If that doesn't help,
please submit a bug report to their bug tracker.

[SOLVED] (was: emacs hebrew script bidi nikud rendering bugs)

[Uwe Brauer]

    > Yes.


    > Right, it seems to be a problem with the shaping engine.  Try
    > upgrading libotf, libm17n-flt, and m17n-db.  If that doesn't help,
    > please submit a bug report to their bug tracker.


I upgraded libotf 0.9.13, libm17n-flt, and m17n-db to version 1.6.X
and the niqqud are displayed correctly.

Thanks

Uwe 

Re: [SOLVED]

[Benjamin Riefenstahl]

Hi Uwe,

Uwe Brauer writes:
> I upgraded libotf 0.9.13, libm17n-flt, and m17n-db to version 1.6.X
> and the niqqud are displayed correctly.

Good!

Just for the record, this is still with "DejaVu Sans" and relatively
standard stuff like alef+qamats and alef+qubuts specifically, right?
Did you try adding dagesh or the yiddish ligatures, like the things that
Mark has?

benny

Re: emacs hebrew script bidi nikud rendering bugs

[Benjamin Riefenstahl]

Hi there,

Uwe Brauer writes:
>     -  First of all the ones which work
>        for example aleph+qamats: I obtain
> [...]
> |             character: א‎ (displayed as א‎) (codepoint 1488, #o2720, #x5d0)
> [...]
> |               display: composed to form "אָ" (see below)
> | 
> | Composed with the following character(s) "ָ" using this font:
> |   xft:-unknown-DejaVu Sans-normal-normal-normal-*-19-*-*-*-*-0-iso10646-1
> [...]
>     -  now to aleph+qubuts where qubuts is not displayed directly under
>        the aleph. Cursor on the aleph
> [...]
> |             character: א‎ (displayed as א‎) (codepoint 1488, #o2720, #x5d0)
> [...]
> |               display: composed to form "אֻ" (see below)
> | 
> | Composed with the following character(s) "ֻ" using this font:
> |   xft:-unknown-DejaVu Sans-normal-normal-normal-*-19-*-*-*-*-0-iso10646-1

Interesting.  On my system both of these combinations seem to work fine
with "DejaVu Sans".  I get that font from the package fonts-dejavu-core,
version 2.34-1 (on Debian GNU/Linux 8.2).

You may want to add your configuration and problems to the bug report at
<https://savannah.nongnu.org/bugs/index.php?46399>.

benny