Re: Composing Hebrew diacriticals

Re: Composing Hebrew diacriticals

[Kenichi Handa]

In article <AANLkTinkfapIXNSnij20psfpKU1ZKS-6wJsVIDbVaQ7i@mail.gmail.com>, Yair F <yair.f.lists@gmail.com> writes:

> On Fri, May 28, 2010 at 3:42 AM, Kenichi Handa <handa@m17n.org> wrote:
> > Then please find why maybe_otf and otf are set to zero by
> > stepping through the code of ftfont_get_otf which is called
> > from ftfont_shape.

> ftfont_get_otf sets otf only if maybe_otf != 0.

> maybe_otf is initialized from ft_face->face_flags in xftfont_open.
> For David CLM maybe_otf = 0 because ft_face->face_flags = 2577.
> For Dejavu Sans maybe_otf = 8 because ft_face->face_flags = 2649.

That's very strange.  Perhaps your David CLM font is
different from mine.

In freetype.h, FT_FACE_FLAG_SFNT is explained as this:

  /*    FT_FACE_FLAG_SFNT ::                                               */
  /*      Indicates that the face uses the `sfnt' storage scheme.  For     */
  /*      now, this means TrueType and OpenType.                           */

So, if the font doesn't have this flag set, it means that
the font is surely not OTF.

This is some info about my David CLM font.

% ls -l DavidCLM-Medium.ttf
-rw-r--r-- 1 handa handa 24156 2010-06-15 09:48 DavidCLM-Medium.ttf
% fc-list 'david clm' capability
:capability=otlayout\:hebr
% od -t x1 DavidCLM-Medium.ttf |head
0000000 00 01 00 00 00 10 01 00 00 04 00 00 46 46 54 4d
0000020 4f 58 4a 2a 00 00 5e 40 00 00 00 1c 47 44 45 46
0000040 08 87 07 9c 00 00 50 24 00 00 00 6e 47 50 4f 53
0000060 c3 06 cd 7e 00 00 55 34 00 00 09 0a 47 53 55 42
0000100 48 82 52 49 00 00 50 94 00 00 04 9e 4f 53 2f 32
0000120 89 5b 2c ee 00 00 01 88 00 00 00 56 63 6d 61 70
0000140 ae 86 db a7 00 00 05 3c 00 00 02 0a 63 76 74 20
0000160 00 28 02 f8 00 00 07 48 00 00 00 04 67 61 73 70
0000200 ff ff 00 03 00 00 50 1c 00 00 00 08 67 6c 79 66
0000220 62 9d 8f 85 00 00 08 fc 00 00 3c 34 68 65 61 64

---
Kenichi Handa
handa@m17n.org

PS.  I got WiFi (WiMAX) now, and the Internet access has
been much improved. :-)

Re: Composing Hebrew diacriticals

[Kenichi Handa]

Yair, could you please check your David CLM font with these
commands?

% ls -l DavidCLM-Medium.ttf
% fc-list 'david clm' capability
% od -t x1 DavidCLM-Medium.ttf |head

---
Kenichi Handa
handa@m17n.org

PS.  I left the hospital yesterday. :-)

In article <tl7eig8pnim.fsf@m17n.org>, Kenichi Handa <handa@m17n.org> writes:

> In article <AANLkTinkfapIXNSnij20psfpKU1ZKS-6wJsVIDbVaQ7i@mail.gmail.com>, Yair F <yair.f.lists@gmail.com> writes:
> > On Fri, May 28, 2010 at 3:42 AM, Kenichi Handa <handa@m17n.org> wrote:
> > > Then please find why maybe_otf and otf are set to zero by
> > > stepping through the code of ftfont_get_otf which is called
> > > from ftfont_shape.

> > ftfont_get_otf sets otf only if maybe_otf != 0.

> > maybe_otf is initialized from ft_face->face_flags in xftfont_open.
> > For David CLM maybe_otf = 0 because ft_face->face_flags = 2577.
> > For Dejavu Sans maybe_otf = 8 because ft_face->face_flags = 2649.

> That's very strange.  Perhaps your David CLM font is
> different from mine.

> In freetype.h, FT_FACE_FLAG_SFNT is explained as this:

>   /*    FT_FACE_FLAG_SFNT ::                                               */
>   /*      Indicates that the face uses the `sfnt' storage scheme.  For     */
>   /*      now, this means TrueType and OpenType.                           */

> So, if the font doesn't have this flag set, it means that
> the font is surely not OTF.

> This is some info about my David CLM font.

> % ls -l DavidCLM-Medium.ttf
> -rw-r--r-- 1 handa handa 24156 2010-06-15 09:48 DavidCLM-Medium.ttf
> % fc-list 'david clm' capability
> :capability=otlayout\:hebr
> % od -t x1 DavidCLM-Medium.ttf |head
> 0000000 00 01 00 00 00 10 01 00 00 04 00 00 46 46 54 4d
> 0000020 4f 58 4a 2a 00 00 5e 40 00 00 00 1c 47 44 45 46
> 0000040 08 87 07 9c 00 00 50 24 00 00 00 6e 47 50 4f 53
> 0000060 c3 06 cd 7e 00 00 55 34 00 00 09 0a 47 53 55 42
> 0000100 48 82 52 49 00 00 50 94 00 00 04 9e 4f 53 2f 32
> 0000120 89 5b 2c ee 00 00 01 88 00 00 00 56 63 6d 61 70
> 0000140 ae 86 db a7 00 00 05 3c 00 00 02 0a 63 76 74 20
> 0000160 00 28 02 f8 00 00 07 48 00 00 00 04 67 61 73 70
> 0000200 ff ff 00 03 00 00 50 1c 00 00 00 08 67 6c 79 66
> 0000220 62 9d 8f 85 00 00 08 fc 00 00 3c 34 68 65 61 64

> ---
> Kenichi Handa
> handa@m17n.org

> PS.  I got WiFi (WiMAX) now, and the Internet access has
> been much improved. :-)

Re: Composing Hebrew diacriticals

[Eli Zaretskii]

> From: Kenichi Handa <handa@m17n.org>
> Date: Thu, 24 Jun 2010 15:33:06 +0900
> Cc: yair.f.lists@gmail.com, emacs-devel@gnu.org
> 
> PS.  I left the hospital yesterday. :-)

Glad to hear that.

Re: Composing Hebrew diacriticals

[Yair F]

Sorry for the late response.

Apparently the Culmus fonts are type1:
/usr/share/fonts/X11/Type1/DavidCLM-Medium.pfa: PostScript Type 1 font
text (DavidCLM-Medium 0.101)

But MS fonts are ttf, and they doesn't compose either.
/usr/share/fonts/truetype/msttcorefonts/Arial.ttf: TrueType font data


On Thu, Jun 24, 2010 at 9:33 AM, Kenichi Handa <handa@m17n.org> wrote:
> Yair, could you please check your David CLM font with these
> commands?
>
> % ls -l DavidCLM-Medium.ttf
> % fc-list 'david clm' capability
> % od -t x1 DavidCLM-Medium.ttf |head
>
> ---
> Kenichi Handa
> handa@m17n.org
>
> PS.  I left the hospital yesterday. :-)

This the best news!

Re: Composing Hebrew diacriticals

[Kenichi Handa]

In article <AANLkTilenRSGCRXJNj8TtdXqUlyoBOuk-PGld8geCah1@mail.gmail.com>, Yair F <yair.f.lists@gmail.com> writes:

> Sorry for the late response.
> Apparently the Culmus fonts are type1:
> /usr/share/fonts/X11/Type1/DavidCLM-Medium.pfa: PostScript Type 1 font
> text (DavidCLM-Medium 0.101)

How did you install that font?  I donwloaded
culmus-0.104.tar.gz from this page:
   http://sourceforge.net/projects/culmus/files/culmus/0.104/
and extracted DavidCLM-Medium.ttf from that tarball, and put
it under ~/.fonts.

Please try that (and uninstall the above type1 font), and
check if Emacs can use TrueType version of that font
correctly.

> But MS fonts are ttf, and they doesn't compose either.
> /usr/share/fonts/truetype/msttcorefonts/Arial.ttf: TrueType font data

I tried that font too.  That font doesn't have OpenType
tables for hebrew script.

% fc-list arial family capability
Arial
Arial:capability=otlayout\:arab

But, the function hebrew-shape-gstring has workaround code
for such fonts, and in my environment, hebrew diacriticals
are surely composed (although the positioning is not
optimal).

---
Kenichi Handa
handa@m17n.org

Re: Composing Hebrew diacriticals

[Yair F]

[-- Attachment #1: Type: text/plain, Size: 1806 bytes --]

On Tue, Jun 29, 2010 at 11:07 AM, Kenichi Handa <handa@m17n.org> wrote:
> In article <AANLkTilenRSGCRXJNj8TtdXqUlyoBOuk-PGld8geCah1@mail.gmail.com>, Yair F <yair.f.lists@gmail.com> writes:
>
>> Sorry for the late response.
>> Apparently the Culmus fonts are type1:
>> /usr/share/fonts/X11/Type1/DavidCLM-Medium.pfa: PostScript Type 1 font
>> text (DavidCLM-Medium 0.101)
>
> How did you install that font?  I donwloaded
> culmus-0.104.tar.gz from this page:

This is from culmus package on ubuntu (and debian as well as most
distributions as well). I would assume most Hebrew speakers on X based
paltform will have these two packages installed. Most Hebrew based
remixes package it.

>   http://sourceforge.net/projects/culmus/files/culmus/0.104/
> and extracted DavidCLM-Medium.ttf from that tarball, and put
> it under ~/.fonts.
>
> Please try that (and uninstall the above type1 font), and
> check if Emacs can use TrueType version of that font
> correctly.
I Tried with Keter-YG which is IMO the best Hebrew font, and Indeed
the rendring looks OK with my sample (See attached). This font comes
from culmus-ancient. The "problem" with that fornt that it is indeed
have an ancient look.

>
>> But MS fonts are ttf, and they doesn't compose either.
>> /usr/share/fonts/truetype/msttcorefonts/Arial.ttf: TrueType font data
>
> I tried that font too.  That font doesn't have OpenType
> tables for hebrew script.
>
> % fc-list arial family capability
> Arial
> Arial:capability=otlayout\:arab
>
> But, the function hebrew-shape-gstring has workaround code
> for such fonts, and in my environment, hebrew diacriticals
> are surely composed (although the positioning is not
> optimal).

I would say that the positioning is not sufficient See attached of same file.

[-- Attachment #2: arial.png --]
[-- Type: image/png, Size: 23621 bytes --]

[-- Attachment #3: keter.png --]
[-- Type: image/png, Size: 29097 bytes --]

Re: Composing Hebrew diacriticals

[Kenichi Handa]

In article <AANLkTintXoyqvqO5Mqqbyci-AKuBqMYRyp7TBnVUKT-Z@mail.gmail.com>, Yair F <yair.f.lists@gmail.com> writes:

>>> But MS fonts are ttf, and they doesn't compose either.
>>> /usr/share/fonts/truetype/msttcorefonts/Arial.ttf: TrueType font data
> >
> > I tried that font too. =A0That font doesn't have OpenType
> > tables for hebrew script.
> >
> > % fc-list arial family capability
> > Arial
> > Arial:capability=3Dotlayout\:arab
> >
> > But, the function hebrew-shape-gstring has workaround code
> > for such fonts, and in my environment, hebrew diacriticals
> > are surely composed (although the positioning is not
> > optimal).

> I would say that the positioning is not sufficient See attached of same fil=
> e.

Comparing images of different font of unfamiliar (for me)
script is very difficult.  Please tell me exactly what
character sequence requires more than positioning, and show
me images of only that sequence.

Anyway, for fonts that don't have OpenType tables for Hebrew
script, we can do nothing other than artificially adjusting
glyph position.  Have you seen any other application
rendering Hebrew well with that Arial font?

---
Kenichi Handa
handa@m17n.org

Fwd: Composing Hebrew diacriticals

[Yair F]

[-- Attachment #1: Type: text/plain, Size: 2226 bytes --]

Here is a shorter version for the list


---------- Forwarded message ----------
From: Yair F <yair.f.lists@gmail.com>
Date: Thu, Jul 1, 2010 at 12:28 AM
Subject: Re: Composing Hebrew diacriticals
To: Kenichi Handa <handa@m17n.org>
Cc: emacs-devel@gnu.org


I apologize for the size of this message.

> Comparing images of different font of unfamiliar (for me)
> script is very difficult.  Please tell me exactly what
> character sequence requires more than positioning, and show
> me images of only that sequence.

Sorry about that Please find hebrew-sample2.txt the source file.
Arial-anottated.png is this file displayed using emacs with Arial font.
The numbers in red refer to the following comments the general flow is
top-bottom right-left:
1. Shin-Dot should be rendered near the right leg. currently it is
rendered above the centre leg, this is unreradable.
2. All points below should be horizontally centred relative to the
base letter. Currently it seems that they are align to the left.
Exception for this rule is letters that have a single leg downward
such as ו, ר, ד, ז the points should be rendered directly under the
leg for these letters.
3. The Shva point touches Qof's leg. the result is unreadable.
4. The Dagesh point is hidden within the Shin letter.
5. This is not Hebrew, but the combining dot above should be composed
with the letter A.
6. The Holam point should be left to the leg, and not right. Result is
unreadable.
7. Shuruq point should be left to the vav letter, and not right.
Result is unreadable.

For reference on correct rendering I also attach The same file using Keter YG.

>
> Anyway, for fonts that don't have OpenType tables for Hebrew
> script, we can do nothing other than artificially adjusting
> glyph position.  Have you seen any other application
> rendering Hebrew well with that Arial font?
Openoffice and Firefox correctly render Hebrew points. The poetry site
you mentioned http://www.zemer.co.il/song.asp?id=393 uses David and
being correctly rendered.
Kate (using pango?) also better render using Arial, David-CLM. It has
some other issues though, but the result is mostly readable.
See attached sample under Kate.

[-- Attachment #2: hebrew-sample2.txt --]
[-- Type: text/plain, Size: 339 bytes --]

שָׁלוֹם לְמִשְׁתַּמְּשֵׁי אִמַאקְס

A "אֲעוֹלֵל 123 כַּגֶּפֶן" B.

עַשֶּׁשֶׁת 

Ȧ

לֹא שַׁרְתִּי לָךְ אַרְצִי,
וְלֹא פֵּאַרְתִּי שְׁמֵךְ

רַק קוֹל תְּרוּעַת הַגִּיל
בְּיוֹם יִגַּהּ הָאוֹר


[-- Attachment #3: Arial-anottated.png --]
[-- Type: image/png, Size: 50244 bytes --]

Re: Composing Hebrew diacriticals

[Kenichi Handa]

[-- Attachment #1: Type: text/plain, Size: 9138 bytes --]

In article <AANLkTim3sQzyJ4YQkOzfRHCFhztgLG-CA2vlM84lbwoq@mail.gmail.com>, Yair F <yair.f.lists@gmail.com> writes:

> Sorry about that Please find hebrew-sample2.txt the source file.
> Arial-anottated.png is this file displayed using emacs with Arial font.
> The numbers in red refer to the following comments the general flow is
> top-bottom right-left:
> 1. Shin-Dot should be rendered near the right leg. currently it is
> rendered above the centre leg, this is unreradable.
> 2. All points below should be horizontally centred relative to the
> base letter. Currently it seems that they are align to the left.
> Exception for this rule is letters that have a single leg downward
> such as =D7=95, =D7=A8, =D7=93, =D7=96 the points should be rendered direct=
> ly under the
> leg for these letters.
> 3. The Shva point touches Qof's leg. the result is unreadable.
> 4. The Dagesh point is hidden within the Shin letter.
> 5. This is not Hebrew, but the combining dot above should be composed
> with the letter A.
> 6. The Holam point should be left to the leg, and not right. Result is
> unreadable.
> 7. Shuruq point should be left to the vav letter, and not right.
> Result is unreadable.

All those are glyph positioning problems and can be improved
by adding more code to hebrew-shape-gstring.

> > Anyway, for fonts that don't have OpenType tables for Hebrew
> > script, we can do nothing other than artificially adjusting
> > glyph position. =C2=A0Have you seen any other application
> > rendering Hebrew well with that Arial font?
> Openoffice and Firefox correctly render Hebrew points.

??? When I open your hebrew-sample2.txt with oowriter, and
specify Arial font, the rendering is almost (exactly?) the
same as that of Emacs (see the attached image).

I confirmed that Firefox (and all applications using
Pango/harfbuzz; e.g. gedit) surely do better hebrew
rendering with Arial.  By reading the code of Pango, I found
that it has a fallback shaping engine that is used for a
font of no hebrew GPOS OpenType tables.  Here's the excerpt
from pango/module/hebrew-shaper.c.  You'll see that it
checks various character combinations and adjust glyph
offsets properly.  But the code has many magic numbers
(e.g. 3.5, 0.7, 0.5, 1/3, 3/5, ...).  I think it's a dirty &
ad-hoc hack.

Theoretically, it is possible to do the same thing in the
function hebrew-shape-gstring.  But, is it really worth
doing that?  Isn't it enough to tell Hebrew users to use
properly desinged OpenType fonts?

============================================================
void
hebrew_shaper_get_cluster_kerning(gunichar            *cluster,
				  gint                cluster_length,
				  PangoRectangle      ink_rect[],

				  /* input and output */
				  gint                width[],
				  gint                x_offset[],
				  gint                y_offset[])
{
  int i;
  int base_ink_x_offset, base_ink_y_offset, base_ink_width, base_ink_height;
  gunichar base_char = cluster[0];

  x_offset[0] = 0;
  y_offset[0] = 0;

  if (cluster_length == 1)
    {
      /* Make lone 'vav dot' have zero width */
      if (base_char == UNI_SHIN_DOT
	  || base_char == UNI_SIN_DOT
	  || base_char == UNI_HOLAM
	  ) {
	x_offset[0] = -ink_rect[0].x - ink_rect[0].width;
	width[0] = 0;
      }

      return;
    }

  base_ink_x_offset = ink_rect[0].x;
  base_ink_y_offset = ink_rect[0].y;
  base_ink_width = ink_rect[0].width;
  base_ink_height = ink_rect[0].height;

  /* Do heuristics */
  for (i=1; i<cluster_length; i++)
    {
      int gl = cluster[i];
      x_offset[i] = 0;
      y_offset[i] = 0;

      /* Check if it is a point */
      if (gl < 0x5B0 || gl >= 0x05D0)
	continue;

      /* Center dot of VAV */
      if (gl == UNI_MAPIQ && base_char == UNI_VAV)
	{
	  x_offset[i] = base_ink_x_offset - ink_rect[i].x;

	  /* If VAV is a vertical bar without a roof, then we
	     need to make room for the dot by increasing the
	     cluster width. But how can I check if that is the
	     case??
	  */
	  /* This is wild, but it does the job of differentiating
	     between two M$ fonts... Base the decision on the
	     aspect ratio of the vav...
	  */
	  if (base_ink_height > base_ink_width * 3.5)
	    {
	      int j;
	      double space = 0.7;
	      double kern = 0.5;

	      /* Shift all characters to make place for the mapiq */
	      for (j=0; j<i; j++)
		  x_offset[j] += ink_rect[i].width*(1+space-kern);

	      width[cluster_length-1] += ink_rect[i].width*(1+space-kern);
	      x_offset[i] -= ink_rect[i].width*(kern);
	    }
	}

      /* Dot over SHIN */
      else if (gl == UNI_SHIN_DOT && base_char == UNI_SHIN)
	{
	  x_offset[i] = base_ink_x_offset + base_ink_width
	    - ink_rect[i].x - ink_rect[i].width;
	}

      /* Dot over SIN */
      else if (gl == UNI_SIN_DOT && base_char == UNI_SHIN)
	{
	  x_offset[i] = base_ink_x_offset - ink_rect[i].x;
	}

      /* VOWEL DOT above to any other character than
	 SHIN or VAV should stick out a bit to the left. */
      else if ((gl == UNI_SIN_DOT || gl == UNI_HOLAM)
	       && base_char != UNI_SHIN && base_char != UNI_VAV)
	{
	  x_offset[i] = base_ink_x_offset -ink_rect[i].x - ink_rect[i].width * 3/ 2;
	}

      /* VOWELS under resh or vav are right aligned, if they are
	 narrower than the characters. Otherwise they are centered.
       */
      else if ((base_char == UNI_VAV
		|| base_char == UNI_RESH
		|| base_char == UNI_YOD
		|| base_char == UNI_DALED
		)
	       && ((gl >= UNI_SHEVA && gl <= UNI_QAMATS) ||
		   gl == UNI_QUBUTS)
	       && ink_rect[i].width < base_ink_width
	       )
	{
	  x_offset[i] = base_ink_x_offset + base_ink_width
	    - ink_rect[i].x - ink_rect[i].width;
	}

      /* VOWELS under FINAL KAF are offset centered and offset in
	 y */
      else if ((base_char == UNI_FINAL_KAF
		)
	       && ((gl >= UNI_SHEVA && gl <= UNI_QAMATS) ||
		   gl == UNI_QUBUTS))
	{
	  /* x are at 1/3 to take into accoun the stem */
	  x_offset[i] = base_ink_x_offset - ink_rect[i].x
	    + base_ink_width * 1/3 - ink_rect[i].width/2;

	  /* Center in y */
	  y_offset[i] = base_ink_y_offset - ink_rect[i].y
	    + base_ink_height * 1/2 - ink_rect[i].height/2;
	}


      /* MAPIQ in PE or FINAL PE */
      else if (gl == UNI_MAPIQ
	       && (base_char == UNI_PE || base_char == UNI_FINAL_PE))
	{
	  x_offset[i]= base_ink_x_offset - ink_rect[i].x
	    + base_ink_width * 2/3 - ink_rect[i].width/2;

	  /* Another option is to offset the MAPIQ in y...
	     glyphs->glyphs[cluster_start_idx+i].geometry.y_offset
	     -= base_ink_height/5; */
	}

      /* MAPIQ in SHIN should be moved a bit to the right */
      else if (gl == UNI_MAPIQ
	       && base_char == UNI_SHIN)
	{
	  x_offset[i]=  base_ink_x_offset - ink_rect[i].x
	    + base_ink_width * 3/5 - ink_rect[i].width/2;
	}

      /* MAPIQ in YUD is right aligned */
      else if (gl == UNI_MAPIQ
	       && base_char == UNI_YOD)
	{
	  x_offset[i]=  base_ink_x_offset - ink_rect[i].x;

	  /* Lower left in y */
	  y_offset[i] = base_ink_y_offset - ink_rect[i].y
	    + base_ink_height - ink_rect[i].height*1.75;

	  if (base_ink_height > base_ink_width * 2)
	    {
	      int j;
	      double space = 0.7;
	      double kern = 0.5;

	      /* Shift all cluster characters to make space for mapiq */
	      for (j=0; j<i; j++)
		x_offset[j] += ink_rect[i].width*(1+space-kern);

	      width[cluster_length-1] += ink_rect[i].width*(1+space-kern);
	    }

	}

      /* VOWEL DOT next to any other character */
      else if ((gl == UNI_SIN_DOT || gl == UNI_HOLAM)
	       && (base_char != UNI_VAV))
	{
	  x_offset[i] = base_ink_x_offset -ink_rect[i].x;
	}

      /* Move nikud of taf a bit ... */
      else if (base_char == UNI_TAV && gl == UNI_MAPIQ)
	{
	  x_offset[i] = base_ink_x_offset - ink_rect[i].x
	    + base_ink_width * 5/8 - ink_rect[i].width/2;
	}

      /* Move center dot of characters with a right stem and no
	 left stem. */
      else if (gl == UNI_MAPIQ &&
	       (base_char == UNI_BET
		|| base_char == UNI_DALED
		|| base_char == UNI_KAF
		|| base_char == UNI_GIMMEL
		))
	{
	  x_offset[i] = base_ink_x_offset - ink_rect[i].x
	    + base_ink_width * 3/8 - ink_rect[i].width/2;
	}

      /* Right align wide nikud under QOF */
      else if (base_char == UNI_QOF &&
	       ( (gl >= UNI_HATAF_SEGOL
		  && gl <= UNI_HATAF_QAMATZ)
		 || (gl >= UNI_TSERE
		     && gl<= UNI_QAMATS)
		 || (gl == UNI_QUBUTS)))
	{
	  x_offset[i] = base_ink_x_offset + base_ink_width
	    - ink_rect[i].x - ink_rect[i].width;
	}

      /* Center by default */
      else
	{
	  x_offset[i] = base_ink_x_offset - ink_rect[i].x
	    + base_ink_width/2 - ink_rect[i].width/2;
	}
    }

}
============================================================

> The poetry site
> you mentioned http://www.zemer.co.il/song.asp?id=3D393 uses David and
> being correctly rendered.
> Kate (using pango?) also better render using Arial, David-CLM. It has
> some other issues though, but the result is mostly readable.

As Kate is a KDE application, I think it's not using Pango.
But, if it renders Hebrew with Arial well, it (or rendering
module of KDE/Qt) should have the similar ad-hoc code.

---
Kenichi Handa
handa@m17n.org


[-- Attachment #2: oowriter-arial.png --]
[-- Type: image/png, Size: 79797 bytes --]

Re: Composing Hebrew diacriticals

[Miles Bader]

Just FYI, the Emacs rendering of your sample text looks correct in my
Gnus buffer, using the Truetype version of "Lucida Sans".

[by "correct" I mean, (1) it handles all the points you describe
correctly, and (2) everything looks "nice".]

-Miles

-- 
Suburbia: where they tear out the trees and then name streets after them.

Re: Composing Hebrew diacriticals

[Yair F]

On Thu, Jul 1, 2010 at 8:52 AM, Kenichi Handa <handa@m17n.org> wrote:
> All those are glyph positioning problems and can be improved
>by adding more code to hebrew-shape-gstring.

What else problem do you expect? So far I see no other problems
regarding bidi or compositions.


> ??? When I open your hebrew-sample2.txt with oowriter, and
> specify Arial font, the rendering is almost (exactly?) the
> same as that of Emacs (see the attached image).
 urrent oo p
You are right. Maybe it was with a special Hebrew oo version I don't
have it now, or maybe on other OS. current oo practice is "use proper
fonts" :(


>I think it's a dirty &
> ad-hoc hack.
>
> Theoretically, it is possible to do the same thing in the
> function hebrew-shape-gstring.  But, is it really worth
> doing that?  Isn't it enough to tell Hebrew users to use
> properly desinged OpenType fonts?

The sad answer on free systems is that there are nealy no such fonts.
The common answer for "Why is Hebrew so ugly on Linux?" is "Install
Culmus and msttcorefonts".
I guess that is the reason for the twaks you mentioned.

>
> As Kate is a KDE application, I think it's not using Pango.
> But, if it renders Hebrew with Arial well, it (or rendering
> module of KDE/Qt) should have the similar ad-hoc code.

Maybe, as you can see I don't know much about rending engines.

An additional and possibly less ugly path is to use presentation forms
when available.(UFB20) There are additional forms in the private use
area.

Re: Composing Hebrew diacriticals

[Kenichi Handa]

In article <AANLkTil7SZlvtMBLmfz3DG_wHKKki72LwSIITx53w0tf@mail.gmail.com>, Yair F <yair.f.lists@gmail.com> writes:

> On Thu, Jul 1, 2010 at 8:52 AM, Kenichi Handa <handa@m17n.org> wrote:
> > All those are glyph positioning problems and can be improved
> >by adding more code to hebrew-shape-gstring.

> What else problem do you expect?

Sorry, I just misread what you wrote "I would say that the
positioning is not sufficient" as "there should be more work
other than positioning".

> >I think it's a dirty &
> > ad-hoc hack.
> >
> > Theoretically, it is possible to do the same thing in the
> > function hebrew-shape-gstring.  But, is it really worth
> > doing that?  Isn't it enough to tell Hebrew users to use
> > properly desinged OpenType fonts?

> The sad answer on free systems is that there are nealy no such fonts.
> The common answer for "Why is Hebrew so ugly on Linux?" is "Install
> Culmus and msttcorefonts".
> I guess that is the reason for the twaks you mentioned.

Sign...

> An additional and possibly less ugly path is to use presentation forms
> when available.(UFB20) There are additional forms in the private use
> area.

Hmmm, that seems to be a practical approach provided that
the presentation forms covers most of frequently used
character combinations.  I'll try to implement it.

---
Kenichi Handa
handa@m17n.org

Re: Composing Hebrew diacriticals

[Kenichi Handa]

In article <tl7hbki48zt.fsf@m17n.org>, Kenichi Handa <handa@m17n.org> writes:

> > An additional and possibly less ugly path is to use presentation forms
> > when available.(UFB20) There are additional forms in the private use
> > area.

> Hmmm, that seems to be a practical approach provided that
> the presentation forms covers most of frequently used
> character combinations.  I'll try to implement it.

I've just comitted the code to do that.  I tested with the
Arial font and it seems that the most of points you listed
are solved now except for this:

5. This is not Hebrew, but the combining dot above should be composed
with the letter A.

It seems that Arial font doesn't have a glyph of #x307.

When you set both the default font and the font for #x307 to
"dejavu sans mono", #x307 is composed with the preceding
"A".

---
Kenichi Handa
handa@m17n.org

Re: Composing Hebrew diacriticals

[Yair F]

On Mon, Jul 12, 2010 at 11:17 AM, Kenichi Handa <handa@m17n.org> wrote:

> I've just comitted the code to do that.  I tested with the
> Arial font and it seems that the most of points you listed
> are solved now except for this:

Now it's much much better, Thank you!
Here are some more improvements needed:
The placement of Holam (05B9) point seems to be top-center. It should
be top-left instead.
Specifically for Lamed (0CDC) base letter it should be to the left of
the top vertical leg.
Some fonts have presentation-form for that at  E804.

Sheva (05B0) and Qamats (05B8) points should be shifted above baseline
to approximatly
center-center position when composed with Final Kaf (05DA).
Again some fonts pre-compose it at E802 and E803 respectively.

Currently I'm trying to hunt-down a problem when sometimes when
transient-mode is
active some characters suddenly stop composing. Once I get a recepie,
I'll let you know.

Thanks Again to you end Eli,
Yair

Re: Composing Hebrew diacriticals

[Kenichi Handa]

In article <AANLkTimYSLAA4UTDjZ2MF5NhqOxtX_m7_oqQ6TanAMZl@mail.gmail.com>, Yair F <yair.f.lists@gmail.com> writes:

> Now it's much much better, Thank you!
> Here are some more improvements needed:
> The placement of Holam (05B9) point seems to be top-center. It should
> be top-left instead.
> Specifically for Lamed (0CDC) base letter it should be to the left of
> the top vertical leg.
> Some fonts have presentation-form for that at  E804.

But, E804 is in a Private Use Area, and there's no way to
check if the glyph there (if any) is a Hebrew glyph or not.

Or, are there any consensus among Hebrew font designers?

> Currently I'm trying to hunt-down a problem when sometimes when
> transient-mode is
> active some characters suddenly stop composing. Once I get a recepie,
> I'll let you know.

I see.

---
Kenichi Handa
handa@m17n.org

Re: Composing Hebrew diacriticals

[Yair F]

On Tue, Jul 13, 2010 at 7:11 AM, Kenichi Handa <handa@m17n.org> wrote:
> But, E804 is in a Private Use Area, and there's no way to
> check if the glyph there (if any) is a Hebrew glyph or not.
By glyph name?

>
> Or, are there any consensus among Hebrew font designers?
It is available on some font, some of them those who don't give enough
information to do proper rendring.

Re: Composing Hebrew diacriticals

[Eli Zaretskii]

> Date: Tue, 13 Jul 2010 00:10:04 +0300
> From: Yair F <yair.f.lists@gmail.com>
> Cc: emacs-devel@gnu.org
> 
> Currently I'm trying to hunt-down a problem when sometimes when
> transient-mode is active some characters suddenly stop composing.

Is this in a buffer that's bidi-reordered for display?  If so, does
the problem go away if you turn off bidi-display-reordering?

When bidi reordering is in effect, both face resolution and character
composition need to examine buffer text backwards, because text
properties and character compositions are still defined in logical
order.  It's possible that face resolution somehow interferes with
character composition in that case.

Let me know if I can help.