[emacs-unicode-2] incorrectly displayed cyrillic symbols

[emacs-unicode-2] incorrectly displayed cyrillic symbols

[Philip Belemezov]

Hello!

I'm not sure if this is the right mailing list or if this is the right
way to report a problem.

I've been using emacs-unicode-2 for some time (maybe a year), updating
from CVS every week. About a month or two ago, Emacs started displaying
cyrillic symbols in a very ugly way [1], regardless of (font)
customizations [2]. Current Emacs 22 does not have this problem [3]. 

Emacs 23 was compiled using `configure --enable-font-backend
--with-freetype --with-gtk --with-jpeg --with-png --with-tiff
--with-toolkit-scroll-bars --with-xft --with-xpm --without-gif'

Is this a known issue? Is there a fix available?

Thanks!

[1] Emacs 23 using DejaVu Sans Mono with enabled font backend:
http://www.informatik.uni-bremen.de/~iblender/emacs/emacs-23.png
[2] Emacs 23 with no customizations, i.e. default:
http://www.informatik.uni-bremen.de/~iblender/emacs/emacs-23-default.png
[3] Emacs 22:
http://www.informatik.uni-bremen.de/~iblender/emacs/emacs-22.png

-- 

Best regards,
Philip Belemezov

Re: [emacs-unicode-2] incorrectly displayed cyrillic symbols

[Kenichi Handa]

Very sorry for the late response.

In article <1185031803.2604.18.camel@localhost>, Philip Belemezov <philip.belemezov@googlemail.com> writes:

> I've been using emacs-unicode-2 for some time (maybe a year), updating
> from CVS every week. About a month or two ago, Emacs started displaying
> cyrillic symbols in a very ugly way [1], regardless of (font)
> customizations [2]. Current Emacs 22 does not have this problem [3]. 

> Emacs 23 was compiled using `configure --enable-font-backend
> --with-freetype --with-gtk --with-jpeg --with-png --with-tiff
> --with-toolkit-scroll-bars --with-xft --with-xpm --without-gif'

> Is this a known issue? Is there a fix available?

> Thanks!

> [1] Emacs 23 using DejaVu Sans Mono with enabled font backend:
> http://www.informatik.uni-bremen.de/~iblender/emacs/emacs-23.png
> [2] Emacs 23 with no customizations, i.e. default:
> http://www.informatik.uni-bremen.de/~iblender/emacs/emacs-23-default.png
> [3] Emacs 22:
> http://www.informatik.uni-bremen.de/~iblender/emacs/emacs-22.png

Please move cursor on a Cyrillic character, type C-u C-x =,
and check which font is used for displaying it for the above
cases.

And, how did you specify "Dejavu Sans Mono" in case [1]?

---
Kenichi Handa
handa@m17n.org

Re: [emacs-unicode-2] incorrectly displayed cyrillic symbols

[Philip Belemezov]

Hello Kenichi!

Thank you for the response!

Emacs is started with the following command:
emacs --enable-font-backend --font "Dejavu Sans Mono-8"

You're right about this being a font issue. The output of `C-u C-x =' on
a Cyrillic letter is 
----
        character: к (1082, #o2072, #x43a)
preferred charset: iso-8859-5 (ISO/IEC 8859/5)
       code point: 0xDA
           syntax: w 	which means: word
         category: Y:Cyrillic characters of 2-byte character sets
c:Chinese h:Korean j:Japanese
				   y:Cyrillic
      buffer code: #xD0 #xBA
        file code: #xD0 #xBA (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
     -unknown-freeserif-medium-r-normal--11-79-100-100-p-64-iso8859-5
(#xDA)

Character code properties are not shown: customize what to show

There are text properties here:
  auto-composed        t
  fontified            t
----

I don't understand why Emacs is using freeserif to render the glyph.
It's definitely not caused by a customization because the problem also
occurs if I run Emacs from a different login with no .emacs file.

I just tried `C-u C-x =' on a Greek letter (capital omega):
----
        character: Ω (937, #o1651, #x3a9)
preferred charset: iso-8859-7 (ISO/IEC 8859/7)
       code point: 0xD9
           syntax: w 	which means: word
         category: G:Greek characters of 2-byte character sets c:Chinese
g:Greek h:Korean
				   j:Japanese
      buffer code: #xCE #xA9
        file code: #xCE #xA9 (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
     -dejavu-dejavu sans
mono-medium-r-normal--11-79-100-100-m-70-iso8859-7 (#xD9)

Character code properties are not shown: customize what to show

There are text properties here:
  auto-composed        t
  fontified            t
  rear-nonsticky       t
----

And the following is the output of `C-u C-x =' on a katakana letter
(GI):
----
        character: ギ (12462, #o30256, #x30ae)
preferred charset: chinese-gb2312 (GB2312 Chinese simplified: ISO-IR-58)
       code point: 0x252E
           syntax: w 	which means: word
         category: K:Japanese Katakana characters of 2-byte character
sets c:Chinese
				   h:Korean j:Japanese
				   |:While filling, we can break a line at this character.
      buffer code: #xE3 #x82 #xAE
        file code: #xE3 #x82 #xAE (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
     -ISAS-Song ti-Medium-R-Normal--24-240-72-72-c-240-GB2312.1980-0
(#x252E)

Character code properties are not shown: customize what to show

There are text properties here:
  auto-composed        t
  fontified            t
  rear-nonsticky       t
----

I started Emacs with "Dejavu Sans Mono-16" and took a screenshot of how
the glyphs are rendered [1].

Best regards,
Philip

[1]
http://www.informatik.uni-bremen.de/~iblender/emacs/emacs-23-large-font.png


On Mon, 2007-08-20 at 10:21 +0900, Kenichi Handa wrote:
> Very sorry for the late response.
> 
> In article <1185031803.2604.18.camel@localhost>, Philip Belemezov <philip.belemezov@googlemail.com> writes:
> 
> > I've been using emacs-unicode-2 for some time (maybe a year), updating
> > from CVS every week. About a month or two ago, Emacs started displaying
> > cyrillic symbols in a very ugly way [1], regardless of (font)
> > customizations [2]. Current Emacs 22 does not have this problem [3]. 
> 
> > Emacs 23 was compiled using `configure --enable-font-backend
> > --with-freetype --with-gtk --with-jpeg --with-png --with-tiff
> > --with-toolkit-scroll-bars --with-xft --with-xpm --without-gif'
> 
> > Is this a known issue? Is there a fix available?
> 
> > Thanks!
> 
> > [1] Emacs 23 using DejaVu Sans Mono with enabled font backend:
> > http://www.informatik.uni-bremen.de/~iblender/emacs/emacs-23.png
> > [2] Emacs 23 with no customizations, i.e. default:
> > http://www.informatik.uni-bremen.de/~iblender/emacs/emacs-23-default.png
> > [3] Emacs 22:
> > http://www.informatik.uni-bremen.de/~iblender/emacs/emacs-22.png
> 
> Please move cursor on a Cyrillic character, type C-u C-x =,
> and check which font is used for displaying it for the above
> cases.
> 
> And, how did you specify "Dejavu Sans Mono" in case [1]?
> 
> ---
> Kenichi Handa
> handa@m17n.org

Re: [emacs-unicode-2] incorrectly displayed cyrillic symbols

[Kenichi Handa]

In article <1187583076.5399.26.camel@localhost>, Philip Belemezov <philip.belemezov@googlemail.com> writes:

> Emacs is started with the following command:
> emacs --enable-font-backend --font "Dejavu Sans Mono-8"

> You're right about this being a font issue. The output of `C-u C-x =' on
> a Cyrillic letter is 
> ----
>         character: к (1082, #o2072, #x43a)
> preferred charset: iso-8859-5 (ISO/IEC 8859/5)
>        code point: 0xDA
>            syntax: w 	which means: word
>          category: Y:Cyrillic characters of 2-byte character sets
> c:Chinese h:Korean j:Japanese
> 				   y:Cyrillic
>       buffer code: #xD0 #xBA
>         file code: #xD0 #xBA (encoded by coding system utf-8-unix)
>           display: by this font (glyph code)
>      -unknown-freeserif-medium-r-normal--11-79-100-100-p-64-iso8859-5
> (#xDA)
[...]
> I don't understand why Emacs is using freeserif to render the glyph.
> It's definitely not caused by a customization because the problem also
> occurs if I run Emacs from a different login with no .emacs file.

Didn't Emacs start to use freeserif for Cyrillic just after
you installed that font and made it available as X font?

Please show me the result of:
% xlsfonts|grep -i iso8859-5

---
Kenichi Handa
handa@m17n.org

Re: [emacs-unicode-2] incorrectly displayed cyrillic symbols

[Philip Belemezov]

Hello Kenichi!

On Mon, 2007-08-20 at 14:03 +0900, Kenichi Handa wrote:
> Didn't Emacs start to use freeserif for Cyrillic just after
> you installed that font and made it available as X font?

I don't really know if Emacs started using that font immediately after
the installation of the freeserif font. Or if the font has always been
installed and Emacs started using it after a CVS update.
(see below about uninstalling the font)


> Please show me the result of:
> % xlsfonts|grep -i iso8859-5

Here's the list:
$ xlsfonts | grep -i iso8859-5
-misc-fixed-bold-r-normal--0-0-100-100-c-0-iso8859-5
-misc-fixed-bold-r-normal--0-0-100-100-c-0-iso8859-5
-misc-fixed-bold-r-normal--0-0-75-75-c-0-iso8859-5
-misc-fixed-bold-r-normal--0-0-75-75-c-0-iso8859-5
-misc-fixed-bold-r-normal--13-120-75-75-c-70-iso8859-5
-misc-fixed-bold-r-normal--13-120-75-75-c-70-iso8859-5
-misc-fixed-bold-r-normal--13-120-75-75-c-80-iso8859-5
-misc-fixed-bold-r-normal--13-120-75-75-c-80-iso8859-5
-misc-fixed-bold-r-normal--14-130-75-75-c-70-iso8859-5
-misc-fixed-bold-r-normal--14-130-75-75-c-70-iso8859-5
-misc-fixed-bold-r-normal--15-140-75-75-c-90-iso8859-5
-misc-fixed-bold-r-normal--15-140-75-75-c-90-iso8859-5
-misc-fixed-bold-r-normal--18-120-100-100-c-90-iso8859-5
-misc-fixed-bold-r-normal--18-120-100-100-c-90-iso8859-5
-misc-fixed-bold-r-semicondensed--0-0-75-75-c-0-iso8859-5
-misc-fixed-bold-r-semicondensed--0-0-75-75-c-0-iso8859-5
-misc-fixed-bold-r-semicondensed--13-120-75-75-c-60-iso8859-5
-misc-fixed-bold-r-semicondensed--13-120-75-75-c-60-iso8859-5
-misc-fixed-medium-o-normal--0-0-75-75-c-0-iso8859-5
-misc-fixed-medium-o-normal--0-0-75-75-c-0-iso8859-5
-misc-fixed-medium-o-normal--13-120-75-75-c-70-iso8859-5
-misc-fixed-medium-o-normal--13-120-75-75-c-70-iso8859-5
-misc-fixed-medium-o-normal--13-120-75-75-c-80-iso8859-5
-misc-fixed-medium-o-normal--13-120-75-75-c-80-iso8859-5
-misc-fixed-medium-o-semicondensed--0-0-75-75-c-0-iso8859-5
-misc-fixed-medium-o-semicondensed--0-0-75-75-c-0-iso8859-5
-misc-fixed-medium-o-semicondensed--13-120-75-75-c-60-iso8859-5
-misc-fixed-medium-o-semicondensed--13-120-75-75-c-60-iso8859-5
-misc-fixed-medium-r-normal--0-0-100-100-c-0-iso8859-5
-misc-fixed-medium-r-normal--0-0-100-100-c-0-iso8859-5
-misc-fixed-medium-r-normal--0-0-75-75-c-0-iso8859-5
-misc-fixed-medium-r-normal--0-0-75-75-c-0-iso8859-5
-misc-fixed-medium-r-normal--10-100-75-75-c-60-iso8859-5
-misc-fixed-medium-r-normal--10-100-75-75-c-60-iso8859-5
-misc-fixed-medium-r-normal--13-120-75-75-c-70-iso8859-5
-misc-fixed-medium-r-normal--13-120-75-75-c-70-iso8859-5
-misc-fixed-medium-r-normal--13-120-75-75-c-80-iso8859-5
-misc-fixed-medium-r-normal--13-120-75-75-c-80-iso8859-5
-misc-fixed-medium-r-normal--14-130-75-75-c-70-iso8859-5
-misc-fixed-medium-r-normal--14-130-75-75-c-70-iso8859-5
-misc-fixed-medium-r-normal--15-140-75-75-c-90-iso8859-5
-misc-fixed-medium-r-normal--15-140-75-75-c-90-iso8859-5
-misc-fixed-medium-r-normal--18-120-100-100-c-90-iso8859-5
-misc-fixed-medium-r-normal--18-120-100-100-c-90-iso8859-5
-misc-fixed-medium-r-normal--20-200-75-75-c-100-iso8859-5
-misc-fixed-medium-r-normal--20-200-75-75-c-100-iso8859-5
-misc-fixed-medium-r-normal--6-60-75-75-c-40-iso8859-5
-misc-fixed-medium-r-normal--6-60-75-75-c-40-iso8859-5
-misc-fixed-medium-r-normal--7-70-75-75-c-50-iso8859-5
-misc-fixed-medium-r-normal--7-70-75-75-c-50-iso8859-5
-misc-fixed-medium-r-normal--8-80-75-75-c-50-iso8859-5
-misc-fixed-medium-r-normal--8-80-75-75-c-50-iso8859-5
-misc-fixed-medium-r-normal--9-90-75-75-c-60-iso8859-5
-misc-fixed-medium-r-normal--9-90-75-75-c-60-iso8859-5
-misc-fixed-medium-r-semicondensed--0-0-75-75-c-0-iso8859-5
-misc-fixed-medium-r-semicondensed--0-0-75-75-c-0-iso8859-5
-misc-fixed-medium-r-semicondensed--12-110-75-75-c-60-iso8859-5
-misc-fixed-medium-r-semicondensed--12-110-75-75-c-60-iso8859-5
-misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso8859-5
-misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso8859-5
-schumacher-clean-medium-r-normal--0-0-75-75-c-0-iso8859-5
-schumacher-clean-medium-r-normal--0-0-75-75-c-0-iso8859-5
-schumacher-clean-medium-r-normal--12-120-75-75-c-60-iso8859-5
-schumacher-clean-medium-r-normal--12-120-75-75-c-60-iso8859-5
-unknown-freemono-bold-o-normal--0-0-0-0-c-0-iso8859-5
-unknown-freemono-bold-o-normal--0-0-0-0-c-0-iso8859-5
-unknown-freemono-bold-o-normal--0-0-0-0-m-0-iso8859-5
-unknown-freemono-bold-o-normal--0-0-0-0-m-0-iso8859-5
-unknown-freemono-bold-r-normal--0-0-0-0-c-0-iso8859-5
-unknown-freemono-bold-r-normal--0-0-0-0-c-0-iso8859-5
-unknown-freemono-bold-r-normal--0-0-0-0-m-0-iso8859-5
-unknown-freemono-bold-r-normal--0-0-0-0-m-0-iso8859-5
-unknown-freemono-medium-o-normal--0-0-0-0-c-0-iso8859-5
-unknown-freemono-medium-o-normal--0-0-0-0-c-0-iso8859-5
-unknown-freemono-medium-o-normal--0-0-0-0-m-0-iso8859-5
-unknown-freemono-medium-o-normal--0-0-0-0-m-0-iso8859-5
-unknown-freemono-medium-r-normal--0-0-0-0-c-0-iso8859-5
-unknown-freemono-medium-r-normal--0-0-0-0-c-0-iso8859-5
-unknown-freemono-medium-r-normal--0-0-0-0-m-0-iso8859-5
-unknown-freemono-medium-r-normal--0-0-0-0-m-0-iso8859-5
-unknown-freesans-bold-o-normal--0-0-0-0-p-0-iso8859-5
-unknown-freesans-bold-o-normal--0-0-0-0-p-0-iso8859-5
-unknown-freesans-bold-r-normal--0-0-0-0-p-0-iso8859-5
-unknown-freesans-bold-r-normal--0-0-0-0-p-0-iso8859-5
-unknown-freesans-medium-o-normal--0-0-0-0-p-0-iso8859-5
-unknown-freesans-medium-o-normal--0-0-0-0-p-0-iso8859-5
-unknown-freesans-medium-r-normal--0-0-0-0-p-0-iso8859-5
-unknown-freesans-medium-r-normal--0-0-0-0-p-0-iso8859-5
-unknown-freeserif-bold-i-normal--0-0-0-0-p-0-iso8859-5
-unknown-freeserif-bold-i-normal--0-0-0-0-p-0-iso8859-5
-unknown-freeserif-bold-r-normal--0-0-0-0-p-0-iso8859-5
-unknown-freeserif-bold-r-normal--0-0-0-0-p-0-iso8859-5
-unknown-freeserif-medium-i-normal--0-0-0-0-p-0-iso8859-5
-unknown-freeserif-medium-i-normal--0-0-0-0-p-0-iso8859-5
-unknown-freeserif-medium-r-normal--0-0-0-0-p-0-iso8859-5
-unknown-freeserif-medium-r-normal--0-0-0-0-p-0-iso8859-5

I just _uninstalled_ the free* fonts and typed some Cyrillic text in
Emacs - it's OK, i.e. glyphs are rendered correctly and, obviously using
DejaVu.
Here's the output of `C-u C-x =' on a Cyrillic letter:
----
        character: д (1076, #o2064, #x434)
preferred charset: iso-8859-5 (ISO/IEC 8859/5)
       code point: 0xD4
           syntax: w 	which means: word
         category: Y:Cyrillic characters of 2-byte character sets
c:Chinese h:Korean j:Japanese
				   y:Cyrillic
      buffer code: #xD0 #xB4
        file code: #xD0 #xB4 (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
     dejavu sans
mono:pixelsize=11:foundry=unknown:weight=medium:slant=r:width=normal
(#x358)

Character code properties are not shown: customize what to show

There are text properties here:
  auto-composed        t
  fontified            t
----

So it's using DejaVu now. Shouldn't it be using fixed or clean, as
they're the only iso-8859-5 fonts left?


-- 

Best regards,
Philip

Re: [emacs-unicode-2] incorrectly displayed cyrillic symbols

[Kenichi Handa]

In article <1187612430.7289.19.camel@localhost>, Philip Belemezov <philip.belemezov@googlemail.com> writes:

> > Please show me the result of:
> > % xlsfonts|grep -i iso8859-5

> Here's the list:
> $ xlsfonts | grep -i iso8859-5
> -misc-fixed-bold-r-normal--0-0-100-100-c-0-iso8859-5
[...]
> -unknown-freeserif-medium-r-normal--0-0-0-0-p-0-iso8859-5
> -unknown-freeserif-medium-r-normal--0-0-0-0-p-0-iso8859-5

Hmmm, somehow (perhaps via xfs) your freeserif truetype font
is recognized as X core fonts.  Emacs' default fontset has
an entry for Cyrillic as below:

     (cyrillic (nil . "ISO8859-5")
	       (nil . "microsoft-cp1251")
	       (nil . "koi8-r"))

So, Emacs tries fonts of those registies in turn, and finds
that freeserif best matches with your requested font size.
And the ugliness is because the font is used as an X font;
i.e. antialias is not supported.

> I just _uninstalled_ the free* fonts and typed some Cyrillic text in
> Emacs - it's OK, i.e. glyphs are rendered correctly and, obviously using
> DejaVu.
[...]
> So it's using DejaVu now. Shouldn't it be using fixed or clean, as
> they're the only iso-8859-5 fonts left?

Should be.  And, in my environment, Emacs surely uses
"Clean" even if I start it as:
% emacs --font "Dejavu Sans Mono-8"

        character: д (1076, #o2064, #x434)
preferred charset: iso-8859-5 (ISO/IEC 8859/5)
       code point: 0xD4
           syntax: w 	which means: word
         category: Y:Cyrillic characters of 2-byte character sets c:Chinese
		   h:Korean j:Japanese y:Cyrillic
      buffer code: #xD0 #xB4
        file code: #xD0 #xB4 (encoded by coding system utf-8-unix)
          display: by this font (glyph code)
     -Schumacher-Clean-Medium-R-Normal--12-120-75-75-C-60-ISO8859-5 (#xD4)

I don't know why you have the different result.  I'll try to
add some debugging facility for font selection.

---
Kenichi Handa
handa@m17n.org

PS. "--enable-font-backend" is not necessary now.

Re: [emacs-unicode-2] incorrectly displayed cyrillic symbols

[Philip Belemezov]

Hello Kenichi!

On Tue, 2007-08-21 at 19:58 +0900, Kenichi Handa wrote:
> I don't know why you have the different result.  I'll try to
> add some debugging facility for font selection.
Great!
If there's a way to debug it on my system or any other way I can help,
I'd be happy to.

> ---
> Kenichi Handa
> handa@m17n.org
> 
> PS. "--enable-font-backend" is not necessary now.
Thanks, didn't know that. I'll change my start script.

Regards,
Philip