charset problems in CVS emacs

charset problems in CVS emacs

[Dmitri Minaev]

Until recently, I used emacs-unicode2 (23.0.0), which was the first
version where cyrillic letters could be correctly pasted from X
selection. Now, lured by multi-tty, I installed the CVS version ("GNU
Emacs 23.0.60.6 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of
2008-02-13 on cera"). The first problem I faced was the incorrect
representation of cyrillic characters. A little investigation shown
that the roots of the problem are in the incorrect charset used for
rendering all (?) non-ASCII characters, including Cyrillic, Greek,
German, etc. For example:

character: й (1081, #o2071, #x439)
preferred charset: gb18030 (GB18030)
code point: 0xA7DB
syntax: w 	which means: word
category: Y:Cyrillic characters of 2-byte character sets c:Chinese
h:Korean j:Japanese y:Cyrillic
to input: type "q" with cyrillic-jcuken
buffer code: #xD0 #xB9
file code: #xD0 #xB9 (encoded by coding system utf-8-unix)
display: by this font (glyph code)
arial black:pixelsize=16:foundry=monotype:weight=medium:slant=r:width=normal
(#x217)

With some fonts this erroneous charset preference causes the non-latin
characters to be displayed in a different font, even though the
current one could represent the character in question.

In emacs-unicode2 describe-char gives the following output for the
same character:

character: й (1081, #o2071, #x439)
preferred charset: iso-8859-5 (ISO/IEC 8859/5)
code point: 0xD9
syntax: w 	which means: word
category: Y:Cyrillic characters of 2-byte character sets c:Chinese
h:Korean j:Japanese y:Cyrillic
buffer code: #xD0 #xB9
file code: #xD0 #xB9 (encoded by coding system utf-8-unix)
display: by this font (glyph code)
-ETL-fixed-medium-r-normal--16-160-72-72-C-80-ISO8859-5 (#xD9)

Both examples were taken with the current fontset
-etl-*-medium-r-normal-*-16-*-*-*-*-*-fontset-16. However, the new
Emacs uses a completely different font to render the same letter.

Is there a workaround to fix this behaviour before the developers find
the proper solution?

-- 
With best regards,
Dmitri Minaev

Russian history blog: http://minaev.blogspot.com

Re: charset problems in CVS emacs

[Peter Dyballa]

Am 13.02.2008 um 13:46 schrieb Dmitri Minaev:

> Is there a workaround to fix this behaviour before the developers find
> the proper solution?


I think there are a few possible.

	arial  
black:pixelsize=16:foundry=monotype:weight=medium:slant=r:width=normal

is a specification from libfontconfig2. You could edit /etc/fonts/ 
fonts.conf (or the equivalent on your system) to make it know all  
font resources of your system (and don't forget 'sudo fc-cache'). You  
could also try to find a TrueType or OpenType font with all four  
variants (regular, bold, italic, bold-italic) that has well-shaped  
mono-spaced Cyrillic and Latin glyphs. You can set this font as the  
default font. (fc-list :lang=ru, or such, could reveal the fonts with  
some Cyrillic support.)

You could create a fontset that specifies Latin and Cyrillic and  
Unicode encodings. Then make this fontset the default.


You could set LANG or LC_CTYPE to the proper value? Or use prefer- 
coding-system?

You could launch GNU Emacs with --disable-font-backend. It will show  
a slightly different behaviour.


--
Mit friedvollen Grüßen

   Pete

It isn't pollution that's harming the environment. It's the  
impurities in our air and water that are doing it.

Re: charset problems in CVS emacs

[Dmitri Minaev]

On Wed, Feb 13, 2008 at 6:16 PM, Peter Dyballa <Peter_Dyballa@web.de> wrote:
>  is a specification from libfontconfig2. You could edit /etc/fonts/
>  fonts.conf (or the equivalent on your system) to make it know all
>  font resources of your system (and don't forget 'sudo fc-cache').

Hmm. Until today, I didn't even know it was called 'fontconfig'. I
only cursed trying to find the way to add a fontpath to the missing
font server :) So, I will postpone this solution till I learn how
fontconfig works.

>  You
>  could also try to find a TrueType or OpenType font with all four
>  variants (regular, bold, italic, bold-italic) that has well-shaped
>  mono-spaced Cyrillic and Latin glyphs.

So I did. Unfortunately, I can't find the way to disable antialiasing,
which makes me feel like a hedgehog in the fog and strain my eyes
trying to discern the shapes through the haze of font smoothing. I
would prefer a crisp and clear X font.

>  You could create a fontset that specifies Latin and Cyrillic and
>  Unicode encodings. Then make this fontset the default.

I tried to do so, but it didn't work. Perhaps, I wasn't sufficiently diligent.

>  You could set LANG or LC_CTYPE to the proper value? Or use prefer-
>  coding-system?

If I only knew which value is proper :/... I tried setting
prefer-coding-system to utf-8, but to no avail.

>  You could launch GNU Emacs with --disable-font-backend. It will show
>  a slightly different behaviour.

Bingo! This one works. Now, I've got both Unicode support and
multi-tty without blurred fonts. Thank you!

-- 
With best regards,
Dmitri Minaev

Russian history blog: http://minaev.blogspot.com