Strange font used for Cyrillic characters

Strange font used for Cyrillic characters

[cmr.pent]

Hello!

I've just updated my OS and I see that Emacs uses very strange font for Cyrillic characters. For example, in C-h h window most words are rendered via system default Mono font, but Cyrillic characters are rendered via non-monotone serif font.

It's reproducible with emacs -Q.
In which direction should I dig?

GNU Emacs 24.2.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.10)
 of 2012-09-09 on trouble, modified by Debian

Best wishes,
Andrey Paramonov

Re: Strange font used for Cyrillic characters

[Peter Dyballa]

Am 19.12.2012 um 22:52 schrieb cmr.pent@gmail.com:

> In which direction should I dig?

Get yourself a monospaced Cyrillic font! If you already have one: does your fonts service list it (with fc-list, fc-match)? If it does, then it might be a problem in the setup of libfontconfig, i.e. the contents of ~/.fonts.conf and /etc/fonts/fonts.conf and the other configuration files in that area (/etc/fonts/conf.avail, /etc/fonts/conf.d).

OTOH it's also possible that GNU Emacs font family names from the fonts service which this one cannot handle correctly because its setup says to substitute the requested monospaced fonts with proportional ones.

You can set FC_DEBUG to some value (see the fontconfig-user documents in PDF or HTML or plain text form) and then launch GNU Emacs from this environment. The debug output might explain what's happening.

Another option is to set the X resource Emacs.FontBackend to the value x. This will, hopefully, disable the use of libfontconfig and allow GNU Emacs to use only fonts known by the X server.

--
Greetings

  Pete

If you're not confused, you're not paying attention.

Re: Strange font used for Cyrillic characters

[Андрей Парамонов]

[-- Attachment #1: Type: text/plain, Size: 1612 bytes --]

Thank you for your response.

I'm pretty sure I do have Cyrillic font. Moreover, the monospace font used
to display Latin characters in Emacs does display Cyrillic characters in
other programs.
FC_DEBUG works but output a little bit too much of info. It would help if
Emacs could tell in which font particular symbol is rendered. What is the
command for it?

Best wishes,
Andrey Paramonov



2012/12/20 Peter Dyballa <Peter_Dyballa@web.de>

>
> Am 19.12.2012 um 22:52 schrieb cmr.pent@gmail.com:
>
> > In which direction should I dig?
>
> Get yourself a monospaced Cyrillic font! If you already have one: does
> your fonts service list it (with fc-list, fc-match)? If it does, then it
> might be a problem in the setup of libfontconfig, i.e. the contents of
> ~/.fonts.conf and /etc/fonts/fonts.conf and the other configuration files
> in that area (/etc/fonts/conf.avail, /etc/fonts/conf.d).
>
> OTOH it's also possible that GNU Emacs font family names from the fonts
> service which this one cannot handle correctly because its setup says to
> substitute the requested monospaced fonts with proportional ones.
>
> You can set FC_DEBUG to some value (see the fontconfig-user documents in
> PDF or HTML or plain text form) and then launch GNU Emacs from this
> environment. The debug output might explain what's happening.
>
> Another option is to set the X resource Emacs.FontBackend to the value x.
> This will, hopefully, disable the use of libfontconfig and allow GNU Emacs
> to use only fonts known by the X server.
>
> --
> Greetings
>
>   Pete
>
> If you're not confused, you're not paying attention.
>
>

[-- Attachment #2: Type: text/html, Size: 2074 bytes --]

Re: Strange font used for Cyrillic characters

[Jason Rumney]

On Thursday, 20 December 2012 13:28:26 UTC+8, Андрей Парамонов  wrote:

> FC_DEBUG works but output a little bit too much of info. It would help if
> Emacs could tell in which font particular symbol is rendered. What is the
> command for it?

C-u C-x =

Re: Strange font used for Cyrillic characters

[cmr.pent]

Hmm, I've tried FC_DEBUG and C-u C-x =, but still no luck. Emacs seems to send different queries to fontconfig for different symbols. Could someone please point me to some documentation for how exactly this works?

What bothers me is that my Emacs is on fresh Debian system. On my previous Debian system Monospace font was correctly used both for Latin and Cyrillic characters. But I fail to spot the difference :-(
Could you please someone take a look at the following debug info?

Below is the output for C-u C-x =

             position: 764 of 3233 (24%), column: 54
            character: З (displayed as З) (codepoint 1047, #o2027, #x417)
    preferred charset: mule-unicode-0100-24ff (Unicode characters of the range U+0100..U+24FF.)
code point in charset: 0x2837
               syntax: w 	which means: word
             category: .:Base, L:Left-to-right (strong), Y:2-byte Cyrillic, c:Chinese, h:Korean, j:Japanese, y:Cyrillic
          buffer code: #xD0 #x97
            file code: ESC #x24 #x2C #x31 #x28 #x37 (encoded by coding system iso-2022-7bit-unix)
              display: by this font (glyph code)
    xft:-urw-Nimbus Roman No9 L-normal-normal-normal-*-15-*-*-*-*-0-iso10646-1 (#x144)

Character code properties: customize what to show
  name: CYRILLIC CAPITAL LETTER ZE
  general-category: Lu (Letter, Uppercase)
  decomposition: (1047) ('З')

There are text properties here:
  charset              mule-unicode-0100-24ff

Here is the relevant piece of fontconfig log, I think

Match Pattern has 28 elts (size 32)
	family: "Nimbus Roman No9 L"(s) "Times"(w) "Nimbus Roman No9 L"(w) "Times New Roman"(w) "Liberation Serif"(w) "Thorndale"(w) "Thorndale AMT"(w) "Times"(w) "Bitstream Vera Serif"(w) "DejaVu Serif"(w) "Times New Roman"(w) "Thorndale AMT"(w) "Luxi Serif"(w) "Nimbus Roman No9 L"(w) "Times"(w) "Artsounk"(w) "BPG UTF8 M"(w) "Kinnari"(w) "Norasi"(w) "Frank Ruehl"(w) "Dror"(w) "JG LaoTimes"(w) "Saysettha Unicode"(w) "Pigiarniq"(w) "B Davat"(w) "B Compset"(w) "Kacst-Qr"(w) "Urdu Nastaliq Unicode"(w) "Raghindi"(w) "Mukti Narrow"(w) "malayalam"(w) "Sampige"(w) "padmaa"(w) "Hapax Berbère"(w) "MS Mincho"(w) "SimSun"(w) "PMingLiu"(w) "WenQuanYi Zen Hei"(w) "WenQuanYi Bitmap Song"(w) "AR PL ShanHeiSun Uni"(w) "AR PL New Sung"(w) "ZYSong18030"(w) "HanyiSong"(w) "MgOpen Canonica"(w) "Sazanami Mincho"(w) "IPAMonaMincho"(w) "IPAMincho"(w) "Kochi Mincho"(w) "AR PL SungtiL GB"(w) "AR PL Mingti2L Big5"(w) "AR PL Zenkai Uni"(w) "ＭＳ 明朝"(w) "ZYSong18030"(w) "NanumMyeongjo"(w) "UnBatang"(w) "Baekmuk Batang"(w) "KacstQura"(w) "Frank Ruehl CLM"(w) "Lohit Bengali"(w) "Lohit Gujarati"(w) "Lohit Hindi"(w) "Lohit Marathi"(w) "Lohit Maithili"(w) "Lohit Kashmiri"(w) "Lohit Konkani"(w) "Lohit Nepali"(w) "Lohit Sindhi"(w) "Lohit Punjabi"(w) "Lohit Tamil"(w) "Meera"(w) "Lohit Malayalam"(w) "Lohit Kannada"(w) "Lohit Telugu"(w) "Lohit Oriya"(w) "LKLUG"(w) "FreeSerif"(w) "Code2000"(w) "Code2001"(w) "serif"(w) "Nazli"(w) "Lotoos"(w) "Mitra"(w) "Ferdosi"(w) "Badr"(w) "Zar"(w) "serif"(w) "serif"(w) "serif"(w) "serif"(w) "serif"(w)
	slant: 0(i)(s)
	weight: 100(i)(s)
	width: 100(i)(s)
	pixelsize: 15(f)(s)
	foundry: "urw"(s)
	antialias: FcTrue(s)
	hintstyle: 3(i)(s)
	hinting: FcTrue(s)
	verticallayout: FcFalse(s)
	autohint: FcFalse(s)
	globaladvance: FcTrue(s)
	file: "/usr/share/fonts/X11/Type1/n021003l.pfb"(s)
	index: 0(i)(s)
	scalable: FcTrue(s)
	dpi: 96.0946(f)(s)
	rgba: 1(i)(s)
	scale: 1(f)(s)
	minspace: FcFalse(s)
	charset: 
	0000: 00000000 ffffffff ffffffff 7fffffff 00000000 ffffffff ffffffff ffffffff
	0001: ffffffff ffffffff ffffffff ffffffff 00040000 00000000 00000000 00000000
	0002: 0f000000 00000000 00000000 00000000 00000000 00000000 3f0002c0 00000000
	0003: 00000000 00000000 00000000 00000000 00100000 10000000 00000000 00000000
	0004: ffffffff ffffffff ffffffff 00000000 fffff000 ffffffff ffff199f 033fffff
	0020: 77180000 06010047 00000010 00000000 00000000 00001000 00000000 00000000
	0021: 00400000 00000004 00000000 00000000 00000000 00000000 00000000 00000000
	0022: 46260044 00000000 00000000 00000031 00000000 00000000 00000000 00000000
	0025: 00000000 00000000 00000000 00000000 00000000 00000000 00000400 00000000
	00f6: 00000000 00000000 00000000 00000000 00000000 00000000 000001f8 00000000
	00fb: 00000006 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(s)
	lang: "ru-RU"(s)
	fontversion: 2147483647(i)(s)
	embolden: FcFalse(s)
	embeddedbitmap: FcTrue(s)
	decorative: FcFalse(s)
	lcdfilter: 1(i)(s)
	render: FcTrue(s)
	maxglyphmemory: 1048576(i)(s)

Best score 1 1 1 1001 3 0 0 0 1 20001 1 1 0 0 0 2.14748e+12
Pattern has 15 elts (size 15)
	family: "Nimbus Roman No9 L"(w)
	style: "Regular"(w)
	slant: 0(i)(w)
	weight: 80(i)(w)
	width: 100(i)(w)
	foundry: "urw"(w)
	file: "/usr/share/fonts/X11/Type1/n021003l.pfb"(w)
	index: 0(i)(w)
	outline: FcTrue(w)
	scalable: FcTrue(w)
	charset: 
	0000: 00000000 ffffffff ffffffff 7fffffff 00000000 ffffffff ffffffff ffffffff
	0001: ffffffff ffffffff ffffffff ffffffff 00040000 00000000 00000000 00000000
	0002: 0f000000 00000000 00000000 00000000 00000000 00000000 3f0002c0 00000000
	0003: 00000000 00000000 00000000 00000000 00100000 10000000 00000000 00000000
	0004: ffffffff ffffffff ffffffff 00000000 fffff000 ffffffff ffff199f 033fffff
	0020: 77180000 06010047 00000010 00000000 00000000 00001000 00000000 00000000
	0021: 00400000 00000004 00000000 00000000 00000000 00000000 00000000 00000000
	0022: 46260044 00000000 00000000 00000031 00000000 00000000 00000000 00000000
	0025: 00000000 00000000 00000000 00000000 00000000 00000000 00000400 00000000
	00f6: 00000000 00000000 00000000 00000000 00000000 00000000 000001f8 00000000
	00fb: 00000006 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(w)
	lang: aa|ab|af|av|ay|ba|be|bg|bi|br|bs|bua|ca|ce|ch|chm|co|cs|cv|da|de|en|eo|es|et|eu|fi|fj|fo|fr|fur|fy|gd|gl|gv|ho|hr|hu|ia|id|ie|ik|io|is|it|kaa|ki|kk|kl|kum|kv|ky|la|lb|lez|lt|lv|mg|mh|mk|mo|mt|nb|nds|nl|nn|no|nr|nso|ny|oc|om|os|pl|pt|rm|ro|ru|sah|se|sel|sh|sk|sl|sma|smj|smn|so|sq|sr|ss|st|sv|sw|tg|tk|tl|tn|tr|ts|tt|tyv|uk|uz|vo|vot|wa|wen|wo|xh|yap|zu|an|crh|csb|fil|hsb|ht|jv|kj|ku-tr|kwm|lg|li|mn-mn|ms|na|ng|pap-an|pap-aw|rn|rw|sc|sg|sn|su|za(w)
	fontversion: 0(i)(w)
	fontformat: "Type 1"(w)
	decorative: FcFalse(w)

Re: Strange font used for Cyrillic characters

[Oleksandr Gavenko]

On 2012-12-19, cmr.pent@gmail.com wrote:

> Hello!
>
> I've just updated my OS and I see that Emacs uses very strange font for
> Cyrillic characters. For example, in C-h h window most words are rendered via
> system default Mono font, but Cyrillic characters are rendered via
> non-monotone serif font.
>
> It's reproducible with emacs -Q.
> In which direction should I dig?
>
> GNU Emacs 24.2.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.10)
>  of 2012-09-09 on trouble, modified by Debian
>

Please follow recent discussion:

  http://thread.gmane.org/gmane.emacs.help/85735
                 Re: Emacs do not show RU and IPA chars in monospaced font.

It have technique for troubleshooting your problem in X Window.

-- 
Best regards!

Re: Strange font used for Cyrillic characters

[cmr.pent]

The problem was solved for me by installing Liberation fonts :-)

The behavior I was observing seems bizarre (and Liberation is not in Debian default install), but I'm not sure now whether it qualifies as bug. I'm glad the problem is solved.

Thanks everyone,
Andrey Paramonov

Re: Strange font used for Cyrillic characters

[Peter Dyballa]

Am 20.12.2012 um 20:10 schrieb cmr.Pent@gmail.com:

> Here is the relevant piece of fontconfig log, I think

No, this is a bit too late. This excerpt shows proportional fonts. The question is why it comes to the use of these inadequate fonts. This must be buried somewhere in the debug output. A shorter test is this:

	fc-match monospace:lang=ru_RU

(The "ru_RU" is presumingly the value of your LANG environment variable.) The output of fc-match should tell you which font will be given (or recommended) by libfontconfig. If this is a proportional font then you'll have your culprit: your system. Otherwise it's GNU Emacs.

--
Greetings

  Pete

Upgraded, adj.:
	Didn't work the first time.

Re: Strange font used for Cyrillic characters

[Peter Dyballa]

Am 20.12.2012 um 22:19 schrieb cmr.pent@gmail.com:

> The behavior I was observing seems bizarre (and Liberation is not in Debian default install), but I'm not sure now whether it qualifies as bug. 

This looks more like an user option… (or an a bit too Latin centric system, worth to report as feedback)

--
Greetings

  Pete

There are very few jobs that actually require a penis or vagina. All other jobs should be open to everybody.
				– Florynce Kennedy

Re: Strange font used for Cyrillic characters

[Андрей Парамонов]

[-- Attachment #1: Type: text/plain, Size: 1230 bytes --]

As far as I can see the problem is in Debian fontconfig priorities/rules?

On more-or-less fresh system I get:
pent@sol:~$ fc-match monospace:lang=ru_RU
n022003l.pfb: "Nimbus Mono L" "Regular"

After installing Liberation I get:
pent@sol:~$ fc-match monospace:lang=ru_RU
DejaVuSansMono.ttf: "DejaVu Sans Mono" "Book"

It seems really strange that Numbus got higher than say FreeMono which is
system default. I'm not sure I can even submit a helpful bug report as I
have very weak understanding about how fontconfig is supposed to work :-/

Anyway, Emacs "must be above suspicion".

Best wishes,
Andrey Paramonov



2012/12/21 Peter Dyballa <Peter_Dyballa@web.de>

>
> Am 20.12.2012 um 22:19 schrieb cmr.pent@gmail.com:
>
> > The behavior I was observing seems bizarre (and Liberation is not in
> Debian default install), but I'm not sure now whether it qualifies as bug.
>
> This looks more like an user option… (or an a bit too Latin centric
> system, worth to report as feedback)
>
> --
> Greetings
>
>   Pete
>
> There are very few jobs that actually require a penis or vagina. All other
> jobs should be open to everybody.
>                                 – Florynce Kennedy
>
>

[-- Attachment #2: Type: text/html, Size: 1745 bytes --]

Re: Strange font used for Cyrillic characters

[Peter Dyballa]

Am 20.12.2012 um 22:40 schrieb Андрей Парамонов:

> It seems really strange that Numbus got higher than say FreeMono which is
> system default. I'm not sure I can even submit a helpful bug report as I
> have very weak understanding about how fontconfig is supposed to work :-/

It's obviously not a setup problem, I think. It looks more like there is initially on a debian install no monospaced font to provide Cyrillic characters (Russian, Ukrainian, Serbian, Bulgarian, Mongolian?) – or you've missed to install support for Russian. After installing Linux Libertine you have Cyrillic monospaced characters and so libfontconfig recommends to use Libertine.

Other options are the GNU Freefonts and particularly DejaVu (based on Bitstream Vera), or the RedHat/Fedora Liberation fonts.

> 
> Anyway, Emacs "must be above suspicion".

This gives a good feeling…

--
Greetings

  Pete

Government is actually the worst failure of civilized man. There has never been a really good one, and even those that are most tolerable are arbitrary, cruel, grasping and unintelligent.
				– H. L. Mencken

Re: Strange font used for Cyrillic characters

[Jason Rumney]

On Friday, 21 December 2012 03:10:07 UTC+8, cmr....@gmail.com  wrote:
> 
> Below is the output for C-u C-x =

>             character: З (displayed as З) (codepoint 1047, #o2027, #x417)
> 
>     preferred charset: mule-unicode-0100-24ff (Unicode characters of the range U+0100..U+24FF.)

This is suspicious.  The mule-unicode-* psuedo-charsets are obsolete since Emacs 23. Perhaps it is some left over customization from a pre-Unicode version of Emacs?  On both Windows and Ubuntu, I get: 

    preferred charset: cyrillic-iso8859-5
		       (Right-Hand Part of ISO/IEC 8859/5 (Latin/Cyrillic): ISO-IR-144)

Re: Strange font used for Cyrillic characters

[Jason Rumney]

On Friday, 21 December 2012 05:19:50 UTC+8, cmr....@gmail.com  wrote:
> The problem was solved for me by installing Liberation fonts :-)
> 
> 
> 
> The behavior I was observing seems bizarre (and Liberation is not in Debian default install), but I'm not sure now whether it qualifies as bug. I'm glad the problem is solved.

DejaVu Sans Mono should also have Cyrillic characters. Perhaps Debian is only installing the original Bitstream Vera Sans Mono by default, which was Latin only?

Re: Strange font used for Cyrillic characters

[cmr.pent]

пятница, 21 декабря 2012 г., 13:10:02 UTC+4 пользователь Jason Rumney написал:
> On Friday, 21 December 2012 03:10:07 UTC+8, cmr....@gmail.com  wrote:
> 
> > 
> 
> > Below is the output for C-u C-x =
> 
> 
> 
> >             character: З (displayed as З) (codepoint 1047, #o2027, #x417)
> 
> > 
> 
> >     preferred charset: mule-unicode-0100-24ff (Unicode characters of the range U+0100..U+24FF.)
> 
> 
> 
> This is suspicious.  The mule-unicode-* psuedo-charsets are obsolete since Emacs 23. Perhaps it is some left over customization from a pre-Unicode version of Emacs?  On both Windows and Ubuntu, I get: 
> 
> 
> 
>     preferred charset: cyrillic-iso8859-5
> 
> 		       (Right-Hand Part of ISO/IEC 8859/5 (Latin/Cyrillic): ISO-IR-144)

Please note I got the above with emacs -Q.

And I believe DejaVu fonts do cover Cyrillics, at least Russian subset. In all other programs (firefox, gnome-terminal) the same font was used for Cyrillics as for Latin letters, out-of-the-box.

For the record, I could reproduce the same behavior with Debian Emacs 23.

Best wishes,
Andrey Paramonov