Re: MULE shows gibberish; now what?

Re: MULE shows gibberish; now what?

[Stefan Monnier <foo@acm.com>]

>>>>> "Ilya" == Ilya Zakharevich <nospam-abuse@ilyaz.org> writes:
> It looks like pointing-fingers was not dead-on-the-target; while Emacs
> is faulty, the principal blame should go to xrus-1251 and (allegedly)
> Netscape.  An excerpt from
>   http://www.siber.org/sib/russify/x-windows/#win-fonts
>    Yes, now you can have CP-1251 fonts on your X Window system.
>    We had to put them under the iso8859-5 because buggy Netscape would
>    not allow these fonts to work under official CP-1251 name.
> So what I see may be a very widespread situation.  Emacs can easily
> detect that the font *is not* a iso8859-5 font (since iso8859-5 has
> very few characters, and cp1251 has many).  At least it could have
> given a warning...

It could also look at the CHARSET_REGISTRY and CHARSET_ENCODING property
of the font (rather than assuming it's the same as in the fontname) ?

BTW, I agree that the fontset management could be made simpler (I think
it's not just a question of the doc being dense or incomplete).
Especially in your case, there should be some way to say something like:

   (fontset-prefer-font 'cyrillic-iso8859-5 "-etl-fixed-*-iso8859-5")


-- Stefan

Re: MULE shows gibberish; now what?

[Ilya Zakharevich]

On Wed, Oct 09, 2002 at 02:20:15PM -0400, Stefan Monnier <foo@acm.com> wrote:
> > So what I see may be a very widespread situation.  Emacs can easily
> > detect that the font *is not* a iso8859-5 font (since iso8859-5 has
> > very few characters, and cp1251 has many).  At least it could have
> > given a warning...
> 
> It could also look at the CHARSET_REGISTRY and CHARSET_ENCODING property
> of the font (rather than assuming it's the same as in the fontname) ?

Good idea.

> BTW, I agree that the fontset management could be made simpler (I think
> it's not just a question of the doc being dense or incomplete).

Given that I cannot deduce what fontsets (all these -0, -1 etc) are
*doing*, it is hard to deduce where what they are doing can be made
simpler.  ;-)

Thanks for your help,
Ilya

P.S.  Apparently, the configuration of the site cannot be changed
      (given that Netscape can break).  Thus without a clean fix which
      can be put in the site-file Cyrillic is not going to be usable in
      Emacs...

Re: MULE shows gibberish; now what?

[Stefan Monnier]

> > BTW, I agree that the fontset management could be made simpler (I think
> > it's not just a question of the doc being dense or incomplete).
> 
> Given that I cannot deduce what fontsets (all these -0, -1 etc) are

I'm not sure what -0 and -1 you're referring to.
I have the impression that you're referring to the -1 in
-misc-fixed-bold-*-*-*-*-140-75-75-*-*-*-1, in which case it has nothing
to do with Emacs or with fontsets.  It's in your Xresource setting and
specifies a font whose CHARSET_ENCODING is 1 (I suspect that it it intended
to only match fonts whose CHARSET_REGISTRY is iso8859, so that together
you get iso8859-1 which I'm sure you recognize).
But maybe that was obvious and you're actually talking about something else.

> P.S.  Apparently, the configuration of the site cannot be changed
>       (given that Netscape can break).  Thus without a clean fix which
>       can be put in the site-file Cyrillic is not going to be usable in
>       Emacs...

Maybe something like

(create-fontset-from-fontset-spec
 "-misc-fixed-bold-*-*-*-*-*-*-*-*-*-fontset-ilya,\
cyrillic-iso8859-5:-etl-*-*-*-*-*-*-*-*-*-*-*-iso8859-5")
(set-face-font 'default "fontset-ilya")


-- Stefan

Re: MULE shows gibberish; now what?

[Kenichi Handa]

In article <5lu1jvfp9c.fsf@rum.cs.yale.edu>, "Stefan Monnier <foo@acm.com>" <monnier+gnu.emacs.help/news/@rum.cs.yale.edu> writes:
>>>>>>  "Ilya" == Ilya Zakharevich <nospam-abuse@ilyaz.org> writes:
>>  It looks like pointing-fingers was not dead-on-the-target; while Emacs
>>  is faulty, the principal blame should go to xrus-1251 and (allegedly)
>>  Netscape.  An excerpt from
>>    http://www.siber.org/sib/russify/x-windows/#win-fonts
>>     Yes, now you can have CP-1251 fonts on your X Window system.
>>     We had to put them under the iso8859-5 because buggy Netscape would
>>     not allow these fonts to work under official CP-1251 name.
>>  So what I see may be a very widespread situation.  Emacs can easily
>>  detect that the font *is not* a iso8859-5 font (since iso8859-5 has
>>  very few characters, and cp1251 has many).  At least it could have
>>  given a warning...

As I didn't receive the original masage from Ilya, this is
my guess.

The current problem is:

(1) Now people install CP-1252 fonts and give alias names
    "*-iso8859-5".
(2) Emacs finds such a font for displaying
    cyrillic-iso8859-5 chars.

Correct?

Then, if such a font can be distinguished from correct
iso8859-5 fonts (perhaps, by family or foundary fields), the
easiest solution is to ignore such a font by setting the
variable `face-ignored-fonts' properly.

> It could also look at the CHARSET_REGISTRY and CHARSET_ENCODING property
> of the font (rather than assuming it's the same as in the fontname) ?

To get those properties, we must call XListFontsWithInfo
which, I think, is very slow compared with XListFonts.
Perhaps, we must introduce a variable, for instance,
`unreliable-font-registry-encoding-name-list, use
XListFontsWithInfo only to find a font whose
registry-encoding matches with this variable.

> BTW, I agree that the fontset management could be made simpler (I think
> it's not just a question of the doc being dense or incomplete).
> Especially in your case, there should be some way to say something like:

>    (fontset-prefer-font 'cyrillic-iso8859-5 "-etl-fixed-*-iso8859-5")

Is that intention different from this?

(dolist (fontset (fontest-list))
  (set-fontset-font fontset 'cyrillic-iso8859-5
		    '("etl-fixed" . "iso8859-5")))

---
Ken'ichi HANDA
handa@m17n.org

Re: MULE shows gibberish; now what?

[Stefan Monnier]

> As I didn't receive the original message from Ilya, this is my guess.
> The current problem is:
> 
> (1) Now people install CP-1252 fonts and give alias names "*-iso8859-5".
> (2) Emacs finds such a font for displaying cyrillic-iso8859-5 chars.
> 
> Correct?

Right on!

> Then, if such a font can be distinguished from correct
> iso8859-5 fonts (perhaps, by family or foundary fields), the
> easiest solution is to ignore such a font by setting the
> variable `face-ignored-fonts' properly.

Ah, I didn't remember that one.  It sounds just perfect.

> > It could also look at the CHARSET_REGISTRY and CHARSET_ENCODING property
> > of the font (rather than assuming it's the same as in the fontname) ?
> 
> To get those properties, we must call XListFontsWithInfo
> which, I think, is very slow compared with XListFonts.
> Perhaps, we must introduce a variable, for instance,
> `unreliable-font-registry-encoding-name-list, use
> XListFontsWithInfo only to find a font whose
> registry-encoding matches with this variable.

I was indeed thinking that it would probably be too costly preformancewise.
Although maybe we could do a sanity-check late in the game (after we have
settled on a particular font) and if the sanity check fails, we add
the fontname to face-ignored-fonts and try again.

> > BTW, I agree that the fontset management could be made simpler (I think
> > it's not just a question of the doc being dense or incomplete).
> > Especially in your case, there should be some way to say something like:
> >    (fontset-prefer-font 'cyrillic-iso8859-5 "-etl-fixed-*-iso8859-5")
> 
> Is that intention different from this?
> 
> (dolist (fontset (fontest-list))
>   (set-fontset-font fontset 'cyrillic-iso8859-5
> 		    '("etl-fixed" . "iso8859-5")))

No, that sounds just right.  But I think such things should be
documented in the Emacs Manual (and thus understandable to someone
who's not proficient in ELisp).



	Stefan

Re: MULE shows gibberish; now what?

[Ilya Zakharevich]

On Wed, Oct 09, 2002 at 05:39:54PM -0400, Stefan Monnier wrote:
> > > BTW, I agree that the fontset management could be made simpler (I think
> > > it's not just a question of the doc being dense or incomplete).
> > 
> > Given that I cannot deduce what fontsets (all these -0, -1 etc) are

> I'm not sure what -0 and -1 you're referring to.
> I have the impression that you're referring to the -1 in
> -misc-fixed-bold-*-*-*-*-140-75-75-*-*-*-1,

No, it is what the info-page mentions.

     Emacs.Fontset-0: -*-fixed-medium-r-normal-*-24-*-*-*-*-*-fontset-24,\
             chinese-gb2312:-*-*-medium-r-normal-*-24-*-gb2312*-*

Ilya

Re: MULE shows gibberish; now what?

[Ilya Zakharevich]

On Thu, Oct 10, 2002 at 09:29:12AM +0900, Kenichi Handa wrote:
> > It could also look at the CHARSET_REGISTRY and CHARSET_ENCODING property
> > of the font (rather than assuming it's the same as in the fontname) ?
> 
> To get those properties, we must call XListFontsWithInfo
> which, I think, is very slow compared with XListFonts.

> Perhaps, we must introduce a variable, for instance,
> `unreliable-font-registry-encoding-name-list, use
> XListFontsWithInfo only to find a font whose
> registry-encoding matches with this variable.

OK, if you do this, please put '-cronix-*' fonts there.  It is a
horrible thing what they did.

> > BTW, I agree that the fontset management could be made simpler (I think
> > it's not just a question of the doc being dense or incomplete).
> > Especially in your case, there should be some way to say something like:
> 
> >    (fontset-prefer-font 'cyrillic-iso8859-5 "-etl-fixed-*-iso8859-5")
> 
> Is that intention different from this?
> 
> (dolist (fontset (fontest-list))
>   (set-fontset-font fontset 'cyrillic-iso8859-5
> 		    '("etl-fixed" . "iso8859-5")))

Would not work if there is no etl-fixed fonts on the site?

Ilya

Re: MULE shows gibberish; now what?

[Ilya Zakharevich]

On Thu, Oct 10, 2002 at 09:29:12AM +0900, Kenichi Handa wrote:
> (1) Now people install CP-1252 fonts and give alias names
>     "*-iso8859-5".
> (2) Emacs finds such a font for displaying
>     cyrillic-iso8859-5 chars.
> 
> Correct?
> 
> Then, if such a font can be distinguished from correct
> iso8859-5 fonts (perhaps, by family or foundary fields), the
> easiest solution is to ignore such a font by setting the
> variable `face-ignored-fonts' properly.

It is not clear what "properly" means.

 (setq face-ignored-fonts '(".*RAWIN-R"))

does not fix the things;

  (setq face-ignored-fonts '(".*Cronyx.*"))

does fix the things...  The problematic font is

 cyrillic-iso8859-5	-*-*-*-iso8859-5
     [-Cronyx-Fixed-Medium-R-Normal--15-140-75-75-C-90-RAWIN-R]

(as output by describe-fontset).

> > It could also look at the CHARSET_REGISTRY and CHARSET_ENCODING property
> > of the font (rather than assuming it's the same as in the fontname) ?
> 
> To get those properties, we must call XListFontsWithInfo
> which, I think, is very slow compared with XListFonts.

Hmm, is it possible to get the actual font name without getting the info?

Ilya

P.S.  In my previous message Cronix was misspelled.  Sorry!

Re: MULE shows gibberish; now what?

[Kenichi Handa]

In article <20021010005818.GC3346@math.berkeley.edu>, Ilya Zakharevich <ilya@Math.Berkeley.EDU> writes:
>>  Perhaps, we must introduce a variable, for instance,
>>  `unreliable-font-registry-encoding-name-list, use
>>  XListFontsWithInfo only to find a font whose
>>  registry-encoding matches with this variable.

> OK, if you do this, please put '-cronix-*' fonts there.  It is a
> horrible thing what they did.

Then, could you please try the first method I wrote like this:
	(setq face-ignored-fonts '("-cronix"))

>>  > BTW, I agree that the fontset management could be made simpler (I think
>>  > it's not just a question of the doc being dense or incomplete).
>>  > Especially in your case, there should be some way to say something like:
>>  
>>  >    (fontset-prefer-font 'cyrillic-iso8859-5 "-etl-fixed-*-iso8859-5")
>>  
>>  Is that intention different from this?
>>  
>>  (dolist (fontset (fontest-list))
>>    (set-fontset-font fontset 'cyrillic-iso8859-5
>>  		    '("etl-fixed" . "iso8859-5")))

> Would not work if there is no etl-fixed fonts on the site?

In such a case, Emacs tries to find fonts ignoring that
foundry/family spec.  So, you must use face-ignored-fonts
as above.

---
Ken'ichi HANDA
handa@m17n.org

Re: MULE shows gibberish; now what?

[Kenichi Handa]

In article <200210100055.g9A0tQg04263@rum.cs.yale.edu>, "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes:
>>  To get those properties, we must call XListFontsWithInfo
>>  which, I think, is very slow compared with XListFonts.
>>  Perhaps, we must introduce a variable, for instance,
>>  `unreliable-font-registry-encoding-name-list, use
>>  XListFontsWithInfo only to find a font whose
>>  registry-encoding matches with this variable.

> I was indeed thinking that it would probably be too costly preformancewise.
> Although maybe we could do a sanity-check late in the game (after we have
> settled on a particular font) and if the sanity check fails, we add
> the fontname to face-ignored-fonts and try again.

Ah, yes, that method is less costy.

>>  Is that intention different from this?
>>  
>>  (dolist (fontset (fontest-list))
>>    (set-fontset-font fontset 'cyrillic-iso8859-5
>>  		    '("etl-fixed" . "iso8859-5")))

> No, that sounds just right.  But I think such things should be
> documented in the Emacs Manual (and thus understandable to someone
> who's not proficient in ELisp).

Isn't such a thing too cryptic to go in Emacs Manual.

It's a long term todo item to provide a good cutomization
method of font related things.  The problem is that no one
have a time to investigate it and work on it.  :-(

---
Ken'ichi HANDA
handa@m17n.org

Re: MULE shows gibberish; now what?

[Kenichi Handa]

In article <20021010010840.GD3346@math.berkeley.edu>, Ilya Zakharevich <ilya@Math.Berkeley.EDU> writes:
>>  Then, if such a font can be distinguished from correct
>>  iso8859-5 fonts (perhaps, by family or foundary fields), the
>>  easiest solution is to ignore such a font by setting the
>>  variable `face-ignored-fonts' properly.

> It is not clear what "properly" means.

>  (setq face-ignored-fonts '(".*RAWIN-R"))

> does not fix the things;

>   (setq face-ignored-fonts '(".*Cronyx.*"))

> does fix the things...  The problematic font is

>  cyrillic-iso8859-5	-*-*-*-iso8859-5
>      [-Cronyx-Fixed-Medium-R-Normal--15-140-75-75-C-90-RAWIN-R]

Ok, then, I guess the original font name is:
	-Cronyx-Fixed-Medium-R-Normal--15-140-75-75-C-90-RAWIN-R
and the alias name is
	-Cronyx-Fixed-Medium-R-Normal--15-140-75-75-C-90-ISO8859-5

Thus,
  (setq face-ignored-fonts '(".*RAWIN-R"))
can't reject the alias name but
  (setq face-ignored-fonts '(".*Cronyx.*"))
can.

The reason why M-x describe-fontset shows the original font
name is that Emacs calls XLoadQueryFont to get also the info
once a font is decided.  Then Emacs extracts "FONT" property
(if any) from the info and remember it as the correct font
name.

>>  > It could also look at the CHARSET_REGISTRY and CHARSET_ENCODING property
>>  > of the font (rather than assuming it's the same as in the fontname) ?
>>  
>>  To get those properties, we must call XListFontsWithInfo
>>  which, I think, is very slow compared with XListFonts.

> Hmm, is it possible to get the actual font name without getting the info?

No, as far as I know.

---
Ken'ichi HANDA
handa@m17n.org

Re: MULE shows gibberish; now what?

[Ilya Zakharevich]

On Thu, Oct 10, 2002 at 11:02:18AM +0900, Kenichi Handa wrote:
> In article <20021010010840.GD3346@math.berkeley.edu>, Ilya Zakharevich <ilya@Math.Berkeley.EDU> writes:
> >  (setq face-ignored-fonts '(".*RAWIN-R"))
> 
> > does not fix the things;
> 
> >   (setq face-ignored-fonts '(".*Cronyx.*"))
> 
> > does fix the things...  The problematic font is
> 
> >  cyrillic-iso8859-5	-*-*-*-iso8859-5
> >      [-Cronyx-Fixed-Medium-R-Normal--15-140-75-75-C-90-RAWIN-R]
> 
> Ok, then, I guess the original font name is:
> 	-Cronyx-Fixed-Medium-R-Normal--15-140-75-75-C-90-RAWIN-R
> and the alias name is
> 	-Cronyx-Fixed-Medium-R-Normal--15-140-75-75-C-90-ISO8859-5

This is the result of grep Stefan recommended:

  /local/share/X11/fonts/xrus-1251/fonts.alias:-cronyx-fixed-medium-r-normal--2-20-75-75-c-12-iso8859-5 -cronyx-fixed-medium-r-normal--2-20-75-75-c-12-rawin-r

So your assumption looks correct, and I need to set the REx to

  "-cronyx-.*-iso8859-5$"

The document I presented on the newsgroup,

  http://www.siber.org/sib/russify/x-windows/#win-fonts

shows that this particular misconfiguration may be very much
widespread; it is needed to make Netscape "work".  I recommend putting
the above REx in the default value of face-ignored-fonts.

BTW, It would help if the docs mentioned that the REx is matched
against the *alias*.  It would also help to mention that the match is
not anchored, and is case-insensitive...

BTW, this grep (of WHATEVER/fonts.*) did not pick up any fonts.dir
file, only fonts.alias file.  I'm puzzled...

Ilya