enforcing double-width CJK character display

enforcing double-width CJK character display

[Werner LEMBERG]

Folks,


within, say, konsole 2.6.4 (from KDE 4.6.5), the character width of
double-width CJK characters is forced to be exactly twice the width of
the used console font, as expected.  Is there a similar option to
emacs?  For example, my main Emacs font is `DejaVu Sans Mono', and the
font selected for Japanese is `Sazanami Mincho', and the latter's
character width is smaller...


    Werner

Re: enforcing double-width CJK character display

[Eli Zaretskii]

> Date: Sun, 24 Jul 2011 14:40:05 +0200 (CEST)
> From: Werner LEMBERG <wl@gnu.org>
> 
> within, say, konsole 2.6.4 (from KDE 4.6.5), the character width of
> double-width CJK characters is forced to be exactly twice the width of
> the used console font, as expected.  Is there a similar option to
> emacs?  For example, my main Emacs font is `DejaVu Sans Mono', and the
> font selected for Japanese is `Sazanami Mincho', and the latter's
> character width is smaller...

Does "C-x =" tell you that Emacs counts each Japanese character as 2
columns?  If it does, then the issue is with selecting a better font,
is that right?

Re: enforcing double-width CJK character display

[Werner LEMBERG]

>> within, say, konsole 2.6.4 (from KDE 4.6.5), the character width of
>> double-width CJK characters is forced to be exactly twice the width
>> of the used console font, as expected.  Is there a similar option
>> to emacs?  For example, my main Emacs font is `DejaVu Sans Mono',
>> and the font selected for Japanese is `Sazanami Mincho', and the
>> latter's character width is smaller...
> 
> Does "C-x =" tell you that Emacs counts each Japanese character as 2
> columns?

Yes.

> If it does, then the issue is with selecting a better font, is that
> right?

Essentially yes.  However, the question is what you define as `better'
-- I'm talking about scalable fonts, of course, not about bitmap
fonts.  There are two possibilities:

  1. scale the CJK font so that its (CJK) glyphs are exactly the
     double width of a given mono-width font

  2. pad the CJK font horizontally with space around each CJK
     character

I tend towards solution 2 and I wonder whether Emacs already provides
a means to do that, or maybe this can be handled on the FontConfig
level.


    Werner

Re: enforcing double-width CJK character display

[Kan-Ru Chen]

Werner LEMBERG <wl@gnu.org> writes:

>   1. scale the CJK font so that its (CJK) glyphs are exactly the
>      double width of a given mono-width font
>
>   2. pad the CJK font horizontally with space around each CJK
>      character
>
> I tend towards solution 2 and I wonder whether Emacs already provides
> a means to do that, or maybe this can be handled on the FontConfig
> level.

+1 for this.

Without this I will be always bound to tty emacs. You can see how gvim
handle this. But I'm thinking of the case where many different size
font-face was used on the same buffer.

Ex: http://farm1.static.flickr.com/107/312993958_fb13a395f5_o.png

How do you decide the proper font size?

-- 
Kanru

Re: enforcing double-width CJK character display

[Werner LEMBERG]

> [...] I'm thinking of the case where many different size font-face
> was used on the same buffer.
> 
> Ex: http://farm1.static.flickr.com/107/312993958_fb13a395f5_o.png
> 
> How do you decide the proper font size?

What exactly do you mean?


    Werner

Re: enforcing double-width CJK character display

[Kan-Ru Chen]

Werner LEMBERG <wl@gnu.org> writes:

>> [...] I'm thinking of the case where many different size font-face
>> was used on the same buffer.
>> 
>> Ex: http://farm1.static.flickr.com/107/312993958_fb13a395f5_o.png
>> 
>> How do you decide the proper font size?
>
> What exactly do you mean?

Oh oh.. I confused it with aligning fonts with different size on the
same buffer. Of course you can make the Chinese font exactly
double-width of the default font, then the aligning of the same font
size will be correct :)

-- 
Kanru

Re: enforcing double-width CJK character display

[Štěpán Němec]

On Sunday, 24.07.2011, Kan-Ru Chen wrote: 

> Werner LEMBERG <wl@gnu.org> writes:
>
>>   1. scale the CJK font so that its (CJK) glyphs are exactly the
>>      double width of a given mono-width font
>>
>>   2. pad the CJK font horizontally with space around each CJK
>>      character
>>
>> I tend towards solution 2 and I wonder whether Emacs already provides
>> a means to do that, or maybe this can be handled on the FontConfig
>> level.
>
> +1 for this.

+2

Currently I'm manually doing the "solution" 1 -- find a font that is
exactly twice as wide as the prevalent latin font. Unfortunately such a
font will also be higher, so it doesn't look quite as well.

=== related but off-subject ===
I'd also like to reiterate that Emacs' current font selection is
obviously suboptimal in more ways than this -- there are cases where
Emacs displays boxes instead of characters, although it

1) has access to the appropriate font (and other applications s.a.
Firefox select it properly).

2) There is no other font which includes the code point, so the
selection should be obvious, right?

3) I have no related customisation that would prevent Emacs from
selecting it automatically.

4) It works (as in, displays properly in Emacs) when I select the font
manually.

I've posted about that to emacs.help[1] before some time, but got no reply.
I'll file a bug report, but thought I'd try to mention it here as well
first in case anyone had any related idea or experience to share.

[1] <http://permalink.gmane.org/gmane.emacs.help/79766>
=== thanks for reading ===

-- 
Štěpán

Re: enforcing double-width CJK character display

[Eli Zaretskii]

> Date: Sun, 24 Jul 2011 15:17:58 +0200 (CEST)
> Cc: emacs-devel@gnu.org
> From: Werner LEMBERG <wl@gnu.org>
> 
> > If it does, then the issue is with selecting a better font, is that
> > right?
> 
> Essentially yes.  However, the question is what you define as `better'
> -- I'm talking about scalable fonts, of course, not about bitmap
> fonts.  There are two possibilities:
> 
>   1. scale the CJK font so that its (CJK) glyphs are exactly the
>      double width of a given mono-width font
> 
>   2. pad the CJK font horizontally with space around each CJK
>      character
> 
> I tend towards solution 2 and I wonder whether Emacs already provides
> a means to do that, or maybe this can be handled on the FontConfig
> level.

I hope someone who is expert on fontconfig and font selection in Emacs
will be able to answer that.

Re: enforcing double-width CJK character display

[Kan-Ru Chen]

Werner LEMBERG <wl@gnu.org> writes:

> Essentially yes.  However, the question is what you define as `better'
> -- I'm talking about scalable fonts, of course, not about bitmap
> fonts.  There are two possibilities:
>
>   1. scale the CJK font so that its (CJK) glyphs are exactly the
>      double width of a given mono-width font

I found setting the variable `face-font-rescale-alist' can achieve this,
if you know the scale factor. My current config looks like:

#+begin_src elisp
  ;; Set fallback fonts so I can use this across different system
  (customize-set-variable
   'face-font-family-alternatives
   (cons '("Han" "WenQuanYi Micro Hei" "AR Heiti Light B5" "cwTexYen")
         face-font-family-alternatives))
  ;; Set scale ratio
  (setq face-font-rescale-alist
        '(("WenQuanYi Micro Hei" . 1.2)
          ("cwTexYen" . 1.35)
          ("AR Heiti Light B5" . 1.2)))
  ;; Set default font for han script
  (set-fontset-font t 'han "Han")
  ;; Setting `face-font-rescale-alist' somehow changes the default font
  ;; for new frame.
  (add-to-list 'frame-inherited-parameters 'font)
#+end_src

It would be great if we can automatically set the scale ratio according
to the :avgwidth attribute of the fonts to match the default ascii font,
but I haven't find a good way to manipulate font-object in elisp.

My org-mode tags finally are aligned again.

-- 
Kanru

Re: enforcing double-width CJK character display

[Werner LEMBERG]

A late follow-up:

> I found setting the variable `face-font-rescale-alist' can achieve
> this, if you know the scale factor.  [...]

Nice idea!

> It would be great if we can automatically set the scale ratio
> according to the :avgwidth attribute of the fonts to match the
> default ascii font, but I haven't find a good way to manipulate
> font-object in elisp.

I ask you to file a bug report so that both your work-around and the
original problem are archived properly.


    Werner

Re: enforcing double-width CJK character display

[Kan-Ru Chen]

Werner LEMBERG <wl@gnu.org> writes:

> A late follow-up:
>
>> I found setting the variable `face-font-rescale-alist' can achieve
>> this, if you know the scale factor.  [...]
>
> Nice idea!
>
>> It would be great if we can automatically set the scale ratio
>> according to the :avgwidth attribute of the fonts to match the
>> default ascii font, but I haven't find a good way to manipulate
>> font-object in elisp.
>
> I ask you to file a bug report so that both your work-around and the
> original problem are archived properly.

Done.

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10179

Re: enforcing double-width CJK character display

[Werner LEMBERG]

>> I ask you to file a bug report so that both your work-around and the
>> original problem are archived properly.
> 
> Done.
> 
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10179

Thanks!


    Werner

Re: enforcing double-width CJK character display

[Ivan Kanis]

Werner LEMBERG <wl@gnu.org> wrote:

> within, say, konsole 2.6.4 (from KDE 4.6.5), the character width of
> double-width CJK characters is forced to be exactly twice the width of
> the used console font, as expected.

There is no easy way with the new display engine. On my maybe list is
to resurrect the bdf font. If I started this work would there be any
chance of it being included in trunk?

Take care
-- 
Ivan Kanis

Computer games don't affect kids; I mean if Pac-Man affected us as
kids, we'd all be running around in darkened rooms, munching magic
pills and listening to repetitive electronic music.
    -- Kristian Wilson 

Re: enforcing double-width CJK character display

[Werner LEMBERG]

>> within, say, konsole 2.6.4 (from KDE 4.6.5), the character width of
>> double-width CJK characters is forced to be exactly twice the width
>> of the used console font, as expected.
> 
> There is no easy way with the new display engine.

This is bad news, since I consider this as a quite important feature.
I really wonder why no CJK users have ever complained (but maybe I've
missed such rants).


    Werner

Re: enforcing double-width CJK character display

[Eli Zaretskii]

> Date: Tue, 26 Jul 2011 22:13:40 +0200 (CEST)
> From: Werner LEMBERG <wl@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> >> within, say, konsole 2.6.4 (from KDE 4.6.5), the character width of
> >> double-width CJK characters is forced to be exactly twice the width
> >> of the used console font, as expected.
> > 
> > There is no easy way with the new display engine.
> 
> This is bad news, since I consider this as a quite important feature.

I would appreciate if we could have some details about why "there is
no easy way with the new display engine" to accomplish this.  What
prevents this, exactly?

Re: enforcing double-width CJK character display

[David De La Harpe Golden]

On 26/07/11 21:13, Werner LEMBERG wrote:

> This is bad news, since I consider this as a quite important feature.
> I really wonder why no CJK users have ever complained (but maybe I've
> missed such rants).

Well, there are AFAIUI dual-width fonts with both western and asian 
glyphs which line up and are stylistically consistent, such as IPAGothic 
(debian package otf-ipafont-gothic).  I suspect they may cover a lot of 
people's needs. Try

fc-list :spacing=dual

to see if you have any currently installed.

One rather brute-force way (with the distinct advantage that it works) 
to force use of _only_ particular well-matched fonts in emacs is to 
limit the fonts visible to emacs at all, at the libfontconfig level.

You can do that using an alternate fontconfig configuration file that 
points at a directory with only the particular fonts you want used in 
it.  The libfontconfig override mechanism is documented to be via an 
environment variable FC_CONFIG_FILE, but the actual environment variable 
is apparently FONTCONFIG_FILE.  (If you're doing that, you also need to 
force use of _only_ the xft font rendering, not both xft and core x, 
with "Emacs.FontBackend: xft" in your x resource db (~/.Xdefaults))

Re: enforcing double-width CJK character display

[Werner LEMBERG]

>> This is bad news, since I consider this as a quite important
>> feature.  I really wonder why no CJK users have ever complained
>> (but maybe I've missed such rants).
> 
> Well, there are AFAIUI dual-width fonts with both western and asian
> glyphs which line up and are stylistically consistent, such as
> IPAGothic (debian package otf-ipafont-gothic).  I suspect they may
> cover a lot of people's needs. Try
> 
> fc-list :spacing=dual
> 
> to see if you have any currently installed.

Thanks for sharing your ideas.

However, in most cases, the latin characters of CJK fonts are really
ugly.  Additionally, normal CJK fonts only cover either Chinese,
Japanese, or Korean, but not all three languages at the same time, so
we have again a similar problem if we want a generic solution for
Unicode display.


    Werner