Re: utf-8 cjk translation bug?

[Dave Love <d.love@dl.ac.uk>]

Kenichi Handa <handa@m17n.org> writes:

> So, #xFF?? are excluded from ucs-unicode-to-mule-cjk, thus
> they are not translated to japanese-jisx0208 on decoding.
> If you have a ISO10646-1 font that contains full width
> glyphs for those characters, you can see correct glyphs.

Or you can display them with a jisx font, for instance.

> I think the reason why they are excluded from the
> translation is that they are representable by the charset
> mule-unicode-e000-ffff, thus there's no need of translation.

That was part of the reason for it -- the hash-based translation code
is only relevant because we more-or-less used up the code space for
the BMP.  I also chose the boundaries to avoid breaking the region
between the mule-unicode and CJK charsets.

> It seems to be a reasonable decision, but considering that
> most users don't have an ISO10646-1 font containing those
> glyphs,

I thought they typically did if they had 10646 fonts at all.  Is the
problem that in recent XFree86, for instance, the double-width
characters are in different fonts which have `adstyl' `ja' or `ko'?
As far as I remember, the fontset code doesn't deal with that yet.
(So many special cases, sigh.)

> and that those characters can also be regarded as
> CJK components (only CJK users uses them), I think we had
> better not exclude them from the translation.

I'm not really convinced, but I don't feel strongly about it.  (If the
extra charsets hadn't been added before mule-unicode, we'd just have
covered the BMP with more mule-unicode ones.)

> So, I suggest changing the above line (and similar lines in
> the other subst-XXX.el) to:
>
>      (if (>= unicode #x2e80)
> 	 (puthash unicode  char ucs-unicode-to-mule-cjk))
>
> and modify ccl-decode-mule-utf-8 to check translation also
> for those characters.
>
> Dave, what do you think?  Does such a change leads to any
> problem?

As far as I remember, it includes too much, and you end up displaying
some characters double width that probably shouldn't be, but I don't
remember which.  How about including the ranges of the double-width
Western characters and the high CJK stuff explicitly?  I guess it
doesn't expand the tables greatly.