From: Kenichi Handa <handa@m17n.org>
Cc: d.love@dl.ac.uk, emacs-devel@gnu.org
Subject: Re: utf-8 cjk translation bug?
Date: Tue, 30 Sep 2003 21:59:42 +0900 (JST) [thread overview]
Message-ID: <200309301259.VAA01304@etlken.m17n.org> (raw)
In-Reply-To: <buo7k3q3cgu.fsf@mcspd15.ucom.lsi.nec.co.jp> (message from Miles Bader on 30 Sep 2003 17:30:25 +0900)
In article <buo7k3q3cgu.fsf@mcspd15.ucom.lsi.nec.co.jp>, Miles Bader <miles@lsi.nec.co.jp> writes:
> I have `utf-translate-cjk-mode' enabled.
> I have the following string in a buffer:
> NECエレクトロニクス(株)
> If I write it using say `euc-jp' coding system, no problem. According
> to `C-u C-x =', all the japanese characters are in the charset
> japanese-jisx0208.
> However, if I save it using utf-8, I get no complaints, but when I read
> it back in, the first 3 characters show up as little boxes. `C-u C-x ='
> shows the boxes as being in charset mule-unicode-e000-ffff; the rest of
> the characters are still listed as being in japanese-jisx0208.
> I presume this is representable utf-8, because unicode is supposed to be
> able to represent all characters in any component character set
> simultaneously, so it would seem to be a bug in utf-translate-cjk-mode.
The first three letters are "FULL WIDTH LATIN ?? LETTER"
(U+FF??). Yes, they are representable in utf-8. But, in
subst-jis.el, we have this code:
(mapc
(lambda (pair)
(let ((unicode (car pair))
(char (cadr pair)))
;; exclude non-CJK components from decode table
(if (and (>= unicode #x2e80) (<= unicode #xd7a3))
(puthash unicode char ucs-unicode-to-mule-cjk))
(puthash char unicode ucs-mule-cjk-to-unicode)))
So, #xFF?? are excluded from ucs-unicode-to-mule-cjk, thus
they are not translated to japanese-jisx0208 on decoding.
If you have a ISO10646-1 font that contains full width
glyphs for those characters, you can see correct glyphs.
I think the reason why they are excluded from the
translation is that they are representable by the charset
mule-unicode-e000-ffff, thus there's no need of translation.
It seems to be a reasonable decision, but considering that
most users don't have an ISO10646-1 font containing those
glyphs, and that those characters can also be regarded as
CJK components (only CJK users uses them), I think we had
better not exclude them from the translation.
So, I suggest changing the above line (and similar lines in
the other subst-XXX.el) to:
(if (>= unicode #x2e80)
(puthash unicode char ucs-unicode-to-mule-cjk))
and modify ccl-decode-mule-utf-8 to check translation also
for those characters.
Dave, what do you think? Does such a change leads to any
problem? Aren't there anything else we should change?
---
Ken'ichi HANDA
handa@m17n.org
next prev parent reply other threads:[~2003-09-30 12:59 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2003-09-30 8:30 utf-8 cjk translation bug? Miles Bader
2003-09-30 9:50 ` Jason Rumney
2003-09-30 10:05 ` Miles Bader
2003-09-30 12:59 ` Kenichi Handa [this message]
2003-10-01 12:44 ` Dave Love
2003-10-02 1:08 ` Kenichi Handa
2003-10-03 16:04 ` Dave Love
2003-10-03 16:34 ` Jason Rumney
2003-10-06 2:29 ` Miles Bader
2003-10-06 20:00 ` Miles Bader
2003-10-06 20:53 ` Jason Rumney
2003-10-06 23:18 ` Kenichi Handa
2003-10-07 9:57 ` Stephen J. Turnbull
2003-10-07 11:41 ` Dave Love
2003-10-07 11:40 ` Dave Love
2003-10-06 23:53 ` Kenichi Handa
2003-10-10 16:56 ` Dave Love
2003-10-13 23:55 ` Kenichi Handa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=200309301259.VAA01304@etlken.m17n.org \
--to=handa@m17n.org \
--cc=d.love@dl.ac.uk \
--cc=emacs-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).