problem with mule-utf-8 ?

problem with mule-utf-8 ?

[Pascal Bourguignon]

Hello, 

I'm trying to make an UTF-8 file containing some katakana characters.

So, I take a new buffer, type M-x set-input-method RET japanese-katakana RET
a SPACE e SPACE i SPACE o SPACE C-x C-s, and then it says:

------------------------------------------------------------------------
These default coding systems were tried to encode text
in the buffer `test':
  utf-8 iso-latin-1
However, each of them encountered these problematic characters:
  iso-latin-1: ア エ イ オ ウ ヤ
  utf-8: ア エ イ オ ウ ヤ
The first problematic character is at point in the displayed buffer,
and C-u C-x = will give information about it.

Select one of the following safe coding systems, or edit the buffer:
  euc-jp shift_jis iso-2022-jp japanese-iso-7bit-1978-irv
  iso-2022-7bit
Or specify any other coding system
on your risk of losing the problematic characters.
------------------------------------------------------------------------

So,  what's  the  matter?  I  thought that  unicode  inclued  all  the
characters,  and  that  utf-8  was  able  to  transcribe  all  unicode
character, or not?

Of course, I insist and save  it with utf-8 encoding, then when I load
this utf-8 file later, I get rectangle frames instead of katakana...


Is this a problem with mule, or is it with unicode?


-- 
__Pascal_Bourguignon__                   http://www.informatimago.com/
----------------------------------------------------------------------
Do not adjust your mind, there is a fault in reality.

Re: problem with mule-utf-8 ?

[Eli Zaretskii]

> From: Pascal Bourguignon <spam@thalassa.informatimago.com>
> Newsgroups: gnu.emacs.help
> Date: 06 Sep 2003 16:59:24 +0200
> 
> So,  what's  the  matter?  I  thought that  unicode  inclued  all  the
> characters,  and  that  utf-8  was  able  to  transcribe  all  unicode
> character, or not?
> 
> Of course, I insist and save  it with utf-8 encoding, then when I load
> this utf-8 file later, I get rectangle frames instead of katakana...

In what version of Emacs?  All released versions of Emacs support
only part of the BMP.  Specifically, these ranges of Unicode
codepoints are supported:

  0100..33ff
  e000..ffff

If katakana characters are not in these ranges, you cannot have them
in unicode.

The CVS version of Emacs can (I think) convert katakana to Unicode
when encoding and decoding text.  I also think you can have this with
released versions if you install ucs-tables (look on
gnu.emacs.sources).

Re: problem with mule-utf-8 ?

[Alex Schroeder]

Pascal Bourguignon <spam@thalassa.informatimago.com> writes:

> Is this a problem with mule, or is it with unicode?

Emacs doesn't do Han-Unification by default.  The CVS version has code
to do that, but until then, if you type Japanese characters using the
Japanese input methods, they can only be represented in the iso-2022
encodings, or in specific Japanese encodings such as euc-jp.

Alex.
-- 
http://www.emacswiki.org/alex/
There is no substitute for experience.

Re: problem with mule-utf-8 ?

[Janusz S. Bień]

On Sat, 06 Sep 2003  "Eli Zaretskii" <eliz@elta.co.il> wrote:

> > From: Pascal Bourguignon <spam@thalassa.informatimago.com>
> > Newsgroups: gnu.emacs.help
> > Date: 06 Sep 2003 16:59:24 +0200
> > 
> > So,  what's  the  matter?  I  thought that  unicode  inclued  all  the
> > characters,  and  that  utf-8  was  able  to  transcribe  all  unicode
> > character, or not?
> > 
> > Of course, I insist and save  it with utf-8 encoding, then when I load
> > this utf-8 file later, I get rectangle frames instead of katakana...
> 
> In what version of Emacs?  
> All released versions of Emacs support
> only part of the BMP.  Specifically, these ranges of Unicode
> codepoints are supported:
> 
>   0100..33ff
>   e000..ffff
> 
> If katakana characters are not in these ranges, you cannot have them
> in unicode.

You can extend Unicode support with mule-ucs. This is the excerpt
from the output of Help -> Find Extra Packages.

 * Mule-UCS: Universal Encoding System:
   <URL:ftp://ftp.m17n.org/pub/mule/Mule-UCS/>
   Extended coding systems for Mule, specifically for reading and
   writing UTF-8 encoded Unicode.  This does more than the built-in
   utf-8 coding system, specifically covering a lot of Far Eastern
   characters.  (See the entry in PROBLEMS concerning slow startup of
   Mule-UCS.)


> 
> The CVS version of Emacs can (I think) convert katakana to Unicode
> when encoding and decoding text.  I also think you can have this with
> released versions if you install ucs-tables (look on
> gnu.emacs.sources).

If you are going to use CVS, you may as well try emacs-unicode :-).

Regards

Janusz

-- 
                     ,   
dr hab. Janusz S. Bien, prof. UW
Prof. Janusz S. Bien, Warsaw Uniwersity
jsbien@mimuw.edu.pl, jsbien@uw.edu.pl
http://www.orient.uw.edu.pl/~jsbien/
http://www.mimuw.edu.pl/~jsbien/

Re: problem with mule-utf-8 ?

[Eli Zaretskii]

> From: jsbien@mimuw.edu.pl (=?iso-8859-2?q?Janusz_S._Bie=F1?=)
> Date: 08 Sep 2003 07:47:24 +0200
> 
> If you are going to use CVS, you may as well try emacs-unicode :-).

AFAIK, the unicode branch is not yet stable enough to give such an
advice, but perhaps my information is outdated.

Re: problem with mule-utf-8 ?

[Pascal Bourguignon]

Alex Schroeder <alex@emacswiki.org> writes:

> Pascal Bourguignon <spam@thalassa.informatimago.com> writes:
> 
> > Is this a problem with mule, or is it with unicode?
> 
> Emacs doesn't do Han-Unification by default.  The CVS version has code
> to do that, but until then, if you type Japanese characters using the
> Japanese input methods, they can only be represented in the iso-2022
> encodings, or in specific Japanese encodings such as euc-jp.

I see. Thank you.

-- 
__Pascal_Bourguignon__
http://www.informatimago.com/
Do not adjust your mind, there is a fault in reality.