cp949 support

cp949 support

[Eduard Wiebe]

[-- Attachment #1: Type: text/plain, Size: 330 bytes --]


  Hi,

the following patch adds cp949 charset support to Emacs.
Btw, i compared etc/charsets/CP949-2BYTE.map with
http://unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP949.TXT and
found no descrepances.

2009-06-15  Eduard Wiebe  <usenet@pusto.de>

	* international/mule-conf.el: Add cp949-2-byte, cp949.

-- 
Eduard Wiebe

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: mule-conf.el.cp949.diff --]
[-- Type: text/x-patch, Size: 738 bytes --]

diff --git a/lisp/international/mule-conf.el b/lisp/international/mule-conf.el
old mode 100644
new mode 100755
index 8bd9301..f8cfa09
--- a/lisp/international/mule-conf.el
+++ b/lisp/international/mule-conf.el
@@ -486,7 +486,19 @@
   :code-offset #x27c218			; ... #x280839
   :unify-map "BIG5-HKSCS")
 
-;; Fixme: Korean cp949/UHC
+(define-charset 'cp949-2-byte
+  "2-byte part of CP949"
+  :dimension 2
+  :map "CP949-2BYTE"
+  :code-space [#x41 #xFE #x81 #xFD]
+  :supplementary-p t)
+
+(define-charset 'cp949
+  "CP949 (Korean)"
+  :short-name "CP949"
+  :long-name  "CP949 (Korean)"
+  :code-space [#x00 #xFE #x00 #xFD]
+  :superset '(ascii cp949-2-byte))
 
 (define-charset 'chinese-sisheng
   "SiSheng characters for PinYin/ZhuYin"

Re: cp949 support

[Kenichi Handa]

In article <861vpldwy5.fsf@nirvana.pusto.de>, Eduard Wiebe <usenet@pusto.de> writes:

> the following patch adds cp949 charset support to Emacs.
> Btw, i compared etc/charsets/CP949-2BYTE.map with
> http://unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP949.TXT and
> found no descrepances.

> 2009-06-15  Eduard Wiebe  <usenet@pusto.de>

> 	* international/mule-conf.el: Add cp949-2-byte, cp949.

Thank you.  I installed your change.

---
Kenichi Handa
handa@m17n.org

Re: cp949 support

[Jihyun Cho]

In the past, someone has submitted a patch that is similar with this.
(http://lists.gnu.org/archive/html/emacs-devel/2008-03/msg01700.html)

They have different code-space between this patch and the old patch.
Which is the correct code-space?

I applied this patch, but some Hangul letters in CP949 region are
shown incorrectly.
For example, the Hangul letter "HANGUL SYLLABLE HAEH" is shown "HANGUL
SYLLABLE JWIG" in this patch.
But the old patch works well.


2009/6/16 Kenichi Handa <handa@m17n.org>:
> In article <861vpldwy5.fsf@nirvana.pusto.de>, Eduard Wiebe <usenet@pusto.de> writes:
>
>> the following patch adds cp949 charset support to Emacs.
>> Btw, i compared etc/charsets/CP949-2BYTE.map with
>> http://unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP949.TXT and
>> found no descrepances.
>
>> 2009-06-15  Eduard Wiebe  <usenet@pusto.de>
>
>>       * international/mule-conf.el: Add cp949-2-byte, cp949.
>
> Thank you.  I installed your change.
>
> ---
> Kenichi Handa
> handa@m17n.org
>
>
>

Re: cp949 support

[Kenichi Handa]

In article <9d644d9b0906170107o2e7f5d82j46882620b5a968b1@mail.gmail.com>, Jihyun Cho <jihyun.jo@gmail.com> writes:

> In the past, someone has submitted a patch that is similar with this.
> (http://lists.gnu.org/archive/html/emacs-devel/2008-03/msg01700.html)

Oops, I have forgotten about that mail.

> They have different code-space between this patch and the old patch.
> Which is the correct code-space?

Af far as I see /usr/share/i18n/charmaps/CP949.gz, it seems
that the current one is correct.

> I applied this patch, but some Hangul letters in CP949 region are
> shown incorrectly.
> For example, the Hangul letter "HANGUL SYLLABLE HAEH" is shown "HANGUL
> SYLLABLE JWIG" in this patch.
> But the old patch works well.

How did you show those letters in Emacs?  By
list-charset-chars?  Please tell me the Unicode code point
of those two characters.

---
Kenichi Handa
handa@m17n.org

Re: cp949 support

[Eduard Wiebe]

Kenichi Handa <handa@m17n.org> writes:

> In article <9d644d9b0906170107o2e7f5d82j46882620b5a968b1@mail.gmail.com>, Jihyun Cho <jihyun.jo@gmail.com> writes:
>
>> In the past, someone has submitted a patch that is similar with this.
>> (http://lists.gnu.org/archive/html/emacs-devel/2008-03/msg01700.html)
>
> Oops, I have forgotten about that mail.
>
>> They have different code-space between this patch and the old patch.
>> Which is the correct code-space?
>
> Af far as I see /usr/share/i18n/charmaps/CP949.gz, it seems
> that the current one is correct.
>
>> I applied this patch, but some Hangul letters in CP949 region are
>> shown incorrectly.
>> For example, the Hangul letter "HANGUL SYLLABLE HAEH" is shown "HANGUL
>> SYLLABLE JWIG" in this patch.
>> But the old patch works well.

Hmm..!?

> How did you show those letters in Emacs?  By
> list-charset-chars?  

The output of M-x list-charset-chars looks pretty good to me and accords
with http://msdn.microsoft.com/de-de/goglobal/cc305154(en-us).aspx

> Please tell me the Unicode code point of those two characters.

@Jihyun, have you following chars in mind?

HANGUL SYLLABLE HAEH:  Unicode 0xd58f, CP949 0xc164
HANGUL SYLLABLE JWIG:  Unicode 0xc951, CP949 0xc1e4

-- 
Eduard Wiebe

Re: cp949 support

[Jihyun Cho]

2009/6/18 Eduard Wiebe <usenet@pusto.de>:
> Kenichi Handa <handa@m17n.org> writes:
>
>> In article <9d644d9b0906170107o2e7f5d82j46882620b5a968b1@mail.gmail.com>, Jihyun Cho <jihyun.jo@gmail.com> writes:
>>
>>> In the past, someone has submitted a patch that is similar with this.
>>> (http://lists.gnu.org/archive/html/emacs-devel/2008-03/msg01700.html)
>>
>> Oops, I have forgotten about that mail.
>>
>>> They have different code-space between this patch and the old patch.
>>> Which is the correct code-space?
>>
>> Af far as I see /usr/share/i18n/charmaps/CP949.gz, it seems
>> that the current one is correct.
>>
>>> I applied this patch, but some Hangul letters in CP949 region are
>>> shown incorrectly.
>>> For example, the Hangul letter "HANGUL SYLLABLE HAEH" is shown "HANGUL
>>> SYLLABLE JWIG" in this patch.
>>> But the old patch works well.
>
> Hmm..!?
>
>> How did you show those letters in Emacs?  By
>> list-charset-chars?
>
> The output of M-x list-charset-chars looks pretty good to me and accords
> with http://msdn.microsoft.com/de-de/goglobal/cc305154(en-us).aspx
>
>> Please tell me the Unicode code point of those two characters.
>
> @Jihyun, have you following chars in mind?
>
> HANGUL SYLLABLE HAEH:  Unicode 0xd58f, CP949 0xc164
> HANGUL SYLLABLE JWIG:  Unicode 0xc951, CP949 0xc1e4


Yes. It looks very good to me, too.

I ran the following test.
First, I applied a old patch.
I saved a file with VIM with the option "set fenc=cp949".
Then I loaded the file with EMACS. It is broken in my UTF-8 environment.
So I was typing "M-x revert-buffer-with-coding-system", "cp949", then
it looks well.

After applying this patch, I ran a same test.
It shows a wrong letter.

I guess the problem is related to coding system.

The problem that "HANGUL SYLLABLE HAEH" is shown "HANGUL SYLLABLE
JWIG" occured in EUC-KR coding system.
Because "HANGUL SYLLABLE HAEH" is not contained in EUC-KR region.
But CP949 contains "HANGUL SYLLABLE HAEH".
This patch could not fix it.

Re: cp949 support

[Kenichi Handa]

In article <9d644d9b0906171919x3b05cc2eu6af3db8d113ee6cd@mail.gmail.com>, Jihyun Cho <jihyun.jo@gmail.com> writes:

> I ran the following test.
> First, I applied a old patch.
> I saved a file with VIM with the option "set fenc=cp949".
> Then I loaded the file with EMACS. It is broken in my UTF-8 environment.
> So I was typing "M-x revert-buffer-with-coding-system", "cp949", then
> it looks well.

> After applying this patch, I ran a same test.
> It shows a wrong letter.

> I guess the problem is related to coding system.

> The problem that "HANGUL SYLLABLE HAEH" is shown "HANGUL SYLLABLE
> JWIG" occured in EUC-KR coding system.
> Because "HANGUL SYLLABLE HAEH" is not contained in EUC-KR region.
> But CP949 contains "HANGUL SYLLABLE HAEH".
> This patch could not fix it.

Wiebe's patch doesn't contain the change of cp949
"coding-system".  So, what you did is the same as reading a
cp949 file by euc-kr coding-system.

By the way, decoding "\xc1\x64" as U+C951 is a bug of
decoding routine for EUC-type coding systems.  It should
treat that sequence as invalid as Emacs 22 does.  I've just
installed a fix.

With the following additional patch, you should be able to
read a cp949 file by cp949 coding-system correctly.

Index: korean.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/language/korean.el,v
retrieving revision 1.41
retrieving revision 1.42
diff -u -r1.41 -r1.42
--- korean.el	5 Jan 2009 03:22:27 -0000	1.41
+++ korean.el	18 Jun 2009 01:15:32 -0000	1.42
@@ -43,7 +43,6 @@
 
 (define-coding-system-alias 'euc-kr 'korean-iso-8bit)
 (define-coding-system-alias 'euc-korea 'korean-iso-8bit)
-(define-coding-system-alias 'cp949 'korean-iso-8bit)
 
 (define-coding-system 'iso-2022-kr
   "ISO 2022 based 7-bit encoding for Korean KSC5601 (MIME:ISO-2022-KR)."
@@ -58,6 +57,14 @@
 
 (define-coding-system-alias 'korean-iso-7bit-lock 'iso-2022-kr)
 
+(define-coding-system 'korean-cp949
+  "CP949 (Microsoft Unified Hangul Code)"
+  :coding-type 'charset
+  :mnemonic ?K
+  :charset-list '(ascii cp949))
+
+(define-coding-system-alias 'cp949 'korean-cp949)
+
 (set-language-info-alist
  "Korean" '((setup-function . setup-korean-environment-internal)
 	    (exit-function . exit-korean-environment)

Yidong and Stefan, I have not yet installed this change
because it is a fix for the bug existing in Emacs 22.  But,
the change itself is very safe and the effect is that Emacs
can correctly decode all CP949 files some of which were not
decodable previously.  In addition, it seems that CP949 is
very important for Korean Windows users.  Shall I install it
now?

---
Kenichi Handa
handa@m17n.org

Re: cp949 support

[Chong Yidong]

Kenichi Handa <handa@m17n.org> writes:

> Yidong and Stefan, I have not yet installed this change
> because it is a fix for the bug existing in Emacs 22.  But,
> the change itself is very safe and the effect is that Emacs
> can correctly decode all CP949 files some of which were not
> decodable previously.  In addition, it seems that CP949 is
> very important for Korean Windows users.  Shall I install it
> now?

Please do.

Re: cp949 support

[User Ew]

[-- Attachment #1: Type: text/plain, Size: 574 bytes --]

Chong Yidong <cyd@stupidchicken.com> writes:

> Kenichi Handa <handa@m17n.org> writes:
>
>> Yidong and Stefan, I have not yet installed this change
>> because it is a fix for the bug existing in Emacs 22.  But,
>> the change itself is very safe and the effect is that Emacs
>> can correctly decode all CP949 files some of which were not
>> decodable previously.  In addition, it seems that CP949 is
>> very important for Korean Windows users.  Shall I install it
>> now?
>
> Please do.

How about the following patch, to complete korean language information
(C-h L Korean)?

[-- Attachment #2: korean.diff --]
[-- Type: text/x-patch, Size: 830 bytes --]

diff --git a/lisp/language/korean.el b/lisp/language/korean.el
index e8e78bf..07382df 100644
--- a/lisp/language/korean.el
+++ b/lisp/language/korean.el
@@ -70,11 +70,11 @@
 	    (exit-function . exit-korean-environment)
 	    (iso639-language . ko)
 	    (tutorial . "TUTORIAL.ko")
-	    (charset korean-ksc5601)
-	    (coding-system iso-2022-kr korean-iso-8bit)
+	    (charset korean-ksc5601 cp949)
+	    (coding-system iso-2022-kr korean-iso-8bit korean-cp949)
 	    (input-method . "korean-hangul")
 	    (features korea-util)
-	    (coding-priority korean-iso-8bit iso-2022-kr)
+	    (coding-priority korean-iso-8bit korean-cp949 iso-2022-kr)
 	    (sample-text . "Hangul (한글)	안녕하세요, 안녕하십니까")
 	    (documentation . "\
 The following key bindings are available for controlling Korean input methods:

[-- Attachment #3: Type: text/plain, Size: 18 bytes --]


-- 
Eduard Wiebe

Re: cp949 support

[Kenichi Handa]

As Yidong has already installed the fix for cp949
coding-system, could you please verify that it works
correctly with the latest code.

---
Kenichi Handa
handa@m17n.org

Re: cp949 support

[Eduard Wiebe]

[-- Attachment #1: Type: text/plain, Size: 458 bytes --]

Kenichi Handa <handa@m17n.org> writes:

> As Yidong has already installed the fix for cp949
> coding-system, could you please verify that it works
> correctly with the latest code.

It works as expected. Thank you.

PS: Sorry when i recur, but can we add following patch to improve
information about korean language/environment?

2009-06-20  Eduard Wiebe  <usenet@pusto.de>

	* language/korean.el (set-language-info-alist): Add korean-cp949,
	cp949 to spec.

[-- Attachment #2: korean.diff --]
[-- Type: text/x-patch, Size: 830 bytes --]

diff --git a/lisp/language/korean.el b/lisp/language/korean.el
index e8e78bf..07382df 100644
--- a/lisp/language/korean.el
+++ b/lisp/language/korean.el
@@ -70,11 +70,11 @@
 	    (exit-function . exit-korean-environment)
 	    (iso639-language . ko)
 	    (tutorial . "TUTORIAL.ko")
-	    (charset korean-ksc5601)
-	    (coding-system iso-2022-kr korean-iso-8bit)
+	    (charset korean-ksc5601 cp949)
+	    (coding-system iso-2022-kr korean-iso-8bit korean-cp949)
 	    (input-method . "korean-hangul")
 	    (features korea-util)
-	    (coding-priority korean-iso-8bit iso-2022-kr)
+	    (coding-priority korean-iso-8bit korean-cp949 iso-2022-kr)
 	    (sample-text . "Hangul (한글)	안녕하세요, 안녕하십니까")
 	    (documentation . "\
 The following key bindings are available for controlling Korean input methods:

[-- Attachment #3: Type: text/plain, Size: 17 bytes --]

-- 
Eduard Wiebe

Re: cp949 support

[Kenichi Handa]

In article <863a9vb8jg.fsf@nirvana.pusto.de>, Eduard Wiebe <usenet@pusto.de> writes:

> > As Yidong has already installed the fix for cp949
> > coding-system, could you please verify that it works
> > correctly with the latest code.

> It works as expected. Thank you.

Thank you for testing it.

> PS: Sorry when i recur, but can we add following patch to improve
> information about korean language/environment?

> 2009-06-20  Eduard Wiebe  <usenet@pusto.de>

> 	* language/korean.el (set-language-info-alist): Add korean-cp949,
> 	cp949 to spec.

As the suggested change is quite safe, I don't object to
installing it.  Yidong and Stefan?

---
Kenichi Handa
handa@m17n.org

> --=-=-=
> Content-Type: text/x-patch; charset=utf-8
> Content-Disposition: inline; filename=korean.diff
> Content-Transfer-Encoding: 8bit

> diff --git a/lisp/language/korean.el b/lisp/language/korean.el
> index e8e78bf..07382df 100644
> --- a/lisp/language/korean.el
> +++ b/lisp/language/korean.el
> @@ -70,11 +70,11 @@
>  	    (exit-function . exit-korean-environment)
>  	    (iso639-language . ko)
>  	    (tutorial . "TUTORIAL.ko")
> -	    (charset korean-ksc5601)
> -	    (coding-system iso-2022-kr korean-iso-8bit)
> +	    (charset korean-ksc5601 cp949)
> +	    (coding-system iso-2022-kr korean-iso-8bit korean-cp949)
>  	    (input-method . "korean-hangul")
>  	    (features korea-util)
> -	    (coding-priority korean-iso-8bit iso-2022-kr)
> +	    (coding-priority korean-iso-8bit korean-cp949 iso-2022-kr)
>  	    (sample-text . "Hangul (한글)	안녕하세요, 안녕하십니까")
>  	    (documentation . "\
>  The following key bindings are available for controlling Korean input methods:

> --=-=-=

> -- 
> Eduard Wiebe

> --=-=-=--

Re: cp949 support

[Kenichi Handa]

In article <863a9vb8jg.fsf@nirvana.pusto.de>, Eduard Wiebe <usenet@pusto.de> writes:

> PS: Sorry when i recur, but can we add following patch to improve
> information about korean language/environment?

> 2009-06-20  Eduard Wiebe  <usenet@pusto.de>

> 	* language/korean.el (set-language-info-alist): Add korean-cp949,
> 	cp949 to spec.

I've just installed that change to the trunk and the 23.1
branch.

---
Kenichi Handa
handa@m17n.org

> --=-=-=
> Content-Type: text/x-patch; charset=utf-8
> Content-Disposition: inline; filename=korean.diff
> Content-Transfer-Encoding: 8bit

> diff --git a/lisp/language/korean.el b/lisp/language/korean.el
> index e8e78bf..07382df 100644
> --- a/lisp/language/korean.el
> +++ b/lisp/language/korean.el
> @@ -70,11 +70,11 @@
>  	    (exit-function . exit-korean-environment)
>  	    (iso639-language . ko)
>  	    (tutorial . "TUTORIAL.ko")
> -	    (charset korean-ksc5601)
> -	    (coding-system iso-2022-kr korean-iso-8bit)
> +	    (charset korean-ksc5601 cp949)
> +	    (coding-system iso-2022-kr korean-iso-8bit korean-cp949)
>  	    (input-method . "korean-hangul")
>  	    (features korea-util)
> -	    (coding-priority korean-iso-8bit iso-2022-kr)
> +	    (coding-priority korean-iso-8bit korean-cp949 iso-2022-kr)
>  	    (sample-text . "Hangul (한글)	안녕하세요, 안녕하십니까")
>  	    (documentation . "\
>  The following key bindings are available for controlling Korean input methods:

> --=-=-=

> -- 
> Eduard Wiebe

> --=-=-=--