coding systems and input methods are non-intuitive stuff

coding systems and input methods are non-intuitive stuff

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 961 bytes --]

C-x b *scratch* RET
C-x RET f latin-1 RET          ; buffer coding system = latin-1
C-u C-\ romanian-prefix RET    ; input method: romanian-prefix
,s                             ; character: ş (2362, #o4472, #x93a,
                               ;   U+015F)
<left> M-x quail-show-key RET  ; To input `ş', type ",s"
<right>
C-x RET f utf-8 RET            ; buffer coding system = utf-8
                               ; input method: the same as before
,s                             ; character: ş (331903, #o1210177,
                               ;   #x5107f, U+015F)
<left> M-x quail-show-key RET  ; ş can't be input by the current
                               ;   input method

Now, I understand that the buffer code for these characters is not the
same... but it is quite weird nonetheless to input a character with
the current input method, and afterwards be told that it "can't be
input by the current input method".

                    /L/e/k/t/u

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: coding systems and input methods are non-intuitive stuff

[Kevin Rodgers]

Juanma Barranquero wrote:
> C-x b *scratch* RET
> C-x RET f latin-1 RET          ; buffer coding system = latin-1
> C-u C-\ romanian-prefix RET    ; input method: romanian-prefix
> ,s                             ; character: ş (2362, #o4472, #x93a,
>                               ;   U+015F)
> <left> M-x quail-show-key RET  ; To input `ş', type ",s"
> <right>
> C-x RET f utf-8 RET            ; buffer coding system = utf-8
>                               ; input method: the same as before
> ,s                             ; character: ş (331903, #o1210177,
>                               ;   #x5107f, U+015F)
> <left> M-x quail-show-key RET  ; ş can't be input by the current
>                               ;   input method
> 
> Now, I understand that the buffer code for these characters is not the
> same... but it is quite weird nonetheless to input a character with
> the current input method, and afterwards be told that it "can't be
> input by the current input method".

The thing that confuses me is that the ISO 8859-1 character set (which
is what the latin-1 coding system encodes, right?) only contains U+0000
- U+00FF.  So how does U+015F get inserted into a latin-1 buffer?

-- 
Kevin Rodgers
Denver, Colorado, USA

Re: coding systems and input methods are non-intuitive stuff

[David Kastrup]

Kevin Rodgers <kevin.d.rodgers@gmail.com> writes:

> Juanma Barranquero wrote:
>> C-x b *scratch* RET
>> C-x RET f latin-1 RET          ; buffer coding system = latin-1
>> C-u C-\ romanian-prefix RET    ; input method: romanian-prefix
>> ,s                             ; character: ş (2362, #o4472, #x93a,
>>                               ;   U+015F)
>> <left> M-x quail-show-key RET  ; To input `ş', type ",s"
>> <right>
>> C-x RET f utf-8 RET            ; buffer coding system = utf-8
>>                               ; input method: the same as before
>> ,s                             ; character: ş (331903, #o1210177,
>>                               ;   #x5107f, U+015F)
>> <left> M-x quail-show-key RET  ; ş can't be input by the current
>>                               ;   input method
>>
>> Now, I understand that the buffer code for these characters is not the
>> same... but it is quite weird nonetheless to input a character with
>> the current input method, and afterwards be told that it "can't be
>> input by the current input method".
>
> The thing that confuses me is that the ISO 8859-1 character set (which
> is what the latin-1 coding system encodes, right?) only contains U+0000
> - U+00FF.  So how does U+015F get inserted into a latin-1 buffer?

There is no such thing as a "latin-1 buffer" in Emacs.  Buffers are
always encoded in emacs-mule (well, we still have something called
"unibyte" buffers, but they are really only for binary data).  The
buffer contains characters.  Code points of a particular coding system
are only associated when saving, loading, communicating with
processes, X selections, networks, keyboards, terminals.  All those
operations have their own coding systems.  The buffer itself hasn't.

-- 
David Kastrup

Re: coding systems and input methods are non-intuitive stuff

[Juanma Barranquero]

On 1/30/07, Kevin Rodgers <kevin.d.rodgers@gmail.com> wrote:

> So how does U+015F get inserted into a latin-1 buffer?

C-x RET f is `set-buffer-file-coding-system'. In other words, it will
apply when I try to save the buffer; and then possibly I'll get a
message saying that the buffer's contents cannot be saved with that
coding system. But that is irrelevant. There's nothing wrong with

 ;; insert whatever stuff
 ;; set buffer coding system 1
 ;; insert stuff not representable in coding system 1
 ;; set buffer coding system 2
 ;; save the file

There's no requirement that the contents of a buffer *must* be
writable with the current buffer coding system, unless and until I try
to save it...

But that's a digression. What I'm interested in knowing is whether
being able to do

  ;; insert a char with the current input method
  ;; M-x quail-show-key => "X can't be input with the current input method"

(as in my previous example) is a bug or a "feature" of the
buffer-coding-system / input method interaction.

                    /L/e/k/t/u

Re: coding systems and input methods are non-intuitive stuff

[Kenichi Handa]

Very sorry for the late response.

In article <f7ccd24b0701260911k6209e858y54c948d52ba1d2d0@mail.gmail.com>, "Juanma Barranquero" <lekktu@gmail.com> writes:

> C-x b *scratch* RET
> C-x RET f latin-1 RET          ; buffer coding system = latin-1
> C-u C-\ romanian-prefix RET    ; input method: romanian-prefix
> ,s                             ; character: ş (2362, #o4472, #x93a,
>                                ;   U+015F)
> <left> M-x quail-show-key RET  ; To input `ş', type ",s"
> <right>
> C-x RET f utf-8 RET            ; buffer coding system = utf-8
>                                ; input method: the same as before
> ,s                             ; character: ş (331903, #o1210177,
>                                ;   #x5107f, U+015F)
> <left> M-x quail-show-key RET  ; ş can't be input by the current
>                                ;   input method

> Now, I understand that the buffer code for these characters is not the
> same... but it is quite weird nonetheless to input a character with
> the current input method, and afterwards be told that it "can't be
> input by the current input method".

The reason why the second quail-show-key fails is that the
input method is going to insert latin-2 character but as the
buffer-file-coding-system is changed to utf-8, the character
is translated by translation-table-for-input into
mule-unicode-0100-24ff character.  Unfortunately as the data
structure of the input method still contains only latin-2
character, the current code of quail-show-key thinks the
translated character is not in the data structure.

I'm now trying to find a good way to solve this problem.

---
Kenichi Handa
handa@m17n.org

Re: coding systems and input methods are non-intuitive stuff

[Juanma Barranquero]

On 1/30/07, Kenichi Handa <handa@m17n.org> wrote:

> Very sorry for the late response.

There was no hurry.

> I'm now trying to find a good way to solve this problem.

Good luck, and thanks for looking into it.

                    /L/e/k/t/u

Re: coding systems and input methods are non-intuitive stuff

[Kenichi Handa]

In article <f7ccd24b0701300458x3b43111cp4d0facd20a5e9d52@mail.gmail.com>, "Juanma Barranquero" <lekktu@gmail.com> writes:

> On 1/30/07, Kenichi Handa <handa@m17n.org> wrote:
> > Very sorry for the late response.

> There was no hurry.

> > I'm now trying to find a good way to solve this problem.

> Good luck, and thanks for looking into it.

I've just installed a fix that re-generate a decode map if
necessary.

---
Kenichi Handa
handa@m17n.org

Re: coding systems and input methods are non-intuitive stuff

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 395 bytes --]

On 1/31/07, Kenichi Handa <handa@m17n.org> wrote:

> I've just installed a fix that re-generate a decode map if
> necessary.

Cool, thanks.

The UCS input method (which "is not really a Quail method", according
to its comments) still acts weird, though:

C-u C-\ latin-1-postfix RET
a'
C-u C-\ ucs RET
<left>
M-x quail-show-key RET

 => "To input `á', type "a'"

                    /L/e/k/t/u

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: coding systems and input methods are non-intuitive stuff

[Kenichi Handa]

In article <f7ccd24b0701301812q7d877346kce19096f7b17c496@mail.gmail.com>, "Juanma Barranquero" <lekktu@gmail.com> writes:

> The UCS input method (which "is not really a Quail method", according
> to its comments) still acts weird, though:

> C-u C-\ latin-1-postfix RET
> a'
> C-u C-\ ucs RET
> <left>
> M-x quail-show-key RET

>>> "To input `á', type "a'"

Ummm, I've just installed a change that signals this error
in such a case:
	The current input method is not using Quail

---
Kenichi Handa
handa@m17n.org

Re: coding systems and input methods are non-intuitive stuff

[Juanma Barranquero]

On 2/1/07, Kenichi Handa <handa@m17n.org> wrote:

> Ummm, I've just installed a change that signals this error
> in such a case:

Thanks.
                    /L/e/k/t/u