kbd vs read-key-sequence

kbd vs read-key-sequence

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

I'm trying to bind a key which includes the character º (that's U+00BA
[MASCULINE ORDINAL INDICATOR], which is easy to access in a Spanish
keyboard).

ELISP> (event-basic-type (aref (kbd "C-M-º") 0))
2234

;; Now we type C-M-º
ELISP> (event-basic-type (aref (read-key-sequence "Try: ") 0))
186

ELISP> ?º
2234

ELISP> (format "%04X" 186)
"00BA"

This makes things interesting, because

(global-set-key (kbd "C-M-º") 'my-command)

produces a binding which is not recognized by `describe-key'.

             Juanma

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: kbd vs read-key-sequence

[Andreas Schwab]

"Juanma Barranquero" <lekktu@gmail.com> writes:

> ;; Now we type C-M-º
> ELISP> (event-basic-type (aref (read-key-sequence "Try: ") 0))
> 186

Looks like your keyboard does not actually produce C-M-º.  I tried the
same experiment with C-M-ß on a German keyboard, and it works as expected.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: kbd vs read-key-sequence

[Lennart Borgman (gmail)]

Andreas Schwab wrote:
> "Juanma Barranquero" <lekktu@gmail.com> writes:
> 
>> ;; Now we type C-M-º
>> ELISP> (event-basic-type (aref (read-key-sequence "Try: ") 0))
>> 186
> 
> Looks like your keyboard does not actually produce C-M-º.  I tried the
> same experiment with C-M-ß on a German keyboard, and it works as expected.


What OS etc?

Re: kbd vs read-key-sequence

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 336 bytes --]

On 2/28/07, Andreas Schwab <schwab@suse.de> wrote:

> Looks like your keyboard does not actually produce C-M-º.

Hmm, 186 is º, in a way... (186 = 0x00ba)

> I tried the
> same experiment with C-M-ß on a German keyboard, and it works as expected.

I see the same problem with other non-ASCII keys, like ¡ or ¿

             Juanma

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: kbd vs read-key-sequence

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 362 bytes --]

On 3/1/07, Juanma Barranquero <lekktu@gmail.com> wrote:

> I see the same problem with other non-ASCII keys, like ¡ or ¿

This is on Windows XP, BTW. On the same hardware (or, to be fair, on a
VMWare VM running on the same system), Kubuntu 6.10 gets it right:
both `kbd' and `read-key-sequence' returns an event with a basic type
of 2234.

             Juanma

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: kbd vs read-key-sequence

[Stefan Monnier]

>> I see the same problem with other non-ASCII keys, like ¡ or ¿
> This is on Windows XP, BTW.

Looks like a bug in our w32 code, where we somehow fail to decode the chars
before returning them from read-key-sequence.  It sill kinda works probably
thanks to the unibyte-char-to-multibyte hack that takes place in
self-insert-command.


        Stefan

Re: kbd vs read-key-sequence

[Juanma Barranquero]

On 3/1/07, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> Looks like a bug in our w32 code, where we somehow fail to decode the chars
> before returning them from read-key-sequence.

I honestly cannot say whether it is a bug in the w32 code, or the
consequence of the complexity of input decoding (with
keyboard-translate-table, translation-table-for-input,
function-key-map, key-translation-map, coding systems and whatnot).

In any case:

;; Kubuntu 6.10
read-char / read-event => 2234
(aref translation-table-for-input 2234) => 2234
(lookup-key key-translation-map [186]) => nil

;; Windows XP SP2
read-char / read-event => 186
(aref translation-table-for-input 2234) => nil
(lookup-key key-translation-map [186]) => encoded-kb-self-insert-ccl

             Juanma

Re: kbd vs read-key-sequence

[Stefan Monnier]

>> Looks like a bug in our w32 code, where we somehow fail to decode the chars
>> before returning them from read-key-sequence.

> I honestly cannot say whether it is a bug in the w32 code, or the
> consequence of the complexity of input decoding (with
> keyboard-translate-table, translation-table-for-input,
> function-key-map, key-translation-map, coding systems and whatnot).

> In any case:

> ;; Kubuntu 6.10
> read-char / read-event => 2234
> (aref translation-table-for-input 2234) => 2234
> (lookup-key key-translation-map [186]) => nil

> ;; Windows XP SP2
> read-char / read-event => 186
> (aref translation-table-for-input 2234) => nil
> (lookup-key key-translation-map [186]) => encoded-kb-self-insert-ccl

[ BTW, please always specify whether this is in "console mode" or not,
because the way keys are translated can be different in each case. ]

Hmm... so w32 does decoding using keyboard-coding-system (aka
encoded-kb)?

But then why does read-key-sequence return 186 as well?  read-key-sequence
should be using key-translation-map and hence turn it into 2234.

What does keyboard-coding-system say?


        Stefan

Re: kbd vs read-key-sequence

[Juanma Barranquero]

On 3/1/07, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> [ BTW, please always specify whether this is in "console mode" or not,
> because the way keys are translated can be different in each case. ]

Sorry. No console; Windows or X (with GTK, BTW :).

> Hmm... so w32 does decoding using keyboard-coding-system (aka
> encoded-kb)?

I suppose so.

> What does keyboard-coding-system say?

cp1252 on Windows, nil on Kubuntu.

             Juanma

Re: kbd vs read-key-sequence

[Stefan Monnier]

>> [ BTW, please always specify whether this is in "console mode" or not,
>> because the way keys are translated can be different in each case. ]

> Sorry. No console; Windows or X (with GTK, BTW :).

OK, good, that's what I assumed.

>> Hmm... so w32 does decoding using keyboard-coding-system (aka
>> encoded-kb)?
> I suppose so.

>> What does keyboard-coding-system say?
> cp1252 on Windows, nil on Kubuntu.

Hmm... so I'm still wondering why does read-key-sequence return 186 as well?
What does M-: (keyboard-coding-system) say?
What does M-: (char-to-string 186) say?
What does the following piece of code say:

  (let ((str (char-to-string 186))
	(ccl (car (aref (coding-system-spec (keyboard-coding-system)) 4)))
	(vec [nil nil nil nil nil nil nil nil nil])
	result)
    (while (= (length (setq result (ccl-execute-on-string ccl vec str t))) 0)
      (dotimes (i 9) (aset vec i nil))
      (setq str (format "%s%c" str (read-char-exclusive))))
    (vector (aref result 0))))


-- Stefan

Re: kbd vs read-key-sequence

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 662 bytes --]

On 3/2/07, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> Hmm... so I'm still wondering why does read-key-sequence return 186 as well?

> What does M-: (keyboard-coding-system) say?

cp1252

> What does M-: (char-to-string 186) say?

"\272"

> What does the following piece of code say:

[2234]

And this is weird:

          read-char   read-key-sequence

º              186           2234
S-º            170           2218
C-º       67109050       67109050
C-S-º     67109034       67109034
M-º      134217914      134217914
S-M-º    134217898      134217898
C-M-º    201326778      201326778
C-S-M-º  201326762      201326762

             Juanma

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: kbd vs read-key-sequence

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 361 bytes --]

On 3/2/07, Juanma Barranquero <lekktu@gmail.com> wrote:

> º              186           2234
> S-º            170           2218

S-º is of course ª in a Spanish keyboard. The point is,
read-key-sequence differs from read-char on non-modified non-ASCII
chars (ºª¡¿ñÑçÇ etc.), but it produces the same output for modified
chars.

             Juanma

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: kbd vs read-key-sequence

[Jason Rumney]

Juanma Barranquero wrote:
>> º              186           2234
>> S-º            170           2218

Is one of these the cp1252 code point (186/170) and the other the 
internal emacs-mule codepoint?

Re: kbd vs read-key-sequence

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 1664 bytes --]

On 3/2/07, Jason Rumney <jasonr@gnu.org> wrote:

> Juanma Barranquero wrote:
> >> º              186           2234
> >> S-º            170           2218
>
> Is one of these the cp1252 code point (186/170) and the other the
> internal emacs-mule codepoint?

2234 and 2218 are internal emacs-mule codepoints, while 186 and 170
are cp1252 (and also iso-8859-1) codepoints.

      character: º (2234, #o4272, #x8ba, U+00BA)
        charset: latin-iso8859-1 (Right-Hand Part of Latin Alphabet 1
(ISO/IEC 8859-1): ISO-IR-100.)
     code point: #x3A
         syntax: w 	which means: word
       category: l:Latin
    buffer code: #x81 #xBA
      file code: #xC2 #xBA (encoded by coding system mule-utf-8)
        display: by this font (glyph code)
     -outline-DejaVu Sans
Mono-normal-r-normal-normal-13-97-96-96-c-*-iso8859-1 (#xBA)
   Unicode data:
           Name: MASCULINE ORDINAL INDICATOR
       Category: lowercase letter
Combining class: Spacing
  Bidi category: Left-to-Right
  Decomposition: supero

      character: ª (2218, #o4252, #x8aa, U+00AA)
        charset: latin-iso8859-1 (Right-Hand Part of Latin Alphabet 1
(ISO/IEC 8859-1): ISO-IR-100.)
     code point: #x2A
         syntax: w 	which means: word
       category: l:Latin
    buffer code: #x81 #xAA
      file code: #xC2 #xAA (encoded by coding system mule-utf-8)
        display: by this font (glyph code)
     -outline-DejaVu Sans
Mono-normal-r-normal-normal-13-97-96-96-c-*-iso8859-1 (#xAA)
   Unicode data:
           Name: FEMININE ORDINAL INDICATOR
       Category: lowercase letter
Combining class: Spacing
  Bidi category: Left-to-Right
  Decomposition: supera

             Juanma

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: kbd vs read-key-sequence

[Stefan Monnier]

>> Hmm... so I'm still wondering why does read-key-sequence return 186 as well?

>> What does M-: (keyboard-coding-system) say?

> cp1252

>> What does M-: (char-to-string 186) say?

> "\272"

>> What does the following piece of code say:

> [2234]

> And this is weird:

>          read-char   read-key-sequence

> º              186           2234
> S-º            170           2218

Oh, so I misunderstood the original report: read-key-sequence does not
return 186, but 2234, i.e. it's working correctly.

> C-º       67109050       67109050
> C-S-º     67109034       67109034
> M-º      134217914      134217914
> S-M-º    134217898      134217898
> C-M-º    201326778      201326778
> C-S-M-º  201326762      201326762

OK, now I understand.
The fix for it is to change w32 so that it does the decoding earlier, as is
done in X, rather than via encoded-kb.  I believe that changing encoded-kb
to do the decoding on events-with-modifiers is a bit tricky because it's
not always desirable IIRC.



        Stefan

Re: kbd vs read-key-sequence

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 703 bytes --]

On 3/2/07, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> Oh, so I misunderstood the original report: read-key-sequence does not
> return 186, but 2234, i.e. it's working correctly.

Yes. On my first message I wasn't talking about º, but C-M-º, and

  (event-basic-type (aref (read-key-sequence "") 0)) => 186

when you type C-M-º (but 2234 when you type º).

> The fix for it is to change w32 so that it does the decoding earlier, as is
> done in X, rather than via encoded-kb.

Before the release?

It is not the most serious of bugs, but it is a bug nonetheless that:

  (global-set-key (kbd "C-M-º") 'my-command)
  M-x describe-key RET C-M-º => "C-M-º is undefined"

             Juanma

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: kbd vs read-key-sequence

[Jason Rumney]

Juanma Barranquero wrote:
>> The fix for it is to change w32 so that it does the decoding earlier, 
>> as is
>> done in X, rather than via encoded-kb.
>
> Before the release?

I think the time is better spent getting Unicode input working properly 
for Emacs 23. Decoding Windows codepages before it hits Lisp is not my 
idea of time well spent.

Re: kbd vs read-key-sequence

[Juanma Barranquero]

On 3/2/07, Jason Rumney <jasonr@gnu.org> wrote:

> Decoding Windows codepages before it hits Lisp is not my
> idea of time well spent.

Fair enough.

But we should then add a note in PROBLEMS or somewhere saying that
binding some non-ASCII key combinations can be problematic.

             Juanma

Re: kbd vs read-key-sequence

[David Kastrup]

"Juanma Barranquero" <lekktu@gmail.com> writes:

> On 3/2/07, Jason Rumney <jasonr@gnu.org> wrote:
>
>> Decoding Windows codepages before it hits Lisp is not my
>> idea of time well spent.
>
> Fair enough.
>
> But we should then add a note in PROBLEMS or somewhere saying that
> binding some non-ASCII key combinations can be problematic.

IIRC, this is not a problem restricted to Windows.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: kbd vs read-key-sequence

[Stefan Monnier]

> But we should then add a note in PROBLEMS or somewhere saying that
> binding some non-ASCII key combinations can be problematic.

Yeah, probably.  But it's been the case sine Emacs-20.  And some of the
bindings which worked in Emacs-20 failed in Emacs-21, and some of the
binding that work in Emacs-21 will fail in Emacs-22, and the same for
Emacs-23 most likely.  And if you add modifiers it's even more likely
to fail.


        Stefan

Re: kbd vs read-key-sequence

[Juanma Barranquero]

On 3/2/07, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> Yeah, probably.  But it's been the case sine Emacs-20.  And some of the
> bindings which worked in Emacs-20 failed in Emacs-21, and some of the
> binding that work in Emacs-21 will fail in Emacs-22, and the same for
> Emacs-23 most likely.  And if you add modifiers it's even more likely
> to fail.

Still, a note would've saved me a few hours. :)

             Juanma

Re: kbd vs read-key-sequence

[Richard Stallman]

    Yeah, probably.  But it's been the case sine Emacs-20.  And some of the
    bindings which worked in Emacs-20 failed in Emacs-21, and some of the
    binding that work in Emacs-21 will fail in Emacs-22, and the same for
    Emacs-23 most likely.  And if you add modifiers it's even more likely
    to fail.

I was not following this thread.  Can someone summarize the problem
as it is known?  And is it only on Windows, or is it for all systems?
(I've seen both statements made.)

Re: kbd vs read-key-sequence

[Stefan Monnier]

>     Yeah, probably.  But it's been the case sine Emacs-20.  And some of the
>     bindings which worked in Emacs-20 failed in Emacs-21, and some of the
>     binding that work in Emacs-21 will fail in Emacs-22, and the same for
>     Emacs-23 most likely.  And if you add modifiers it's even more likely
>     to fail.

> I was not following this thread.  Can someone summarize the problem
> as it is known?

Here is the problem: when a key is pressed which causes a non-ASCII char to
be inserted in a buffer, the translation from the key-event to the non-ASCII
char can happen at many different levels:
- before the event is passed to Lisp code
- in input-method
- in key-translation-map (using encoded-kb typically)
- in self-insert-command (using unibyte-char-to-multibyte)

Usually, the earlier the better, but historically we started by doing the
most case of the translation (e.g. latin-1) in self-insert-command.
Also the place where it's done depends on the kind of char, the kind of
input method (e.g. XIM or LEIM), whether we're in console mode or w32, or
X11, or ...

On the other side of the problem we have code written in the .emacs file
which tries to bind something to this key.  To work, this has to satisfy
the condition that when Emacs loads the .emacs file, the key it finds in the
define-key call corresponds to the form of the event after
key-translation-map (but before unibyte-char-to-multibyte if applicable).

An additional piece of the puzzle is that the key that Emacs finds in the
define-key call might not be the one that the user sees on her screen when
she edits her .emacs, depending on how she specifies the file's encoding.

Another source of fun is of course the lack/presence of unification which
means that the same glyph or char can have different codes.

And if you want it to work with modifiers, it's yet a bit more interesting
because the base-event is not always decoded (partly because of bugs,
I guess, but in some cases it's done on purpose.  IIRC an example of
purposefully keeping the base event untranslated is for cyrillic
input-method where `c' might map to a cyrillic char, but C-c and M-c should
stay unchanged).


        Stefan

Re: kbd vs read-key-sequence

[Richard Stallman]

    On the other side of the problem we have code written in the .emacs file
    which tries to bind something to this key.  To work, this has to satisfy
    the condition that when Emacs loads the .emacs file, the key it finds in the
    define-key call corresponds to the form of the event after
    key-translation-map (but before unibyte-char-to-multibyte if applicable).

Except for the case of unibyte characters, that is the same as the
character that gets inserted in the buffer if you type the character,
right?  So it should be possible to type in the character code that
C-x = shows you, right?

And you should also be able to insert that character in .emacs,
then save it in whichever coding system.  Why won't that work?

    And if you want it to work with modifiers, it's yet a bit more interesting
    because the base-event is not always decoded

I don't understand that.  What does it mean to "decode the base-event"?

Anyway, if you want to bind a character with modifiers, you can just use
a construct in .emacs that applies the modifier to the desired base
character, such as \M- in a string, or (meta CHAR).

Is there a case where that doesn't work?

Re: kbd vs read-key-sequence

[Stefan Monnier]

>     On the other side of the problem we have code written in the .emacs
>     file which tries to bind something to this key.  To work, this has to
>     satisfy the condition that when Emacs loads the .emacs file, the key
>     it finds in the define-key call corresponds to the form of the event
>     after key-translation-map (but before unibyte-char-to-multibyte if
>     applicable).

> Except for the case of unibyte characters, that is the same as the
> character that gets inserted in the buffer if you type the character,
> right?  So it should be possible to type in the character code that
> C-x = shows you, right?

Almost:
1 - the unibyte case is very common, so the "exception" is important.
2 - there's also translation-table-for-input.  I had suggested to apply this
    directly in read-char but it was rejected, so it's still applied much
    later, in self-insert-command (and a few other commands).

> And you should also be able to insert that character in .emacs,
> then save it in whichever coding system.  Why won't that work?

Off the top of my head:
- saving may change the char thanks to unify-8859-on-encoding
  (e.g. you copy a iso-8859-15 char to your .emacs which you then save in
  iso-8859-1).
- the coding-system may not be properly detected, especially if you
  configure your auto-dtection in your .emacs: the configuration will apply
  to all files you open but not to the loading of .emacs

>     And if you want it to work with modifiers, it's yet a bit more
>     interesting because the base-event is not always decoded
> I don't understand that.  What does it mean to "decode the base-event"?

Turn ?\M-<encoded-é> into ?\M-é.

> Anyway, if you want to bind a character with modifiers, you can just use
> a construct in .emacs that applies the modifier to the desired base
> character, such as \M- in a string, or (meta CHAR).

> Is there a case where that doesn't work?

Because the way the event is decoded through read-key-sequence is not
necessarily the same.  E.g. while ?<encoded-é> may get turned into ?é, it
may be the case that ?\M-<encoded-é> stays unchanged.


        Stefan

Re: kbd vs read-key-sequence

[Richard Stallman]

    Off the top of my head:
    - saving may change the char thanks to unify-8859-on-encoding
      (e.g. you copy a iso-8859-15 char to your .emacs which you then save in
      iso-8859-1).

Is there a procedure we can recommend for the user to avoid this?

    - the coding-system may not be properly detected, especially if you
      configure your auto-dtection in your .emacs: the configuration will apply
      to all files you open but not to the loading of .emacs

Can you avoid this by specifying the coding-system explicitly in .emacs?
We could recommend that users do so.

    > Anyway, if you want to bind a character with modifiers, you can just use
    > a construct in .emacs that applies the modifier to the desired base
    > character, such as \M- in a string, or (meta CHAR).

    > Is there a case where that doesn't work?

    Because the way the event is decoded through read-key-sequence is not
    necessarily the same.  E.g. while ?<encoded-é> may get turned into ?é, it
    may be the case that ?\M-<encoded-é> stays unchanged.

How does that happen?  Maybe that is a bug; if so, we should fix it.

We could recommend that people write (meta ?<encoded-é>).
That would eliminate this particular problem, right?


These do not make things perfect, but I think they might be enough
to enable users to get reliable results.  What do you think?

Re: kbd vs read-key-sequence

[Stefan Monnier]

>     Off the top of my head:
>     - saving may change the char thanks to unify-8859-on-encoding
>     (e.g. you copy a iso-8859-15 char to your .emacs which you then save
>     in iso-8859-1).

> Is there a procedure we can recommend for the user to avoid this?

I guess "only ever use one and only one encoding".  E.g. don't mix latin-1
and latin-9.
(with-silly-mode
 "use utf-8" or better yet "use emacs-unicode")
(although, maybe utf-8 won't solve it all either, because we may end up with
similar problem where one side uses the single char é while the other uses
a composed "e" and "'").

>     - the coding-system may not be properly detected, especially if you
>       configure your auto-dtection in your .emacs: the configuration will
>       apply to all files you open but not to the loading of .emacs

> Can you avoid this by specifying the coding-system explicitly in .emacs?
> We could recommend that users do so.

Yes, with -*- coding -*-.  It should also work correctly as long as they
*don't* specify the encoding via elisp code in their .emacs (with calls to
set-language-environment, for example).

>> Anyway, if you want to bind a character with modifiers, you can just use
>> a construct in .emacs that applies the modifier to the desired base
>> character, such as \M- in a string, or (meta CHAR).

>> Is there a case where that doesn't work?

>     Because the way the event is decoded through read-key-sequence is not
>     necessarily the same.  E.g. while ?<encoded-?> may get turned into ??, it
>     may be the case that ?\M-<encoded-?> stays unchanged.

> How does that happen?

See the OP.  It seems that encoded-kb.el has such a limitation (it should
probably be considered as a bug, but there may be good reasons for it,
I don't know).

> Maybe that is a bug; if so, we should fix it.

That's what this thread is about supposedly.

> We could recommend that people write (meta ?<encoded-?>).
> That would eliminate this particular problem, right?

No.  It would make it work on those systems which have the bug (e.g. w32
right now), but would not work on systems where read-key-sequence correctly
decodes such a key-combination into ?\M-é.

> These do not make things perfect, but I think they might be enough
> to enable users to get reliable results.  What do you think?

I don't think it'll really be reliable, but we can definitely provide
guidelines and hints.


        Stefan

Re: kbd vs read-key-sequence

[Kenichi Handa]

I'm very sorry to being late to join this thread.

In article <jwvk5xmv6sm.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> writes:

> >     Because the way the event is decoded through read-key-sequence is not
> >     necessarily the same.  E.g. while ?<encoded-?> may get turned into ??, it
> >     may be the case that ?\M-<encoded-?> stays unchanged.

> > How does that happen?

> See the OP.  It seems that encoded-kb.el has such a limitation (it should
> probably be considered as a bug, but there may be good reasons for it,
> I don't know).

At first, encoded-kbd-mode is designed to be used in the
case that Emacs is invoked with -nw, and it uses
key-translation-map to handle character events from 0 to 255
(that's all what happens with -nw).

I think the following diagram is the right thing to do:

       window-system        terminal
             |                 |
             | window event    | character event 0..255
             |                 |
       handle_one_xevent    keyboad-coding-system decoder
              \               /
               \ lispy event /
                \           /
                 \         /              
                  read_char 
                      |
                 input method   
                      |
                   keymap

And, it's in my todo list to abolish encoded-kbd-mode and
implement keyboard-coding-system decoder in C for long, but
unfortunately I still don't have a time to do that.

> > Maybe that is a bug; if so, we should fix it.

> That's what this thread is about supposedly.

> > We could recommend that people write (meta ?<encoded-?>).
> > That would eliminate this particular problem, right?

> No.  It would make it work on those systems which have the bug (e.g. w32
> right now), but would not work on systems where read-key-sequence correctly
> decodes such a key-combination into ?\M-é.

As shown in the previous diagram, I think it's a bug of w32
code if it doesn't convert such an event as ?\M-<encoded-?>
into ?\M-é because only a window system generates such an
event and knows how to encode it.

> > These do not make things perfect, but I think they might be enough
> > to enable users to get reliable results.  What do you think?

> I don't think it'll really be reliable, but we can definitely provide
> guidelines and hints.

One guideline is not to use encoded-kbd-mode on window
system.

---
Kenichi Handa
handa@m17n.org

Re: kbd vs read-key-sequence

[Stefan Monnier]

>> >     Because the way the event is decoded through read-key-sequence is
>> >     not necessarily the same.  E.g. while ?<encoded-?> may get turned
>> >     into ??, it may be the case that ?\M-<encoded-?> stays unchanged.

>> > How does that happen?

>> See the OP.  It seems that encoded-kb.el has such a limitation (it should
>> probably be considered as a bug, but there may be good reasons for it,
>> I don't know).

> At first, encoded-kbd-mode is designed to be used in the case that Emacs
> is invoked with -nw, and it uses key-translation-map to handle character
> events from 0 to 255 (that's all what happens with -nw).

That's true, so it's probably the reason for the limitation: all meta keys
are expected to come with an ESC prefix, so the code has no good reason to
handle ?\M-<foo> chars.

> I think the following diagram is the right thing to do:

>        window-system        terminal
>              |                 |
>              | window event    | character event 0..255
>              |                 |
>        handle_one_xevent    keyboad-coding-system decoder
>               \               /
>                \ lispy event /
>                 \           /
>                  \         /
>                   read_char
>                       |
>                  input method
>                       |
>                    keymap

Agreed.

> And, it's in my todo list to abolish encoded-kbd-mode and
> implement keyboard-coding-system decoder in C for long, but
> unfortunately I still don't have a time to do that.

Does it have to be done in C?

> As shown in the previous diagram, I think it's a bug of w32
> code if it doesn't convert such an event as ?\M-<encoded-?>
> into ?\M-é because only a window system generates such an
> event and knows how to encode it.

Agreed.  The bug is that w32 shouldn't be using encoded-kb.

> One guideline is not to use encoded-kbd-mode on window system.

But that's the only available way in w32, IIUC.


        Stefan

Re: kbd vs read-key-sequence

[Kenichi Handa]

In article <jwv7itl7sfl.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> writes:

> > And, it's in my todo list to abolish encoded-kbd-mode and
> > implement keyboard-coding-system decoder in C for long, but
> > unfortunately I still don't have a time to do that.

> Does it have to be done in C?

What I'm thinking is to implement it in read_avail_input()
somewhere before it calls kbd_buffer_store_event () in the
case of ! read_socket_hook.  It may be possible to implement
the decoder in Lisp, but it seems far easier to do that in
C.

> > As shown in the previous diagram, I think it's a bug of w32
> > code if it doesn't convert such an event as ?\M-<encoded-?>
> > into ?\M-é because only a window system generates such an
> > event and knows how to encode it.

> Agreed.  The bug is that w32 shouldn't be using encoded-kb.

> > One guideline is not to use encoded-kbd-mode on window system.

> But that's the only available way in w32, IIUC.

I think the bug should be fixed.  Or, do you mean that it's
impossible for w32 code to do what done in
handle_one_xevent?

---
Kenichi Handa
handa@m17n.org

Re: kbd vs read-key-sequence

[Stefan Monnier]

>> > One guideline is not to use encoded-kbd-mode on window system.
>> But that's the only available way in w32, IIUC.
> I think the bug should be fixed.  Or, do you mean that it's
> impossible for w32 code to do what done in
> handle_one_xevent?

We agree.  But there's noone to implement it, AFAICT.


        Stefan

Re: kbd vs read-key-sequence

[Richard Stallman]

    At first, encoded-kbd-mode is designed to be used in the
    case that Emacs is invoked with -nw, 

I am not sure what that statement means.  Does it mean that

   encoded-kbd-mode was designed originally to be used in -nw

or does it mean that

   encoded-kbd-mode is intended originally to be used in -nw

Which one?

    And, it's in my todo list to abolish encoded-kbd-mode and
    implement keyboard-coding-system decoder in C for long, but
    unfortunately I still don't have a time to do that.

That might be a good thing, but since you can't do it now,
can we fix encoded-kbd-mode to handle modifiers correctly?
It probably is not very hard.

    > > We could recommend that people write (meta ?<encoded-?>).
    > > That would eliminate this particular problem, right?

    > No.  It would make it work on those systems which have the bug (e.g. w32
    > right now), but would not work on systems where read-key-sequence correctly
    > decodes such a key-combination into ?\M-é.

I do not follow that argument.  If users write (meta ?<encoded-?>),
why would that cause any problem anywhere?

    > I don't think it'll really be reliable, but we can definitely provide
    > guidelines and hints.

    One guideline is not to use encoded-kbd-mode on window
    system.

What should be used instead?  If there is nothing that works,
I suggest that people who care about Windows support either fix this
now, or implement something else now.

Re: kbd vs read-key-sequence

[Kenichi Handa]

In article <E1HTACo-0003Q9-5f@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:

>     At first, encoded-kbd-mode is designed to be used in the
>     case that Emacs is invoked with -nw, 

> I am not sure what that statement means.  Does it mean that

>    encoded-kbd-mode was designed originally to be used in -nw

> or does it mean that

>    encoded-kbd-mode is intended originally to be used in -nw

> Which one?

Both.  It was designed to be used in -nw and it still keeps
that design, and thus it is intended to be used in -nw.
Please note that "decoding" means to convert a byte sequence
to a character sequence.  On terminal, what Emacs receives
is surely a byte sequnece.  But on windows, what Emacs
receive is an window event, and it's a responsibility of
window-event handler to convert it to a character event.
And what window-event handler returns is a character,
applying "decoding" on it is not the right thing.

>     And, it's in my todo list to abolish encoded-kbd-mode and
>     implement keyboard-coding-system decoder in C for long, but
>     unfortunately I still don't have a time to do that.

> That might be a good thing, but since you can't do it now,
> can we fix encoded-kbd-mode to handle modifiers correctly?
> It probably is not very hard.

Implementing it in C doesn't solve the current problem.
Still, Windows code should be fixed to generate a correct
(multibyte) character event.  And, once Windows code is
fixed, we don't have to touch encoded-kbd-mode now.

And, changing encoded-kbd-mode is not very hard, but hard
enough to make me hesitate to do before the release.

> > We could recommend that people write (meta ?<encoded-?>).
> > That would eliminate this particular problem, right?

> No.  It would make it work on those systems which have the bug (e.g. w32
> right now), but would not work on systems where read-key-sequence correctly
> decodes such a key-combination into ?\M-é.

> I do not follow that argument.  If users write (meta ?<encoded-?>),
> why would that cause any problem anywhere?

Here, I think what he meant by "(meta ?<encoded-?>)" is a
code something like this ("\351" part is of 4 chars):

(global-set-key [(meta ?\351)] ...)

Then, it works only for w32.  Another way is to visit .emacs
as unibyte file, put "coding: raw-text;" tag, write as below
("\351" part is of 1 eight-bit char), and save it as
raw-text.

(global-set-key [(meta ?\351)] ...)

But, this also works only for w32.

Both depends on that the event ?\M-\351 is generated, but X
generates ?\M-é.

> I don't think it'll really be reliable, but we can definitely provide
> guidelines and hints.

>     One guideline is not to use encoded-kbd-mode on window
>     system.

> What should be used instead?  If there is nothing that works,
> I suggest that people who care about Windows support either fix this
> now, or implement something else now.

I do suggest the former.

I don't know how Windows port handles a keyboad event, but
considering that it depends on encoded-kbd-mode to work,
perhaps it returns a raw-byte (maybe with modifiers)
sequence as a character event sequence.  What it should do
is to decode those raw-bytes by locale-coding-system.

In emacs-unicode-2, I think it is much easier because I
believe Windows itself has a facility to convert keyboard
events to Unicode character.

---
Kenichi Handa
handa@m17n.org

Re: kbd vs read-key-sequence

[Kim F. Storm]

Kenichi Handa <handa@m17n.org> writes:

> (global-set-key [(meta ?\351)] ...)
>
> Then, it works only for w32.  

So it works!!!

As long as there is _some_ way to specify those bindings, I think it
is ok if some crude bindings have to be tweaked a little to work on
w32 with Emacs 22, as was the case for Emacs 20 and 21, too.

After all, I suppose this is a user writing those bindings into his
emacs on w32, and it is trivial to condition those bindings for w32
in case the user is sharing the same .emacs across multiple systems
and platforms.

I'm not even sure we need to document this; if there's an interest,
someone could write about it on the Emacs wiki (it has a W32 section).

> In emacs-unicode-2, I think it is much easier because I
> believe Windows itself has a facility to convert keyboard
> events to Unicode character.

Since the problem will fix itself "real soon now", I propse that
we do nothing about it for now -- it's obviously not a new
problem, and people have managed without a fix for years!!

IOW: Don't delay the release just to fix some minor w32 specific
problem, for which there is a work-around!

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: kbd vs read-key-sequence

[Richard Stallman]

    Here, I think what he meant by "(meta ?<encoded-?>)" is a
    code something like this ("\351" part is of 4 chars):

    (global-set-key [(meta ?\351)] ...)

    Then, it works only for w32.  Another way is to visit .emacs
    as unibyte file, put "coding: raw-text;" tag, write as below
    ("\351" part is of 1 eight-bit char), and save it as
    raw-text.

    (global-set-key [(meta ?\351)] ...)

    But, this also works only for w32.

    Both depends on that the event ?\M-\351 is generated, but X
    generates ?\M-é.

What I have in mind is writing (meta ?é).

The idea is to put the non-ASCII character in .emacs, without
modifiers.  If the user is using a method that handles non-modified
characters correctly, this should extend the method to handle modified
characters as well.

Isn't that valid?

Re: kbd vs read-key-sequence

[Kenichi Handa]

In article <E1HVgGK-00072z-KG@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:

> What I have in mind is writing (meta ?é).

> The idea is to put the non-ASCII character in .emacs, without
> modifiers.  If the user is using a method that handles non-modified
> characters correctly, this should extend the method to handle modified
> characters as well.

> Isn't that valid?

What does "that" refer to; to write (meta ?é), or your
reasoning above?

Anyway, it doesn't work on Windows if .emacs is editted and
saved as a multibyte buffer.

---
Kenichi Handa
handa@m17n.org

Re: kbd vs read-key-sequence

[Richard Stallman]

    > The idea is to put the non-ASCII character in .emacs, without
    > modifiers.  If the user is using a method that handles non-modified
    > characters correctly, this should extend the method to handle modified
    > characters as well.

    > Isn't that valid?

    What does "that" refer to; to write (meta ?é), or your
    reasoning above?

Both.

    Anyway, it doesn't work on Windows if .emacs is editted and
    saved as a multibyte buffer.

Why not?  It seems that the character should be decoded by visiting
the file, and converted into the proper character code.

Re: kbd vs read-key-sequence

[Kenichi Handa]

In article <E1HVyOM-0001Le-H0@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:

>     Anyway, it doesn't work on Windows if .emacs is editted and
>     saved as a multibyte buffer.

> Why not?  It seems that the character should be decoded by visiting
> the file, and converted into the proper character code.

That part is ok.  The problem is that the current Windows
code generates an event M-\351 instead of M-é.

"\351" will be different depending on one's locale.

---
Kenichi Handa
handa@m17n.org

Re: kbd vs read-key-sequence

[Richard Stallman]

    > Why not?  It seems that the character should be decoded by visiting
    > the file, and converted into the proper character code.

    That part is ok.  The problem is that the current Windows
    code generates an event M-\351 instead of M-é.

Isn't that a bug?  It ought to handle the é the same regardless of
the modifiers.

Re: kbd vs read-key-sequence

[Kenichi Handa]

In article <E1HWEki-0007Bx-Lf@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:

> Why not?  It seems that the character should be decoded by visiting
> the file, and converted into the proper character code.

>     That part is ok.  The problem is that the current Windows
>     code generates an event M-\351 instead of M-é.

> Isn't that a bug?  It ought to handle the é the same regardless of
> the modifiers.

I think it's a bug.  It surely handles the é the same
regardless of the modifiers.  In both cases, it doesn't
decode the key event \351 to é.  Windows code ask
encoded-kbd-mode to decode it.  And, although
Encoded-kbd-mode can decode \351 to é, it can't decode
M-\351 to M-é.

---
Kenichi Handa
handa@m17n.org

Re: kbd vs read-key-sequence

[Richard Stallman]

    I think it's a bug.  It surely handles the é the same
    regardless of the modifiers.  In both cases, it doesn't
    decode the key event \351 to é.  Windows code ask
    encoded-kbd-mode to decode it.  And, although
    Encoded-kbd-mode can decode \351 to é, it can't decode
    M-\351 to M-é.

That sounds like a clear bug in encoded-kbd-mode.  Can you fix it
easily?

Re: kbd vs read-key-sequence

[Stefan Monnier]

>     I think it's a bug.  It surely handles the ? the same
>     regardless of the modifiers.  In both cases, it doesn't
>     decode the key event \351 to ?.  Windows code ask
>     encoded-kbd-mode to decode it.  And, although
>     Encoded-kbd-mode can decode \351 to ?, it can't decode
>     M-\351 to M-?.

> That sounds like a clear bug in encoded-kbd-mode.

No, the bug in the use of encoded-kbd-mode.  The decoding should be
done elsewhere, as is done in X11.


        Stefan

Re: kbd vs read-key-sequence

[Richard Stallman]

    >     I think it's a bug.  It surely handles the ? the same
    >     regardless of the modifiers.  In both cases, it doesn't
    >     decode the key event \351 to ?.  Windows code ask
    >     encoded-kbd-mode to decode it.  And, although
    >     Encoded-kbd-mode can decode \351 to ?, it can't decode
    >     M-\351 to M-?.

    > That sounds like a clear bug in encoded-kbd-mode.

    No, the bug in the use of encoded-kbd-mode.

Maybe they are both bugs.

    The decoding should be
    done elsewhere, as is done in X11.

Perhaps it should be, and perhaps after the release we will change
that.  But that has no effect on what to do now.  Right now, this
is done with encoded-kbd-mode, and we are not planning to change that now.

I've argued that encoded-kbd-mode is doing it wrong.
Handa, do you see what I mean?

Re: kbd vs read-key-sequence

[Stefan Monnier]

>     No, the bug in the use of encoded-kbd-mode.

> Maybe they are both bugs.

I don't think so: encoded-kbd-mode was specifically written to handle
terminal input (i.e. sequences of bytes), so it fulfills this role fine.
It's just that the w32 code abuses it by passing it non-terminal events
(i.e. non-bytes).

>     The decoding should be
>     done elsewhere, as is done in X11.

> Perhaps it should be, and perhaps after the release we will change
> that.  But that has no effect on what to do now.  Right now, this
> is done with encoded-kbd-mode, and we are not planning to change that now.

Why should we add hacks to encoded-kbd-mode to work around a bug in the w32
support, rather than fix this w32 code?  Especially since encoded-kbd-mode
is used on every platform, so we would risk introducing bugs for users on
favored platforms like GNU/Linux for the sake of users on w32???

Fixing it right shouldn't be hard, really.  No harder than changing
encoded-kbd-mode.  We just need someone to do it.

If we can't find this someone, then we should probably leave this bug.
After all it's been there in one form or another since Emacs-20.


        Stefan

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Fri, 30 Mar 2007 20:38:24 -0400
> Cc: lekktu@gmail.com, jasonr@gnu.org, emacs-devel@gnu.org, handa@m17n.org
> 
> Why should we add hacks to encoded-kbd-mode to work around a bug in the w32
> support, rather than fix this w32 code?  Especially since encoded-kbd-mode
> is used on every platform, so we would risk introducing bugs for users on
> favored platforms like GNU/Linux for the sake of users on w32???

Full agreement.

On top of this, if Kenichi says that it wouldn't be easy, I submit
that it shouldn't be done at all before the release.

> Fixing it right shouldn't be hard, really.  No harder than changing
> encoded-kbd-mode.  We just need someone to do it.

Actually, I don't think it's as easy as you think (since obviously the
way Windows does non-ASCII keyboard input is radically different from
X11), but it doesn't matter.

> If we can't find this someone, then we should probably leave this bug.
> After all it's been there in one form or another since Emacs-20.

Agreed.  And that's why I installed the PROBLEMS entry suggested by
Kenichi.

Re: kbd vs read-key-sequence

[Richard Stallman]

    On top of this, if Kenichi says that it wouldn't be easy, I submit
    that it shouldn't be done at all before the release.

You are asking for a rush to a decision, based on surmises.
That is the wrong way to handle a problem.  I want to understand
what is happening.

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: monnier@iro.umontreal.ca, lekktu@gmail.com, jasonr@gnu.org,
> 	emacs-devel@gnu.org, handa@m17n.org
> Date: Sat, 31 Mar 2007 19:20:56 -0400
> 
>     On top of this, if Kenichi says that it wouldn't be easy, I submit
>     that it shouldn't be done at all before the release.
> 
> You are asking for a rush to a decision, based on surmises.

That Kenichi said it was not easy isn't a surmise: it's a fact.

Re: kbd vs read-key-sequence

[Richard Stallman]

    > You are asking for a rush to a decision, based on surmises.

    That Kenichi said it was not easy isn't a surmise: it's a fact.

It looks like there are lots of directions to explore.
Past experience leads me to suspect that only some of them
have been considered.  If they are all difficult, another
direction not yet considered might be easier.

Re: kbd vs read-key-sequence

[Stefan Monnier]

> It looks like there are lots of directions to explore.
> Past experience leads me to suspect that only some of them
> have been considered.  If they are all difficult, another
> direction not yet considered might be easier.

There's no indication that doing it right (i.e. doing the coding-system
decoding somewhere in the w32 code which turns w32 events into elisp events)
is difficult.  AFAICT nobody has really tried to do it.


        Stefan

Re: kbd vs read-key-sequence

[Jason Rumney]

Stefan Monnier wrote:
> There's no indication that doing it right (i.e. doing the coding-system
> decoding somewhere in the w32 code which turns w32 events into elisp events)
> is difficult.  AFAICT nobody has really tried to do it.
>   

Anything to do with coding-systems in C code is difficult. Especially 
when almost every encoding on Windows involves autoloaded CCL based 
coding-systems.

Re: kbd vs read-key-sequence

[Stefan Monnier]

>> There's no indication that doing it right (i.e. doing the coding-system
>> decoding somewhere in the w32 code which turns w32 events into elisp events)
>> is difficult.  AFAICT nobody has really tried to do it.

> Anything to do with coding-systems in C code is difficult. Especially when
> almost every encoding on Windows involves autoloaded CCL based
> coding-systems.

It doesn't have to be done in C.  Since decoding may run elisp code, it has
to be done somewhere where lisp can be run, so you may as well do it by
calling an elisp routine from C.  Of course, you have to figure out where to
do it, and make sure that it's indeed a place where lisp can be run.


        Stefan

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> Cc: rms@gnu.org,  Eli Zaretskii <eliz@gnu.org>,  handa@m17n.org,
> 	  emacs-devel@gnu.org
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Sun, 01 Apr 2007 16:35:30 -0400
> 
> > Anything to do with coding-systems in C code is difficult. Especially when
> > almost every encoding on Windows involves autoloaded CCL based
> > coding-systems.
> 
> It doesn't have to be done in C.  Since decoding may run elisp code, it has
> to be done somewhere where lisp can be run, so you may as well do it by
> calling an elisp routine from C.

In the middle of keyboard-reading code, this is a highly complicated
way of committing suicide.  Especially since the w32 keyboard
processing is split between two threads, unlike on other platforms.

> Of course, you have to figure out where to
> do it, and make sure that it's indeed a place where lisp can be run.

Exactly.

Re: kbd vs read-key-sequence

[Kenichi Handa]

In article <4610139D.3060909@gnu.org>, Jason Rumney <jasonr@gnu.org> writes:

> Stefan Monnier wrote:
> > There's no indication that doing it right (i.e. doing the coding-system
> > decoding somewhere in the w32 code which turns w32 events into elisp events)
> > is difficult.  AFAICT nobody has really tried to do it.
> >   

> Anything to do with coding-systems in C code is difficult. Especially 
> when almost every encoding on Windows involves autoloaded CCL based 
> coding-systems.

We don't have to use the mechanism of "decode-coding".
Windows code is already using MultiByteToWideChar() which
seems to convert codepage codes to Unicode character, right?
Then, can't we use it to get a Unicode character code from
keyevent(s) and do something like this?

        MultiByteToWideChar (codepage, 0, inbuf, inlen, outbuf, outlen)
	lisp_char = call2 (intern ("decode-char"), intern ("ucs"),
			    make_number (outbuf[0]));

And, in emacs-unicode-2, you don't have to call "decode-char".

---
Kenichi Handa
handa@m17n.org

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> From: Kenichi Handa <handa@m17n.org>
> CC: monnier@iro.umontreal.ca, rms@gnu.org, eliz@gnu.org, emacs-devel@gnu.org
> Date: Mon, 02 Apr 2007 11:27:41 +0900
> 
> Windows code is already using MultiByteToWideChar() which
> seems to convert codepage codes to Unicode character, right?

Not to Unicode, to UTF-16.

> Then, can't we use it to get a Unicode character code from
> keyevent(s) and do something like this?
> 
>         MultiByteToWideChar (codepage, 0, inbuf, inlen, outbuf, outlen)
> 	lisp_char = call2 (intern ("decode-char"), intern ("ucs"),
> 			    make_number (outbuf[0]));

I don't think this is the right way of getting Unicode codepoints for
keyboard input on Windows; see

     http://msdn2.microsoft.com/en-us/library/ms646267.aspx

(under "Nonsystem Character Messages"), and

     http://msdn2.microsoft.com/en-us/library/ms633586.aspx

for the gory details.

But surely, such a drastic change in the w32 keyboard input processing
is not something to consider before the release!  Especially when the
problems we are trying to fix are so minor, almost invisible.

Re: kbd vs read-key-sequence

[Kenichi Handa]

In article <utzvz56of.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:

> > Then, can't we use it to get a Unicode character code from
> > keyevent(s) and do something like this?
> > 
> >         MultiByteToWideChar (codepage, 0, inbuf, inlen, outbuf, outlen)
> > 	lisp_char = call2 (intern ("decode-char"), intern ("ucs"),
> > 			    make_number (outbuf[0]));

> I don't think this is the right way of getting Unicode codepoints for
> keyboard input on Windows; see

>      http://msdn2.microsoft.com/en-us/library/ms646267.aspx

> (under "Nonsystem Character Messages"), and

>      http://msdn2.microsoft.com/en-us/library/ms633586.aspx

> for the gory details.

Ummm, by reading that page briefly, it seems that getting
Unicode codepoints is much easier.

> But surely, such a drastic change in the w32 keyboard input processing
> is not something to consider before the release!  Especially when the
> problems we are trying to fix are so minor, almost invisible.

I basicly agree.  But, such a change is better than managing
to modify encoded-kbd-mode to do what it shouldn't do.

---
Kenichi Handa
handa@m17n.org

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> From: Kenichi Handa <handa@m17n.org>
> CC: emacs-devel@gnu.org, monnier@iro.umontreal.ca, rms@gnu.org, jasonr@gnu.org
> Date: Mon, 02 Apr 2007 21:31:38 +0900
> 
> >      http://msdn2.microsoft.com/en-us/library/ms646267.aspx
> 
> > (under "Nonsystem Character Messages"), and
> 
> >      http://msdn2.microsoft.com/en-us/library/ms633586.aspx
> 
> > for the gory details.
> 
> Ummm, by reading that page briefly, it seems that getting
> Unicode codepoints is much easier.

Maybe it is (experience tells me that nothing is ever easy on Windows
wrt international support), but some Windows versions (which we still
support) don't have Unicode support built-in.  We don't yet want to
drop support for those systems, surely not because of an obscure
problem of binding META-non-ASCII characters!

> > But surely, such a drastic change in the w32 keyboard input processing
> > is not something to consider before the release!  Especially when the
> > problems we are trying to fix are so minor, almost invisible.
> 
> I basicly agree.  But, such a change is better than managing
> to modify encoded-kbd-mode to do what it shouldn't do.

Well, I think we shouldn't be doing anything at all, neither in
encoded-kbd-mode nor in w32 keyboard input.  I already documented in
PROBLEMS that this is a known problem with Emacs 22.1 on Windows.

Re: kbd vs read-key-sequence

[Stefan Monnier]

> Well, I think we shouldn't be doing anything at all, neither in
> encoded-kbd-mode nor in w32 keyboard input.  I already documented in
> PROBLEMS that this is a known problem with Emacs 22.1 on Windows.

100% agreement.


        Stefan

Re: kbd vs read-key-sequence

[Jason Rumney]

Kenichi Handa wrote:
>>      http://msdn2.microsoft.com/en-us/library/ms633586.aspx
>>     
>> for the gory details.
>>     
>
> Ummm, by reading that page briefly, it seems that getting
> Unicode codepoints is much easier.
>   

It seems easier, but in fact to DTRT according to the documentation 
above means dropping support for Windows 95, 98 and ME, and modifying 
ALL system messages to use Unicode, which is major work. I think it is 
worth doing that for Emacs 22, because Windows 98 and ME are almost dead 
now, and probably will be by the time Emacs 22 s released, and also with 
the change of Emacs' internal representation, Unicode fits better. But 
now I think the best approach for this problem is to add an entry to 
PROBLEMS and get on with the release.

Re: kbd vs read-key-sequence

[Davis Herring]

> It seems easier, but in fact to DTRT according to the documentation
> above means dropping support for Windows 95, 98 and ME, and modifying
> ALL system messages to use Unicode, which is major work. I think it is
> worth doing that for Emacs 22, because Windows 98 and ME are almost dead
> now, and probably will be by the time Emacs 22 s released, and also with
> the change of Emacs' internal representation, Unicode fits better. But
> now I think the best approach for this problem is to add an entry to
> PROBLEMS and get on with the release.

Just to be certain:
Surely you mean 23 instead of 22, especially with the mention of Unicode?

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: kbd vs read-key-sequence

[Jason Rumney]

Davis Herring wrote:

> Just to be certain:
> Surely you mean 23 instead of 22, especially with the mention of Unicode?
>   

Yes, my mistake.

Re: kbd vs read-key-sequence

[Richard Stallman]

Lots of people in poor countries run versions of Windows prior to XP.
XP has activation features that cause trouble for them.

Whether this means they run Windows 98 or Windows ME, I don't know.
I don't remember all the versions of Windows; was there one in between
ME and XP?

We do not want to encourage them to upgrade.

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: handa@m17n.org, eliz@gnu.org, monnier@iro.umontreal.ca,
> 	emacs-devel@gnu.org
> Date: Tue, 03 Apr 2007 03:46:51 -0400
> 
> Lots of people in poor countries run versions of Windows prior to XP.
> XP has activation features that cause trouble for them.
> 
> Whether this means they run Windows 98 or Windows ME, I don't know.
> I don't remember all the versions of Windows; was there one in between
> ME and XP?

There was Windows 2000, but I don't think many users in poor countries
use it.  I think Windows 9x/ME is the most widespread OS there.

Re: kbd vs read-key-sequence

[Stefan Monnier]

>> Lots of people in poor countries run versions of Windows prior to XP.
>> XP has activation features that cause trouble for them.
>> 
>> Whether this means they run Windows 98 or Windows ME, I don't know.
>> I don't remember all the versions of Windows; was there one in between
>> ME and XP?

> There was Windows 2000, but I don't think many users in poor countries
> use it.  I think Windows 9x/ME is the most widespread OS there.

They'd still have may options left:
1 - Upgrade to a Free operating system.
2 - Use the Cygwin version of Emacs.
3 - Stick to the older Emacs (i.e. presumably Emacs-22).


-- Stefan

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> Cc: rms@gnu.org,  jasonr@gnu.org,  handa@m17n.org,  emacs-devel@gnu.org
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 03 Apr 2007 10:58:08 -0400
> 
> 2 - Use the Cygwin version of Emacs.

Not really an option, since Cygwin doesn't support Windows 9x very
well (if at all), AFAIK.

Re: kbd vs read-key-sequence

[Stefan Monnier]

>> 2 - Use the Cygwin version of Emacs.
> Not really an option, since Cygwin doesn't support Windows 9x very
> well (if at all), AFAIK.

Hmm... oh but wait, don't they get to be able to use the MS-DOS
port instead?


        Stefan

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 03 Apr 2007 13:59:20 -0400
> 
> >> 2 - Use the Cygwin version of Emacs.
> > Not really an option, since Cygwin doesn't support Windows 9x very
> > well (if at all), AFAIK.
> 
> Hmm... oh but wait, don't they get to be able to use the MS-DOS
> port instead?

Isn't that 2 days late?

(MS-DOS port supports only a single non-ASCII locale, and on W9x that
locale cannot be changed without rebooting the machine.)

Re: kbd vs read-key-sequence

[Stefan Monnier]

>> >> 2 - Use the Cygwin version of Emacs.
>> > Not really an option, since Cygwin doesn't support Windows 9x very
>> > well (if at all), AFAIK.
>> 
>> Hmm... oh but wait, don't they get to be able to use the MS-DOS
>> port instead?

> Isn't that 2 days late?

> (MS-DOS port supports only a single non-ASCII locale, and on W9x that
> locale cannot be changed without rebooting the machine.)

Still: if they want to use Emacs>22 under Windows 9x, it's an option.


        Stefan

Re: kbd vs read-key-sequence

[Richard Stallman]

    Hmm... oh but wait, don't they get to be able to use the MS-DOS
    port instead?

You forgot the smiley.

Re: kbd vs read-key-sequence

[Richard Stallman]

    3 - Stick to the older Emacs (i.e. presumably Emacs-22).

That is the bad consequence I want to avoid.

Re: kbd vs read-key-sequence

[Jason Rumney]

Richard Stallman wrote:
> Lots of people in poor countries run versions of Windows prior to XP.
> XP has activation features that cause trouble for them.
>   
I got the Emacs version numbers wrong. I wasn't suggesting to drop 
support for them immediately, but it is increasingly difficult to 
continue to support them when no developers are using them, and we 
probably need to make some major changes for Emacs 23 which are easier 
and produce better results if we use Unicode APIs.

> Whether this means they run Windows 98 or Windows ME, I don't know.
> I don't remember all the versions of Windows; was there one in between
> ME and XP?
>   
There were two strands of Windows, 95/98/ME which is not Unicode based 
and supports unicode only for a few limited uses, and Windows 
NT/2000/XP/Vista, which is Unicode based. From Windows XP, the first 
strand was dropped and replaced with a crippled version of the second 
strand.


> We do not want to encourage them to upgrade.

Surely we do want to encourage them to upgrade, to a Free OS?

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=failed 
> 	version=3.1.0
> Date: Tue, 03 Apr 2007 10:05:14 +0100
> From: Jason Rumney <jasonr@gnu.org>
> Cc: handa@m17n.org, eliz@gnu.org, monnier@iro.umontreal.ca,
> 	emacs-devel@gnu.org
> 
> There were two strands of Windows, 95/98/ME which is not Unicode based 
> and supports unicode only for a few limited uses, and Windows 
> NT/2000/XP/Vista, which is Unicode based.

Windows 9x has a Unicode add-on that can be downloaded from the
Microsoft site and installed.  Does that add-on (MSLU) supply the
functionality that we need for Unicode keyboard input?  From what I
glean from the MS documentation, it does, but maybe I'm missing
something.

> From Windows XP, the first strand was dropped and replaced with a
> crippled version of the second strand.

I don't think I understand what you mean by this.  Are you referring
to MSLU that is the ``crippled version'' of Unicode support that MS
now suggest that all Windows applications use?

Re: kbd vs read-key-sequence

[Jason Rumney]

Eli Zaretskii wrote:

>> From Windows XP, the first strand was dropped and replaced with a
>> crippled version of the second strand.
>>     
>
> I don't think I understand what you mean by this.  Are you referring
> to MSLU that is the ``crippled version'' of Unicode support that MS
> now suggest that all Windows applications use?
>   

No, I mean "XP Home" is a crippled version of the NT strand of Windows, 
which replaced the 95 strand.

Re: kbd vs read-key-sequence

[Richard Stallman]

We are hoping to release Emacs 23 in not terribly long -- and the situation
with widespread use of pre-XP versions of Windows is likely not to change
for years.

    > We do not want to encourage them to upgrade.

    Surely we do want to encourage them to upgrade, to a Free OS?

If we see, in the coming months, that the tendency is to update to a
free OS rather than downgrade to Windows Vista, then we might conclude
it is good to encouage them.

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> Date: Mon, 02 Apr 2007 17:14:51 +0100
> From: Jason Rumney <jasonr@gnu.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org,
> 	monnier@iro.umontreal.ca, rms@gnu.org
> 
> Kenichi Handa wrote:
> >>      http://msdn2.microsoft.com/en-us/library/ms633586.aspx
> >>     
> >> for the gory details.
> >>     
> >
> > Ummm, by reading that page briefly, it seems that getting
> > Unicode codepoints is much easier.
> >   
> 
> It seems easier, but in fact to DTRT according to the documentation 
> above means dropping support for Windows 95, 98 and ME, and modifying 
> ALL system messages to use Unicode, which is major work. I think it is 
> worth doing that for Emacs 22, because Windows 98 and ME are almost dead 
> now, and probably will be by the time Emacs 22 s released, and also with 
> the change of Emacs' internal representation, Unicode fits better.

I added an entry to etc/TODO based on this.

Re: kbd vs read-key-sequence

[Richard Stallman]

    > It seems easier, but in fact to DTRT according to the documentation 
    > above means dropping support for Windows 95, 98 and ME, and modifying 
    > ALL system messages to use Unicode, which is major work. I think it is 
    > worth doing that for Emacs 22, because Windows 98 and ME are almost dead 
    > now, and probably will be by the time Emacs 22 s released, and also with 
    > the change of Emacs' internal representation, Unicode fits better.

    I added an entry to etc/TODO based on this.

As I've already said, I think this change would probably be a bad
thing.  People can try it and see whether it works well enough
on the old Windows versions.  But if it doesn't, I think it should
not be installed.

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: jasonr@gnu.org, emacs-devel@gnu.org
> Date: Wed, 04 Apr 2007 10:02:08 -0400
> 
>     > It seems easier, but in fact to DTRT according to the documentation 
>     > above means dropping support for Windows 95, 98 and ME, and modifying 
>     > ALL system messages to use Unicode, which is major work. I think it is 
>     > worth doing that for Emacs 22, because Windows 98 and ME are almost dead 
>     > now, and probably will be by the time Emacs 22 s released, and also with 
>     > the change of Emacs' internal representation, Unicode fits better.
> 
>     I added an entry to etc/TODO based on this.
> 
> As I've already said, I think this change would probably be a bad
> thing.  People can try it and see whether it works well enough
> on the old Windows versions.  But if it doesn't, I think it should
> not be installed.

If you've read what I added to TODO, you know that I never meant to
encourage a change that will not work on Windows 9x.

Re: kbd vs read-key-sequence

[Richard Stallman]

    It's just that the w32 code abuses it by passing it non-terminal events
    (i.e. non-bytes).

I don't understand what you mean by that.  Would you please explain?

    Why should we add hacks to encoded-kbd-mode to work around a bug in the w32
    support, rather than fix this w32 code?

In those abstract terms, both sound good.  But I am not sure
what concrete things you are talking about.  Is one of them
the same as what Handa said was too difficult to do now?

If you propose a different solution that people are
willing to implement now, fine and good.  Otherwise,
you're just arguing for a nonexistent solution, so stop.

    If we can't find this someone, then we should probably leave this bug.

No.

    After all it's been there in one form or another since Emacs-20.

That doesn't change anything.

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Sat, 31 Mar 2007 16:42:31 -0400
> Cc: lekktu@gmail.com, jasonr@gnu.org, emacs-devel@gnu.org, handa@m17n.org
> 
>     After all it's been there in one form or another since Emacs-20.
> 
> That doesn't change anything.

Yes, it does: it tells us that the way things work now, they do the
job quite well, and that the problem which sparked this thread is a
marginal one which users are unlikely to bump into.

IOW, it should not hold the release.

Re: kbd vs read-key-sequence

[Juanma Barranquero]

On 3/31/07, Eli Zaretskii <eliz@gnu.org> wrote:

> and that the problem which sparked this thread is a
> marginal one which users are unlikely to bump into.

I think that anyone who encounters the problem can likely find the workaround...

> IOW, it should not hold the release.

Agreed.

Also, since when a bug in the Windows port is allowed to delay a release? :)

             Juanma

Re: kbd vs read-key-sequence

[Kenichi Handa]

In article <E1HWwcA-0000bQ-L3@fencepost.gnu.org>, Richard Stallman <rms@gnu.org> writes:

>     I think it's a bug.  It surely handles the é the same
>     regardless of the modifiers.  In both cases, it doesn't
>     decode the key event \351 to é.  Windows code ask
>     encoded-kbd-mode to decode it.  And, although
>     Encoded-kbd-mode can decode \351 to é, it can't decode
>     M-\351 to M-é.

> That sounds like a clear bug in encoded-kbd-mode.

Why do you think so?  The task of encoded-kbd-mode is to do
"decode-coding" according to keyboard-coding-system, and
"decode-coding" is to decode a byte sequence into a
character sequence.  But M-\351 is not a byte.

> Can you fix it easily?

It's possible but not easy.

As the other people say, I strongly recommend to add
something like the following text in etc/PROBLEMS.

On Windows, to bind a non-ASCII character with some modifier
to a command, you must specify a raw byte code.  For
instance, if you want to bind meta-a-grave to a command:
  (global-set-key [?\M-\340] ...)
That is for the case of Latin-1 environment where the byte
code of a-grave is \340.

---
Kenichi Handa
handa@m17n.org

Re: kbd vs read-key-sequence

[Stefan Monnier]

> On Windows, to bind a non-ASCII character with some modifier
> to a command, you must specify a raw byte code.  For
> instance, if you want to bind meta-a-grave to a command:
>   (global-set-key [?\M-\340] ...)
> That is for the case of Latin-1 environment where the byte
> code of a-grave is \340.

That sounds right, except I'd add a "currently" somewhere in there.


        Stefan

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 29 Mar 2007 23:25:38 -0400
> Cc: lekktu@gmail.com, jasonr@gnu.org, rms@gnu.org, emacs-devel@gnu.org
> 
> > On Windows, to bind a non-ASCII character with some modifier
> > to a command, you must specify a raw byte code.  For
> > instance, if you want to bind meta-a-grave to a command:
> >   (global-set-key [?\M-\340] ...)
> > That is for the case of Latin-1 environment where the byte
> > code of a-grave is \340.
> 
> That sounds right, except I'd add a "currently" somewhere in there.

I added the entry in the section whose header is "Known problems with
the MS-Windows port of Emacs 22.1".  Since it mentions v22.1
explicitly, I don't think we need to say "currently".

Re: kbd vs read-key-sequence

[Stefan Monnier]

> I added the entry in the section whose header is "Known problems with
> the MS-Windows port of Emacs 22.1".  Since it mentions v22.1
> explicitly, I don't think we need to say "currently".

Great, thanks,


        Stefan

Re: kbd vs read-key-sequence

[Richard Stallman]

      And, although
    >     Encoded-kbd-mode can decode \351 to é, it can't decode
    >     M-\351 to M-é.

    > That sounds like a clear bug in encoded-kbd-mode.

    Why do you think so?

Because it gives the wrong output, and the result is incorrect
execution.

      The task of encoded-kbd-mode is to do
    "decode-coding" according to keyboard-coding-system, and
    "decode-coding" is to decode a byte sequence into a
    character sequence.

In some sense, yes.

			 But M-\351 is not a byte.

So what?  Some sequence of bytes comes in when the user
types M-é.  What sequence is it?

    > Can you fix it easily?

    It's possible but not easy.

Can you explain why it is hard?

    On Windows, to bind a non-ASCII character with some modifier
    to a command, you must specify a raw byte code.  For
    instance, if you want to bind meta-a-grave to a command:
      (global-set-key [?\M-\340] ...)
    That is for the case of Latin-1 environment where the byte
    code of a-grave is \340.

That is really ugly.   Documenting a bug should be the last resort.
First try fixing it.

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> From: Kenichi Handa <handa@m17n.org>
> Date: Fri, 30 Mar 2007 09:47:06 +0900
> Cc: lekktu@gmail.com, emacs-devel@gnu.org, monnier@iro.umontreal.ca,
> 	jasonr@gnu.org
> 
> As the other people say, I strongly recommend to add
> something like the following text in etc/PROBLEMS.
> 
> On Windows, to bind a non-ASCII character with some modifier
> to a command, you must specify a raw byte code.  For
> instance, if you want to bind meta-a-grave to a command:
>   (global-set-key [?\M-\340] ...)
> That is for the case of Latin-1 environment where the byte
> code of a-grave is \340.

Done.

Re: kbd vs read-key-sequence

[Richard Stallman]

    I guess "only ever use one and only one encoding".  E.g. don't mix latin-1
    and latin-9.

Do you mean don't mix encodings in .emacs?  Or do you mean only use
one encoding for all the files you edit?  The former sounds like a fine
solution: we just have to recommend it in the manual.

    > Can you avoid this by specifying the coding-system explicitly in .emacs?
    > We could recommend that users do so.

    Yes, with -*- coding -*-.  It should also work correctly as long as they
    *don't* specify the encoding via elisp code in their .emacs (with calls to
    set-language-environment, for example).

That sounds like a practical recommendation.
Can you (or someone) write it up for the Emacs manual,
then ack?

Re: kbd vs read-key-sequence

[Richard Stallman]

[I sent this message a week ago but did not get a response.
Could we get this moving again?]

    I guess "only ever use one and only one encoding".  E.g. don't mix latin-1
    and latin-9.

Do you mean don't mix encodings in .emacs?  Or do you mean only use
one encoding for all the files you edit?  The former sounds like a fine
solution: we just have to recommend it in the manual.

    > Can you avoid this by specifying the coding-system explicitly in .emacs?
    > We could recommend that users do so.

    Yes, with -*- coding -*-.  It should also work correctly as long as they
    *don't* specify the encoding via elisp code in their .emacs (with calls to
    set-language-environment, for example).

That sounds like a practical recommendation.
Can you (or someone) write it up for the Emacs manual,
then ack?

Re: kbd vs read-key-sequence

[Stefan Monnier]

>     I guess "only ever use one and only one encoding".  E.g. don't mix
>     latin-1 and latin-9.
> Do you mean don't mix encodings in .emacs?

No.  Not just in .emacs.

> Or do you mean only use one encoding for all the files you edit?

And not only that: also for keyboard-coding-system, locale-coding-system,
...

>> Can you avoid this by specifying the coding-system explicitly in .emacs?
>> We could recommend that users do so.

>     Yes, with -*- coding -*-.  It should also work correctly as long as
>     they *don't* specify the encoding via elisp code in their .emacs (with
>     calls to set-language-environment, for example).

> That sounds like a practical recommendation.
> Can you (or someone) write it up for the Emacs manual, then ack?

I have no idea where to put such a thing.


        Stefan

Re: kbd vs read-key-sequence

[Richard Stallman]

    > Or do you mean only use one encoding for all the files you edit?

    And not only that: also for keyboard-coding-system, locale-coding-system,
    ...

keyboard-coding-system says how your keyboard encodes characters.
Of course you should not change that, unless you change your keyboard.

How is the encoding of files you edit relevant to this at all?
How is locale-coding-system relevant to this at all?
They do not affect the decoding of your input.

    >> Can you avoid this by specifying the coding-system explicitly in .emacs?
    >> We could recommend that users do so.

    >     Yes, with -*- coding -*-.  It should also work correctly as long as
    >     they *don't* specify the encoding via elisp code in their .emacs (with
    >     calls to set-language-environment, for example).

    > That sounds like a practical recommendation.
    > Can you (or someone) write it up for the Emacs manual, then ack?

    I have no idea where to put such a thing.

In custom.texi, node Init File.  Perhaps as a new subnode.

Re: kbd vs read-key-sequence

[Stefan Monnier]

>> Or do you mean only use one encoding for all the files you edit?
>     And not only that: also for keyboard-coding-system, locale-coding-system,
>     ...

> keyboard-coding-system says how your keyboard encodes characters.
> Of course you should not change that, unless you change your keyboard.

I didn't mean to change it, but to use the same coding system for it than
the one used for files etc...

> How is the encoding of files you edit relevant to this at all?

It has to match: the event is decoded with keyboard-coding-system whereas
the define-key statement is decoded via the .emacs file's coding system.

> How is locale-coding-system relevant to this at all?

IIRC in some circumstances Emacs uses it to decode keyboard input (IIRC
this is not the case any more in Emacs-22, but I wouldn't bet on it).


        Stefan

Re: kbd vs read-key-sequence

[Richard Stallman]

    > How is the encoding of files you edit relevant to this at all?

    It has to match: the event is decoded with keyboard-coding-system whereas
    the define-key statement is decoded via the .emacs file's coding system.

At most, this could mean that the coding system used for .emacs has to
match keyboard-coding-system.  It has nothing to do with any other
file's coding system.

But I don't even see why keyboard-coding-system has to match the coding
system used for .emacs.  Once .emacs is decoded, what you have is
a non-ASCII character, in Emacs representation.  As long as you reliably
get the character you intended, the coding system of .emacs has no
further relevance to how that character is handled.

So I think your contention is not established.

Re: kbd vs read-key-sequence

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

>     >> Can you avoid this by specifying the coding-system explicitly in .emacs?
>     >> We could recommend that users do so.
>
>     >     Yes, with -*- coding -*-.  It should also work correctly as long as
>     >     they *don't* specify the encoding via elisp code in their .emacs (with
>     >     calls to set-language-environment, for example).
>
>     > That sounds like a practical recommendation.
>     > Can you (or someone) write it up for the Emacs manual, then ack?
>
>     I have no idea where to put such a thing.
>
> In custom.texi, node Init File.  Perhaps as a new subnode.

The Emacs manual already contained advice to use a -*- coding -*-
string in several places.  I have consolidated these, with a few other
recommendations from this thread, into a new "Init Non-ASCII" subnode.

I didn't add any mention of the Windows-specific problems discussed
elsewhere in this thread, though.

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Fri, 30 Mar 2007 19:47:47 -0400
> Cc: lekktu@gmail.com, emacs-devel@gnu.org,
> 	Stefan Monnier <monnier@iro.umontreal.ca>, jasonr@gnu.org
> 
> The Emacs manual already contained advice to use a -*- coding -*-
> string in several places.  I have consolidated these, with a few other
> recommendations from this thread, into a new "Init Non-ASCII" subnode.

Thanks.  But please always run "make info" after such changes to
verify that your changes work.

Re: kbd vs read-key-sequence

[Richard Stallman]

    But we should then add a note in PROBLEMS or somewhere saying that
    binding some non-ASCII key combinations can be problematic.

Yes, would someone please add that, then ack?

Re: kbd vs read-key-sequence

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 477 bytes --]

On 3/2/07, Jason Rumney <jasonr@gnu.org> wrote:

> I think the time is better spent getting Unicode input working properly
> for Emacs 23.

And, in fact, the problem does *not* happen in the Unicode branch. But
the weird thing is:

ELISP> (read-key-sequence "")         ; º
[186]

so either read-key-sequence fails there, or Stefan was wrong about it
working right on the trunk (where it returns [2234])... Or I just
don't understand anything anymore :-)

             Juanma

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: kbd vs read-key-sequence

[Jason Rumney]

Juanma Barranquero wrote:
> ELISP> (read-key-sequence "")         ; º
> [186]
>
> so either read-key-sequence fails there, or Stefan was wrong about it
> working right on the trunk (where it returns [2234])... Or I just
> don't understand anything anymore :-)


The only difference is that Emacs internal encoding is no longer 
emacs-mule, but utf-8. So the code points just happen to coincide in 
this case.

Re: kbd vs read-key-sequence

[Eli Zaretskii]

> Date: Fri, 2 Mar 2007 18:29:33 +0100
> From: "Juanma Barranquero" <lekktu@gmail.com>
> Cc: Emacs Devel <emacs-devel@gnu.org>
> 
> On 3/2/07, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
> > Oh, so I misunderstood the original report: read-key-sequence does not
> > return 186, but 2234, i.e. it's working correctly.
> 
> Yes. On my first message I wasn't talking about º, but C-M-º, and
> 
>   (event-basic-type (aref (read-key-sequence "") 0)) => 186
> 
> when you type C-M-º (but 2234 when you type º).
> 
> > The fix for it is to change w32 so that it does the decoding earlier, as is
> > done in X, rather than via encoded-kb.
> 
> Before the release?
> 
> It is not the most serious of bugs, but it is a bug nonetheless that:
> 
>   (global-set-key (kbd "C-M-º") 'my-command)
>   M-x describe-key RET C-M-º => "C-M-º is undefined"

Handa-san, could you please comment on the findings in this thread?
Are the diagnosis and the proposed solution correct, in your opinion?

TIA