Problem with viper-replace-char and Swedish characters on w32

Problem with viper-replace-char and Swedish characters on w32

[Lennart Borgman]

This is for Emacs on w32. There is a problem in viper when replacing a 
character (using the "r" command - `viper-repace-char') with a Swedish 
character. (For example ö which is an o with two dots above.)

I am using Swedish keyboard layout. Under w2k this is set in Control 
Panel - Regional Settings - Input Locales - Keyboard Layout. I have 
tested two different locales, which gives different results below. The 
locales are found under ... - Regional Settings - General - Your Locale.

I start with

emacs -Q

enter some characters and then do

M-x viper-mode <RET>

I move point to somewhere in the text and then press "r" and a 
character. This replaces a single character with the new one I typed. I 
am doing this in the *scratch* buffer. I get the following results:

1) When I use local Swedish the correct character is inserted (o with 
two dots above). Trying to save the buffer (C-x C-w) I get a message saying:

 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
These default coding systems were tried to encode text
in the buffer `oe-local-sv.txt':
iso-latin-1-dos
However, each of them encountered these problematic characters:
iso-latin-1-dos: ⃶ ෶਀
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Where the question marks I see above in Thunderbird looks as o with two 
dots in Emacs.

2) When I use local English (United States) I see \366 in the buffer. 
Trying to save the buffer (C-x C-w) I get a message saying:

 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
These default coding systems were tried to encode text
in the buffer `oe-local-en.txt':
iso-latin-1
However, each of them encountered these problematic characters:
iso-latin-1: ෶਀
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Here the question marks looks like \366 in Emacs.

-------------------------------
I am not sure where the problem is. I suggested earlier a change in 
`viper-special-read-and-insert-char'. This seems to cure the problem, 
but I suspects something totally different might be the correct cure.

diff -c \eclean\bld\emacs\lisp\emulation/viper-cmd.el ./viper-cmd.el
> *** \eclean\bld\emacs\lisp\emulation/viper-cmd.el	2005-12-09 21:29:12.848928500 +0100
> --- ./viper-cmd.el	2006-01-27 22:38:43.472822400 +0100
> ***************
> *** 872,879 ****
>   	       (if (eq ch ?\C-m) (setq ch ?\n))
>   	       ;; Make sure ^V and ^Q work as quotation chars
>   	       (if (memq ch '(?\C-v ?\C-q))
> ! 		   (setq ch (read-char)))
> ! 	       (insert ch))
>   	      )
>   	(setq last-command-event
>   	      (viper-copy-event (if viper-xemacs-p
> --- 872,882 ----
>   	       (if (eq ch ?\C-m) (setq ch ?\n))
>   	       ;; Make sure ^V and ^Q work as quotation chars
>   	       (if (memq ch '(?\C-v ?\C-q))
> ! 		   (setq ch (read-char-exclusive)))
> ! 	       ;;(insert ch)
> !                (let ((last-command-char ch))
> !                  (self-insert-command 1))
> !                )
>   	      )
>   	(setq last-command-event
>   	      (viper-copy-event (if viper-xemacs-p
> 

Re: Problem with viper-replace-char and Swedish characters on w32

[Lennart Borgman]

Lennart Borgman wrote:
> This is for Emacs on w32. There is a problem in viper when replacing a 
> character (using the "r" command - `viper-repace-char') with a Swedish 
> character. (For example ö which is an o with two dots above.)
>
> I am using Swedish keyboard layout. Under w2k this is set in Control 
> Panel - Regional Settings - Input Locales - Keyboard Layout. I have 
> tested two different locales, which gives different results below. The 
> locales are found under ... - Regional Settings - General - Your Locale.
>
> ...
Since no one has answered I will try to restate the question in a more 
general and simple manner: The user is prompted for a single character 
and this character should then be inserted in the buffer. The question 
is how this should be done. The current code in viper does AFAICS 
something similar to

    (insert (read-char "char: "))

This gives incorrect display of character if it is a Swedish character 
(like o with two dots above) under the conditions I explained before 
(Swedish keyboard, Local English). The code below gives the desired result:

    (let ((last-command-char (read-char "c:"))) (self-insert-command 1))


Is this the correct way to do it?

Re: Problem with viper-replace-char and Swedish characters on w32

[Stefan Monnier]

> This gives incorrect display of character if it is a Swedish character (like
> o with two dots above) under the conditions I explained before (Swedish
> keyboard, Local English). The code below gives the desired result:

>    (let ((last-command-char (read-char "c:"))) (self-insert-command 1))

This will not obey keyboard-coding-system but
nonascii-translation-table instead.  So it will work OK sometimes, but is
not what you want.  But note that this is also what quoted-insert uses, so
it's no worse than the normal C-q.


        Stefan

Re: Problem with viper-replace-char and Swedish characters on w32

[Lennart Borgman]

Stefan Monnier wrote:
>> This gives incorrect display of character if it is a Swedish character (like
>> o with two dots above) under the conditions I explained before (Swedish
>> keyboard, Local English). The code below gives the desired result:
>>     
>
>   
>>    (let ((last-command-char (read-char "c:"))) (self-insert-command 1))
>>     
>
> This will not obey keyboard-coding-system but
> nonascii-translation-table instead.  So it will work OK sometimes, but is
> not what you want.  But note that this is also what quoted-insert uses, so
> it's no worse than the normal C-q.
>
>
>         Stefan
>   
Thanks, Stefan. Is something like this what should be done then:

    (let* ((ch (read-char "cs:")))
      (insert (decode-coding-string (char-to-string ch)
                                    (keyboard-coding-system))))

--lennart

Re: Problem with viper-replace-char and Swedish characters on w32

[Stefan Monnier]

> Thanks, Stefan.  Is something like this what should be done then:

>    (let* ((ch (read-char "cs:")))
>      (insert (decode-coding-string (char-to-string ch)
>                                    (keyboard-coding-system))))

This also has its share of problems since keyboard-coding-system may be
multibyte, in which case you might need to read more than 1 char ;-)

  (insert (aref (read-key-sequence) 0))

is another approch which will correctly solve those problems, but will
suffer from yet others: typically after hitting things like C-c or C-x Emacs
will just sit still, waiting for more input before processing it.
You can get closer with

  (insert (let ((overriding-terminal-local-map (make-sparse-keymap)))
            (aref (read-key-sequence) 0)))

but C-x will still hang because of the C-x 8 prefix in key-translation-map,
and similarly for ESC because of the bindings in function-key-map.

I'm not really sure what should be the right answer... I'm not even sure
what should be the right behavior.


        Stefan

Re: Problem with viper-replace-char and Swedish characters on w32

[Lennart Borgman]

Stefan Monnier wrote:
>> Thanks, Stefan.  Is something like this what should be done then:
>>     
>
>   
>>    (let* ((ch (read-char "cs:")))
>>      (insert (decode-coding-string (char-to-string ch)
>>                                    (keyboard-coding-system))))
>>     
>
> This also has its share of problems since keyboard-coding-system may be
> multibyte, in which case you might need to read more than 1 char ;-)
>
>   (insert (aref (read-key-sequence) 0))
>   
Thanks for the lesson. However it should be (read-key-sequence nil).

> is another approch which will correctly solve those problems, but will
> suffer from yet others: typically after hitting things like C-c or C-x Emacs
> will just sit still, waiting for more input before processing it.
> You can get closer with
>
>   (insert (let ((overriding-terminal-local-map (make-sparse-keymap)))
>             (aref (read-key-sequence) 0)))
>
> but C-x will still hang because of the C-x 8 prefix in key-translation-map,
> and similarly for ESC because of the bindings in function-key-map.
>   
Could key sequences like those produce "readable" characters?

> I'm not really sure what should be the right answer... I'm not even sure
> what should be the right behavior.
>
>
>         Stefan
>   


-- 
Lennart
från Lund
lennart.borgman.073@student.lu.se

Re: Problem with viper-replace-char and Swedish characters on w32

[Stefan Monnier]

>> (insert (aref (read-key-sequence) 0))
> Thanks for the lesson. However it should be (read-key-sequence nil).

Indeed, thank you.

>> but C-x will still hang because of the C-x 8 prefix in key-translation-map,
>> and similarly for ESC because of the bindings in function-key-map.
> Could key sequences like those produce "readable" characters?

ESC maybe not, but C-x 8 very much so: C-x e ' generates an é


        Stefan

Re: Problem with viper-replace-char and Swedish characters on w32

[Lennart Borgman]

Stefan Monnier wrote:
>>> (insert (aref (read-key-sequence) 0))
>>>       
>> Thanks for the lesson. However it should be (read-key-sequence nil).
>>     
>
> Indeed, thank you.
>
>   
>>> but C-x will still hang because of the C-x 8 prefix in key-translation-map,
>>> and similarly for ESC because of the bindings in function-key-map.
>>>       
>> Could key sequences like those produce "readable" characters?
>>     
>
> ESC maybe not, but C-x 8 very much so: C-x e ' generates an é
>
>
>         Stefan
>   
I did not know. I guess you mean C-x 8 ' e. But that actually seems to 
work with your suggestion above.

Is not key-translation-map used before read-key-sequence can see 
anything? I do not know at the moment, but it seems logical.

Re: Problem with viper-replace-char and Swedish characters on w32

[Stefan Monnier]

>> ESC maybe not, but C-x 8 very much so: C-x e ' generates an é
> I did not know. I guess you mean C-x 8 ' e.

Yes, sorry, that's what I meant.

> But that actually seems to work with your suggestion above.

The problem is when the user doesn't want to insert é but C-x (aka ASCII
code 24).


        Stefan

Re: Problem with viper-replace-char and Swedish characters on w32

[Lennart Borgman]

Stefan Monnier wrote:
>> But that actually seems to work with your suggestion above.
>>     
>
> The problem is when the user doesn't want to insert é but C-x (aka ASCII
> code 24).
>
>
>         Stefan
>   
Ah, I misunderstood you. But I think that is not any big problem for the 
"r" command in viper since it is mostly meant to be used for visible 
characters. So maybe a good enough solution is the one you proposed:

    (insert (aref (read-key-sequence nil) 0))

Michael, what do you think?

Re: Problem with viper-replace-char and Swedish characters on w32

[Lennart Borgman]

Lennart Borgman wrote:
> Stefan Monnier wrote:
>>> But that actually seems to work with your suggestion above.
>>>     
>>
>> The problem is when the user doesn't want to insert é but C-x (aka ASCII
>> code 24).
>>
>>
>>         Stefan
>>   
> Ah, I misunderstood you. But I think that is not any big problem for 
> the "r" command in viper since it is mostly meant to be used for 
> visible characters. So maybe a good enough solution is the one you 
> proposed:
>
>    (insert (aref (read-key-sequence nil) 0))
>
> Michael, what do you think?


Hi Michael, did you see this? I suggest changing it to something like 
the above. This at least fixes the bug I have seen with national Swedish 
characters and the vi r command.

It would be very nice if we could fix this before the pretest.