Issues with quail.el

Issues with quail.el

[Amit Ramon]

I believe I found some bugs in quail.el that occurs when the keyboard
layout is set to a layout that is different from the standard
"quail-keyboard-layout-standard". I'd like to try and fix them myself,
and I have some questions.

First issue

Assume that the keyboard layout is set to "atari-german" and that the
input method is "Hebrew". When I type C-h I (thus invoking
"describe-input-method"), the drawing of the keyboard layout is
incorrect. The keys that appear in the shifted level, which should be,
in this input method, similar to those in the standard layout, are
actually taken from the current keyboard layout.

For example, the Hebrew letter TET is on the (standard) y key, and
pressing this key together with shift when Hebrew input method is
active should produce Y. This works fine when typing, but in the map
drawn by "describe-input-method" the letter that appears on this key
in the shifted state is Z, which is what this key does in
"atari-german" layout. There are some other keys not in the right
place - all those that in "atari-german" have a meaning that is
different than the one in the standard layout.

Second issue

The "quail-show-key" function should describe the keys need to be
pressed in order to produce the character at point. When the input
method is Hebrew and the keyboard layout is not the standard, this
functions fails to find the keys for a Hebrew letter that sits on a
key that its meaning in the current keyboard layout is different from
its meaning in the standard layout.

For example, if the keyboard layout is "atari-german", invoking
"quail-show-key" for the Hebrew letter TET produce the message "TET
can't be input by the current input method".

This is because the key on which TET sits is Y in the standard layout
but Z in "atari-german".

Questions and Comments

For the first issue, the problem is with the "quail-insert-kbd-layout"
function. In its first part this functions generate a list of
characters to be drawn, based in the input method. It walks along the
characters of the keyboard layout, try to translate each using
"quail-map" to obtain the input method character. If translation is
not found for a specific key, its value is taken from the *current
layout* given as an argument to the function. This, as far as I
understand, is an error - input methods are based on the standard
keyboard layout, or in other words, they are based on a fixed known
physical layout. Thus, if a certain input method omits specifying a
translation for a key, it expects that the "default" value will be
taken - i.e., the value from the standard layout.

I'd appreciate an advice here - if I'm right about this, it's an easy
fix.

I also have a more general question. This function takes an argument
which is a keyboard layout, and is used in quail exactly in two
places: one for drawing the layout of an input method (this is what I
was referring to so far), and the other is for drawing the layout of a
*keyboard layout*.

The first thing the function does is to translate the given layout to
the standard one, if "(quail-kbd-layout)" is t.  I don't really
understand why this is needed - if input methods are based on the
standard layout, it seems to me that the "quail-insert-kbd-layout"
function don't need to receive a keyboard layout as an argument at
all. It should work directly with the standard layout and generate the
drawing from it (and that would solve the issue I raised).

So what is the situation in which the given keyboard layout should not
be translated back to the standard layout? If this is for drawing a
keyboard layout (not for an input method), then I would suggest to
break this function down to two separate functions.

I would appreciate your input about it.

For the second issue, I believe the problem is with the
"quail-find-key" function. This function takes a character and try to
back-translate it using quail's "decode-map". However, it seems the
"decode-map" is always based on the standard keyboard layout, no
matter what is the currently set layout. Thus, when the current layout
is not the standard, this function will fail for some characters.

I tried to fix it in a simple way - I take the key(s) returned from
the "decode-map", and translate them from the current layout to the
standard layout (a function for the other direction exists, I had to
write a new one). This seem to work.

However, as seen from the code, the decoding can return either a
string or a different type of object (cons maybe?). I'm not sure in
which cases a non-string type is returned and how to handle it.

With your help, I'd be happy to try and write a fix.

Thanks, and sorry for the length of this mail...

Amit

Re: Issues with quail.el

[Eli Zaretskii]

> Date: Sat, 5 May 2018 14:53:57 +0300
> From: Amit Ramon <amit.ramon@riseup.net>
> 
> Assume that the keyboard layout is set to "atari-german" and that the
> input method is "Hebrew". When I type C-h I (thus invoking
> "describe-input-method"), the drawing of the keyboard layout is
> incorrect. The keys that appear in the shifted level, which should be,
> in this input method, similar to those in the standard layout, are
> actually taken from the current keyboard layout.

I'm no expert on this, but does the fact that Dvorak keyboards need
special input methods tell anything useful for your use case?

Re: Issues with quail.el

[Amit Ramon]

Eli Zaretskii <eliz@gnu.org> [2018-05-07 22:33 +0300]:

>
>I'm no expert on this, but does the fact that Dvorak keyboards need
>special input methods tell anything useful for your use case?

Perhaps I wasn't clear enough. I've been using Dvorak keyboard layout
(defined at X level, physical keyboard is standard one) with Hebrew
input method for quite many years, and as for typing, i.e. inputting
Hebrew characters, everything works just fine. Dvorak keyboard doesn't
need special input method - AFAIK quail is designed to work with any
keyboard layout - you just have to tell quail about it.

The problems I'm talking about are "second order" - they are related to
functions that display information about input methods and keyboard
layouts.

I spent quiet a lot of time trying to resolve the exact problem and
how to fix it, but I have to admit that quail is quiet complex so I'd
appreciate some directions from people who understand it better than
me.

--- Amit



-- 

Re: Issues with quail.el

[Eli Zaretskii]

> Date: Tue, 8 May 2018 11:41:40 +0300
> From: Amit Ramon <amit.ramon@riseup.net>
> 
> Eli Zaretskii <eliz@gnu.org> [2018-05-07 22:33 +0300]:
> 
> >
> >I'm no expert on this, but does the fact that Dvorak keyboards need
> >special input methods tell anything useful for your use case?
> 
> Perhaps I wasn't clear enough. I've been using Dvorak keyboard layout
> (defined at X level, physical keyboard is standard one) with Hebrew
> input method for quite many years, and as for typing, i.e. inputting
> Hebrew characters, everything works just fine. Dvorak keyboard doesn't
> need special input method - AFAIK quail is designed to work with any
> keyboard layout - you just have to tell quail about it.
> 
> The problems I'm talking about are "second order" - they are related to
> functions that display information about input methods and keyboard
> layouts.

My point was that maybe the "second-order" problems you bumped into do
need something special like Dvorak input methods do.

And we do have special input methods for Dvorak keyboard layouts,
AFAIK: e.g., see programmer-dvorak and English-dvorak.

> I spent quiet a lot of time trying to resolve the exact problem and
> how to fix it, but I have to admit that quail is quiet complex so I'd
> appreciate some directions from people who understand it better than
> me.

I'm afraid all the experts on this matter are quite silent for several
years.  But maybe someone will chime in...

Re: Issues with quail.el

[Amit Ramon]

Hi Eli,

Eli Zaretskii <eliz@gnu.org> [2018-05-08 20:28 +0300]:

>> Date: Tue, 8 May 2018 11:41:40 +0300
>> From: Amit Ramon <amit.ramon@riseup.net>
>>
>> Eli Zaretskii <eliz@gnu.org> [2018-05-07 22:33 +0300]:
>>
>> >
>> >I'm no expert on this, but does the fact that Dvorak keyboards need
>> >special input methods tell anything useful for your use case?
>>
>> Perhaps I wasn't clear enough. I've been using Dvorak keyboard layout
>> (defined at X level, physical keyboard is standard one) with Hebrew
>> input method for quite many years, and as for typing, i.e. inputting
>> Hebrew characters, everything works just fine. Dvorak keyboard doesn't
>> need special input method - AFAIK quail is designed to work with any
>> keyboard layout - you just have to tell quail about it.
>>
>> The problems I'm talking about are "second order" - they are related to
>> functions that display information about input methods and keyboard
>> layouts.
>
>My point was that maybe the "second-order" problems you bumped into do
>need something special like Dvorak input methods do.

No, this is not what I think. There are two things that one should be
able to set independently - input method and keyboard layout. One
should be able to use her favorite (or available) keyboard layout,
whether it is a physical keyboard or a layout defined in code (e.g., X
window keyboard layout), and this would usually be the 'constant'
factor. Then, inside Emacs, one should be able to choose different input
methods according to her needs. As far as I can tell from my own
experience and from investigating the code (quail.el), this is how
Emacs is designed to work.

>
>And we do have special input methods for Dvorak keyboard layouts,
>AFAIK: e.g., see programmer-dvorak and English-dvorak.

These input methods are for someone who's (physical) keyboard has the
standard layout, but who wish to use (in Emacs) a Dvorak layout. It's
not the situation I described.

(also, I just looked at the English-dvorak input method, and it also
has some issues, or perhaps features... more things to think about)

>I'm afraid all the experts on this matter are quite silent for several
>years.  But maybe someone will chime in...

Lets hope so...

In the meantime I'll try to continue my investigation, as my time will
permit. If things are still not clear enough and you would like me to
try and write a more coherent and detailed description of the way I
see these issues let me know.

Best,

Amit

P.s. Eli, I'm receiving your replies twice, I guess it is because you
send them to the list but also cc me directly. Unless this is
intentional for a reason that I'm missing, I think it would be best to send
them only to the list.

Re: Issues with quail.el

[Michael Welsh Duggan]

Amit Ramon <amit.ramon@riseup.net> writes:

> I believe I found some bugs in quail.el that occurs when the keyboard
> layout is set to a layout that is different from the standard
> "quail-keyboard-layout-standard". I'd like to try and fix them myself,
> and I have some questions.

I'm not a quail expert, but I looked through a lot of the code recently
to try to figure out how to do something else, so maybe I can figure out
how to help here.

> First issue
>
> Assume that the keyboard layout is set to "atari-german" and that the
> input method is "Hebrew". When I type C-h I (thus invoking
> "describe-input-method"), the drawing of the keyboard layout is
> incorrect. The keys that appear in the shifted level, which should be,
> in this input method, similar to those in the standard layout, are
> actually taken from the current keyboard layout.

Okay, let's verify some things.  When I type 
`M-x quail-show-keyboard-layout RET atari-german RET', I get the
following:

Keyboard layout (keyboard type: atari-german)

     +----------------------------------------------------------------+
     | 1! | 2" | 3§ | 4$ | 5% | 6& | 7/ | 8( | 9) | 0= | ß? | '` | #^ |
     +----------------------------------------------------------------+
       | qQ | wW | eE | rR | tT | zZ | uU | iI | oO | pP | üÜ | +* |
       +------------------------------------------------------------+
        | aA | sS | dD | fF | gG | hH | jJ | kK | lL | öÖ | äÄ | ~| |
     +--------------------------------------------------------------+
     | <> | yY | xX | cC | vV | bB | nN | mM | ,; | .: | -_ |
     +------------------------------------------------------+
		    +-----------------------------+
		    |          space bar          |
		    +-----------------------------+


Is this what you get as well?  Does this match your actual keyboard?


> For example, the Hebrew letter TET is on the (standard) y key, and
> pressing this key together with shift when Hebrew input method is
> active should produce Y. This works fine when typing, but in the map
> drawn by "describe-input-method" the letter that appears on this key
> in the shifted state is Z, which is what this key does in
> "atari-german" layout. There are some other keys not in the right
> place - all those that in "atari-german" have a meaning that is
> different than the one in the standard layout.

Also to verify, the 'y' key is the one to diagonally down and to the
right of the 'a' key?  Like in the layout above?  When I try to recreate
what you are showing, I get the following.  Does this match?

     +----------------------------------------------------------------+
     | 1! | 2" | 3§ | 4$ | 5% | 6& | 7/ | 8( | 9) | 0( | ß? | '` | ;^ |
     +----------------------------------------------------------------+
       | qQ | 'W | ק‎E | ר‎R | א‎T | ט‎Z | ו‎U | ן‎I | ם‎O | פ‎P | ]} | [{ |
       +------------------------------------------------------------+
        | ש‎A | ד‎S | ג‎D | כ‎F | ע‎G | י‎H | ח‎J | ל‎K | ך‎L | ף‎Ö | ,Ä | ~| |
     +--------------------------------------------------------------+
     | >< | ז‎Y | ס‎X | ב‎C | ה‎V | נ‎B | מ‎N | צ‎M | ת‎> | ץ‎< | ._ |
     +------------------------------------------------------+
                    +-----------------------------+
                    |          space bar          |
                    +-----------------------------+

Finally, if it does match, what should it look like instead?

> Second issue

[...]

Snipped until I have a clearer idea what the problem is.

-- 
Michael Welsh Duggan
(md5i@md5i.com)

Re: Issues with quail.el

[Amit Ramon]

Michael Welsh Duggan <mwd@md5i.com> [2018-05-08 23:19 -0400]:

>I'm not a quail expert, but I looked through a lot of the code recently
>to try to figure out how to do something else, so maybe I can figure out
>how to help here.

Thanks, any help is appreciated.

>
>> First issue
>>
>> Assume that the keyboard layout is set to "atari-german" and that the
>> input method is "Hebrew". When I type C-h I (thus invoking
>> "describe-input-method"), the drawing of the keyboard layout is
>> incorrect. The keys that appear in the shifted level, which should be,
>> in this input method, similar to those in the standard layout, are
>> actually taken from the current keyboard layout.
>
>Okay, let's verify some things.  When I type
>`M-x quail-show-keyboard-layout RET atari-german RET', I get the
>following:
>
>Keyboard layout (keyboard type: atari-german)
>
>     +----------------------------------------------------------------+
>     | 1! | 2" | 3§ | 4$ | 5% | 6& | 7/ | 8( | 9) | 0= | ß? | '` | #^ |
>     +----------------------------------------------------------------+
>       | qQ | wW | eE | rR | tT | zZ | uU | iI | oO | pP | üÜ | +* |
>       +------------------------------------------------------------+
>        | aA | sS | dD | fF | gG | hH | jJ | kK | lL | öÖ | äÄ | ~| |
>     +--------------------------------------------------------------+
>     | <> | yY | xX | cC | vV | bB | nN | mM | ,; | .: | -_ |
>     +------------------------------------------------------+
>		    +-----------------------------+
>		    |          space bar          |
>		    +-----------------------------+
>
>
>Is this what you get as well?

Yes.

>Does this match your actual keyboard?

No, but my actual keyboard isn't atari-german -- I gave it just as an
example. My physical keyboard is standard English QWERTY keyboard. On
my Linux system I setup a Dvorak layout. I made tests mostly with my
Dvorak setup[1] (there are footnotes at the end), but because Dvorak
keyboard layout is not one of Emacs' defined layouts I used
atari-german for the example.

>> For example, the Hebrew letter TET is on the (standard) y key, and
>> pressing this key together with shift when Hebrew input method is
>> active should produce Y. This works fine when typing, but in the map
>> drawn by "describe-input-method" the letter that appears on this key
>> in the shifted state is Z, which is what this key does in
>> "atari-german" layout. There are some other keys not in the right
>> place - all those that in "atari-german" have a meaning that is
>> different than the one in the standard layout.
>
>Also to verify, the 'y' key is the one to diagonally down and to the
>right of the 'a' key?  Like in the layout above?

Yes.

> When I try to recreate what you are showing, I get the following.
> Does this match?
>
>     +----------------------------------------------------------------+
>     | 1! | 2" | 3§ | 4$ | 5% | 6& | 7/ | 8( | 9) | 0( | ß? | '` | ;^ |
>     +----------------------------------------------------------------+
>       | qQ | 'W | ק‎E | ר‎R | א‎T | ט‎Z | ו‎U | ן‎I | ם‎O | פ‎P | ]} | [{ |
>       +------------------------------------------------------------+
>        | ש‎A | ד‎S | ג‎D | כ‎F | ע‎G | י‎H | ח‎J | ל‎K | ך‎L | ף‎Ö | ,Ä | ~| |
>     +--------------------------------------------------------------+
>     | >< | ז‎Y | ס‎X | ב‎C | ה‎V | נ‎B | מ‎N | צ‎M | ת‎> | ץ‎< | ._ |
>     +------------------------------------------------------+
>                    +-----------------------------+
>                    |          space bar          |
>                    +-----------------------------+
>

Yes.

>Finally, if it does match, what should it look like instead?

This is what it should look like:

     +----------------------------------------------------------------+
     | 1! | 2@ | 3# | 4$ | 5% | 6^ | 7& | 8* | 9) | 0( | -_ | =+ | `~ |
     +----------------------------------------------------------------+
       | /Q | 'W | ק‎E | ר‎R | א‎T | ט‎Y | ו‎U | ן‎I | ם‎O | פ‎P | ]} | [{ |
       +------------------------------------------------------------+
        | ש‎A | ד‎S | ג‎D | כ‎F | ע‎G | י‎H | ח‎J | ל‎K | ך‎L | ף‎: | ," | \| |
        +-----------------------------------------------------------+
          | ז‎Z | ס‎X | ב‎C | ה‎V | נ‎B | מ‎N | צ‎M | ת‎> | ץ‎< | .? |
          +-------------------------------------------------+
		    +-----------------------------+
		    |          space bar          |
		    +-----------------------------+

Most of the differences are in the first shift level.

Note, for example, the location of the "Y" and "Z", and the two keys
right of the "L". This layout is the standard for a Hebrew keyboard
layout (the Israeli standard, at least), and this is what a Hebrew
writer would expect when Hebrew input method is set. (you can get it
simply if you don't change your keyboard layout -- assuming it is the
standard one -- and then set input method to Hebrew, and do C-h I.)

The point is that *there are no problems with actual typing* -- you
press Shift with the key that has the "Z" in your drawing (to remove
any possible confusion, this is the key that is right below 6 and 7
and on the center between them), and you'll get "Y".

The problem is only with the drawing created by C-h I -- I believe the
algorithm for creating this drawing is wrong, but it could also be a
problem of the way the Hebrew input method is defined[2] or maybe
both.

>> Second issue
>
>[...]
>
>Snipped until I have a clearer idea what the problem is.

Fair enough. Here is how to reproduce it.

I assume that your real keyboard is standard and that you're using the
standard keyboard layout (this is the default of Emacs).

1. Open a new buffer and set the input method to Hebrew (M-x
set-input-method).

2. Press the keys t, y, u (as on the standard keyboard, t is under 5 and
6, and then the two following keys). This will type the Hebrew letters
אטו (this is ALEF TEV VAV).

3. Place the pointer on the first letter, and do:

   M-x quail-show-key

You should get:

   To input 'א', type "t"

4. Now place the pointer on the second letter and do the same. You should
get:

   To input 'ט', type "y"

5. For the third letter, you should get:

   To input 'ו', type "u"

6. Now switch to atari-german keyboard layout and perform the same steps.

For the first and third letters you should get the same as above.

What I get for the second letter (TET, ט) is:

   ט can't be input by the current input method

Now, to prove that this message is incorrect, press the z on your
keyboard (physical z). This is kind of cheating, but because y and z
are swapped in atari-german, pressing z is like really giving Emacs y,
and vice versa. So try pressing either keys, y and z, and you'll see
that you can input TET.

I hope this helps to understand the problems.

Best,

--- Amit

[1] I set my Linux X window system to Dvorak layout with the following
shell command:

   $ setxkbmap -rules evdev -model pc104 -layout "us" -variant "dvorak"

So from Emacs' point of view the keyboard is really Dvorak. Note that
if the real keyboard layout is not the standard one, you should call
"quail-set-keyboard-layout" to set it for Emacs. However, for a reason
I don't understand Emacs doesn't have a definition for Dvorak layout[*]
so you have to use a hommade one. You can do that by evaluating the
following elisp code:

   (require 'quail)
   (add-to-list 'quail-keyboard-layout-alist
             `("dvorak" . ,(concat "                              "
				   "  1!2@3#4$5%6^7&8*9(0)[{]}`~  "
				   "  '\",<.>pPyYfFgGcCrRlL/?=+    "
				   "  aAoOeEuUiIdDhHtTnNsS-_\\|    "
				   "  ;:qQjJkKxXbBmMwWvVzZ        "
				   "                              ")))

   (quail-set-keyboard-layout "dvorak")

If you actually evaluate this elisp code, you could call
"quail-set-keyboard-layout" once with "dvorak" and once with
"standard" and compare the results of C-h I (and for that you don't
have to change the actual layout of the keyboard).

* Emacs has (see quail.el) definitions for sun-type3, atari-german,
pc102-de, jp106 and pc105-uk keyboard layouts only, but not Dvorak
(anyone has an idea why? perhaps it is time to add it?)


[2] The definitions for the standard Hebrew input method ("hebrew" and
"hebrew-new") do not explicitly define the first shift level. Perhap
the author of it trusted quail to take this from the English standard
layout (the first shift level is identical, at least most of
it). However, perhaps it would be safer to completly define the Hebrew
layout, including the first shift level.

Re: Issues with quail.el

[Van L]

> Amit Ramon writes:
> 
> * Emacs has (see quail.el) definitions for sun-type3, atari-german,
> pc102-de, jp106 and pc105-uk keyboard layouts only, but not Dvorak
> (anyone has an idea why? perhaps it is time to add it?)

Is there an input method for Southern China’s Cantonese for 

 - Jyutping w/Digits
 - JP w/Superscripts
 - Yale w/Marks
 - Yale w/Numbers 

phonetic system?

I have started to use chinese-tonepy and chinese-py-punct-b5 and the other possible completions to chinese- don’t look a match.

Re: Issues with quail.el

[Amit Ramon]

Van L <van@scratch.space> [2018-05-11 13:18 +1000]:

>Is there an input method for Southern China’s Cantonese for
>
> - Jyutping w/Digits
> - JP w/Superscripts
> - Yale w/Marks
> - Yale w/Numbers
>
>phonetic system?
>
>I have started to use chinese-tonepy and chinese-py-punct-b5 and the other possible completions to chinese- don’t look a match.

Hope I'm not saying the obvious, but

M-x list-input-methods

will display the list of all input methods.

Re: Issues with quail.el

[Michael Welsh Duggan]

Amit Ramon <amit.ramon@riseup.net> writes:

> Michael Welsh Duggan <mwd@md5i.com> [2018-05-08 23:19 -0400]:
>
>>I'm not a quail expert, but I looked through a lot of the code recently
>>to try to figure out how to do something else, so maybe I can figure out
>>how to help here.
>
> Thanks, any help is appreciated.
>
>>
>>> First issue
>>>
>>> Assume that the keyboard layout is set to "atari-german" and that the
>>> input method is "Hebrew". When I type C-h I (thus invoking
>>> "describe-input-method"), the drawing of the keyboard layout is
>>> incorrect. The keys that appear in the shifted level, which should be,
>>> in this input method, similar to those in the standard layout, are
>>> actually taken from the current keyboard layout.
>>
>>Okay, let's verify some things.  When I type
>>`M-x quail-show-keyboard-layout RET atari-german RET', I get the
>>following:
>>
>>Keyboard layout (keyboard type: atari-german)
>>
>>     +----------------------------------------------------------------+
>>     | 1! | 2" | 3§ | 4$ | 5% | 6& | 7/ | 8( | 9) | 0= | ß? | '` | #^ |
>>     +----------------------------------------------------------------+
>>       | qQ | wW | eE | rR | tT | zZ | uU | iI | oO | pP | üÜ | +* |
>>       +------------------------------------------------------------+
>>        | aA | sS | dD | fF | gG | hH | jJ | kK | lL | öÖ | äÄ | ~| |
>>     +--------------------------------------------------------------+
>>     | <> | yY | xX | cC | vV | bB | nN | mM | ,; | .: | -_ |
>>     +------------------------------------------------------+
>>		    +-----------------------------+
>>		    |          space bar          |
>>		    +-----------------------------+
>>
>>
>>Is this what you get as well?
>
> Yes.
>
>>Does this match your actual keyboard?
>
> No, but my actual keyboard isn't atari-german -- I gave it just as an
> example. My physical keyboard is standard English QWERTY keyboard. On
> my Linux system I setup a Dvorak layout. I made tests mostly with my
> Dvorak setup[1] (there are footnotes at the end), but because Dvorak
> keyboard layout is not one of Emacs' defined layouts I used
> atari-german for the example.
>
>>> For example, the Hebrew letter TET is on the (standard) y key, and
>>> pressing this key together with shift when Hebrew input method is
>>> active should produce Y. This works fine when typing, but in the map
>>> drawn by "describe-input-method" the letter that appears on this key
>>> in the shifted state is Z, which is what this key does in
>>> "atari-german" layout. There are some other keys not in the right
>>> place - all those that in "atari-german" have a meaning that is
>>> different than the one in the standard layout.
>>
>>Also to verify, the 'y' key is the one to diagonally down and to the
>>right of the 'a' key?  Like in the layout above?
>
> Yes.
>
>> When I try to recreate what you are showing, I get the following.
>> Does this match?
>>
>>     +----------------------------------------------------------------+
>>     | 1! | 2" | 3§ | 4$ | 5% | 6& | 7/ | 8( | 9) | 0( | ß? | '` | ;^ |
>>     +----------------------------------------------------------------+
>>       | qQ | 'W | ק‎E | ר‎R | א‎T | ט‎Z | ו‎U | ן‎I | ם‎O | פ‎P | ]} | [{ |
>>       +------------------------------------------------------------+
>>        | ש‎A | ד‎S | ג‎D | כ‎F | ע‎G | י‎H | ח‎J | ל‎K | ך‎L | ף‎Ö | ,Ä | ~| |
>>     +--------------------------------------------------------------+
>>     | >< | ז‎Y | ס‎X | ב‎C | ה‎V | נ‎B | מ‎N | צ‎M | ת‎> | ץ‎< | ._ |
>>     +------------------------------------------------------+
>>                    +-----------------------------+
>>                    |          space bar          |
>>                    +-----------------------------+
>>
>
> Yes.
>
>>Finally, if it does match, what should it look like instead?
>
> This is what it should look like:
>
>     +----------------------------------------------------------------+
>     | 1! | 2@ | 3# | 4$ | 5% | 6^ | 7& | 8* | 9) | 0( | -_ | =+ | `~ |
>     +----------------------------------------------------------------+
>       | /Q | 'W | ק‎E | ר‎R | א‎T | ט‎Y | ו‎U | ן‎I | ם‎O | פ‎P | ]} | [{ |
>       +------------------------------------------------------------+
>        | ש‎A | ד‎S | ג‎D | כ‎F | ע‎G | י‎H | ח‎J | ל‎K | ך‎L | ף‎: | ," | \| |
>        +-----------------------------------------------------------+
>          | ז‎Z | ס‎X | ב‎C | ה‎V | נ‎B | מ‎N | צ‎M | ת‎> | ץ‎< | .? |
>          +-------------------------------------------------+
> 		    +-----------------------------+
> 		    |          space bar          |
> 		    +-----------------------------+
>
> Most of the differences are in the first shift level.
>
> Note, for example, the location of the "Y" and "Z", and the two keys
> right of the "L". This layout is the standard for a Hebrew keyboard
> layout (the Israeli standard, at least), and this is what a Hebrew
> writer would expect when Hebrew input method is set. (you can get it
> simply if you don't change your keyboard layout -- assuming it is the
> standard one -- and then set input method to Hebrew, and do C-h I.)
>
> The point is that *there are no problems with actual typing* -- you
> press Shift with the key that has the "Z" in your drawing (to remove
> any possible confusion, this is the key that is right below 6 and 7
> and on the center between them), and you'll get "Y".

Strangely, though, that is not what I experience.  Here are the
unshifted keys for me, atari-german, hebrew input mode:

  ‎‪+1234567890.(\
  ‭׳קראזוןםפ][
  ‭שדגכעיחלך>=
  ‭טסבהנמצתץ&

and the shifted ones:

  \!@;$%~^*)?[|
  QWERTZUIOP}{
  ASDFGHJKL<@
  YXCVBNM><_

There are a few that don't match, but the Y and Z are not two of them.
This is not the case for you?

-- 
Michael Welsh Duggan
(md5i@md5i.com)

Re: Issues with quail.el

[Amit Ramon]

Hi,

Michael Welsh Duggan <mwd@md5i.com> [2018-05-12 00:13 -0400]:
>>
>> Note, for example, the location of the "Y" and "Z", and the two keys
>> right of the "L". This layout is the standard for a Hebrew keyboard
>> layout (the Israeli standard, at least), and this is what a Hebrew
>> writer would expect when Hebrew input method is set. (you can get it
>> simply if you don't change your keyboard layout -- assuming it is the
>> standard one -- and then set input method to Hebrew, and do C-h I.)
>>
>> The point is that *there are no problems with actual typing* -- you
>> press Shift with the key that has the "Z" in your drawing (to remove
>> any possible confusion, this is the key that is right below 6 and 7
>> and on the center between them), and you'll get "Y".
>
>Strangely, though, that is not what I experience.  Here are the
>unshifted keys for me, atari-german, hebrew input mode:
>
>  ‎‪+1234567890.(\
>  ‭׳קראזוןםפ][
>  ‭שדגכעיחלך>=
>  ‭טסבהנמצתץ&
>
>and the shifted ones:
>
>  \!@;$%~^*)?[|
>  QWERTZUIOP}{
>  ASDFGHJKL<@
>  YXCVBNM><_
>
>There are a few that don't match, but the Y and Z are not two of them.
>This is not the case for you?


This is what you actually get when typing, right? This is also the
case for me, but perhaps I didn't explain the problem clearly enough.

Just to make sure - I assume your physical keyboard is a standard
English (or at least very similar)? (i.e., it is not a real, physical,
atari-german keyboard.)

Also, before going on I suggest that we adopt some "naming
conventions" for keyboard keys, to make communication easier. I'll use
the convention described here:

https://en.wikipedia.org/wiki/ISO/IEC_9995#ISO/IEC_9995-1

It is similar to the way used for X Window keyboard maps. So, for
example, the row under the numbers is row D, and the keys are numbered
D1, D2 and so on from left to right, where "q" is on D1, "w" is on D2
and so on.

Lets first do a simple test. When Hebrew input method is active, you
press Shift-D6 and you get "Z" (per your result above). If you now
deactivate the input method ("toggle-input-method", bound by default
to C-\) and press D6 with and without Shift, do you get "y" and "Y"?

If so, then the "strangeness" is due to the fact that the physical
keyboard is not really atari-german -- if the keyboard was
atari-german, D6 in the test would print "z" and "Z".

Emacs receives just characters, it has no idea about the actual
physical keys that are pressed -- this is why we define for quail the
keyboard layout. It enables Emacs to "compute" the physical key that
was pressed by comparing the active keyboard layout with the standard
one.

We can "emulate" an atari-german real keyboard by pressing B1 when we
really mean D6, and vice versa. Then Emacs will receive "z" for
(emulated) "D6" and "y" for (emulated) "B1". Then we should swap the Z
and Y in your typing results.

I hope I managed to clear this out this time, although I'm not sure
how much I succeeded in this.

Best,

--- Amit

Re: Issues with quail.el

[K. Handa]

Very sorry for the late response.

In article <20180510141234.tpj4vagod6bejmta@isis.luna>, Amit Ramon <amit.ramon@riseup.net> writes:
> > When I try to recreate what you are showing, I get the following.
> > Does this match?
> >
> >     +----------------------------------------------------------------+
> >     | 1! | 2" | 3§ | 4$ | 5% | 6& | 7/ | 8( | 9) | 0( | ß? | '` | ;^ |
> >     +----------------------------------------------------------------+
> >       | qQ | 'W | ק‎E | ר‎R | א‎T | ט‎Z | ו‎U | ן‎I | ם‎O | פ‎P | ]} | [{ |
> >       +------------------------------------------------------------+
> >        | ש‎A | ד‎S | ג‎D | כ‎F | ע‎G | י‎H | ח‎J | ל‎K | ך‎L | ף‎Ö | ,Ä | ~| |
> >     +--------------------------------------------------------------+
> >     | >< | ז‎Y | ס‎X | ב‎C | ה‎V | נ‎B | מ‎N | צ‎M | ת‎> | ץ‎< | ._ |
> >     +------------------------------------------------------+
[...]
> This is what it should look like:

>      +----------------------------------------------------------------+
>      | 1! | 2@ | 3# | 4$ | 5% | 6^ | 7& | 8* | 9) | 0( | -_ | =+ | `~ |
>      +----------------------------------------------------------------+
>        | /Q | 'W | ק‎E | ר‎R | א‎T | ט‎Y | ו‎U | ן‎I | ם‎O | פ‎P | ]} | [{ |
>        +------------------------------------------------------------+
>         | ש‎A | ד‎S | ג‎D | כ‎F | ע‎G | י‎H | ח‎J | ל‎K | ך‎L | ף‎: | ," | \| |
>         +-----------------------------------------------------------+
>           | ז‎Z | ס‎X | ב‎C | ה‎V | נ‎B | מ‎N | צ‎M | ת‎> | ץ‎< | .? |
>           +-------------------------------------------------+
[...]
> Note, for example, the location of the "Y" and "Z", and the two keys
> right of the "L". This layout is the standard for a Hebrew keyboard
> layout (the Israeli standard, at least), and this is what a Hebrew
> writer would expect when Hebrew input method is set. (you can get it
> simply if you don't change your keyboard layout -- assuming it is the
> standard one -- and then set input method to Hebrew, and do C-h I.)

It seems that the difference is at such a key that is different from the
standard layout and that is not mapped to Hebrew.  So, I suspect that
the code (in quail-insert-kbd-layout) handling such a key should be
fixed.  Could you please try the attached patch?

[...]
> What I get for the second letter (TET, ט) is:

>    ט can't be input by the current input method

I confimed this bug, but it seems that fixing take more time.   Please wait.

>    (add-to-list 'quail-keyboard-layout-alist
>              `("dvorak" . ,(concat "                              "
> 				   "  1!2@3#4$5%6^7&8*9(0)[{]}`~  "
> 				   "  '\",<.>pPyYfFgGcCrRlL/?=+    "
> 				   "  aAoOeEuUiIdDhHtTnNsS-_\\|    "
> 				   "  ;:qQjJkKxXbBmMwWvVzZ        "
> 				   "                              ")))

>    (quail-set-keyboard-layout "dvorak")

> If you actually evaluate this elisp code, you could call
> "quail-set-keyboard-layout" once with "dvorak" and once with
> "standard" and compare the results of C-h I (and for that you don't
> have to change the actual layout of the keyboard).

> * Emacs has (see quail.el) definitions for sun-type3, atari-german,
> pc102-de, jp106 and pc105-uk keyboard layouts only, but not Dvorak
> (anyone has an idea why? perhaps it is time to add it?)

I agree.

> [2] The definitions for the standard Hebrew input method ("hebrew" and
> "hebrew-new") do not explicitly define the first shift level. Perhap
> the author of it trusted quail to take this from the English standard
> layout (the first shift level is identical, at least most of
> it). However, perhaps it would be safer to completly define the Hebrew
> layout, including the first shift level.

For that, I don't have a strong opinion.  But, I think it is not good to
add extra code for hiding a bug of quail.

---
K. Handa
handa@gnu.org

diff --git a/lisp/international/quail.el b/lisp/international/quail.el
index eece836354..a81244a1db 100644
--- a/lisp/international/quail.el
+++ b/lisp/international/quail.el
@@ -815,7 +815,7 @@ quail-insert-kbd-layout
                             (aref (cdr translation) 0)
                           " ")))
 	      (setq done-list (cons translation done-list)))
-	  (setq translation (aref kbd-layout i)))
+	  (setq translation ch))
 	(aset layout i translation))
       (setq i (1+ i)))
 

Re: Issues with quail.el

[K. Handa]

Hi,

In article <87po21f4dl.fsf@gnu.org>, handa@gnu.org (K. Handa) writes:

> diff --git a/lisp/international/quail.el b/lisp/international/quail.el
> index eece836354..a81244a1db 100644
> --- a/lisp/international/quail.el
> +++ b/lisp/international/quail.el
> @@ -815,7 +815,7 @@ quail-insert-kbd-layout
>                              (aref (cdr translation) 0)
>                            " ")))
>  	      (setq done-list (cons translation done-list)))
> -	  (setq translation (aref kbd-layout i)))
> +	  (setq translation ch))
>  	(aset layout i translation))
>        (setq i (1+ i)))
 
Have you tried the above patch?  Did it work?

> [...]
> > What I get for the second letter (TET, ט) is:

> >    ט can't be input by the current input method

> I confimed this bug, but it seems that fixing take more time.   Please wait.

Here's the patch to fix it.  Could you please try it?

diff --git a/lisp/international/quail.el b/lisp/international/quail.el
index eece836354..bcbb0c0958 100644
--- a/lisp/international/quail.el
+++ b/lisp/international/quail.el
@@ -786,6 +786,36 @@ quail-keyboard-translate
 		char)
 	    ch))))))
 
+(defun quail-keyboard-untranslate (char)
+  "Translate CHAR back to the one on the current keyboard layout.
+CHAR is the one on the standard keyboard layout.
+If the current keyboard layout does not have the corresponding character,
+return nil."
+  (if (eq quail-keyboard-layout quail-keyboard-layout-standard)
+      ;; All Quail packages are designed based on
+      ;; `quail-keyboard-layout-standard'.
+      char
+    (let ((i 0))
+      ;; Find the key location on the standard keyboard layout.
+      (while (and (< i quail-keyboard-layout-len)
+		  (/= char (aref quail-keyboard-layout-standard i)))
+	(setq i (1+ i)))
+      (if (= i quail-keyboard-layout-len)
+	  ;; CHAR is not in the current keyboard layout, which means
+	  ;; that a user can not type a key which generates CHAR.
+	  ;; Just return nil.
+          nil
+	(let ((ch (aref quail-keyboard-layout i)))
+	  (if (= ch ?\ )
+	      ;; This location not available in the current keyboard
+	      ;; layout.  Check if the location is used to substitute
+	      ;; for the other location of the standard layout.
+	      (if (setq i (cdr (rassq i quail-keyboard-layout-substitution)))
+		  (aref quail-keyboard-layout i)
+		;; Just return il as well as above.
+		nil)
+	    ch))))))
+
 (defun quail-keyseq-translate (keyseq)
   (apply 'string
 	 (mapcar (function (lambda (x) (quail-keyboard-translate x)))
@@ -2843,8 +2873,27 @@ quail-show-key
       (error "No input method is activated"))
   (or (assoc current-input-method quail-package-alist)
       (error "The current input method does not use Quail"))
-  (let* ((char (following-char))
-	 (key-list (quail-find-key char)))
+  (let ((char (following-char))
+        key-list)
+    ;; Get a list of keyseqs (the standard keyboard layout based) to type CHAR.
+    (let ((quail-keyboard-layout quail-keyboard-layout-standard))
+      (setq key-list (quail-find-key char)))
+    ;; Then convert each key (of each keyseq if any) to the key of the
+    ;; current keyboard layout.
+    (if (and (consp key-list)
+             (quail-kbd-translate))
+        (let (untranslated-key-list)
+          (dolist (keyseq key-list)
+            (setq keyseq
+                  (catch 'tag
+                    (mapconcat #'(lambda (key)
+                                   (setq key (quail-keyboard-untranslate key))
+                                   (if key
+                                       (string key)
+                                     (throw 'tag nil)))
+                               keyseq "")))
+            (if keyseq (push keyseq untranslated-key-list)))
+          (setq key-list (nreverse untranslated-key-list))))
     (cond ((consp key-list)
 	   (message "To input `%c', type \"%s\""
 		    char

Re: Issues with quail.el

[Amit Ramon]

Hello,

K. Handa <handa@gnu.org> [2018-05-17 21:31 +0900]:

>In article <87po21f4dl.fsf@gnu.org>, handa@gnu.org (K. Handa) writes:
>
>> [...]

>Have you tried the above patch?  Did it work?

Actually I've started to look it only earlier today, but initial
testing are showing it had fixed the problem. I'll do more tests
during the weekend and let you know. Sorry it took me too long to
reply.

>
>> [...]
>> > What I get for the second letter (TET, ט) is:
>
>> >    ט can't be input by the current input method
>
>> I confimed this bug, but it seems that fixing take more time.   Please wait.
>
>Here's the patch to fix it.  Could you please try it?
>
> [...]

Thanks for this fix. I'll test it this weekend as well and let you
know.

Also, I've actually tried to write a fix for it myself, and after
doing more tests and looking at your fix, if I'll think my code worth
something I might put it here, in case you, or someone else, will find
it useful.

Best,

Amit

Re: Issues with quail.el

[Filipp Gunbin]

On 17/05/2018 21:31 +0900, K. Handa wrote:

> Here's the patch to fix it.  Could you please try it?
...
> +	      ;; This location not available in the current keyboard
> +	      ;; layout.  Check if the location is used to substitute
> +	      ;; for the other location of the standard layout.
> +	      (if (setq i (cdr (rassq i quail-keyboard-layout-substitution)))
> +		  (aref quail-keyboard-layout i)

Cannot understand this part - it seems to look up alist element by cdr
(i), and then set i to the same value.  Should there be `(car (rassq..'
instead?

Sorry if missing something obvious.

Filipp

Re: Issues with quail.el

[Amit Ramon]

Filipp Gunbin <fgunbin@fastmail.fm> [2018-05-17 18:44 +0300]:

>On 17/05/2018 21:31 +0900, K. Handa wrote:
>
>> Here's the patch to fix it.  Could you please try it?
>...
>> +	      ;; This location not available in the current keyboard
>> +	      ;; layout.  Check if the location is used to substitute
>> +	      ;; for the other location of the standard layout.
>> +	      (if (setq i (cdr (rassq i quail-keyboard-layout-substitution)))
>> +		  (aref quail-keyboard-layout i)
>
>Cannot understand this part - it seems to look up alist element by cdr
>(i), and then set i to the same value.  Should there be `(car (rassq..'
>instead?

I agree with Filipp -- I also think it should be set to '(car (rassq
...'.

(this is a 'reverse' translation -- the code in the existing
translation sets i to '(cdr (assq ...', here it should be the
'mirrored image' of it.)

--- Amit

Re: Issues with quail.el

[K. Handa]

In article <m2fu2qs2wv.fsf@fgunbin.playteam.ru>, Filipp Gunbin <fgunbin@fastmail.fm> writes:

> On 17/05/2018 21:31 +0900, K. Handa wrote:
> > Here's the patch to fix it.  Could you please try it?
...
> > +	      ;; This location not available in the current keyboard
> > +	      ;; layout.  Check if the location is used to substitute
> > +	      ;; for the other location of the standard layout.
> > +	      (if (setq i (cdr (rassq i quail-keyboard-layout-substitution)))
> > +		  (aref quail-keyboard-layout i)

> Cannot understand this part - it seems to look up alist element by cdr
> (i), and then set i to the same value.  Should there be `(car (rassq..'
> instead?

You are right.  I made quail-keyboard-untranslate by copying
quail-keyboard-translate and forgot to adjust the above part correctly.

---
K. Handa
handa@gnu.org

Re: Issues with quail.el

[Amit Ramon]

Hi,

K. Handa <handa@gnu.org> [2018-05-17 21:31 +0900]:

>Hi,
>
>In article <87po21f4dl.fsf@gnu.org>, handa@gnu.org (K. Handa) writes:
>
>> diff --git a/lisp/international/quail.el b/lisp/international/quail.el
>> index eece836354..a81244a1db 100644
>> --- a/lisp/international/quail.el
>> +++ b/lisp/international/quail.el
>> @@ -815,7 +815,7 @@ quail-insert-kbd-layout
>>                              (aref (cdr translation) 0)
>>                            " ")))
>>  	      (setq done-list (cons translation done-list)))
>> -	  (setq translation (aref kbd-layout i)))
>> +	  (setq translation ch))
>>  	(aset layout i translation))
>>        (setq i (1+ i)))
>
>Have you tried the above patch?  Did it work?

Yes, I've tried this patch with my Dvorak keyboard layout and Hebrew
input method and it works great. I've also tried it with atari-german
keyboard layout and Hebrew-new input method.

There are still some slight issued, for example the quail's definition
of the standard keyboard layout is based on VT100 layout, which is
different in some minor aspects from a standard PC keyboard (e.g. the
location of the tilde key), but this is not related to the problem
this patch fixes.


>> [...]
>> > What I get for the second letter (TET, ט) is:
>
>> >    ט can't be input by the current input method
>
>> I confimed this bug, but it seems that fixing take more time.   Please wait.
>
>Here's the patch to fix it.  Could you please try it?

I tested this patch with the configurations mentioned above (after
fixing the cdr/car typo mentioned by Filipp Gunbin in a different mail
in this thread) and, again, it fixes the problem.

There are still some issued that are, perhaps, related to a broader
question, I'll try to describe them briefly here, but I think they
should not stop us from applying your fixes.

1. Key sequences

Lets assume I'm using a Dvorak keyboard layout and a Hebrew input
method. This input method include some key sequences for typing some
special Hebrew signs. For example, for inputting HEBREW PUNCTUATION
MAQAF (־), one has to type q-p. When the keyboard layout is standard,
there is no problem. When it is Dvorak, C-h I still says you have to
type q-p, and it works if you press the keys that are q-p in the
standard layout, but since I use Dvoark it is (for me) '-l. Here comes
the question of what the input method really wants -- to keep the
actual location of the keys you need to type a sequence constant, or
that the constant should be the keys themselves (i.e., their meaning).

On the other hand, when quail-show-key is asked to find what to type
for entering HEBREW PUNCTUATION MAQAF, it says '-l -- this is the
actual meaning of the keys need to be typed for a Dvorak layout.

2. Shifted keys

This is somewhat similar to the above. If (Hebrew + Dvorak layout) I
ask quail-show-key to tell what to type for inputting Z, it tells me
"To input 'Z', just type it".

But (again, Hebrew input method is active) this is not really true
since my keyboard layout is Dvorak -- to enter Z in Hebrew input
method I need to press Shift-;, since the key that has ; in Dvorak is
the z key in the standard layout.

These are, maybe, questions of semantics and context -- when one say
that I have to type q-p (to get HEBREW MAQAF), is it in the context of
my real keyboard (then it is not true for Dvoark), or is it in the
context of a standard keyboard (and then it is confusing people who're
using, say, a Dvorak keyboard).

It seems to me that these questions need some more thinking, at least
for me. In the meantime, I'm just putting them here.

Thanks again for the two fixes,

Amit

P.s. Handa san, I have some technical questions and, perhaps, some
suggestions for simplifying some parts of quail.el (assuming I'm
not completely misunderstanding it) -- would you be interested to
hear? If so, shall I post my questions in this thread or, perhaps,
send them directly to you?

Re: Issues with quail.el

[K. Handa]

In article <20180519120124.ee4bompnunw2gxuv@isis.luna>, Amit Ramon <amit.ramon@riseup.net> writes:

> There are still some slight issued, for example the quail's definition
> of the standard keyboard layout is based on VT100 layout, which is
> different in some minor aspects from a standard PC keyboard (e.g. the
> location of the tilde key), but this is not related to the problem
> this patch fixes.

When I first wrote that part (looong ago), we were using Unix (not
GNU/Linux) on workstations (not PC).  At that time, the choice of VT100
as the starndard layout was, I thought, not that bad.

>>> > What I get for the second letter (TET, ט) is:
> >
>>> >    ט can't be input by the current input method
> >
>>> I confimed this bug, but it seems that fixing take more time.   Please wait.
> >
> >Here's the patch to fix it.  Could you please try it?

> I tested this patch with the configurations mentioned above (after
> fixing the cdr/car typo mentioned by Filipp Gunbin in a different mail
> in this thread) and, again, it fixes the problem.

Thank you for testing it.  I'll commit that patch soon.

> There are still some issued that are, perhaps, related to a broader
> question, I'll try to describe them briefly here, but I think they
> should not stop us from applying your fixes.

> 1. Key sequences

> Lets assume I'm using a Dvorak keyboard layout and a Hebrew input
> method. This input method include some key sequences for typing some
> special Hebrew signs. For example, for inputting HEBREW PUNCTUATION
> MAQAF (־), one has to type q-p. When the keyboard layout is standard,
> there is no problem. When it is Dvorak, C-h I still says you have to
> type q-p, and it works if you press the keys that are q-p in the
> standard layout, but since I use Dvoark it is (for me) '-l. Here comes
> the question of what the input method really wants -- to keep the
> actual location of the keys you need to type a sequence constant, or
> that the constant should be the keys themselves (i.e., their meaning).

At first, there's something wrong in the docstring of Hebrew.  It says:
------------------------------------------------------------
 'q' is used to switch levels instead of Alt-Gr.
 Maqaaf (־) is mapped to '/פ'.
------------------------------------------------------------
Why does it say "mapped to '/פ'"?  Shouldn't it be "mapped to 'qp'"? 
[I included yair.f.lists@gmail.com in CC: because it seems that this
part is added by him].

Anyway, to make that part shown correctly in *Help* buffer for Dvorak
users, we need a new mechanism to convert "qp" to "-l".  One idea is to
introduce a new notation, something like "\[q]" and modify
quail-help to convert it to the key on the current layout.

Here, I have a question.  I think 'q' of 'qp' is selected because a user
won't have to insert "q" while typing Hebrew.  But, is it ok to use "/"
for that kind of prefix key?

The following section in *Help* buffer can be fixed rather easily...
============================================================
KEY SEQUENCE
------------
You can also input more characters by the following key sequences:
key char  [type a key sequence to insert the corresponding character]
--- ---- --- ---- --- ---- --- ---- --- ---- --- ---- --- ----
q"  ״	 q0  ׁ	  q4  ִ	   q8  ָ	    q\	ֻ     qh	 ײ    qw  ׳
[...]
============================================================
because the table is generated by a program.  Perhaps, we should fix
quail-insert-decode-map.

> 2. Shifted keys

> This is somewhat similar to the above. If (Hebrew + Dvorak layout) I
> ask quail-show-key to tell what to type for inputting Z, it tells me
> "To input 'Z', just type it".

> But (again, Hebrew input method is active) this is not really true
> since my keyboard layout is Dvorak -- to enter Z in Hebrew input
> method I need to press Shift-;, since the key that has ; in Dvorak is
> the z key in the standard layout.

This part should also be fixed.

> These are, maybe, questions of semantics and context -- when one say
> that I have to type q-p (to get HEBREW MAQAF), is it in the context of
> my real keyboard (then it is not true for Dvoark), or is it in the
> context of a standard keyboard (and then it is confusing people who're
> using, say, a Dvorak keyboard).

As I wrote above, 'qp' should be converted to '-l' in *Help* buffer.

> P.s. Handa san, I have some technical questions and, perhaps, some
> suggestions for simplifying some parts of quail.el (assuming I'm
> not completely misunderstanding it) -- would you be interested to
> hear? If so, shall I post my questions in this thread or, perhaps,
> send them directly to you?

Could you post to emacs-devel with a new proper subject?

---
K. Handa
handa@gnu.org

Re: Issues with quail.el

[Amit Ramon]

K. Handa <handa@gnu.org> [2018-05-23 23:27 +0900]:

>In article <20180519120124.ee4bompnunw2gxuv@isis.luna>, Amit Ramon <amit.ramon@riseup.net> writes:
>
>> There are still some slight issued, for example the quail's definition
>> of the standard keyboard layout is based on VT100 layout, which is
>> different in some minor aspects from a standard PC keyboard (e.g. the
>> location of the tilde key), but this is not related to the problem
>> this patch fixes.
>
>When I first wrote that part (looong ago), we were using Unix (not
>GNU/Linux) on workstations (not PC).  At that time, the choice of VT100
>as the starndard layout was, I thought, not that bad.

Yes, I guessed that is the reason. I really don't know which keyboard
layout is the most prevalent now days (I would guess that a PC
keyboard, but I can't back it up with data), so I don't have a strong
opinion regarding which keyboard layout should be the
default. However, we should at least have a definition of a 'standard
PC' keyboard layout in quail's list of keyboards, as well as, IMHO, a
'standard English Dvorak' (we discussed this in a previous email), so
I suggest that I'll prepare a patch and post it here as soon as I find
time for it.

Also, there is a new standard for Hebrew keyboard layout, and I intend
to prepare a patch for it and post it here.

> [...]

>Thank you for testing it.  I'll commit that patch soon.

Glad I could help with that, thanks.

>> There are still some issued that are, perhaps, related to a broader
>> question, I'll try to describe them briefly here, but I think they
>> should not stop us from applying your fixes.
>
>> 1. Key sequences
>
>> Lets assume I'm using a Dvorak keyboard layout and a Hebrew input
>> method. This input method include some key sequences for typing some
>> special Hebrew signs. For example, for inputting HEBREW PUNCTUATION
>> MAQAF (־), one has to type q-p. When the keyboard layout is standard,
>> there is no problem. When it is Dvorak, C-h I still says you have to
>> type q-p, and it works if you press the keys that are q-p in the
>> standard layout, but since I use Dvoark it is (for me) '-l. Here comes
>> the question of what the input method really wants -- to keep the
>> actual location of the keys you need to type a sequence constant, or
>> that the constant should be the keys themselves (i.e., their meaning).
>
>At first, there's something wrong in the docstring of Hebrew.  It says:
>------------------------------------------------------------
> 'q' is used to switch levels instead of Alt-Gr.
> Maqaaf (־) is mapped to '/פ'.
>------------------------------------------------------------
>Why does it say "mapped to '/פ'"?  Shouldn't it be "mapped to 'qp'"?
>[I included yair.f.lists@gmail.com in CC: because it seems that this
>part is added by him].

I cannot know the reasoning for that, but it does make some sense. If
the keyboard layout is standard, then MAQAF is produced by "qp". But
if the keyboard layout is, say, Dvoark, then the corresponding key
sequence would be "'l". The message you are referring to comes from
hebrew.el so it is "hard coded". If it would say "qp", it will be
incorrect for users of Dvorak and, possibly, other keyboard layouts.

So as long as this message is hard coded using the key system of the
input method is, perhaps, a reasonable compromise -- at least it
refers to keys that are in a fixed location, no matter what is the
keyboard layout.

>Anyway, to make that part shown correctly in *Help* buffer for Dvorak
>users, we need a new mechanism to convert "qp" to "-l".  One idea is to
>introduce a new notation, something like "\[q]" and modify
>quail-help to convert it to the key on the current layout.

I agree that converting it on-the-fly would be the best solution.

>Here, I have a question.  I think 'q' of 'qp' is selected because a user
>won't have to insert "q" while typing Hebrew.  But, is it ok to use "/"
>for that kind of prefix key?

When we are in Hebrew input method, the "q" key produces "/", like you
said, so the question is about the usage of "/". This key is indeed
rarely used in Hebrew, so probably this is the reason for its
selection as the prefix key. Perhaps another reason is the location of
it on the keyboard, which, at least to me, is convenient.

>The following section in *Help* buffer can be fixed rather easily...
>============================================================
>KEY SEQUENCE
>------------
>You can also input more characters by the following key sequences:
>key char  [type a key sequence to insert the corresponding character]
>--- ---- --- ---- --- ---- --- ---- --- ---- --- ---- --- ----
>q"  ״	 q0  ׁ	  q4  ִ	   q8  ָ	    q\	ֻ     qh	 ײ    qw  ׳
>[...]
>============================================================
>because the table is generated by a program.  Perhaps, we should fix
>quail-insert-decode-map.

That would be great.

>> 2. Shifted keys
>
>> This is somewhat similar to the above. If (Hebrew + Dvorak layout) I
>> ask quail-show-key to tell what to type for inputting Z, it tells me
>> "To input 'Z', just type it".
>
>> But (again, Hebrew input method is active) this is not really true
>> since my keyboard layout is Dvorak -- to enter Z in Hebrew input
>> method I need to press Shift-;, since the key that has ; in Dvorak is
>> the z key in the standard layout.
>
>This part should also be fixed.

I am for it.

>> These are, maybe, questions of semantics and context -- when one say
>> that I have to type q-p (to get HEBREW MAQAF), is it in the context of
>> my real keyboard (then it is not true for Dvoark), or is it in the
>> context of a standard keyboard (and then it is confusing people who're
>> using, say, a Dvorak keyboard).
>
>As I wrote above, 'qp' should be converted to '-l' in *Help* buffer.

Yes, it could be that I repeated myself.


>> P.s. Handa san, I have some [...] suggestions for simplifying some
>> parts of quail.el [...]
>
>Could you post to emacs-devel with a new proper subject?

Yes, I would do that. Anyway, these are not functional issues, they
are more of code refactoring, and perhaps we wouldn't want to modify
working code just for 'ascetics'. But I'll post it anyway and let you
decide.

Best,

--- Amit