Re: thoughts on interaction of key bindings and input methods (was Re: wish: right alt/meta to switch keyboard layout while pressed)

[Ben Wing <ben@666.com>]

this is what i call the "russian c-x problem".  at one point i had 
worked out what the correct thing to do is, and there are various 
comments to this effect in XEmacs; but i never finished it.  it looks 
like aidan went ahead and implemented it, though, right, aidan?

in events.h we have:

enum alternative_key_chars
{
  KEYCHAR_CURRENT_LANGENV,
  KEYCHAR_DEFAULT_USER,
  KEYCHAR_DEFAULT_SYSTEM,
  KEYCHAR_UNDERLYING_VIRTUAL_KEY_CURRENT_LANGENV,
  KEYCHAR_UNDERLYING_VIRTUAL_KEY_DEFAULT_USER,
  KEYCHAR_UNDERLYING_VIRTUAL_KEY_DEFAULT_SYSTEM,
  KEYCHAR_QWERTY,
  KEYCHAR_LAST
};

struct Lisp_Key_Data
{
#ifdef EVENT_DATA_AS_OBJECTS
  struct lrecord_header lheader;
#endif /* EVENT_DATA_AS_OBJECTS */
  /* What keysym this is; a character or a symbol. */
  Lisp_Object keysym;
  /* Modifiers held down when key was pressed: control, meta, etc.
     Also includes buttons.  For many keys, Shift is not a bit; that
     is implicit in the keyboard layout. */
  int modifiers;
  /* Alternate character interpretations for this key in different
     keyboard layouts.  This deals with the problem of pressing C-x in
     the Russian layout (the so-called "Russian C-x problem"), for
     example: `x' gets mapped to a Cyrillic character, so what do we
     do?  For that matter, what about `C-x b'?  What we do is look the
     key up in the default locales (current language environment, user
     default, system default), then check to see if the underlying
     virtual key is alphabetic in the same three defaults, then
     finally check US ASCII.  We ignore the underlying virtual key for
     the current layout to avoid the problem of a French speaker
     (AZERTY layout) who temporarily switches to Russian: The virtual
     keys underlying Russian are US-ASCII, so what the French speaker
     things of as C-a (the key just to the right of TAB) appears as
     C-q.

         I've just implemented this in event-stream.c, and I really want to
         see feedback from actual Russians about it, and whether it needs to
         be much more complete than what I've done. E.g, the mapping back to
         US Qwerty is hardcoded on the X11 side of things, and only deals
         with the alphabetic characters.

     Also, I think it's downright confusing for people with Roman
     letters on their keyboard to have random other letters than are
     described as calling some command, call that command. So I want to
     consider enabling it by language environment.

     [[ (#### We should probably ignore the current char and look
     *ONLY* in alt_keychars for all control keys.  What about the
     English speaker who temporarily switches to the French layout and
     finds C-q mapped to C-a?) ]]

         No, we shouldn't. People who use the French layout expect that
         pressing control with the key to the right of tab passes C-a to
         emacs; English speakers (more exactly, Qwerty users) who
         temporarily switch to the French layout encounter that issue in
         every other app too, and they normally remap the keyboard in
         software as soon as they can, or learn to live with Azerty. That
         applies for all the Roman-alphabet keyboard layouts. Aidan Kehoe,
         2005-05-15

         I've taken out the dependency on MULE for this feature because it's
         also useful in a non-Mule XEmacs where the user has set their font
         to something ending in iso8859-5. How many of those users there
         are, is another question. */
  Ichar alt_keychars[KEYCHAR_LAST];
};

the first level of indenting in the comments is from me; stuff i wrote 
back in 2001 or so.  the second level is from aidan.  what i wrote is 
heavily based on microsoft's ui style guide, as they have a long 
discussion of this issue.

note also that this algorithm *does* use the locale of the current 
buffer; this is certainly possible under windows, where there is an 
explicit api to query both the physical and logical keyboard.

it also seems that alphabetic and non-alphabetic keys should (perhaps) 
behave differently.

now, from personal experience: i have had many times when i've been in 
foreign countries and had to log on to the internet.  typically, the 
punctuation is in a completely different place.  i always switched to us 
layout, and found it nearly impossible to use any other layouts.  i 
*definitely* would expect in such a case that keyboard shortcuts 
involving punctuation should follow the logical, not physical, layout -- 
but with the physical layout as a backup, so when i temporarily switch 
to russian, i can still type C-x. (with alphabetic keys, it is 
semi-feasible to search the keyboard in front of me to find the keys, 
but this is just impossible for punctuation.)

ben

Paul Pogonyshev wrote:

>Aidan Kehoe wrote:
>  
>
>>[CCing XEmacs Beta, because despite it originating at emacs-devel@gnu.org,
>>this isn’t directly relevant to the FSF’s Emacs any more, but there’s more
>>value in keeping it visible than in keeping it in private mail.]
>>    
>>
>
>OK, I'll also CC it back to GNU Emacs' list for the same reason.
>
>  
>
>>[...]
>> > 
>> > Sorry, I'm on dial-up and I don't think I want to download megabytes just
>> > to test it.  I use GNU Emacs normally.
>>
>>Sure. I hope you don’t mind if I ask you a few specific questions then --
>>    
>>
>
>Sure.
>
>  
>
>>-- If you see M-; listed as a key binding, is the first thing that occurs to
>>you to type Alt+Shift+8? Or would you go for Alt+ж, since ж is where ; is on
>>the US keyboard? Should we accept both? 
>>    
>>
>
>I can't say which would be the most natural thing to hit, because few apps
>but Emacs use such extravagant shortcuts.
>
>Maybe I'd say like this: if an application uses _localized shortucts_ (i.e.
>is filled with things like `Ctrl-Щ' and stuff), I'd go for `Alt-Shift-8', else
>`Alt-ж' i.e. `Alt-;' on English layout.  Since Emacs is not localized in this
>way (and cannot and mustn't be, I think), typing `M-ж' for `M-;' is probably
>more natural.
>
>Localized key bindings are impossible for Emacs, since you cannot type say ']'
>with the Russian keyboard layout.  So, the only option is to type `C-]' as
>it is on English layout, which implies you must type all the other shortcuts
>like this, for UI consistency and to avoid clashes.
>
>  
>
>>-- More generally, does this need to be done just for the alphabetic
>>characters, or does punctuation need to be handled too? Looking at the
>>Russian key layout, there’s no way to type `, ^, $, so I suspect you’re
>>going to answer “yes” to me on that. 
>>    
>>
>
>I cannot answer `yes' to an `or' question ;)  Based on what I stated above,
>I'd say that all shortcuts, in Emacs at least, must be invariant to physical
>keys, not to logical characters the keys produce with the current input
>method/layout.
>
>This is probably also better ergonomically, since for power users (like me ;)
>key binding combos slip into subconsciousness.  I don't think my internal
>autopilot performs a ``hold the Meta key and type the semicolon, wherever it
>is'' command, rather a ``press the key here with the left thumb, and another
>one there with the third right finger.''  If it indeed works like that,
>shortcuts staying at the same physical keys require less effort to memorize/
>push into subconsciousness.
>
>  
>
>>-- When I switch to the Russian layout in software, and type C-ч и to call
>>“switch-to-buffer” , I then need to switch the keyboard layout back if I am
>>to type *scratch*, which I frequently want to do. Is there a reasonable
>>thing we can do there that doesn’t make it necessary to switch layout? 
>>Accepting “;ыскфеср;” as an equivalent buffer name for “*scratch*” doesn’t
>>really seem like a great idea to me; maybe you _wanted_ to create a buffer
>>with that name. I frequently create buffers named fdlsfdsfds and variations
>>on that, for example.
>>    
>>
>
>Accepting `;ыскфеср;' as `*scratch*' won't work as a generic solution for
>the reason you mentioned.  It may be an optional heuristic, turned off by
>default.  There were some interesting programs for Windows that would
>automagically switch the layout for you once you had started typing gibberish
>in the current layout.  I.e. if you typed ``Heccrbq'', it would consider it
>too weird an English word, backspace it and retype in Russian layout as
>``Русский''.  Likewise, it would automagically switch to English layout if
>you typed ``щчньщкщт'' instead of ``oxymoron''.  However, such heuristical
>things often work incorrectly with artificial languages, like programming
>ones, and must never be forced on the user.
>
>The only plausible generic solution I can think of is to track the layout for
>each buffer separately.  This may be very difficult to impossible to
>implement with external layout switching.  And that's precisely the reason
>why I prefer to use Emacs' own input methods switched by `C-\'.  This means
>that I have to use different layout switching methods in Emacs and elsewhere,
>but the advantages are more important for me.
>
>Another, less important, superiority of Emacs input methods for me is the
>ability to easily activate an otherwise not used input method.  I
>occasionally activate Greek and German input methods.  With Emacs it is
>relatively simple, while with say KDE it would mean that I'd have to either
>constantly scroll through 4 layouts or go into the Conrol Center each time
>I need one of the rarely used layouts.
>
>Paul
>
>  
>