* change in X character input processing @ 2002-10-31 10:55 Dave Love 2002-10-31 15:20 ` Stefan Monnier 2002-10-31 15:31 ` Stefan Monnier 0 siblings, 2 replies; 15+ messages in thread From: Dave Love @ 2002-10-31 10:55 UTC (permalink / raw) I've changed the handling of character input under X to consider the definitions of keysyms rather than trying to decode everything for some locale which may be inappropriate. Thus, for instance, my euro key will work in a Latin-1 environment like the pound key (and doesn't depend on Latin-9 support in the X libraries). I think this DTRT generally, but if some sort of input doesn't work right, please let me know. There seems to be a shortage of useful documentation in this area. Note that a given keysym produces a character from a fixed charset, e.g. Odiaeresis always produces (make-char 'latin-iso8859-1 86). The Kana keysyms produce mule-unicode since I'm not sure how to map them onto CJK charsets from the defined unicodes; if anyone is sure how to translate them, please update the table in term/x-win.el which populates `x-keysym-table'. Note also that input via a Compose key and other sorts of X input methods will be different. Thus with typical XKB rules, `Compose e =' won't produce € in a Latin-1 locale but `AltGr 4' will. (That's using X's Multi_key and Mode_switch.) ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-10-31 10:55 change in X character input processing Dave Love @ 2002-10-31 15:20 ` Stefan Monnier 2002-11-01 13:54 ` Dave Love 2002-10-31 15:31 ` Stefan Monnier 1 sibling, 1 reply; 15+ messages in thread From: Stefan Monnier @ 2002-10-31 15:20 UTC (permalink / raw) Cc: emacs-devel > I've changed the handling of character input under X to consider the > definitions of keysyms rather than trying to decode everything for > some locale which may be inappropriate. Thus, for instance, my euro > key will work in a Latin-1 environment like the pound key (and doesn't > depend on Latin-9 support in the X libraries). IIUC this doesn't prevent the usual Xlib decoding of keysym, but only uses an auxiliary table x-keysym-table to map keysyms to characters in case where Xlib failed to decode the keysym (typically because it can't be represented in the user's locale). I.e. it is similar in behavior to my earlier patch which tried Xutf8LookupString if XmbLookupString failed (except that your patch doesn't rely on the deprecated Xutf8LookupString function or any other extra support in Xlib for that matter). Is that right ? Stefan ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-10-31 15:20 ` Stefan Monnier @ 2002-11-01 13:54 ` Dave Love 2002-11-01 15:54 ` Stefan Monnier 2002-11-01 19:53 ` Stefan Monnier 0 siblings, 2 replies; 15+ messages in thread From: Dave Love @ 2002-11-01 13:54 UTC (permalink / raw) Cc: emacs-devel "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes: > IIUC this doesn't prevent the usual Xlib decoding of keysym, but > only uses an auxiliary table x-keysym-table to map keysyms > to characters in case where Xlib failed to decode the keysym > (typically because it can't be represented in the user's > locale). The idea is to translate known character keysyms directly to the appropriate characters. This should be entirely straightforward, unlike the situation where you have to try to match X's and Emacs's idea of the encoding, potentially with input from a different locale from the one in which Emacs is running. Note that I don't understand XIM -- is there a good introduction? -- and I don't know whether Emacs really DTRT with XIM. I don't think what I've done affects it, anyhow, and if it causes problems it can be fixed or reverted. > I.e. it is similar in behavior to my earlier patch which tried > Xutf8LookupString if XmbLookupString failed (except that your patch > doesn't rely on the deprecated Xutf8LookupString function or any > other extra support in Xlib for that matter). I don't know about that, but presumably not. Avoiding Xlib restrictions is part of the idea, e.g. you can use a euro key without Latin-9 support from Xlib. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-11-01 13:54 ` Dave Love @ 2002-11-01 15:54 ` Stefan Monnier 2002-11-01 19:53 ` Stefan Monnier 1 sibling, 0 replies; 15+ messages in thread From: Stefan Monnier @ 2002-11-01 15:54 UTC (permalink / raw) Cc: Stefan Monnier, emacs-devel > "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes: > > > IIUC this doesn't prevent the usual Xlib decoding of keysym, but > > only uses an auxiliary table x-keysym-table to map keysyms > > to characters in case where Xlib failed to decode the keysym > > (typically because it can't be represented in the user's > > locale). > > The idea is to translate known character keysyms directly to the > appropriate characters. This should be entirely straightforward, > unlike the situation where you have to try to match X's and Emacs's > idea of the encoding, potentially with input from a different locale > from the one in which Emacs is running. Yes, I understand that part. The question is: when is your code used; in which case(s) does it change the previous behavior. > > I.e. it is similar in behavior to my earlier patch which tried > > Xutf8LookupString if XmbLookupString failed (except that your patch > > doesn't rely on the deprecated Xutf8LookupString function or any > > other extra support in Xlib for that matter). > I don't know about that, but presumably not. Actually, having looked at the code some more, I'd say "yes, it does the same thing, except it does it itself rather than with Xlib's help" > Avoiding Xlib > restrictions is part of the idea, e.g. you can use a euro key without > Latin-9 support from Xlib. Yes, but the behavior is the same: if XmbLookupString manages to decode the keysym into a char, then the old code is used. If it doesn't, my code tried Xutf8StringLookup whereas your code uses x-keysym-table. When the two table are similar, you get pretty much the same behavior. Not relying on Xlib is of course an advantage right now since unicode support in Xlib can't be relied upon. Stefan ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-11-01 13:54 ` Dave Love 2002-11-01 15:54 ` Stefan Monnier @ 2002-11-01 19:53 ` Stefan Monnier 2002-11-02 13:51 ` Dave Love 1 sibling, 1 reply; 15+ messages in thread From: Stefan Monnier @ 2002-11-01 19:53 UTC (permalink / raw) Cc: Stefan Monnier, emacs-devel The new code you installed works very differently from the one you had before. Now your x-keysym-table overrides the usual XmbLookupString+decode process This means that when you do LANG=fr_FR@euro emacs -q --no-site-file with your code an eacute will insert a latin-1 eacute whereas with the previous code it inserted a latin-9 eacute. I'm not sure we want to do that just now. I suggest we revert to the code you had before, so that the XmbLookupString+decode is used as before and x-keysym-table is only used if that failed (which is only the case if the key you hit translates into a char that doesn't exist in the coding-system corresponding to your locale, such as when you hit EuroSign in a latin-1 locale). The reason why I'd like to revert is that your code basically does a form of unify-on-keyboard-input and I'm not sure that we want to do that already since we don't have unify-on-decoding turned on by default. I.e. I suggest the patch below (against the version before your latest fix). See after my sig for the patch against the current version. *** src/xterm.c 30 Oct 2002 19:12:37 -0000 1.759 --- src/xterm.c 1 Nov 2002 19:47:22 -0000 *************** *** 10821,10850 **** if (temp_index == sizeof temp_buffer / sizeof (short)) temp_index = 0; temp_buffer[temp_index++] = keysym; ! /* First deal with keysyms which have ! defined translations to characters. */ ! if (keysym >= 32 && keysym < 128) ! /* Avoid explicitly decoding each ASCII ! character. */ { ! bufp->kind = ASCII_KEYSTROKE_EVENT; ! bufp->code = c; ! } ! else if (! EQ ((c = Fgethash (make_number (keysym), ! Vx_keysym_table, ! Qnil)), ! Qnil)) ! { ! bufp->kind = (SINGLE_BYTE_CHAR_P (c) ? ASCII_KEYSTROKE_EVENT : MULTIBYTE_CHAR_KEYSTROKE_EVENT); - bufp->code = c; } else { ! /* Not a character keysym. ! make_lispy_event will convert it to a ! symbolic key. */ bufp->kind = NON_ASCII_KEYSTROKE_EVENT; bufp->code = keysym; } --- 10822,10840 ---- if (temp_index == sizeof temp_buffer / sizeof (short)) temp_index = 0; temp_buffer[temp_index++] = keysym; ! if (NATNUMP (c = Fgethash (make_number (keysym), ! Vx_keysym_table, ! Qnil))) { ! bufp->code = XFASTINT (c); ! bufp->kind = (SINGLE_BYTE_CHAR_P (bufp->code) ? ASCII_KEYSTROKE_EVENT : MULTIBYTE_CHAR_KEYSTROKE_EVENT); } else { ! /* Not a character keysym. make_lispy_event ! will convert it to a symbolic key. */ bufp->kind = NON_ASCII_KEYSTROKE_EVENT; bufp->code = keysym; } As you can see all it does is remove the ASCII handling (because it basically can't happen here and because it uses variable `c' before it's initialized) and fixes some int/Lisp_Object mixup (and uses NATNUMP rather then EQ (..., Qnil)). Stefan Index: src/xterm.c =================================================================== RCS file: /cvsroot/emacs/emacs/src/xterm.c,v retrieving revision 1.760 diff -c -r1.760 xterm.c *** src/xterm.c 31 Oct 2002 17:59:30 -0000 1.760 --- src/xterm.c 1 Nov 2002 19:45:01 -0000 *************** *** 10738,10871 **** if (numchars > 1) { ! Lisp_Object c; ! ! /* First deal with keysyms which have defined ! translations to characters. */ ! if (keysym >= 32 && keysym < 128) ! /* Avoid explicitly decoding each ASCII character. */ ! { ! bufp->kind = ASCII_KEYSTROKE_EVENT; ! bufp->code = keysym; ! XSETFRAME (bufp->frame_or_window, f); ! bufp->arg = Qnil; ! bufp->modifiers ! = x_x_to_emacs_modifiers (FRAME_X_DISPLAY_INFO (f), ! modifiers); ! bufp->timestamp = event.xkey.time; ! bufp++; ! count++; ! numchars--; ! } ! /* Now non-ASCII. */ ! else if (! EQ ((c = Fgethash (make_number (keysym), ! Vx_keysym_table, Qnil)), ! Qnil)) ! { ! bufp->kind = (SINGLE_BYTE_CHAR_P (c) ! ? ASCII_KEYSTROKE_EVENT ! : MULTIBYTE_CHAR_KEYSTROKE_EVENT); ! bufp->code = c; ! XSETFRAME (bufp->frame_or_window, f); ! bufp->arg = Qnil; ! bufp->modifiers ! = x_x_to_emacs_modifiers (FRAME_X_DISPLAY_INFO (f), ! modifiers); ! bufp->timestamp = event.xkey.time; ! bufp++; ! count++; ! numchars--; ! } ! /* Random non-modifier sorts of keysyms. */ ! else if (((keysym >= XK_BackSpace && keysym <= XK_Escape) ! || keysym == XK_Delete #ifdef XK_ISO_Left_Tab ! || (keysym >= XK_ISO_Left_Tab ! && keysym <= XK_ISO_Enter) #endif ! || IsCursorKey (keysym) /* 0xff50 <= x < 0xff60 */ ! || IsMiscFunctionKey (keysym) /* 0xff60 <= x < VARIES */ #ifdef HPUX ! /* This recognizes the "extended function ! keys". It seems there's no cleaner way. ! Test IsModifierKey to avoid handling ! mode_switch incorrectly. */ ! || ((unsigned) (keysym) >= XK_Select ! && (unsigned)(keysym) < XK_KP_Space) #endif #ifdef XK_dead_circumflex ! || orig_keysym == XK_dead_circumflex #endif #ifdef XK_dead_grave ! || orig_keysym == XK_dead_grave #endif #ifdef XK_dead_tilde ! || orig_keysym == XK_dead_tilde #endif #ifdef XK_dead_diaeresis ! || orig_keysym == XK_dead_diaeresis #endif #ifdef XK_dead_macron ! || orig_keysym == XK_dead_macron #endif #ifdef XK_dead_degree ! || orig_keysym == XK_dead_degree #endif #ifdef XK_dead_acute ! || orig_keysym == XK_dead_acute #endif #ifdef XK_dead_cedilla ! || orig_keysym == XK_dead_cedilla #endif #ifdef XK_dead_breve ! || orig_keysym == XK_dead_breve #endif #ifdef XK_dead_ogonek ! || orig_keysym == XK_dead_ogonek #endif #ifdef XK_dead_caron ! || orig_keysym == XK_dead_caron #endif #ifdef XK_dead_doubleacute ! || orig_keysym == XK_dead_doubleacute #endif #ifdef XK_dead_abovedot ! || orig_keysym == XK_dead_abovedot #endif ! || IsKeypadKey (keysym) /* 0xff80 <= x < 0xffbe */ ! || IsFunctionKey (keysym) /* 0xffbe <= x < 0xffe1 */ ! /* Any "vendor-specific" key is ok. */ ! || (orig_keysym & (1 << 28)) ! || (keysym != NoSymbol && nbytes == 0)) ! && ! (IsModifierKey (orig_keysym) #ifndef HAVE_X11R5 #ifdef XK_Mode_switch ! || ((unsigned)(orig_keysym) == XK_Mode_switch) #endif #ifdef XK_Num_Lock ! || ((unsigned)(orig_keysym) == XK_Num_Lock) #endif #endif /* not HAVE_X11R5 */ ! /* The symbols from XK_ISO_Lock ! to XK_ISO_Last_Group_Lock ! don't have real modifiers but ! should be treated similarly to ! Mode_switch by Emacs. */ #if defined XK_ISO_Lock && defined XK_ISO_Last_Group_Lock ! || ((unsigned)(orig_keysym) ! >= XK_ISO_Lock ! && (unsigned)(orig_keysym) ! <= XK_ISO_Last_Group_Lock) #endif ! )) { if (temp_index == sizeof temp_buffer / sizeof (short)) temp_index = 0; temp_buffer[temp_index++] = keysym; ! /* make_lispy_event will convert this to a symbolic ! key. */ ! bufp->kind = NON_ASCII_KEYSTROKE_EVENT; ! bufp->code = keysym; XSETFRAME (bufp->frame_or_window, f); bufp->arg = Qnil; bufp->modifiers --- 10738,10843 ---- if (numchars > 1) { ! if (((keysym >= XK_BackSpace && keysym <= XK_Escape) ! || keysym == XK_Delete #ifdef XK_ISO_Left_Tab ! || (keysym >= XK_ISO_Left_Tab && keysym <= XK_ISO_Enter) #endif ! || (keysym >= XK_Kanji && keysym <= XK_Eisu_toggle) ! || IsCursorKey (keysym) /* 0xff50 <= x < 0xff60 */ ! || IsMiscFunctionKey (keysym) /* 0xff60 <= x < VARIES */ #ifdef HPUX ! /* This recognizes the "extended function keys". ! It seems there's no cleaner way. ! Test IsModifierKey to avoid handling mode_switch ! incorrectly. */ ! || ((unsigned) (keysym) >= XK_Select ! && (unsigned)(keysym) < XK_KP_Space) #endif #ifdef XK_dead_circumflex ! || orig_keysym == XK_dead_circumflex #endif #ifdef XK_dead_grave ! || orig_keysym == XK_dead_grave #endif #ifdef XK_dead_tilde ! || orig_keysym == XK_dead_tilde #endif #ifdef XK_dead_diaeresis ! || orig_keysym == XK_dead_diaeresis #endif #ifdef XK_dead_macron ! || orig_keysym == XK_dead_macron #endif #ifdef XK_dead_degree ! || orig_keysym == XK_dead_degree #endif #ifdef XK_dead_acute ! || orig_keysym == XK_dead_acute #endif #ifdef XK_dead_cedilla ! || orig_keysym == XK_dead_cedilla #endif #ifdef XK_dead_breve ! || orig_keysym == XK_dead_breve #endif #ifdef XK_dead_ogonek ! || orig_keysym == XK_dead_ogonek #endif #ifdef XK_dead_caron ! || orig_keysym == XK_dead_caron #endif #ifdef XK_dead_doubleacute ! || orig_keysym == XK_dead_doubleacute #endif #ifdef XK_dead_abovedot ! || orig_keysym == XK_dead_abovedot #endif ! || IsKeypadKey (keysym) /* 0xff80 <= x < 0xffbe */ ! || IsFunctionKey (keysym) /* 0xffbe <= x < 0xffe1 */ ! /* Any "vendor-specific" key is ok. */ ! || (orig_keysym & (1 << 28)) ! || (keysym != NoSymbol && nbytes == 0)) ! && ! (IsModifierKey (orig_keysym) #ifndef HAVE_X11R5 #ifdef XK_Mode_switch ! || ((unsigned)(orig_keysym) == XK_Mode_switch) #endif #ifdef XK_Num_Lock ! || ((unsigned)(orig_keysym) == XK_Num_Lock) #endif #endif /* not HAVE_X11R5 */ ! /* The symbols from XK_ISO_Lock to ! XK_ISO_Last_Group_Lock doesn't have real ! modifiers but should be treated similarly ! to Mode_switch by Emacs. */ #if defined XK_ISO_Lock && defined XK_ISO_Last_Group_Lock ! || ((unsigned)(orig_keysym) >= XK_ISO_Lock ! && (unsigned)(orig_keysym) <= XK_ISO_Last_Group_Lock) #endif ! )) { + Lisp_Object c; + if (temp_index == sizeof temp_buffer / sizeof (short)) temp_index = 0; temp_buffer[temp_index++] = keysym; ! if (NATNUMP (c = Fgethash (make_number (keysym), ! Vx_keysym_table, ! Qnil))) ! { ! bufp->code = XFASTINT (c); ! bufp->kind = (SINGLE_BYTE_CHAR_P (bufp->code) ! ? ASCII_KEYSTROKE_EVENT ! : MULTIBYTE_CHAR_KEYSTROKE_EVENT); ! } ! else ! { ! /* Not a character keysym. make_lispy_event ! will convert it to a symbolic key. */ ! bufp->kind = NON_ASCII_KEYSTROKE_EVENT; ! bufp->code = keysym; ! } XSETFRAME (bufp->frame_or_window, f); bufp->arg = Qnil; bufp->modifiers *************** *** 10877,10883 **** numchars--; } else if (numchars > nbytes) ! { /* Raw bytes, not keysym. */ register int i; register int c; int nchars, len; --- 10849,10855 ---- numchars--; } else if (numchars > nbytes) ! { register int i; register int c; int nchars, len; ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-11-01 19:53 ` Stefan Monnier @ 2002-11-02 13:51 ` Dave Love 2002-11-03 3:06 ` Stefan Monnier 0 siblings, 1 reply; 15+ messages in thread From: Dave Love @ 2002-11-02 13:51 UTC (permalink / raw) Cc: emacs-devel "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes: > The new code you installed works very differently from the one you had before. > Now your x-keysym-table overrides the usual XmbLookupString+decode process > This means that when you do > > LANG=fr_FR@euro emacs -q --no-site-file > > with your code an eacute will insert a latin-1 eacute whereas with > the previous code it inserted a latin-9 eacute. Sorry, I forgot the input translation table never got used properly. It should affect self-inserting characters, but I couldn't do that originally (in Lisp). The situation above is the same as for Quail methods, and I think it's strictly correct insofar as X uses iso2022 and defines the keysym to be Latin-1, but obviously it's not what you want. I'll sort out the translation table, then you'll get a high probability of translating the input character appropriately for the buffer context, regardless of the locale. I probably can't test that under X before Monday. If the current code is causing anyone problems, setting `x-keysym-table' to an empty hash table should give the same behaviour as before. [The input translation may be useful for more charsets than currently supported. See ucs-tables.el if anyone wants to add more tables that are useful and not too big. If they are big (2-dimensional charsets), they should probably just be added to the translation tables in appropriate language environments.] ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-11-02 13:51 ` Dave Love @ 2002-11-03 3:06 ` Stefan Monnier 2002-11-06 19:04 ` Dave Love 0 siblings, 1 reply; 15+ messages in thread From: Stefan Monnier @ 2002-11-03 3:06 UTC (permalink / raw) Cc: Stefan Monnier, emacs-devel > > The new code you installed works very differently from the one you had before. > > Now your x-keysym-table overrides the usual XmbLookupString+decode process > > This means that when you do > > > > LANG=fr_FR@euro emacs -q --no-site-file > > > > with your code an eacute will insert a latin-1 eacute whereas with > > the previous code it inserted a latin-9 eacute. > > Sorry, I forgot the input translation table never got used properly. > It should affect self-inserting characters, but I couldn't do that > originally (in Lisp). The situation above is the same as for Quail > methods, and I think it's strictly correct insofar as X uses iso2022 > and defines the keysym to be Latin-1, but obviously it's not what you > want. > > I'll sort out the translation table, then you'll get a high > probability of translating the input character appropriately for the > buffer context, regardless of the locale. I probably can't test that > under X before Monday. If I understand correctly, once the input translation table is setup, then the difference I pointed out above between the old code and the new code would mostly vanish: both would return a char in the charset most closely related to the buffer's coding system. Right? > If the current code is causing anyone problems, setting > `x-keysym-table' to an empty hash table should give the same behaviour > as before. That's a good point. I think my main question is: which mapping table should be preferred. The options we have already seen are: - originally, only one table was used: the Xlib table for the current locale. - my utf8 patch temporarily changed it to: first try the Xlib table for the current locale and if that fails, try the Xlib table for a utf-8 locale. - your first patch said: first try the Xlib table for the current locale and if that fails, try the x-keysym-table. - the current code does: try x-keysym-table and if that fails, use the Xlib table for the current locale. The use of x-keysym-table has the advantage of flexibility, since we don't depend on the Xlib at all. But I tend to prefer the Xlib table because I think of it as canonical since every application uses it. Among other things, that means that it is kept uptodate and populated without any work on our part. For that reason I'd prefer using the code that you first installed, which prefers the Xlib table to the x-keysym-table. Of course I also prefer it because it offers better backward compatibility. Stefan ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-11-03 3:06 ` Stefan Monnier @ 2002-11-06 19:04 ` Dave Love 2002-11-06 19:19 ` Stefan Monnier 0 siblings, 1 reply; 15+ messages in thread From: Dave Love @ 2002-11-06 19:04 UTC (permalink / raw) Cc: emacs-devel "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes: > If I understand correctly, once the input translation table is setup, > then the difference I pointed out above between the old code and the new > code would mostly vanish: both would return a char in the charset most > closely related to the buffer's coding system. Right? No. Previously nothing outside Quail tried to conform to the file coding system of the current buffer. > The use of x-keysym-table has the advantage of flexibility, since we > don't depend on the Xlib at all. But I tend to prefer the Xlib > table because I think of it as canonical since every application > uses it. As I understand it, there currently isn't a proper definition of the translation of many keysyms, and thus no guarantee of consistency across platforms. For instance, EuroSign, which my Sun keyboard generates under XFree, isn't defined on Solaris 8. > Among other things, that means that it is kept uptodate > and populated without any work on our part. But there isn't a single Xlib, especially one that can be relied to be at all `up-to-date' on random systems, and the input may be coming from a different system. I don't think it's any more reasonable to depend on Xlib than on iconv routines on the system, say. > For that reason I'd prefer using the code that you > first installed, which prefers the Xlib table to the x-keysym-table. > Of course I also prefer it because it offers better backward compatibility. I don't want it backwards-compatible, because that doesn't DTRT with high probability. Also it doesn't seem sensible to decode every character, though I don't know whether that's a significant inefficiency. Afraid I haven't got the keyboard translation sorted out yet, partly because I realized it should use `keyboard-translate-table'. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-11-06 19:04 ` Dave Love @ 2002-11-06 19:19 ` Stefan Monnier 2002-11-07 10:34 ` Kim F. Storm 2002-11-11 19:59 ` Dave Love 0 siblings, 2 replies; 15+ messages in thread From: Stefan Monnier @ 2002-11-06 19:19 UTC (permalink / raw) Cc: Stefan Monnier, emacs-devel > > If I understand correctly, once the input translation table is setup, > > then the difference I pointed out above between the old code and the new > > code would mostly vanish: both would return a char in the charset most > > closely related to the buffer's coding system. Right? > > No. Previously nothing outside Quail tried to conform to the file > coding system of the current buffer. I meant to compare old xterm.c code with new input translation table setup vs new xterm.c code with new input translation table setup > > The use of x-keysym-table has the advantage of flexibility, since we > > don't depend on the Xlib at all. But I tend to prefer the Xlib > > table because I think of it as canonical since every application > > uses it. > > As I understand it, there currently isn't a proper definition of the > translation of many keysyms, and thus no guarantee of consistency > across platforms. For instance, EuroSign, which my Sun keyboard > generates under XFree, isn't defined on Solaris 8. > > > Among other things, that means that it is kept uptodate > > and populated without any work on our part. > > But there isn't a single Xlib, especially one that can be relied to be > at all `up-to-date' on random systems, and the input may be coming > from a different system. I don't think it's any more reasonable to > depend on Xlib than on iconv routines on the system, say. There is one significant way in which it is more reasonable: 99% of the programs rely on it. So if your Xlib doesn't understand EuroSign, then none of your programs will understand it, so I don't see why Emacs should go through extra trouble: the problem should obviously be fixed in Xlib anyway. > > For that reason I'd prefer using the code that you > > first installed, which prefers the Xlib table to the x-keysym-table. > > Of course I also prefer it because it offers better backward compatibility. > I don't want it backwards-compatible, because that doesn't DTRT with > high probability. Could you give some example where the old code does turn the event into a char but doesn't do it right ? That's obviously the only relevant case since if it fails to turn it into a char we all agree that we should then use x-keysym-table. > Also it doesn't seem sensible to decode every > character, though I don't know whether that's a significant > inefficiency. Indeed, the efficiency aspect is irrelevant. > Afraid I haven't got the keyboard translation sorted out yet, partly > because I realized it should use `keyboard-translate-table'. I think as long as this is not fixed, your new code's behavior is wrong. Preferring the decode rather than the x-keysym-table would not suffer from this problem. That's one of the ways in which backward compatibility is beneficial. Stefan ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-11-06 19:19 ` Stefan Monnier @ 2002-11-07 10:34 ` Kim F. Storm 2002-11-11 20:08 ` Dave Love 2002-11-11 19:59 ` Dave Love 1 sibling, 1 reply; 15+ messages in thread From: Kim F. Storm @ 2002-11-07 10:34 UTC (permalink / raw) Cc: Dave Love, emacs-devel "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes: > There is one significant way in which it is more reasonable: 99% of > the programs rely on it. So if your Xlib doesn't understand EuroSign, > then none of your programs will understand it, so I don't see why > Emacs should go through extra trouble: the problem should obviously > be fixed in Xlib anyway. If you've got emacs, why care about those other 99% of the programs which you probably only use 1% of the time :-) :-) :-) Anyway, on my redhat 6.2 system the Xkeymap (or whatever it is called) doesn't even have a keycode for the AltGr-E on the DK-keymap, so emacs never get any keysym to process - Xlib or no Xlib... -- Kim F. Storm <storm@cua.dk> http://www.cua.dk ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-11-07 10:34 ` Kim F. Storm @ 2002-11-11 20:08 ` Dave Love 0 siblings, 0 replies; 15+ messages in thread From: Dave Love @ 2002-11-11 20:08 UTC (permalink / raw) Cc: Stefan Monnier, emacs-devel storm@cua.dk (Kim F. Storm) writes: > If you've got emacs, why care about those other 99% of the programs > which you probably only use 1% of the time :-) :-) :-) Why the emoticons? I don't use them at all for multilingual input, other than occasional tests of things in Yudit, I guess. (For what it's worth, the major non-English script I'm actually interested in using is maths, which has a lot in common with CJK. I partly pick examples like euro for familiarity, though I also want them to work well.) ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-11-06 19:19 ` Stefan Monnier 2002-11-07 10:34 ` Kim F. Storm @ 2002-11-11 19:59 ` Dave Love 2002-11-11 20:35 ` Stefan Monnier 1 sibling, 1 reply; 15+ messages in thread From: Dave Love @ 2002-11-11 19:59 UTC (permalink / raw) Cc: emacs-devel "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes: > > > If I understand correctly, once the input translation table is setup, > > > then the difference I pointed out above between the old code and the new > > > code would mostly vanish: both would return a char in the charset most > > > closely related to the buffer's coding system. Right? > > > > No. Previously nothing outside Quail tried to conform to the file > > coding system of the current buffer. > > I meant to compare > > old xterm.c code with new input translation table setup > vs > new xterm.c code with new input translation table setup In that case, yes. > > But there isn't a single Xlib, especially one that can be relied to be > > at all `up-to-date' on random systems, and the input may be coming > > from a different system. I don't think it's any more reasonable to > > depend on Xlib than on iconv routines on the system, say. > > There is one significant way in which it is more reasonable: 99% of > the programs rely on it. 99% of programs probably aren't multilingual. I don't know of another one that is, like Emacs, or guidance on writing one. I don't think that invalidates the point anyhow. > So if your Xlib doesn't understand EuroSign, > then none of your programs will understand it, Whether or not they understand it, they probably aren't going to do anything useful with it in a locale whose charset doesn't represent it. That isn't specific to euro. > so I don't see why Emacs should go through extra trouble: I've gone to the trouble anyway. > the problem should obviously be fixed in Xlib anyway. I think that's unrealistic. There are assorted things that doubtless should be done differently outside Emacs for which Emacs compensates. > > I don't want it backwards-compatible, because that doesn't DTRT with > > high probability. > > Could you give some example where the old code does turn the event into > a char but doesn't do it right ? I'm not sure you're talking about the same thing, but what I meant was: typing £ from a Latin-1 keyboard into a Latin-9 buffer didn't DTRT previously. > That's obviously the only relevant case since if it fails to turn it > into a char we all agree that we should then use x-keysym-table. I know how to DTRT at least as well as old Xlibs, the result will be consistent, and it can be customized. There may or may not be Xlibs which translate certain keysyms wrongly in given locales. I don't have examples, but I'd be surprised if they didn't exist. You're not at their mercy. If you think that character conversions are trivial and well-defined, consider alternativnyj as an 8-bit example, and jisx0208 as a multibyte one. Note also that there aren't even unicodes for some of the defined keysyms. > Indeed, the efficiency aspect is irrelevant. What are the figures? > > Afraid I haven't got the keyboard translation sorted out yet, partly > > because I realized it should use `keyboard-translate-table'. > > I think as long as this is not fixed, your new code's behavior > is wrong. I said that should be used. It is now (reverting to the original translation table usage following handa's comment). Are there now cases where it fails to DTRT according to what a user is likely to expect and where it reasonably could? ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-11-11 19:59 ` Dave Love @ 2002-11-11 20:35 ` Stefan Monnier 0 siblings, 0 replies; 15+ messages in thread From: Stefan Monnier @ 2002-11-11 20:35 UTC (permalink / raw) Cc: Stefan Monnier, emacs-devel > > > But there isn't a single Xlib, especially one that can be relied to be > > > at all `up-to-date' on random systems, and the input may be coming > > > from a different system. I don't think it's any more reasonable to > > > depend on Xlib than on iconv routines on the system, say. > > > > There is one significant way in which it is more reasonable: 99% of > > the programs rely on it. > > 99% of programs probably aren't multilingual. I don't know of another > one that is, like Emacs, or guidance on writing one. I don't think > that invalidates the point anyhow. No, indeed it doesn't invalidate the point because the point was not about multilingualism but about what happens when you generate a particular keysym_press event perfectly understood by Xlib in the current locale. > > So if your Xlib doesn't understand EuroSign, > > then none of your programs will understand it, > > Whether or not they understand it, they probably aren't going to do > anything useful with it in a locale whose charset doesn't represent > it. That isn't specific to euro. I'm not talking about the case where the locale cannot represent the character. As I've said repeatedly, in that case clearly the Emacs-21.3 code is not sufficient and something like x-keysym-table is necessary. I don't object at all to that part of your code. What I object to is that your code bypasses Xlib even in the cases where Xlib used to do things just fine. It is unnecessary and subtly changes the behavior of the code, leading to potential unexpected behaviors. My point is that we don't need to make such behavioral changes. > > > I don't want it backwards-compatible, because that doesn't DTRT with > > > high probability. > > > > Could you give some example where the old code does turn the event into > > a char but doesn't do it right ? > > I'm not sure you're talking about the same thing, but what I meant > was: typing £ from a Latin-1 keyboard into a Latin-9 buffer didn't > DTRT previously. I think you know by now that "doesn't DTRT" is not a useful description of a problem. AFAIK, it worked fine: it inserted a latin-1 £ and your x-keysym-table code doesn't make any change here. The translation-table-for-input change does make a difference, but it is unrelated to the topic I'm discussing. (except for the fact that it hides most of the incompatible changes introduced by your x-keysym-table change that I'm objecting to). > and well-defined, consider alternativnyj as an 8-bit example, and > jisx0208 as a multibyte one. Note also that there aren't even > unicodes for some of the defined keysyms. I don't see in what way this is relevant. > > Indeed, the efficiency aspect is irrelevant. > What are the figures? It's done once per keypress. If decoding a couple of bytes takes a significant part of the keypress-handling time, then we're in real trouble. > > > Afraid I haven't got the keyboard translation sorted out yet, partly > > > because I realized it should use `keyboard-translate-table'. > > > > I think as long as this is not fixed, your new code's behavior > > is wrong. > > I said that should be used. It is now (reverting to the original > translation table usage following handa's comment). > > Are there now cases where it fails to DTRT according to what a user is > likely to expect and where it reasonably could? I don't think so, but if the code only used x-keysym-table when Xlib failed it would work just as well, so I really don't see why we should introduce this risk of incompatible change. What is the big deal with reverting to "try Xlib first and if that fails, try x-keysym-table" ? It would turn your x-keysym-table feature into an "obviously safe" thing, whereas right now, it's only "probably safe provided translation-table-for-input is setup corrctly". Stefan PS: the current code still behaves differently in that the events corresponding to a keypress have not gone through translation-table-for-input yet, so in a latin-9 locale, those events used to be latin-9 chars whereas they are now mostly-latin-1-chars. This shouldn't make too much of a difference in practice (except when people create bindings for non-ASCII keys maybe, or a few pieces of code that do something similar to self-insert-command). ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-10-31 10:55 change in X character input processing Dave Love 2002-10-31 15:20 ` Stefan Monnier @ 2002-10-31 15:31 ` Stefan Monnier 2002-11-01 13:58 ` Dave Love 1 sibling, 1 reply; 15+ messages in thread From: Stefan Monnier @ 2002-10-31 15:31 UTC (permalink / raw) Cc: emacs-devel > I've changed the handling of character input under X to consider the > definitions of keysyms rather than trying to decode everything for > some locale which may be inappropriate. Thus, for instance, my euro > key will work in a Latin-1 environment like the pound key (and doesn't > depend on Latin-9 support in the X libraries). BTW, in the code I see you do: /* First deal with keysyms which have defined translations to characters. */ if (keysym >= 32 && keysym < 128) /* Avoid explicitly decoding each ASCII character. */ { bufp->kind = ASCII_KEYSTROKE_EVENT; bufp->code = c; } Have you encountered cases where this can happen ? I thought this branch of the code can never be taken if the keysym is an ASCII char (except for special keysyms explicitly listed, it's taken only if (keysym != NoSymbol && nbytes == 0) which means that XLookupString failed to decode the keysym into a char which seems impossible if the keysym is an ASCII char). Stefan ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: change in X character input processing 2002-10-31 15:31 ` Stefan Monnier @ 2002-11-01 13:58 ` Dave Love 0 siblings, 0 replies; 15+ messages in thread From: Dave Love @ 2002-11-01 13:58 UTC (permalink / raw) Cc: emacs-devel "Stefan Monnier" <monnier+gnu/emacs@rum.cs.yale.edu> writes: > Have you encountered cases where this can happen ? Well spotted. I had completely messed up a merge and didn't realize on a quick test, but I think it's right now. I _had_ exercised it properly in a different context. It's a nightmare keeping several things in sync mostly down a phone line and trying to get it right in a code base I don't run. By the way, why were the event kinds all renamed like that? It would have made more sense to me to remove `_event' suffixes rather than add them. ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2002-11-11 20:35 UTC | newest] Thread overview: 15+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2002-10-31 10:55 change in X character input processing Dave Love 2002-10-31 15:20 ` Stefan Monnier 2002-11-01 13:54 ` Dave Love 2002-11-01 15:54 ` Stefan Monnier 2002-11-01 19:53 ` Stefan Monnier 2002-11-02 13:51 ` Dave Love 2002-11-03 3:06 ` Stefan Monnier 2002-11-06 19:04 ` Dave Love 2002-11-06 19:19 ` Stefan Monnier 2002-11-07 10:34 ` Kim F. Storm 2002-11-11 20:08 ` Dave Love 2002-11-11 19:59 ` Dave Love 2002-11-11 20:35 ` Stefan Monnier 2002-10-31 15:31 ` Stefan Monnier 2002-11-01 13:58 ` Dave Love
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.