From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#10299: Emacs doesn't handle Unicode characters in keyboard layout on MS Windows Date: Sat, 28 Jul 2012 17:50:06 +0300 Message-ID: <836298hrdt.fsf@gnu.org> References: <8739clgapc.fsf@gnu.org> <83zket20xw.fsf@gnu.org> <83vcph0w9t.fsf@gnu.org> <83obv821wv.fsf@gnu.org> <831us31atj.fsf@gnu.org> <83pqflzr1d.fsf@gnu.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE X-Trace: dough.gmane.org 1343487067 23247 80.91.229.3 (28 Jul 2012 14:51:07 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sat, 28 Jul 2012 14:51:07 +0000 (UTC) Cc: lekktu@gmail.com, 10299@debbugs.gnu.org To: Joakim =?UTF-8?Q?H=C3=A5rsman?= Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Jul 28 16:51:07 2012 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Sv8ME-0001Al-9P for geb-bug-gnu-emacs@m.gmane.org; Sat, 28 Jul 2012 16:51:06 +0200 Original-Received: from localhost ([::1]:57759 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Sv8MD-0000mH-Nz for geb-bug-gnu-emacs@m.gmane.org; Sat, 28 Jul 2012 10:51:05 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:54491) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Sv8MA-0000m3-Py for bug-gnu-emacs@gnu.org; Sat, 28 Jul 2012 10:51:04 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Sv8M8-0002NW-D7 for bug-gnu-emacs@gnu.org; Sat, 28 Jul 2012 10:51:02 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:37117) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Sv8M8-0002NP-8R for bug-gnu-emacs@gnu.org; Sat, 28 Jul 2012 10:51:00 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.72) (envelope-from ) id 1Sv8Sw-00054c-5q for bug-gnu-emacs@gnu.org; Sat, 28 Jul 2012 10:58:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 28 Jul 2012 14:58:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 10299 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 10299-submit@debbugs.gnu.org id=B10299.134348746719481 (code B ref 10299); Sat, 28 Jul 2012 14:58:02 +0000 Original-Received: (at 10299) by debbugs.gnu.org; 28 Jul 2012 14:57:47 +0000 Original-Received: from localhost ([127.0.0.1]:46663 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Sv8Sg-00054A-Uq for submit@debbugs.gnu.org; Sat, 28 Jul 2012 10:57:47 -0400 Original-Received: from mtaout22.012.net.il ([80.179.55.172]:36757) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1Sv8Sc-00053z-O8 for 10299@debbugs.gnu.org; Sat, 28 Jul 2012 10:57:44 -0400 Original-Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0M7V00300L2ST800@a-mtaout22.012.net.il> for 10299@debbugs.gnu.org; Sat, 28 Jul 2012 17:49:57 +0300 (IDT) Original-Received: from HOME-C4E4A596F7 ([87.69.210.75]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0M7V003W2L79P420@a-mtaout22.012.net.il>; Sat, 28 Jul 2012 17:49:57 +0300 (IDT) In-reply-to: X-012-Sender: halo1@inter.net.il X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:62502 Archived-At: > Date: Tue, 24 Jan 2012 21:42:46 +0100 > From: Joakim H=E5rsman > Cc: Andreas Schwab , 10299@debbugs.gnu.org >=20 > GetMessage is a macro, many Windows API functions are, and switches > between GetMessageA and GetMessageW based on whether UNICODE is > defined. I really should explicitly use GetMessageA, I just missed = it. >=20 > I switched to Stefan's suggestion, but with explicit use of > GetMessageA. Note that this makes that line longer than 80 chars. >=20 > Here's an amended patch that also moves the definition of the > INIT_WINDOW_CLASS macro to just before use, and makes > w32_set_frame_text static. Thanks. It's been a while since you submitted this, so sorry for the delay. We are now past Emacs 24.1 release, so we are free to install your changes. However, when I applied the patch, I bumped into several problems. First, I don't understand why we need this part: +/* Set text of w32 frame with handle HWND to TEXT. + + We explicitly switch between the Unicode and ANSI version of + SetWindowText because Emacs isn't compiled with UNICODE defined= to + retain compatibility with Windows 95. */ + +static void +w32_set_frame_text (HWND hwnd, LPCSTR text) +{ + if (os_subtype =3D=3D OS_NT) + SetWindowTextW (hwnd, (LPCWSTR)text); + else + SetWindowTextA (hwnd, text); +} /* Change the name of frame F to NAME. If NAME is nil, set F's na= me to =09 w32_id_name. @@ -1697,10 +1711,10 @@ if (FRAME_W32_WINDOW (f)) { =09 if (STRING_MULTIBYTE (name)) - name =3D ENCODE_SYSTEM (name); - + name =3D ENCODE_SYSTEM (name); + =09 BLOCK_INPUT; - SetWindowText (FRAME_W32_WINDOW (f), SDATA (name)); + w32_set_frame_text(FRAME_W32_WINDOW (f), SDATA (name)); =09 UNBLOCK_INPUT; } } @@ -1746,7 +1760,7 @@ =09 name =3D ENCODE_SYSTEM (name); =09 BLOCK_INPUT; - SetWindowText (FRAME_W32_WINDOW (f), SDATA (name)); + w32_set_frame_text(FRAME_W32_WINDOW (f), SDATA (name)); =09 UNBLOCK_INPUT; } } Why do we need to use SetWindowTextW, when all we change is the way w= e request messages for Emacs? With the patch below, which doesn't use this part, I see no problems at all with the titles of Emacs frames. Can you tell me how to reproduce the problem with the title? Moreover, passing 'SDATA (name)' to SetWindowTextW is wrong, since that function expects a wchar_t *, not a char *, as pointer to the text to display in the title. (The compiler should warn about that, didn't it?) So if we do need this part, we will need to convert the frame name into a wide character string. Next, this part, whose need was not entirely understood even when you submitted the patches: @@ -2915,8 +2948,21 @@ case WM_SYSCHAR: case WM_CHAR: - post_character_message (hwnd, msg, wParam, lParam, - w32_get_key_modifiers (wParam, lParam= )); + if (wParam > 255 ) + { + unsigned short lo =3D wParam & 0x0000FFFF; + unsigned short hi =3D (wParam & 0xFFFF0000) >> 8; + wParam =3D hi | lo; + + W32Msg wmsg; + wmsg.dwModifiers =3D w32_get_key_modifiers (wParam, lPar= am); + signal_user_input (); + my_post_msg (&wmsg, hwnd, WM_UNICHAR, wParam, lParam); + + } + else + post_character_message (hwnd, msg, wParam, lParam, + w32_get_key_modifiers (wParam, lPa= ram)); =09 break; This doesn't work in a locale where Unicode codepoints of the characters generated by the localized keyboard are above 255. (I liv= e in one such locale.) In those locales, this snippet produces the wrong characters. The root cause of this is that WM_CHAR's wParam is not really the Unicode codepoint of the character I type, it's some garbled value. After looking into this, my conclusion was that your patches didn't do everything that's needed to get true Unicode character input. This page: http://social.msdn.microsoft.com/forums/en-US/windowssdk/thread/07a= fec87-68c1-4a56-bf46-a38a9c2232e9/ suggests what needs to be done. So I went ahead and made the few additional changes mentioned there, and the Unicode input started working for me. With these additional changes, the funky bit-tweakin= g code is no longer needed, at least for the characters in the 0x05nn range of Unicode that I tried. I hope it will work with your custom keyboard as well. The patches I tried are below. I'd appreciate if you could try them and see if they work with your custom keyboard layouts. If they do, = I will install them. (Juanma, I'd appreciate if you could try this as well.) Thanks. =3D=3D=3D modified file 'src/w32fns.c' --- src/w32fns.c=092012-07-27 09:24:34 +0000 +++ src/w32fns.c=092012-07-28 14:28:29 +0000 @@ -1780,23 +1780,37 @@ w32_load_cursor (LPCTSTR name) =20 static LRESULT CALLBACK w32_wnd_proc (HWND, UINT, WPARAM, LPARAM); =20 +#define INIT_WINDOW_CLASS(WC)=09=09=09 \ + (WC).style =3D CS_HREDRAW | CS_VREDRAW;=09=09 \ + (WC).lpfnWndProc =3D (WNDPROC) w32_wnd_proc; \ + (WC).cbClsExtra =3D 0; \ + (WC).cbWndExtra =3D WND_EXTRA_BYTES; \ + (WC).hInstance =3D hinst; \ + (WC).hIcon =3D LoadIcon (hinst, EMACS_CLASS); \ + (WC).hCursor =3D w32_load_cursor (IDC_ARROW); \ + (WC).hbrBackground =3D NULL; \ + (WC).lpszMenuName =3D NULL; \ + static BOOL w32_init_class (HINSTANCE hinst) { - WNDCLASS wc; =20 - wc.style =3D CS_HREDRAW | CS_VREDRAW; - wc.lpfnWndProc =3D (WNDPROC) w32_wnd_proc; - wc.cbClsExtra =3D 0; - wc.cbWndExtra =3D WND_EXTRA_BYTES; - wc.hInstance =3D hinst; - wc.hIcon =3D LoadIcon (hinst, EMACS_CLASS); - wc.hCursor =3D w32_load_cursor (IDC_ARROW); - wc.hbrBackground =3D NULL; /* GetStockObject (WHITE_BRUSH); */ - wc.lpszMenuName =3D NULL; - wc.lpszClassName =3D EMACS_CLASS; + if (os_subtype =3D=3D OS_NT) + { + WNDCLASSW uwc; + INIT_WINDOW_CLASS(uwc); + uwc.lpszClassName =3D L"Emacs"; + + return RegisterClassW (&uwc); + } + else + { + WNDCLASS wc; + INIT_WINDOW_CLASS(wc); + wc.lpszClassName =3D EMACS_CLASS; =20 - return (RegisterClass (&wc)); + return RegisterClassA (&wc); + } } =20 static HWND @@ -2246,7 +2260,7 @@ w32_msg_pump (deferred_msg * msg_buf) =20 msh_mousewheel =3D RegisterWindowMessage (MSH_MOUSEWHEEL); =20 - while (GetMessage (&msg, NULL, 0, 0)) + while ((os_subtype =3D=3D OS_NT ? GetMessageW : GetMessageA) (&msg= , NULL, 0, 0)) { if (msg.hwnd =3D=3D NULL) =09{ @@ -2341,7 +2355,10 @@ w32_msg_pump (deferred_msg * msg_buf) =09} else =09{ -=09 DispatchMessage (&msg); +=09 if (os_subtype =3D=3D OS_NT) +=09 DispatchMessageW (&msg); +=09 else +=09 DispatchMessageA (&msg); =09} =20 /* Exit nested loop when our deferred message has completed. = */ @@ -2918,8 +2935,18 @@ w32_wnd_proc (HWND hwnd, UINT msg, WPARA =20 case WM_SYSCHAR: case WM_CHAR: - post_character_message (hwnd, msg, wParam, lParam, -=09=09=09 w32_get_key_modifiers (wParam, lParam)); + if (wParam > 255 ) + { + W32Msg wmsg; + + wmsg.dwModifiers =3D w32_get_key_modifiers (wParam, lParam= ); + signal_user_input (); + my_post_msg (&wmsg, hwnd, WM_UNICHAR, wParam, lParam); + + } + else + post_character_message (hwnd, msg, wParam, lParam, + w32_get_key_modifiers (wParam, lPara= m)); break; =20 case WM_UNICHAR: @@ -3801,7 +3828,7 @@ w32_wnd_proc (HWND hwnd, UINT msg, WPARA =09} =20 dflt: - return DefWindowProc (hwnd, msg, wParam, lParam); + return (os_subtype =3D=3D OS_NT ? DefWindowProcW : DefWindowP= rocA) (hwnd, msg, wParam, lParam); } =20 /* The most common default return code for handled messages is 0. = */