From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Joakim =?UTF-8?Q?H=C3=A5rsman?= Newsgroups: gmane.emacs.bugs Subject: bug#10299: Emacs doesn't handle Unicode characters in keyboard layout on MS Windows Date: Mon, 6 Aug 2012 22:20:01 +0200 Message-ID: References: <8739clgapc.fsf@gnu.org> <83zket20xw.fsf@gnu.org> <83vcph0w9t.fsf@gnu.org> <83obv821wv.fsf@gnu.org> <831us31atj.fsf@gnu.org> <83pqflzr1d.fsf@gnu.org> <836298hrdt.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Trace: dough.gmane.org 1344284475 29081 80.91.229.3 (6 Aug 2012 20:21:15 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Mon, 6 Aug 2012 20:21:15 +0000 (UTC) Cc: lekktu@gmail.com, 10299@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Mon Aug 06 22:21:15 2012 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1SyTne-0003dS-Bj for geb-bug-gnu-emacs@m.gmane.org; Mon, 06 Aug 2012 22:21:14 +0200 Original-Received: from localhost ([::1]:54134 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SyTnd-0004PI-KH for geb-bug-gnu-emacs@m.gmane.org; Mon, 06 Aug 2012 16:21:13 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:56607) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SyTnZ-0004P1-9o for bug-gnu-emacs@gnu.org; Mon, 06 Aug 2012 16:21:11 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SyTnX-0004Ik-HE for bug-gnu-emacs@gnu.org; Mon, 06 Aug 2012 16:21:09 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:56772) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SyTnX-0004Ig-DO for bug-gnu-emacs@gnu.org; Mon, 06 Aug 2012 16:21:07 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.72) (envelope-from ) id 1SyTvB-0007cN-Jy for bug-gnu-emacs@gnu.org; Mon, 06 Aug 2012 16:29:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Joakim =?UTF-8?Q?H=C3=A5rsman?= Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 06 Aug 2012 20:29:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 10299 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 10299-submit@debbugs.gnu.org id=B10299.134428488229211 (code B ref 10299); Mon, 06 Aug 2012 20:29:01 +0000 Original-Received: (at 10299) by debbugs.gnu.org; 6 Aug 2012 20:28:02 +0000 Original-Received: from localhost ([127.0.0.1]:38085 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SyTuD-0007ao-1C for submit@debbugs.gnu.org; Mon, 06 Aug 2012 16:28:02 -0400 Original-Received: from mail-gg0-f172.google.com ([209.85.161.172]:48378) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SyTu9-0007ae-7U for 10299@debbugs.gnu.org; Mon, 06 Aug 2012 16:27:59 -0400 Original-Received: by ggnc4 with SMTP id c4so3058207ggn.3 for <10299@debbugs.gnu.org>; Mon, 06 Aug 2012 13:20:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=lHJptTfRIxtW4khXSenQJ8hMuFeSU8+qJu6X+DqnJiE=; b=CF+7aqU3IDPQ4nvCvG3+tVvbheAYiSMCCuc6NB6LcOG9ZokmIg1TYMIPatiGkIQraD LwDstcdXbdIpDWxSThCcYb4gS7pCVJegmzNBLD9Fg2qCHPkO41iUYwG+pcQMeOKCj7C4 QVdv3p0tZT5XRnSZyvgVhtDTETYHJ2toR6/xCOeQ3RqyBY+6DEORoNfQ+GVOTqBZfw1Y MMl3lS+9OQ9r/LePAh5yas4q8YLQTt7Nxuydzv7S96YCzYG/IbhN/WGrxbgCn3fg2HWV XW1Eo9cTDA5/aJM3hA9Y5ZqzK01YYola8AeIKgEtZeIuqLcgejWN0rC/iMYDVXdA04pq BeBg== Original-Received: by 10.50.6.229 with SMTP id e5mr6633193iga.9.1344284401527; Mon, 06 Aug 2012 13:20:01 -0700 (PDT) Original-Received: by 10.50.45.70 with HTTP; Mon, 6 Aug 2012 13:20:01 -0700 (PDT) In-Reply-To: <836298hrdt.fsf@gnu.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:62882 Archived-At: Sorry for the late reply, I've been away on vacation. The change to SetWindowTextW for frame titles was needed because otherwise the frame title was cut short after the first character (presumably because the window class name is a UCS-2 wide string and the second byte therefore is null). This is exaclty the case they talk about at your link. If I change w32_set frame text to always call SetWindowText I get a default frame title of "e". Changing DispatchMessage to use the wide version when appropriate doesn't help either. I'm not surprised the WM_CHAR bit twiddling was wrong since the actual format was so badly documented, your version looks much better. I'll try your new patch tomorrow and get back with details on how it worked= , On 28 July 2012 16:50, Eli Zaretskii wrote: >> Date: Tue, 24 Jan 2012 21:42:46 +0100 >> From: Joakim H=E5rsman >> Cc: Andreas Schwab , 10299@debbugs.gnu.org >> >> GetMessage is a macro, many Windows API functions are, and switches >> between GetMessageA and GetMessageW based on whether UNICODE is >> defined. I really should explicitly use GetMessageA, I just missed it. >> >> I switched to Stefan's suggestion, but with explicit use of >> GetMessageA. Note that this makes that line longer than 80 chars. >> >> Here's an amended patch that also moves the definition of the >> INIT_WINDOW_CLASS macro to just before use, and makes >> w32_set_frame_text static. > > Thanks. It's been a while since you submitted this, so sorry for the > delay. We are now past Emacs 24.1 release, so we are free to install > your changes. > > However, when I applied the patch, I bumped into several problems. > First, I don't understand why we need this part: > > +/* Set text of w32 frame with handle HWND to TEXT. > + > + We explicitly switch between the Unicode and ANSI version of > + SetWindowText because Emacs isn't compiled with UNICODE defined to > + retain compatibility with Windows 95. */ > + > +static void > +w32_set_frame_text (HWND hwnd, LPCSTR text) > +{ > + if (os_subtype =3D=3D OS_NT) > + SetWindowTextW (hwnd, (LPCWSTR)text); > + else > + SetWindowTextA (hwnd, text); > +} > > /* Change the name of frame F to NAME. If NAME is nil, set F's name t= o > w32_id_name. > @@ -1697,10 +1711,10 @@ > if (FRAME_W32_WINDOW (f)) > { > if (STRING_MULTIBYTE (name)) > - name =3D ENCODE_SYSTEM (name); > - > + name =3D ENCODE_SYSTEM (name); > + > BLOCK_INPUT; > - SetWindowText (FRAME_W32_WINDOW (f), SDATA (name)); > + w32_set_frame_text(FRAME_W32_WINDOW (f), SDATA (name)); > UNBLOCK_INPUT; > } > } > @@ -1746,7 +1760,7 @@ > name =3D ENCODE_SYSTEM (name); > > BLOCK_INPUT; > - SetWindowText (FRAME_W32_WINDOW (f), SDATA (name)); > + w32_set_frame_text(FRAME_W32_WINDOW (f), SDATA (name)); > UNBLOCK_INPUT; > } > } > > Why do we need to use SetWindowTextW, when all we change is the way we > request messages for Emacs? With the patch below, which doesn't use > this part, I see no problems at all with the titles of Emacs frames. > Can you tell me how to reproduce the problem with the title? > > Moreover, passing 'SDATA (name)' to SetWindowTextW is wrong, since > that function expects a wchar_t *, not a char *, as pointer to the > text to display in the title. (The compiler should warn about that, > didn't it?) So if we do need this part, we will need to convert the > frame name into a wide character string. > > Next, this part, whose need was not entirely understood even when you > submitted the patches: > > @@ -2915,8 +2948,21 @@ > > case WM_SYSCHAR: > case WM_CHAR: > - post_character_message (hwnd, msg, wParam, lParam, > - w32_get_key_modifiers (wParam, lParam)); > + if (wParam > 255 ) > + { > + unsigned short lo =3D wParam & 0x0000FFFF; > + unsigned short hi =3D (wParam & 0xFFFF0000) >> 8; > + wParam =3D hi | lo; > + > + W32Msg wmsg; > + wmsg.dwModifiers =3D w32_get_key_modifiers (wParam, lParam); > + signal_user_input (); > + my_post_msg (&wmsg, hwnd, WM_UNICHAR, wParam, lParam); > + > + } > + else > + post_character_message (hwnd, msg, wParam, lParam, > + w32_get_key_modifiers (wParam, lParam)= ); > break; > > This doesn't work in a locale where Unicode codepoints of the > characters generated by the localized keyboard are above 255. (I live > in one such locale.) In those locales, this snippet produces the > wrong characters. The root cause of this is that WM_CHAR's wParam is > not really the Unicode codepoint of the character I type, it's some > garbled value. After looking into this, my conclusion was that your > patches didn't do everything that's needed to get true Unicode > character input. This page: > > http://social.msdn.microsoft.com/forums/en-US/windowssdk/thread/07afec8= 7-68c1-4a56-bf46-a38a9c2232e9/ > > suggests what needs to be done. So I went ahead and made the few > additional changes mentioned there, and the Unicode input started > working for me. With these additional changes, the funky bit-tweaking > code is no longer needed, at least for the characters in the 0x05nn > range of Unicode that I tried. I hope it will work with your custom > keyboard as well. > > The patches I tried are below. I'd appreciate if you could try them > and see if they work with your custom keyboard layouts. If they do, I > will install them. > > (Juanma, I'd appreciate if you could try this as well.) > > Thanks. > > > =3D=3D=3D modified file 'src/w32fns.c' > --- src/w32fns.c 2012-07-27 09:24:34 +0000 > +++ src/w32fns.c 2012-07-28 14:28:29 +0000 > @@ -1780,23 +1780,37 @@ w32_load_cursor (LPCTSTR name) > > static LRESULT CALLBACK w32_wnd_proc (HWND, UINT, WPARAM, LPARAM); > > +#define INIT_WINDOW_CLASS(WC) \ > + (WC).style =3D CS_HREDRAW | CS_VREDRAW; \ > + (WC).lpfnWndProc =3D (WNDPROC) w32_wnd_proc; \ > + (WC).cbClsExtra =3D 0; \ > + (WC).cbWndExtra =3D WND_EXTRA_BYTES; \ > + (WC).hInstance =3D hinst; \ > + (WC).hIcon =3D LoadIcon (hinst, EMACS_CLASS); \ > + (WC).hCursor =3D w32_load_cursor (IDC_ARROW); \ > + (WC).hbrBackground =3D NULL; \ > + (WC).lpszMenuName =3D NULL; \ > + > static BOOL > w32_init_class (HINSTANCE hinst) > { > - WNDCLASS wc; > > - wc.style =3D CS_HREDRAW | CS_VREDRAW; > - wc.lpfnWndProc =3D (WNDPROC) w32_wnd_proc; > - wc.cbClsExtra =3D 0; > - wc.cbWndExtra =3D WND_EXTRA_BYTES; > - wc.hInstance =3D hinst; > - wc.hIcon =3D LoadIcon (hinst, EMACS_CLASS); > - wc.hCursor =3D w32_load_cursor (IDC_ARROW); > - wc.hbrBackground =3D NULL; /* GetStockObject (WHITE_BRUSH); */ > - wc.lpszMenuName =3D NULL; > - wc.lpszClassName =3D EMACS_CLASS; > + if (os_subtype =3D=3D OS_NT) > + { > + WNDCLASSW uwc; > + INIT_WINDOW_CLASS(uwc); > + uwc.lpszClassName =3D L"Emacs"; > + > + return RegisterClassW (&uwc); > + } > + else > + { > + WNDCLASS wc; > + INIT_WINDOW_CLASS(wc); > + wc.lpszClassName =3D EMACS_CLASS; > > - return (RegisterClass (&wc)); > + return RegisterClassA (&wc); > + } > } > > static HWND > @@ -2246,7 +2260,7 @@ w32_msg_pump (deferred_msg * msg_buf) > > msh_mousewheel =3D RegisterWindowMessage (MSH_MOUSEWHEEL); > > - while (GetMessage (&msg, NULL, 0, 0)) > + while ((os_subtype =3D=3D OS_NT ? GetMessageW : GetMessageA) (&msg, NU= LL, 0, 0)) > { > if (msg.hwnd =3D=3D NULL) > { > @@ -2341,7 +2355,10 @@ w32_msg_pump (deferred_msg * msg_buf) > } > else > { > - DispatchMessage (&msg); > + if (os_subtype =3D=3D OS_NT) > + DispatchMessageW (&msg); > + else > + DispatchMessageA (&msg); > } > > /* Exit nested loop when our deferred message has completed. */ > @@ -2918,8 +2935,18 @@ w32_wnd_proc (HWND hwnd, UINT msg, WPARA > > case WM_SYSCHAR: > case WM_CHAR: > - post_character_message (hwnd, msg, wParam, lParam, > - w32_get_key_modifiers (wParam, lParam)); > + if (wParam > 255 ) > + { > + W32Msg wmsg; > + > + wmsg.dwModifiers =3D w32_get_key_modifiers (wParam, lParam); > + signal_user_input (); > + my_post_msg (&wmsg, hwnd, WM_UNICHAR, wParam, lParam); > + > + } > + else > + post_character_message (hwnd, msg, wParam, lParam, > + w32_get_key_modifiers (wParam, lParam)); > break; > > case WM_UNICHAR: > @@ -3801,7 +3828,7 @@ w32_wnd_proc (HWND hwnd, UINT msg, WPARA > } > > dflt: > - return DefWindowProc (hwnd, msg, wParam, lParam); > + return (os_subtype =3D=3D OS_NT ? DefWindowProcW : DefWindowProcA= ) (hwnd, msg, wParam, lParam); > } > > /* The most common default return code for handled messages is 0. */ >