From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#10299: Emacs doesn't handle Unicode characters in keyboard layout on MS Windows Date: Sat, 17 Dec 2011 17:23:36 +0200 Message-ID: <831us31atj.fsf@gnu.org> References: <8739clgapc.fsf@gnu.org> <83zket20xw.fsf@gnu.org> <83vcph0w9t.fsf@gnu.org> <83obv821wv.fsf@gnu.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE X-Trace: dough.gmane.org 1324135471 26873 80.91.229.12 (17 Dec 2011 15:24:31 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sat, 17 Dec 2011 15:24:31 +0000 (UTC) Cc: 10299@debbugs.gnu.org To: Joakim =?UTF-8?Q?H=C3=A5rsman?= Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Dec 17 16:24:27 2011 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Rbw7e-0006ID-Kk for geb-bug-gnu-emacs@m.gmane.org; Sat, 17 Dec 2011 16:24:26 +0100 Original-Received: from localhost ([::1]:55455 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Rbw7e-0008Q7-6d for geb-bug-gnu-emacs@m.gmane.org; Sat, 17 Dec 2011 10:24:26 -0500 Original-Received: from eggs.gnu.org ([140.186.70.92]:57012) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Rbw7a-0008Pv-MW for bug-gnu-emacs@gnu.org; Sat, 17 Dec 2011 10:24:23 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Rbw7Z-0005xK-Dc for bug-gnu-emacs@gnu.org; Sat, 17 Dec 2011 10:24:22 -0500 Original-Received: from debbugs.gnu.org ([140.186.70.43]:41546) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Rbw7Z-0005xF-9R for bug-gnu-emacs@gnu.org; Sat, 17 Dec 2011 10:24:21 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.69) (envelope-from ) id 1Rbw9C-0007Qx-Dj for bug-gnu-emacs@gnu.org; Sat, 17 Dec 2011 10:26:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 17 Dec 2011 15:26:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 10299 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 10299-submit@debbugs.gnu.org id=B10299.132413551728523 (code B ref 10299); Sat, 17 Dec 2011 15:26:02 +0000 Original-Received: (at 10299) by debbugs.gnu.org; 17 Dec 2011 15:25:17 +0000 Original-Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Rbw8T-0007Pz-Cx for submit@debbugs.gnu.org; Sat, 17 Dec 2011 10:25:17 -0500 Original-Received: from mtaout20.012.net.il ([80.179.55.166]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Rbw8R-0007Pq-C1 for 10299@debbugs.gnu.org; Sat, 17 Dec 2011 10:25:16 -0500 Original-Received: from conversion-daemon.a-mtaout20.012.net.il by a-mtaout20.012.net.il (HyperSendmail v2007.08) id <0LWC00A00TAIH300@a-mtaout20.012.net.il> for 10299@debbugs.gnu.org; Sat, 17 Dec 2011 17:23:32 +0200 (IST) Original-Received: from HOME-C4E4A596F7 ([77.127.39.203]) by a-mtaout20.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0LWC00ANXTF7CV20@a-mtaout20.012.net.il>; Sat, 17 Dec 2011 17:23:32 +0200 (IST) In-reply-to: X-012-Sender: halo1@inter.net.il X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list Resent-Date: Sat, 17 Dec 2011 10:26:02 -0500 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:55036 Archived-At: > Date: Sat, 17 Dec 2011 13:52:29 +0100 > From: Joakim H=E5rsman > Cc: 10299@debbugs.gnu.org >=20 > I had to switch to iswalpha because isalpha segfaulted when sent > values larger than 255. wParam isn't really the Unicode code point > here either, it's encoded as UTF-16. With this change, Emacs no lon= ger > prints a question mark when I press the special keys, it doesn't pr= int > anything at all. >=20 > I figured this is beacause wParam isn't a valid Unicode codepoint > here, like it is when you get a WM_UNICHAR, so I tried a quick hack > converting from UTF-16 and re-posting as a WM_UNICHAR even. I chang= ed > the handling of WM_CHAR in w32_wnd_proc: >=20 > case WM_CHAR: > if (wParam > 255 ) > { > /* Hacky conversion from UTF-16 to UCS-4. > Doesn't handle surrogate pairs. */ > unsigned short lo =3D wParam & 0x0000FFFF; > unsigned short hi =3D (wParam & 0xFFFF0000) >> 8; > wParam =3D hi | lo; >=20 > W32Msg wmsg; > wmsg.dwModifiers =3D w32_get_key_modifiers (wParam, lPara= m); > signal_user_input (); > my_post_msg (&wmsg, hwnd, WM_UNICHAR, wParam, lParam); >=20 > } > else > post_character_message (hwnd, msg, wParam, lParam, > w32_get_key_modifiers (wParam, lPar= am)); >=20 > This fixed it! I can now enter the characters with a custom keyboar= d > layout so I'm happy. That's good news. However, I'm puzzled: are you saying that the code points passed by Windows to Emacs for the characters generated by MKL= C are outside the Unicode BMP, i.e. larger than 65535? If so, what cod= e points are they? > There's loads of bugs still (e.g. the frame title > is "e" instead of "Emacs@host" because I just pass the frame's name= buf > to CreateWindowW without converting it to a wide string), but at le= ast > the main issue is fixed. If at all possible, I'd like to keep the use of wide APIs to the absolute minimum, because: > I guess these changes need to only happen when running on NT for Em= acs > to still work on Windows 95 or does Emacs use MSLU? We do use MSLU on Windows 9X (because the display routines need that). However, given that none of the developers has easy access to a 9X machine, it is very easy to break Emacs on 9X by excess usage of wide APIs. In fact, all versions of Emacs 23 didn't work on Windows 9X and only recently we were lucky enough to find a 9X user who helpe= d us debug the problems and fix them. So if possible, could you perhaps try finding out why you didn't get UTF-16 codepoints without CreateWindowW and GetMessageW? Maybe it's TranslateMessage that causes that, and the original message Emacs get= s is okay? > Maybe they should depend on some lisp variable being set? Probably just on the value of os_subtype. > Here's my changes in full so far: Thanks. This will probably need to wait until after v24.1 is released.