bug#396: 23.0.60; Quail (swedish postfix) broken in console

bug#396: 23.0.60; Quail (swedish postfix) broken in console

[E. Anders Lannerbäck]

Since the merge of the unicode branch on Feb 1, the quail package
swedish-postfix has been broken in console.  When using X11 it works.
Also, there is no problem to use the Swedish keyboard, so I believe that
the problem is in quail.

How to reproduce:

emacs -q -nw

C-x b f o o RET C-x RET C-\ s w e d i s h - p o s t
f i x RET a a SPC SPC

Instead of the string "å  " I end up with 堠 (#x5820).  

Regards
/Anders L

bug#396: marked as done (23.0.60; Quail (swedish postfix) broken in console)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 889 bytes --]


Your message dated Sun, 15 Jun 2008 00:46:36 -0400
with message-id <jwvve0b73vy.fsf-monnier+emacsbugreports@gnu.org>
and subject line Re: bug#396: 23.0.60; Quail (swedish postfix) broken in console
has caused the Emacs bug report #396,
regarding 23.0.60; Quail (swedish postfix) broken in console
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact don@donarmstrong.com
immediately.)


-- 
396: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=396
Emacs Bug Tracking System
Contact don@donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 2672 bytes --]

From: "E. Anders Lannerbäck" <anders@lannerback.net>
To: emacs-pretest-bug@gnu.org
Subject: 23.0.60; Quail (swedish postfix) broken in console
Date: Thu, 12 Jun 2008 14:56:02 +0200 (CEST)
Message-ID: <20080612125602.842C35C036@sol.lannerback.net>


Since the merge of the unicode branch on Feb 1, the quail package
swedish-postfix has been broken in console.  When using X11 it works.
Also, there is no problem to use the Swedish keyboard, so I believe that
the problem is in quail.

How to reproduce:

emacs -q -nw

C-x b f o o RET C-x RET C-\ s w e d i s h - p o s t
f i x RET a a SPC SPC

Instead of the string "å  " I end up with 堠 (#x5820).  

Regards
/Anders L



[-- Attachment #3: Type: message/rfc822, Size: 4404 bytes --]

From: Stefan Monnier <monnier@iro.umontreal.ca>
To: 396-close@emacsbugs.donarmstrong.com, Kenichi Handa <handa@m17n.org>
Subject: Re: bug#396: 23.0.60; Quail (swedish postfix) broken in console
Date: Sun, 15 Jun 2008 00:46:36 -0400
Message-ID: <jwvve0b73vy.fsf-monnier+emacsbugreports@gnu.org>

> Since the merge of the unicode branch on Feb 1, the quail package
> swedish-postfix has been broken in console.  When using X11 it works.
> Also, there is no problem to use the Swedish keyboard, so I believe that
> the problem is in quail.

> C-x b f o o RET C-x RET C-\ s w e d i s h - p o s t
> f i x RET a a SPC SPC

> Instead of the string "å  " I end up with 堠 (#x5820).  

Yuck!
Here's what seems to be happening: quail takes a sequence of bytes from
your terminal, interprets them as ascii chars, and turns this sequence
into another sequence, which is composed of latin-1 chars, but which
encoded-kbd then takes for bytes coming straight from your terminal, so
it decodes this sequence as a utf-8 encoded char: "a a" turns into #xe5
and somehow encoded-kbd confuses "#xe5 SPC SPC" for "#xE5 #xA0 #xA0".

I've installed the patch below which should fix your problem most of
the time.  But to fix it right, I think we'd need to hook encoded-kbd
earlier in the chain so it gets processed before input-methods are run.


        Stefan


--- encoded-kb.el.~1.45.~	2008-05-06 23:35:23.000000000 -0400
+++ encoded-kb.el	2008-06-15 00:39:58.000000000 -0400
@@ -219,8 +219,9 @@
 
 (defun encoded-kbd-self-insert-utf-8 (arg)
   (interactive "p")
-  (let ((char (encoded-kbd-last-key))
-	len)
+  (let* ((lead (encoded-kbd-last-key))
+         (char lead)
+         len event)
     (cond ((< char #xE0)
 	   (setq len 1 char (logand char #x1F)))
 	  ((< char #xF0)
@@ -230,8 +231,22 @@
 	  (t
 	   (setq len 4 char 0)))
     (while (> len 0)
-      (setq char (logior (lsh char 6) (logand (read-char-exclusive) #x3F))
-	    len (1- len)))
+      (setq event (read-char-exclusive))
+      (if (and (>= event #x80) (< event #xc0))
+          ;; Valid utf-8 sequence.
+          (setq char (logior (lsh char 6) (- event #x80))
+                len (1- len))
+        ;; Invalid utf-8 sequence.  Might be because Quail got involved
+        ;; in-between and the bytes we thought we were reading were actually
+        ;; latin-1 chars.  Let's presume that `event' is the second "byte",
+        ;; i.e. there weren't any "apprently correct" between `lead' and
+        ;; `event': it's easy to recover in this case, and the more general
+        ;; case seems pretty unlikely.
+        ;; FIXME: We should really do encoded-kbd decoding before processing
+        ;; input-methods.
+        (push event unread-command-events)
+        (setq char lead)
+        (setq len 0)))
     (vector char)))
 
 (defun encoded-kbd-setup-keymap (keymap coding)