Non-ASCII chars in quail rules

Non-ASCII chars in quail rules

[Perry E. Metzger]

This is an elisp question, and it *seemed* more appropriate on the
devel list, but if I'm wrong, I'll move the question elsewhere.

Briefly, my problem is this: my caps lock key is never used for
anything on my keyboard, and I thought it might be cool to use it
as a poor man's compose key for entering non-ASCII characters.

I mapped it to F19 and then set that to insert the Unicode character
for the Compose glyph (U+2384) thinking it would be especially
appropriate.

I then realized that "quail-define-rules" really doesn't play well
with KEY strings that aren't pure ASCII.

1) Is there any easy way to fix that? Simply removing the < 127 test
in the code that builds the maps doesn't work. (I'm afraid I don't
fully understand where the maps are used, which is probably a
problem.)

2) If there isn't an easy way to fix that, is there any obvious
simple way to build an input method that either signals the beginning
of a translated series of characters either with a non-ASCII character
(like U+2384) or with a function key like F19 directly?

I note that essentially the only way of building input methods these
days seems to be quail, or at least, almost all the shipped methods
use quail.

BTW, if it isn't obvious, yes, I could just set the key to enter some
ASCII character (see the rfc1345 input method for example) but my
goal here is to *not* require anything special to enter normal ASCII
but to use the caps lock key as some sort of compose key.

Perry
-- 
Perry E. Metzger		perry@piermont.com

Re: Non-ASCII chars in quail rules

[Stefan Monnier]

> 1) Is there any easy way to fix that? Simply removing the < 127 test
> in the code that builds the maps doesn't work. (I'm afraid I don't
> fully understand where the maps are used, which is probably a
> problem.)

AFAIK there is no technical reason for the <127 limit.  So I think the
best way forward is to remove the <127 test and then see what breaks
next, fix that part, lather, rinse, repeat.

If you have trouble with the "fix that part", feel free to send us
a recipe to reproduce the problem (probably includes your
work-in-progress-patch) so we can help along.


        Stefan

Re: Non-ASCII chars in quail rules

[Perry E. Metzger]

On Fri, 29 Aug 2014 21:13:39 -0400 Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
> > 1) Is there any easy way to fix that? Simply removing the < 127
> > test in the code that builds the maps doesn't work. (I'm afraid I
> > don't fully understand where the maps are used, which is probably
> > a problem.)
> 
> AFAIK there is no technical reason for the <127 limit.  So I think
> the best way forward is to remove the <127 test and then see what
> breaks next, fix that part, lather, rinse, repeat.
> 
> If you have trouble with the "fix that part", feel free to send us
> a recipe to reproduce the problem (probably includes your
> work-in-progress-patch) so we can help along.

I've been digging in to this and it isn't nearly as simple as I
thought. As I suspected, it isn't just the test.

Part of the issue is that the whole of quail seems to be driven off
of things arriving via the set input-method-function, and function
key inputs and the like don't end up flowing through
input-method-function at all.

(Indeed, input-method-function seems to be restricted to passing along
information about printable ASCII (or at least, it documents itself
as only dealing with octal 040 to 0176), and directly via
read-char/read-event.)

I think I'll dig at it for a while longer. What I want might require
adding an alternative path to start invoking quail's substitution
behavior, or it might be better if I ignore the input method
machinery entirely.

Perry
-- 
Perry E. Metzger		perry@piermont.com

Re: Non-ASCII chars in quail rules

[Stefan Monnier]

> (Indeed, input-method-function seems to be restricted to passing along
> information about printable ASCII (or at least, it documents itself
> as only dealing with octal 040 to 0176), and directly via
> read-char/read-event.)

There's no doubt that it won't work for function keys under a tty
(where those function keys are transmitted as "escape sequences", which
Emacs translated back into "function key" via input-decode-map which
happens *after* input decoding).

Maybe it could be made to work for that case as well, but for now
I think we should focus on the GUI case, where those problems
shouldn't appear.

> I think I'll dig at it for a while longer.

Do you have a recipe to reproduce your current problem?


        Stefan

Re: Non-ASCII chars in quail rules

[Perry E. Metzger]

On Sun, 31 Aug 2014 08:38:24 -0400 Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
> > (Indeed, input-method-function seems to be restricted to passing
> > along information about printable ASCII (or at least, it
> > documents itself as only dealing with octal 040 to 0176), and
> > directly via read-char/read-event.)
> 
> There's no doubt that it won't work for function keys under a tty
> (where those function keys are transmitted as "escape sequences",
> which Emacs translated back into "function key" via
> input-decode-map which happens *after* input decoding).
> 
> Maybe it could be made to work for that case as well, but for now
> I think we should focus on the GUI case, where those problems
> shouldn't appear.

Well, for either case, it should be possible to make this work by
pushing back the event into the event queue:

(setq unread-command-events (listify-key-sequence [...])))

which I turned into a function and bound to F19. (I substituted the
unicode value for the compose codepoint in where the ... are in that
sample). The unicode value then does arrive back via
input-method-function

I also then manually bound that value to quail-self-insert-command in
the appropriate quail keymaps, and yet everything isn't working right
for me even yet. I think I'm close, though.

> > I think I'll dig at it for a while longer.
> 
> Do you have a recipe to reproduce your current problem?

I'll construct a minimized version of it and forward that later today.

One overall note: this area of emacs is *really* not documented very
well, which makes the problem a lot harder for someone unfamiliar
with it who starts digging in.

-- 
Perry E. Metzger		perry@piermont.com

Re: Non-ASCII chars in quail rules

[Perry E. Metzger]

[-- Attachment #1: Type: text/plain, Size: 952 bytes --]

On Sun, 31 Aug 2014 10:00:35 -0400 "Perry E. Metzger"
<perry@piermont.com> wrote:
> On Sun, 31 Aug 2014 08:38:24 -0400 Stefan Monnier
> <monnier@iro.umontreal.ca> wrote:
> > Do you have a recipe to reproduce your current problem?
> 
> I'll construct a minimized version of it and forward that later
> today.

Attached are two files, hacked and cut down versions of rfc1345.el

Both are identical except that in one & is used as the prefix
character and in the other the prefix character is U+2384

In both, F19 is bound to send the character in question, though
indirectly by pushing it back into unread-command-events

Load each in turn and set the input method to "testmode", and then
try typing something like "<F19>Pd". When & is the prefix character,
everything works as expected and you get a pound sign. When U+2384 is
the character, it gets entered into the buffer and nothing otherwise
happens.

Perry
-- 
Perry E. Metzger		perry@piermont.com

[-- Attachment #2: testmode.el --]
[-- Type: application/octet-stream, Size: 3113 bytes --]

;;; testmode.el --- Quail test method -*- coding: utf-8 -*-

;; Copyright (C) 2001-2014 Free Software Foundation, Inc.

;; Author: Dave Love <fx@gnu.org>
;; Keywords: i18n

;; This file is part of GNU Emacs.

;; GNU Emacs is free software: you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; GNU Emacs is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.

;;; Commentary:

;; The table below was derived from GNU Recode 3.6.

;;; Code:

;; This works with & and not with ⎄

(require 'quail)

;; To test: load this file, set the input method to "testmode" and try
;; hitting F19 P d -- it works fine.
;; Then replace all instances of & with ⎄ and try it again, it fails

(quail-define-package
 "testmode" "UTF-8" "&" t
 "Unicode characters input method using two char mnemonics
E.g. &Pd -> £"
 nil t nil nil nil nil nil nil nil nil t)

;(defun compose-input-synthetic ()
;  (quail-input-method ?\&))

(defun compose-input-synthetic ()
  (setq unread-command-events (listify-key-sequence [?&])))

(global-set-key (kbd "<f19>")
		(lambda () (interactive) (compose-input-synthetic)))

(quail-define-rules
 ("&&" ?\&)  ; XXX Why doesn't this work, either?
 ("&Ct" ?\¢)
 ("&Pd" ?\£)
 ("&Ye" ?\¥)
 ("&Co" ?\©)
 ("&NO" ?\¬)
 ("&+-" ?\±)
 ("&<-" ?\←)
 ("&-!" ?\↑)
 ("&->" ?\→)
 ("&-v" ?\↓)
 ("&<>" ?\↔)
 ("&UD" ?\↕)
 ("&<!!" ?\↖)
 ("&//>" ?\↗)
 ("&!!>" ?\↘)
 ("&<//" ?\↙)
 ("&UD-" ?\↨)
 ("&>V" ?\⇀)
 ("&<=" ?\⇐)
 ("&=>" ?\⇒)
 ("&==" ?\⇔)
 ("&FA" ?\∀)
 ("&dP" ?\∂)
 ("&TE" ?\∃)
 ("&/0" ?\∅)
 ("&DE" ?\∆)
 ("&NB" ?\∇)
 ("&(-" ?\∈)
 ("&-)" ?\∋)
 ("&FP" ?\∎)
 ("&*P" ?\∏)
 ("&+Z" ?\∑)
 ("&-2" ?\−)
 ("&-+" ?\∓)
 ("&.+" ?\∔)
 ("&*-" ?\∗)
 ("&Ob" ?\∘)
 ("&Sb" ?\∙)
 ("&RT" ?\√)
 ("&0(" ?\∝)
 ("&00" ?\∞)
 ("&-L" ?\∟)
 ("&-V" ?\∠)
 ("&PP" ?\∥)
 ("&AN" ?\∧)
 ("&OR" ?\∨)
 ("&(U" ?\∩)
 ("&)U" ?\∪)
 ("&In" ?\∫)
 ("&DI" ?\∬)
 ("&Io" ?\∮)
 ("&.:" ?\∴)
 ("&:." ?\∵)
 ("&:R" ?\∶)
 ("&::" ?\∷)
 ("&?1" ?\∼)
 ("&CG" ?\∾)
 ("&?-" ?\≃)
 ("&?=" ?\≅)
 ("&?2" ?\≈)
 ("&=?" ?\≌)
 ("&HI" ?\≓)
 ("&!=" ?\≠)
 ("&=3" ?\≡)
 ("&=<" ?\≤)
 ("&>=" ?\≥)
 ("&<*" ?\≪)
 ("&*>" ?\≫)
 ("&!<" ?\≮)
 ("&!>" ?\≯)
 ("&(C" ?\⊂)
 ("&)C" ?\⊃)
 ("&(_" ?\⊆)
 ("&)_" ?\⊇)
 ("&0." ?\⊙)
 ("&02" ?\⊚)
 ("&-T" ?\⊥)
 ("&.P" ?\⋅)
 ("&</" ?\〈)
 ("&/>" ?\〉)
)

(define-key quail-translation-keymap [?\&] 'quail-self-insert-command)
(define-key quail-simple-translation-keymap [?\&] 'quail-self-insert-command)
(define-key quail-conversion-keymap [?\&] 'quail-self-insert-command)


(provide 'testmode)

;;; testmode.el ends here

[-- Attachment #3: testmode-replaced.el --]
[-- Type: application/octet-stream, Size: 3293 bytes --]

;;; testmode.el --- Quail test method -*- coding: utf-8 -*-

;; Copyright (C) 2001-2014 Free Software Foundation, Inc.

;; Author: Dave Love <fx@gnu.org>
;; Keywords: i18n

;; This file is part of GNU Emacs.

;; GNU Emacs is free software: you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; GNU Emacs is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.

;;; Commentary:

;; The table below was derived from GNU Recode 3.6.

;;; Code:

;; This works with & and not with ⎄

(require 'quail)

;; To test: load this file, set the input method to "testmode" and try
;; hitting F19 P d -- it works fine.
;; Then replace all instances of & with ⎄ and try it again, it fails

(quail-define-package
 "testmode" "UTF-8" "⎄" t
 "Unicode characters input method using two char mnemonics
E.g. ⎄Pd -> £"
 nil t nil nil nil nil nil nil nil nil t)

;(defun compose-input-synthetic ()
;  (quail-input-method ?\⎄))

(defun compose-input-synthetic ()
  (setq unread-command-events (listify-key-sequence [?⎄])))

(global-set-key (kbd "<f19>")
		(lambda () (interactive) (compose-input-synthetic)))

(quail-define-rules
 ("⎄⎄" ?\⎄)  ; XXX Why doesn't this work, either?
 ("⎄Ct" ?\¢)
 ("⎄Pd" ?\£)
 ("⎄Ye" ?\¥)
 ("⎄Co" ?\©)
 ("⎄NO" ?\¬)
 ("⎄+-" ?\±)
 ("⎄<-" ?\←)
 ("⎄-!" ?\↑)
 ("⎄->" ?\→)
 ("⎄-v" ?\↓)
 ("⎄<>" ?\↔)
 ("⎄UD" ?\↕)
 ("⎄<!!" ?\↖)
 ("⎄//>" ?\↗)
 ("⎄!!>" ?\↘)
 ("⎄<//" ?\↙)
 ("⎄UD-" ?\↨)
 ("⎄>V" ?\⇀)
 ("⎄<=" ?\⇐)
 ("⎄=>" ?\⇒)
 ("⎄==" ?\⇔)
 ("⎄FA" ?\∀)
 ("⎄dP" ?\∂)
 ("⎄TE" ?\∃)
 ("⎄/0" ?\∅)
 ("⎄DE" ?\∆)
 ("⎄NB" ?\∇)
 ("⎄(-" ?\∈)
 ("⎄-)" ?\∋)
 ("⎄FP" ?\∎)
 ("⎄*P" ?\∏)
 ("⎄+Z" ?\∑)
 ("⎄-2" ?\−)
 ("⎄-+" ?\∓)
 ("⎄.+" ?\∔)
 ("⎄*-" ?\∗)
 ("⎄Ob" ?\∘)
 ("⎄Sb" ?\∙)
 ("⎄RT" ?\√)
 ("⎄0(" ?\∝)
 ("⎄00" ?\∞)
 ("⎄-L" ?\∟)
 ("⎄-V" ?\∠)
 ("⎄PP" ?\∥)
 ("⎄AN" ?\∧)
 ("⎄OR" ?\∨)
 ("⎄(U" ?\∩)
 ("⎄)U" ?\∪)
 ("⎄In" ?\∫)
 ("⎄DI" ?\∬)
 ("⎄Io" ?\∮)
 ("⎄.:" ?\∴)
 ("⎄:." ?\∵)
 ("⎄:R" ?\∶)
 ("⎄::" ?\∷)
 ("⎄?1" ?\∼)
 ("⎄CG" ?\∾)
 ("⎄?-" ?\≃)
 ("⎄?=" ?\≅)
 ("⎄?2" ?\≈)
 ("⎄=?" ?\≌)
 ("⎄HI" ?\≓)
 ("⎄!=" ?\≠)
 ("⎄=3" ?\≡)
 ("⎄=<" ?\≤)
 ("⎄>=" ?\≥)
 ("⎄<*" ?\≪)
 ("⎄*>" ?\≫)
 ("⎄!<" ?\≮)
 ("⎄!>" ?\≯)
 ("⎄(C" ?\⊂)
 ("⎄)C" ?\⊃)
 ("⎄(_" ?\⊆)
 ("⎄)_" ?\⊇)
 ("⎄0." ?\⊙)
 ("⎄02" ?\⊚)
 ("⎄-T" ?\⊥)
 ("⎄.P" ?\⋅)
 ("⎄</" ?\〈)
 ("⎄/>" ?\〉)
)

(define-key quail-translation-keymap [?\⎄] 'quail-self-insert-command)
(define-key quail-simple-translation-keymap [?\⎄] 'quail-self-insert-command)
(define-key quail-conversion-keymap [?\⎄] 'quail-self-insert-command)


(provide 'testmode)

;;; testmode.el ends here

Re: Non-ASCII chars in quail rules

[Stefan Monnier]

> Load each in turn and set the input method to "testmode", and then
> try typing something like "<F19>Pd". When & is the prefix character,
> everything works as expected and you get a pound sign. When U+2384 is
> the character, it gets entered into the buffer and nothing otherwise
> happens.

But that's expected because of the restriction documented in the
docstring of input-method-function:

   It's called with one argument, a printing character that was just read.
   (That means a character with code 040...0176.)

The patch below lifts this restriction, after which your code seems
to work.


        Stefan


=== modified file 'src/keyboard.c'
--- src/keyboard.c	2014-08-27 10:51:21 +0000
+++ src/keyboard.c	2014-09-03 02:20:07 +0000
@@ -3048,7 +3048,7 @@
      save the echo area contents for it to refer to.  */
   if (INTEGERP (c)
       && ! NILP (Vinput_method_function)
-      && ' ' <= XINT (c) && XINT (c) < 256 && XINT (c) != 127)
+      && ' ' <= XINT (c) && CHARACTERP (c) && XINT (c) != 127)
     {
       previous_echo_area_message = Fcurrent_message ();
       Vinput_method_previous_message = previous_echo_area_message;
@@ -3078,7 +3078,7 @@
       /* Don't run the input method within a key sequence,
 	 after the first event of the key sequence.  */
       && NILP (prev_event)
-      && ' ' <= XINT (c) && XINT (c) < 256 && XINT (c) != 127)
+      && ' ' <= XINT (c) && CHARACTERP (c) && XINT (c) != 127)
     {
       Lisp_Object keys;
       ptrdiff_t key_count;

Re: Non-ASCII chars in quail rules

[Perry E. Metzger]

On Tue, 02 Sep 2014 22:23:54 -0400 Stefan Monnier
<monnier@IRO.UMontreal.CA> wrote:
> But that's expected because of the restriction documented in the
> docstring of input-method-function:
> 
>    It's called with one argument, a printing character that was
> just read. (That means a character with code 040...0176.)
> 
> The patch below lifts this restriction, after which your code seems
> to work.

Indeed, that is doubtless the problem! Thank you!

One note: CHARACTERP does a bit more than is actually needed here, it
may be sufficient to say XINT (c) <= MAX_CHAR or to use the
CHAR_VALID_P macro.

I am not sure what the intent here of the restriction was, but a more
general solution for the entire line of the test might be
CHAR_PRINTABLE_P depending on what that intent actually is. (I don't
entirely understand the intent, so I can't say.)

Perry

> 
> 
>         Stefan
> 
> 
> === modified file 'src/keyboard.c'
> --- src/keyboard.c	2014-08-27 10:51:21 +0000
> +++ src/keyboard.c	2014-09-03 02:20:07 +0000
> @@ -3048,7 +3048,7 @@
>       save the echo area contents for it to refer to.  */
>    if (INTEGERP (c)
>        && ! NILP (Vinput_method_function)
> -      && ' ' <= XINT (c) && XINT (c) < 256 && XINT (c) != 127)
> +      && ' ' <= XINT (c) && CHARACTERP (c) && XINT (c) != 127)
>      {
>        previous_echo_area_message = Fcurrent_message ();
>        Vinput_method_previous_message = previous_echo_area_message;
> @@ -3078,7 +3078,7 @@
>        /* Don't run the input method within a key sequence,
>  	 after the first event of the key sequence.  */
>        && NILP (prev_event)
> -      && ' ' <= XINT (c) && XINT (c) < 256 && XINT (c) != 127)
> +      && ' ' <= XINT (c) && CHARACTERP (c) && XINT (c) != 127)
>      {
>        Lisp_Object keys;
>        ptrdiff_t key_count;
> 



-- 
Perry E. Metzger		perry@piermont.com

Re: Non-ASCII chars in quail rules

[Stefan Monnier]

> One note: CHARACTERP does a bit more than is actually needed here, it
> may be sufficient to say XINT (c) <= MAX_CHAR or to use the
> CHAR_VALID_P macro.

The only "more" it does, compared to XINT (c) <= MAX_CHAR is that it
checks >=0 which is redundant because we already compared with ' '.

> I am not sure what the intent here of the restriction was, but a more
> general solution for the entire line of the test might be
> CHAR_PRINTABLE_P depending on what that intent actually is. (I don't
> entirely understand the intent, so I can't say.)

The intent is to make sure that `c' is something we can insert in the
buffer (since that's what quail does).  We could also remove this check
completely and allow using any event to trigger input-method-function,
but that does break the current quail.el, so it's clearly more "backward
incompatible".


        Stefan

Re: Non-ASCII chars in quail rules

[Perry E. Metzger]

On Wed, 03 Sep 2014 09:19:18 -0400 Stefan Monnier
<monnier@IRO.UMontreal.CA> wrote:
> > I am not sure what the intent here of the restriction was, but a
> > more general solution for the entire line of the test might be
> > CHAR_PRINTABLE_P depending on what that intent actually is. (I
> > don't entirely understand the intent, so I can't say.)
> 
> The intent is to make sure that `c' is something we can insert in
> the buffer (since that's what quail does).

It would seem that CHAR_PRINTABLE_P is the right thing, then, since
other characters can't be inserted (at least not validly).

Given that, it would seem like this would be a nice patch to get
applied.

> We could also remove this check completely and allow using any
> event to trigger input-method-function, but that does break the
> current quail.el, so it's clearly more "backward incompatible".

Yah, that seems like The Wrong Thing.

Perry
-- 
Perry E. Metzger		perry@piermont.com

Re: Non-ASCII chars in quail rules

[Stephen J. Turnbull]

Perry E. Metzger writes:

 > It would seem that CHAR_PRINTABLE_P is the right thing, then, since
 > other characters can't be inserted (at least not validly).

What makes you think that?  As far as Emacs is concerned, a character
is a character.  For example, in bidi text you might want (in theory
-- in practice it's usually a bad idea) to insert a directional marker
character.  In Japanese or Chinese where the normal practice is to
break on any word component, you might want to use a word joiner
character to save English words from the kinsoku scalpel.

Re: Non-ASCII chars in quail rules

[Perry E. Metzger]

On Thu, 04 Sep 2014 09:02:06 +0900 "Stephen J. Turnbull"
<stephen@xemacs.org> wrote:
> Perry E. Metzger writes:
> 
>  > It would seem that CHAR_PRINTABLE_P is the right thing, then,
>  > since other characters can't be inserted (at least not validly).
> 
> What makes you think that?

Apparently a lack of familiarity, as you've listed reasons that I'm
clearly wrong on that.

In any case, I'll be happy with whatever the people who actually know
what they're doing think the right thing is. I just want it to
work. :)

Perry
-- 
Perry E. Metzger		perry@piermont.com

Re: Non-ASCII chars in quail rules

[Stephen J. Turnbull]

Perry E. Metzger writes:

 > Apparently a lack of familiarity, as you've listed reasons that I'm
 > clearly wrong on that.

Good, I was afraid it was a doc bug!
 > 
 > In any case, I'll be happy with whatever the people who actually know
 > what they're doing think the right thing is. I just want it to
 > work. :)

I'll drink to that!

Steve

Re: Non-ASCII chars in quail rules

[Perry E. Metzger]

On Thu, 04 Sep 2014 23:17:34 +0900 "Stephen J. Turnbull"
<stephen@xemacs.org> wrote:
> Perry E. Metzger writes:
> 
>  > Apparently a lack of familiarity, as you've listed reasons that
>  > I'm clearly wrong on that.
> 
> Good, I was afraid it was a doc bug!

No bug, though there is an issue in so far as there's no manual
covering how to write input methods -- other than doc strings,
you you have to read code to figure out how it works, which is
painful. A manual or a section of the elisp manual would be a big
help.

Is there a list somewhere of "things that need to be documented"
that this could be added to? (Filling in things from such a list might
be a great project for a Summer of Code student some year...)

Perry
-- 
Perry E. Metzger		perry@piermont.com

Re: Non-ASCII chars in quail rules

[Eli Zaretskii]

> Date: Thu, 4 Sep 2014 11:20:52 -0400
> From: "Perry E. Metzger" <perry@piermont.com>
> Cc: Stefan Monnier <monnier@IRO.UMontreal.CA>, emacs-devel@gnu.org
> 
> there is an issue in so far as there's no manual covering how to
> write input methods -- other than doc strings, you you have to read
> code to figure out how it works, which is painful. A manual or a
> section of the elisp manual would be a big help.

Indeed.

> Is there a list somewhere of "things that need to be documented"
> that this could be added to? (Filling in things from such a list might
> be a great project for a Summer of Code student some year...)

etc/TODO is supposed to be it (not only for documentation).

Re: Non-ASCII chars in quail rules

[Stephen J. Turnbull]

Perry E. Metzger writes:

 > Is there a list somewhere of "things that need to be documented"
 > that this could be added to? 

Bug tracker seems like the right place.  IMHO lack of docs is a bug,
not an RFE, but follow Emacs practice on that..

 > (Filling in things from such a list might be a great project for a
 > Summer of Code student some year...)

Unfortunately that's right out.  SoC specifically excludes projects
aimed at improving docs.