Best way to intercept terminal escape sequences?

[Ryan Johnson <ryanjohn@ece.cmu.edu>]

  Hi all,

Terminal escape sequences cause emacs a lot of pain, with problems like
bug #6758 and the contortions performed by xt-mouse.el being just two
examples.

I've been wrestling with xt-mouse.el for the last week and it seems to
me that many or all of these problems arise because escape sequences are
treated like any other string of characters when in fact they are nearly
all encoded events of some sort. For example, the xterm escape sequence
"ESC O D" is eventually converted to <right>, but anybody calling
read-char or read-event will get a string of characters instead (and
probably wish they hadn't). This is why xt-mouse can't do full mouse
handling -- mouse-drag-track (mouse.el) calls read-event to watch for
the mouse-up event. If xt-mouse simply translated mouse escapes into
events, mouse-drag-track would intercept the mouse-up sequence and think
it was is garbage, with xt-mouse never having a chance to decode it. So,
instead, xt-mouse reinvents the wheel (poorly) and has to jump through
hoops to send [down-mouse-1 move-mouse-1 mouse-1] all at once to
simulate a mouse drag after it has finished.

Problems like this could be solved by allowing the system to process
terminal escape sequences early in the food chain (= before read-char
and read-event). The idea would be to let a terminal translator
interpose on the keyboard input before anything else -- even coding
systems -- and filter known escape sequences. The interposition would
have two key features:

1. Recognized key sequences can be absorbed completely or (more likely)
converted to events which appear at read-event in the proper order -- (a
ESC O D b) would make three read-event calls return (a <right> b).
2. Unrecognized sequences are passed through, unchanged, for normal
handling by the rest of the input processing chain -- (ESC x x) would be
ignored

With this interposition in place, terminal events could become first
class citizens. Function key presses would appear to *everyone* as such,
mouse events would be processed by the normal mouse-handling code, etc.

The input-decode-map introduced by v23 is a nice start (it can absorb
the sequence or generate events as needed), but it doesn't pass through
unrecognized sequences and read-char/read-event bypass it.

Unfortunately read-* are written in C and can't be advised effectively
(I tried but the warnings were accurate). Input methods don't seem to
work, even if I knew how to create one (they receive printable chars
only and the read-* have to ask for them). I also tried creating a
keyboard coding system to intercept terminal escape sequences as a
proof-of-concept, but was thwarted by bug #6920 (and that's not the
right place for it anyway).

Alternatively, I toyed with the idea of making mouse.el install an
overriding-terminal-local-map (to silently ignore everything except ^G
and mouse events) instead of calling read-event, but that would require
significant rewriting and isn't a general fix for the escape sequence
problem. I tried source diving for more ideas, but read_filtered_event
(lread.c) calls read_char (keyboard.c), which is 900 lines of Greek to me.

Thoughts? What would be the least intrusive way to support escape
sequences better? Or is there a better way mouse.el should process
events so the existing input-decode-map becomes effective?

(please CC me in all replies -- I'm signed up for daily digests)

Thanks,
Ryan

P.S. Apologies if this double-posts -- my mailer was messed up.