Re: Best way to intercept terminal escape sequences?

Re: Best way to intercept terminal escape sequences?

[Ryan Johnson]

  On Fri, 27 Aug 2010 16:17:14 +0200, David Kastrup wrote:
>>   On Fri, 27 Aug 2010 12:36:52 +0200, David Kastrup wrote:
>>> Ryan Johnson<ryanjohn@ece.cmu.edu>   writes
>>>> I tried setting the keyboard-coding-system to iso-latin-1, but no
>>>> luck.
>>> You should set it to raw-text, do your mouse code preprocessing, and
>>> afterwards decode the remainder using the intended coding system.
>> Like this?
>> (defun xterm-mouse-pos-read ()
>>    (let ((old-coding (keyboard-coding-system)))
>>      (set-keyboard-coding-system 'raw-text)
>>      (unwind-protect
>>          (cons (xterm-mouse-event-read) (xterm-mouse-event-read))
>>        (set-keyboard-coding-system old-coding))))
> You can't go setting the keyboard reading system back and forth.  It
> operates into a buffer even before calling read-char.  You have to put
> it to raw and stick with it.
Makes sense. That's why I was hoping there was a way intercept input 
before coding systems get their claws on it. Otherwise xt-mouse finds 
itself worrying about what coding system the user has requested, whether 
it changed recently, etc.

If coding systems stacked (and if user-defined ones worked properly) it 
might be easier, but AFAIK neither is true.

Is there really no other way to do this?

Ryan

Re: Best way to intercept terminal escape sequences?

[Eli Zaretskii]

> Date: Fri, 27 Aug 2010 16:44:12 +0200
> From: Ryan Johnson <ryanjohn@ece.cmu.edu>
> 
> Is there really no other way to do this?

You could use the :post-read-conversion attribute of a coding-system.
That is, define a new coding-system that has this attribute specifying
a function you will write.  That function will first decode the mouse
stuff, and then decode the rest by the terminal-coding-system set by
the user.

You can see an example of this in ctext-with-extensions.  It is
defined on mule-conf.el and its post-read-conversion function is
defined on mule.el.

Other than that, it's no surprise that this is not easy: we ask Emacs
to read keyboard input that is encoded in two different encodings,
which is not how keyboard input was designed.

I think the only way that is easier and cleaner would be if Emacs
could read the mouse input from a separate file descriptor.  We could
then set that file descriptor to use a different encoding.  Of course,
that would need low-level changes in Emacs, even if it is possible in
xterm.

Re: Best way to intercept terminal escape sequences?

[Ryan Johnson]

  On 8/27/2010 5:40 PM, Eli Zaretskii wrote:
>> Date: Fri, 27 Aug 2010 16:44:12 +0200
>> From: Ryan Johnson<ryanjohn@ece.cmu.edu>
>>
>> Is there really no other way to do this?
> You could use the :post-read-conversion attribute of a coding-system.
> That is, define a new coding-system that has this attribute specifying
> a function you will write.  That function will first decode the mouse
> stuff, and then decode the rest by the terminal-coding-system set by
> the user.
> You can see an example of this in ctext-with-extensions.  It is
> defined on mule-conf.el and its post-read-conversion function is
> defined on mule.el.
That's actually what I tried first, but as I mentioned in the OP, emacs 
doesn't always deliver characters to :post-read-conversion in the 
correct order, which makes it impossible to do anything reliable with 
either escape sequences or utf-8. See 
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=6920 for more details.

The other problem is, I need to convert escape sequences into mouse 
events, not characters. Even if the bug were fixed, and even if it were 
easy to detect the user's desired coding system and piggy-back on it 
reliably, there's a problem with event ordering. I can't put a mouse 
event in the buffer, so if I'm ever given text and mouse-escape-sequence 
I'd have to leave the buffer empty, then load up unread-command-events 
with both text and mouse events. I'm not sure where those rejoin the 
input processing chain, but even with (t . EVT) I suspect they skip 
something (input methods, perhaps?). Also, I had some trouble getting (t 
. EVT) to work from within :post-read-conversion, though that could just 
be a bug in my code.

> Other than that, it's no surprise that this is not easy: we ask Emacs
> to read keyboard input that is encoded in two different encodings,
> which is not how keyboard input was designed.
Not quite... emacs insists on interpreting everything as encoded in some 
way, and I'd really like to just get my hands on a few raw bytes before 
it does so.

> I think the only way that is easier and cleaner would be if Emacs
> could read the mouse input from a separate file descriptor.  We could
> then set that file descriptor to use a different encoding.  Of course,
> that would need low-level changes in Emacs, even if it is possible in
> xterm.
What I had in mind was simpler:

Right now we have input -> coding system -> input method -> read-char -> 
input-decode-map -> ... other keymaps galore ... -> read-key

read-key is nice because it comes after all the interpretation is 
complete (high-level). Read-char is nice for interpreting key sequences 
(e.g. if you're a key map, mid-level), and it makes perfect sense to 
apply coding systems first. However, for things like escape sequences 
(very low-level), which are raw bytes with a specific meaning regardless 
of locale (and even keyboard layout, for xterm), coding systems are 
unnecessary at best and harmful at worst.

I'd propose adding an input filtering mechanism, which allows to 
register functions which pick off the raw input and are expected to put 
back anything they doesn't need. Sort of like a coding system, but 
orthogonal. Then you'd have:

input -> input-filter(s) -> coding system -> input method -> read-char 
-> input-decode-map -> ... other keymaps galore ... -> read-key

That said, if coding systems can really be abused as you suggest, and 
the ordering bug could be fixed, and unread-command-events does the 
right thing, then that would have the same effect and I'd be happy to 
use it. If just seems like taking the long way around.

Ryan

Re: Best way to intercept terminal escape sequences?

[Eli Zaretskii]

> Date: Fri, 27 Aug 2010 20:04:48 +0200
> From: Ryan Johnson <ryanjohn@ece.cmu.edu>
> CC: emacs-devel@gnu.org
> 
>   On 8/27/2010 5:40 PM, Eli Zaretskii wrote:
> >> Date: Fri, 27 Aug 2010 16:44:12 +0200
> >> From: Ryan Johnson<ryanjohn@ece.cmu.edu>
> >>
> >> Is there really no other way to do this?
> > You could use the :post-read-conversion attribute of a coding-system.
> > That is, define a new coding-system that has this attribute specifying
> > a function you will write.  That function will first decode the mouse
> > stuff, and then decode the rest by the terminal-coding-system set by
> > the user.
> > You can see an example of this in ctext-with-extensions.  It is
> > defined on mule-conf.el and its post-read-conversion function is
> > defined on mule.el.
> That's actually what I tried first, but as I mentioned in the OP, emacs 
> doesn't always deliver characters to :post-read-conversion in the 
> correct order, which makes it impossible to do anything reliable with 
> either escape sequences or utf-8. See 
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=6920 for more details.

You need to base your system on raw-text or no-conversion.  Then the
bytes will arrive unscathed.

> Right now we have input -> coding system -> input method -> read-char -> 
> input-decode-map -> ... other keymaps galore ... -> read-key
> 
> read-key is nice because it comes after all the interpretation is 
> complete (high-level). Read-char is nice for interpreting key sequences 
> (e.g. if you're a key map, mid-level), and it makes perfect sense to 
> apply coding systems first. However, for things like escape sequences 
> (very low-level), which are raw bytes with a specific meaning regardless 
> of locale (and even keyboard layout, for xterm), coding systems are 
> unnecessary at best and harmful at worst.

Well, mouse events aren't supposed to arrive by way of keyboard input,
either, you know.  They are supposed to come from a totally different
API.

Re: Best way to intercept terminal escape sequences?

[Stefan Monnier]

> I think the only way that is easier and cleaner would be if Emacs
> could read the mouse input from a separate file descriptor.  We could

Note that under older Emacsen, read-event did not obey the
keyboard-coding-system at all: it only applied to read-key-sequence.
So maybe we should simply change read-event not to try and decoding
keyboard input.


        Stefan

Re: Best way to intercept terminal escape sequences?

[Ryan Johnson]

  On 8/28/2010 1:54 AM, Stefan Monnier wrote:
>> I think the only way that is easier and cleaner would be if Emacs
>> could read the mouse input from a separate file descriptor.  We could
> Note that under older Emacsen, read-event did not obey the
> keyboard-coding-system at all: it only applied to read-key-sequence.
> So maybe we should simply change read-event not to try and decoding
> keyboard input.
That might make sense... the caller could always apply the coding system 
manually before dumping things back in the unread-command-events queue.

Who currently uses read-* that might be affected? xt-mouse.el would love 
it, mouse.el certainly won't care, and other xterm processing will be 
indifferent.

BTW, I've been playing with read-key and it's perfect for making 
mouse.el and xt-mouse.el play nice together! I'm a tad unclear on the 
difference between read-key and read-key-sequence, though, other than 
the latter letting you supply a minibuffer prompt.

Ryan

Re: Best way to intercept terminal escape sequences?

[Stefan Monnier]

> That might make sense... the caller could always apply the coding system
> manually before dumping things back in the unread-command-events queue.

Or coding-system decoding should be applied to events from
unread-command-events.

> Who currently uses read-* that might be affected? xt-mouse.el would love it,
> mouse.el certainly won't care, and other xterm processing will
> be indifferent.

As mentioned, read-event did not do obey keyboard-coding-system in
earlier Emacsen, so any affected package is more likely to be fixed than
broken by making a change that reverts to this previous behavior.

> BTW, I've been playing with read-key and it's perfect for making mouse.el
> and xt-mouse.el play nice together! I'm a tad unclear on the difference
> between read-key and read-key-sequence, though, other than the latter
> letting you supply a minibuffer prompt.

read-key only reads a single event.  I.e. only C-x not C-x C-a.
This event might be the result of processing several raw events
(e.g. via keyboard-coding-system, input-decode-map, ...).


        Stefan

Re: Best way to intercept terminal escape sequences?

[Ryan Johnson]

  On 8/28/2010 4:47 PM, Stefan Monnier wrote:
>> That might make sense... the caller could always apply the coding system
>> manually before dumping things back in the unread-command-events queue.
> Or coding-system decoding should be applied to events from
> unread-command-events.
That would basically be the proposed 'read-byte' behavior, then? Where 
read-char can filter out the raw inputs, and anything that it puts back 
goes through the whole input processing chain? Works for me!
>> Who currently uses read-* that might be affected? xt-mouse.el would love it,
>> mouse.el certainly won't care, and other xterm processing will
>> be indifferent.
> As mentioned, read-event did not do obey keyboard-coding-system in
> earlier Emacsen, so any affected package is more likely to be fixed than
> broken by making a change that reverts to this previous behavior.
Assuming this does get fixed, the xterm mouse events will be vastly more 
reliable.

Meanwhile, I've got a patch mostly ready which makes xt-mouse.el behave 
much more like native mouse. Xterm has a mode which sends mouse motion 
events whenever a button is down, and making mouse.el use read-key lets 
me send those events separately. The result is that clicking and 
dragging are highly responsive, where before you didn't see anything 
happen until after button release. The visual feedback is really helpful 
when dragging to highlight text.

I should also be able to add support for multi-click by emulating the 
behavior described in the elisp manual. I kind of hoped the emacs core 
would do that for me, given a stream of mouse-down and mouse-up events, 
but it doesn't. Oh well... some more code to write but nothing terrible. 
The only thing that would be missing is support for track-mouse and 
mouse-face, because they're hardwired in C.

It would be really nice if there were a way to hook into the native 
mouse subsystem rather than reinventing the wheel... it already computes 
what window/buffer a given coordinate is in, grabs timestamps, detects 
multi-click, and implements track-mouse/mouse-face. Assuming proper 
hooks were available, xterm does have a full mouse-tracking mode which 
we could use. That's way over my head to tackle, though.

Should I clean up the current patch, try to add multi-click, or see if 
somebody wants to expose some native mouse hooks?

Ryan

Re: Best way to intercept terminal escape sequences?

[Ryan Johnson]

  On 8/28/2010 4:47 PM, Stefan Monnier wrote:
>> Who currently uses read-* that might be affected? xt-mouse.el would love it,
>> mouse.el certainly won't care, and other xterm processing will
>> be indifferent.
> As mentioned, read-event did not do obey keyboard-coding-system in
> earlier Emacsen, so any affected package is more likely to be fixed than
> broken by making a change that reverts to this previous behavior.
Hmm... here's a twist: The elisp docs under keymaps -> translation 
keymaps explain that:

If you have enabled keyboard character set decoding using
`set-keyboard-coding-system', decoding is done after the translations
listed above.  See Terminal I/O Encoding.  However, in future Emacs
versions, character set decoding may be done at an earlier stage.

However, the same info node admits that translation keymaps may want to 
read input (which does *not* escape I/O coding). So, suppose we do the 
following:

(define-key
   input-decode-map
   "\M-[M"; CSI M
   '(keymap; pb
     (t keymap; px
        (t keymap; py
           (t . xterm-mouse-translate)))))

In theory, the above matches any three characters following the start of 
the mouse escape sequence. Then inside xterm-mouse-translate 
(this-command-keys) comes close to being raw bytes. It now works great 
for any px I can throw at it, but still something goes wrong for py > #x7f.

If I print (this-command-keys-vector) after a mouse click at (0 . 95), I 
get: [27 91 77 32 33 4194176] -- mouse-down -- and then emacs hangs 
waiting for more input; the next key I type ends up prefixed by \200.  
The lossage buffer shows ESC [ M SPC ! \300\200 ESC [ M # ! \300\200, 
but I don't know where the 'ESC [ M # ! \300' part disappeared to -- it 
doesn't get inserted into any buffer and yet xterm-mouse-translate never 
gets called, either.

The docs seem out of date about where coding systems kick in... 
apparently it still tries to decode utf-8 somehow, even though I never 
call read-*.

Even if I (set-keyboard-coding-system 'no-conversion), the bytes turn 
into all kinds of strange M- and C- versions of characters, which is no 
fun to disentangle. 'raw-text and 'binary give different but equally 
not-fun sets of weirdness.

I'm beginning to think there's actually no way to get raw bytes from the 
terminal...

Ideas?
Ryan

Re: Best way to intercept terminal escape sequences?

[Stefan Monnier]

> As mentioned, read-event did not do obey keyboard-coding-system in
> earlier Emacsen, so any affected package is more likely to be fixed than
> broken by making a change that reverts to this previous behavior.

Handa, could you take a look at the feasibility of moving the
decode_keyboard_code to a later stage such that read-event still returns
raw bytes for ttys?

There is a tension here, because raw events in GUIs are already decoded,
whereas raw events in ttys are just bytes.  You "fixed it" by decoding
tty input in directly in tty_read_avail_input, so that read-event now
always returns decoded input, but that in turns means that read-event
doesn't return raw events any more.  The decoding is desirable for
read-key-sequence (and maybe also for read-char, tho I don't care much
about this case since read-key is generally a better replacement) but
not for read-event, since access to raw events is important for things
like xt-mouse.el.

> Hmm... here's a twist: The elisp docs under keymaps -> translation keymaps
> explain that:

> If you have enabled keyboard character set decoding using
> `set-keyboard-coding-system', decoding is done after the translations
> listed above.  See Terminal I/O Encoding.  However, in future Emacs
> versions, character set decoding may be done at an earlier stage.

This doc is out of date, indeed.


        Stefan

Re: Best way to intercept terminal escape sequences?

[Ryan Johnson]

  On 9/2/2010 12:53 PM, Stefan Monnier wrote:
>> As mentioned, read-event did not do obey keyboard-coding-system in
>> earlier Emacsen, so any affected package is more likely to be fixed than
>> broken by making a change that reverts to this previous behavior.
> Handa, could you take a look at the feasibility of moving the
> decode_keyboard_code to a later stage such that read-event still returns
> raw bytes for ttys?
>
> There is a tension here, because raw events in GUIs are already decoded,
> whereas raw events in ttys are just bytes.  You "fixed it" by decoding
> tty input in directly in tty_read_avail_input, so that read-event now
> always returns decoded input, but that in turns means that read-event
> doesn't return raw events any more.  The decoding is desirable for
> read-key-sequence (and maybe also for read-char, tho I don't care much
> about this case since read-key is generally a better replacement) but
> not for read-event, since access to raw events is important for things
> like xt-mouse.el.
By the way, I noticed while working on something else [1] that read-key 
cannot actually replace all uses of read-event because the latter 
supports timeouts while the former does not. Would it be possible to add 
a timeout to read-key as a third (optional) parameter? I don't know 
whether read-key-delay would provide workarounds to some timeout uses, 
but it seems brittle.

>> If you have enabled keyboard character set decoding using
>> `set-keyboard-coding-system', decoding is done after the translations
>> listed above.  See Terminal I/O Encoding.  However, in future Emacs
>> versions, character set decoding may be done at an earlier stage.
> This doc is out of date, indeed.

It would be really nice to have, somewhere in the emacs docs, a diagram 
showing what processing happens to keyboard input, starting from raw 
bytes and UI events, and tracing them (or their translations) through 
coding systems, input methods, command loop, various keymaps, etc. and 
showing where in that process the different read-* functions intercept 
that data (and where the various unread-*-events reinsert things). A 
similar diagram for reading and writing files would probably also be useful.

This would not only make it easier to figure out how to interface new 
code with emacs, it would probably expose gaps in the API which make 
existing code unnecessarily complex and certain features impossible 
(mouse.el and xt-mouse.el suffer from both of those latter problems).

Unfortunately, even after spending so long on this problem I don't think 
I know enough to generate that diagram...

Ryan

[1] http://www.ece.cmu.edu/~ryanjohn/sticky-control.el

Improve input handling documentation (was Re: Best way to intercept terminal escape sequences?)

[Štěpán Němec]

Ryan Johnson <ryanjohn@ece.cmu.edu> writes:

[...]

> It would be really nice to have, somewhere in the emacs docs, a diagram
> showing what processing happens to keyboard input, starting from raw bytes and
> UI events, and tracing them (or their translations) through coding systems,
> input methods, command loop, various keymaps, etc. and showing where in that
> process the different read-* functions intercept that data (and where the
> various unread-*-events reinsert things). A similar diagram for reading and
> writing files would probably also be useful.

Sounds great indeed!

[...]

> Unfortunately, even after spending so long on this problem I don't think I
> know enough to generate that diagram...

But perhaps you could come up with some initial version that others
could continue improving upon?

Štěpán

Re: Best way to intercept terminal escape sequences?

[Kenichi Handa]

Sorry, I have not read this thread.

In article <jwvy6bfdp23.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> As mentioned, read-event did not do obey keyboard-coding-system in
>>> earlier Emacsen, so any affected package is more likely to be fixed than
>>> broken by making a change that reverts to this previous behavior.
> > Handa, could you take a look at the feasibility of moving the
> > decode_keyboard_code to a later stage such that read-event still returns
> > raw bytes for ttys?

> ping?

"... read-event still returns raw bytes" means that you
can't get, for instance, A-ogonek event by read-event (or
read-char) even if you type A-ogonek from a terminal.  I
don't remember well but one reason of moving
keyboard-coding-system handling from keymap to the current
place was to make read-event on tty work as the same way as
that on graphic terminal.  So, I think we should not change
it.

Is it difficult to make new functions, say tty-getc and
tty-ungetc, to handle responsding escape sequences sent from
terminal?

---
Kenichi Handa
handa@m17n.org

Re: Best way to intercept terminal escape sequences?

[Stefan Monnier]

>>>> As mentioned, read-event did not do obey keyboard-coding-system in
>>>> earlier Emacsen, so any affected package is more likely to be fixed than
>>>> broken by making a change that reverts to this previous behavior.
>> > Handa, could you take a look at the feasibility of moving the
>> > decode_keyboard_code to a later stage such that read-event still returns
>> > raw bytes for ttys?
>> ping?
> "... read-event still returns raw bytes" means that you can't get, for
> instance, A-ogonek event by read-event (or read-char) even if you type
> A-ogonek from a terminal.  I don't remember well but one reason of
> moving keyboard-coding-system handling from keymap to the current
> place was to make read-event on tty work as the same way as that on
> graphic terminal.  So, I think we should not change it.

IIRC the reason to decode earlier was so that things like
input-decode-map and friends get to see real chars rather than bytes,
but none of those reasons require read-event to always return chars.

IIRC one of the main problems with earlier code was that encoded-kb
would sometimes see bytes and sometimes chars (i.e. it would normally
see bytes under a tty except when using leim), so it sometimes
incorrectly took chars for bytes and re-decoded them.

> Is it difficult to make new functions, say tty-getc and tty-ungetc, to
> handle responsding escape sequences sent from terminal?

It doesn't look easy at all, since the current code does decoding before
placing the bytes in the event-queue (i.e. by the time Elisp code gets
a chance to look at the queue, the decoding has already taken place).

It would be better to decode later (i.e. after placing the events on
the event-queue), tho still before leim and input-decode-map.

We can then easily provide a "new" function to read "decoded events".
We could call it ... read-char ;-)


        Stefan

Improve input handling documentation (was Re: Best way to intercept terminal escape sequences?)

[Ryan Johnson]

  On Sat, 04 Sep 2010 15:01:28 +0200 stepnem wrote:
>> >  It would be really nice to have, somewhere in the emacs docs, a diagram
>> >  showing what processing happens to keyboard input, starting from raw bytes and
>> >  UI events, and tracing them (or their translations) through coding systems,
>> >  input methods, command loop, various keymaps, etc. and showing where in that
>> >  process the different read-* functions intercept that data (and where the
>> >  various unread-*-events reinsert things). A similar diagram for reading and
>> >  writing files would probably also be useful.
> Sounds great indeed!
>
> [...]
>> >  Unfortunately, even after spending so long on this problem I don't think I
>> >  know enough to generate that diagram...
> But perhaps you could come up with some initial version that others
> could continue improving upon?
What format should the diagram take? I don't think I've ever seen ascii 
art, let alone an image, in the emacs docs...

Meanwhile, here's what I have inferred, roughly, about input handling. 
Hopefully those familiar with emacs' guts can correct and improve it:

<-------- keyboard input
keyboard-coding-system
<-------- unread-input-method-events
[Input Method (optional, printable ascii chars only)]
<-------- unread-post-input-method-events
???
<-------- non-keyboard (e.g. mouse) events
<-------- unread-command-events (and unread-command-char)
--------> read-char, read-event
input-decode-map
key-translation-map
[local-]function-key-map
--------> read-key
assorted keymaps (global, local, minor mode, emulation, char/text 
properties, overriding)
--------> read-key-sequence
???
read-number, read-string, etc. (???)

(I have no idea where read-passwd fits in this hierarchy, nor how the 
command loop lines up with things).

NOTE: the description of read-key and input-decode-map makes it sound 
like read-event comes before coding systems, but empirically this is not 
true.

Regards,
Ryan