bug#13709: bug#13793: 24.3.50; M-x broken in viper and X

[Frank Fischer <frank-fischer@shadow-soft.de>]

On 02/24, Stefan Monnier wrote:
> The same commit caused a problem in Evil as well (see bug#13709), and
> I hope the problem is the same and can be fixed in the same way, but I'm
> also having trouble figuring out what's happening in that case.
> 
> If someone familiar with Evil or Viper's keymap tricks could investigate
> a bit more to try and see where the behavior changes, I can then
> hopefully see how it relates to my read-key-sequence changes.

Coincidentally, I'm involved with Evil, but I've never tried to debug
Emacs' C code. But I think I succeeded and can explain what's going
on. I will restrict to Evil, because I know it better than viper. I
start with a description of what Evil does and afterwards describe the
problem with the new code.

The problem, as I said, is that Evil needs to differentiate between a
plain ESC key and a meta key sequence. Evil does the following. It has
a special minor mode `evil-esc-mode` along with its keymap
`evil-esc-map`. This keymap has only one binding, (kbd "ESC") is bound
to a function `evil-esc`. This function is very simple (I removed some
code, but they main idea should become clear)

	(defun evil-esc (arg)
	  "Wait for further keys within `evil-esc-delay'.
	Otherwise send [escape]."
	  (interactive "P")
	  (if (sit-for evil-esc-delay t)
		  (push 'escape unread-command-events)
		(push last-command-event unread-command-events)
		;; preserve prefix argument
		(setq prefix-arg arg))
	  ;; disable interception for the next key sequence
	  (evil-esc-mode -1)
	  (setq this-command last-command)
	  (add-hook 'pre-command-hook #'evil-turn-on-esc-mode nil t))

The function waits for the next event. If it arrives, the (kbd "ESC")
event is unread, and evil-esc-mode is disabled so that it does not
call again `evil-esc`. If no next event arrives the event 'escape is
unread so whatever command is bound to 'escape will be called.
`evil-esc-mode` will be reenabled after the next command completed.
Note that an event like M-x (in the terminal) generates two events in
quick succession, namely (kbd "ESC") and "x", so the first case
happens. In GUI mode the function `evil-esc` is never called because
here M-x generates another event (some large integer). This event is
transformed to an ESC sequence, too, but not using `evil-esc` but
within the function `access_keymap_1` in file `keymap.c`.

If this function detects that the event is a meta event, it looks for
the prefix map of (kbd "ESC") in the keymap that has been passed to
the function in the "map" argument. Here comes the problem.

The function `follow_key` has been changed by the problematic commit.
Formerly severall keymaps have been passed in an array. Each keymap
has been checked in turn for a binding. One of the keymaps is
`evil-esc-map`. If this keymap is checked no binding is found. So the
next keymap is checked an it may contain a binding for M-x so this
binding is used.

After the commit all keymaps are collected in one single "super"
keymap, that contains all keymaps as "parents" (is this correct?). Now
`access_keymap_1` gets only called once for this super keymap. Again
this function tries to find (kbd "ESC") in this keymap. This will
return the binding of `evil-esc`, which is not a prefix keymap so no
binding will be found. Because of how keymaps work only the first
binding of (kbd "ESC") in one of the parent keymaps will be returned,
and this is `evil-esc`.

The get the old behaviour, `access_keymap_1` would have to check all
parent keymaps in turn. For each keymap that bind (kbd "ESC") to
another keymap it would have to check the second key.

I hope this helps. Sorry that I do not provide a patch, but currently
my "Emacs-C" is too bad for this.

Anyhow, the real problem is to "multiplex" the (kbd "ESC") event in
the terminal. Any solution that sends 'escape instead of (kbd "ESC")
if another event arrives within a short period should solve the
problem.

If my explanations are unclear, please tell ;)

Best regards,
Frank