M-x mystery

M-x mystery

[Silvio Levy]

Dear emacs folks

in trying to debug some bad interactions between vip.el (VI emulation)
and icicles, I've come against an obstable I can't surmount: M-x
sometimes ceases to be M-x and becomes instead ESC-x.  

Minimal example: I run "emacs -q -l foo.el" where foo.el contains this code:

(defun doit ()
  "run test"
  (interactive)
  (defvar vip-mode-map (make-keymap))
  (define-key vip-mode-map "\e" 'kill-line)
  (use-local-map vip-mode-map)
)

Then I into a scratch buffer and type M-x doit<RET> -- so far so good.
After that, when I type M-x the behavior, it is as if I had typed ESC
and x: the current line is killed and an x is inserted.

Any ideas about what I'm doing wrong?

Silvio

Re: M-x mystery

[Alexander]

[-- Attachment #1: Type: text/plain, Size: 979 bytes --]

On Mon, Jan 2, 2012 at 4:51 AM, Silvio Levy <levy@msri.org> wrote:

> Dear emacs folks
>
> in trying to debug some bad interactions between vip.el (VI emulation)
> and icicles, I've come against an obstable I can't surmount: M-x
> sometimes ceases to be M-x and becomes instead ESC-x.
>
> Minimal example: I run "emacs -q -l foo.el" where foo.el contains this
> code:
>
> (defun doit ()
>  "run test"
>  (interactive)
>  (defvar vip-mode-map (make-keymap))
>  (define-key vip-mode-map "\e" 'kill-line)
>  (use-local-map vip-mode-map)
> )
>
> Then I into a scratch buffer and type M-x doit<RET> -- so far so good.
> After that, when I type M-x the behavior, it is as if I had typed ESC
> and x: the current line is killed and an x is inserted.
>
> Any ideas about what I'm doing wrong?
>
> Silvio
>
>
> I'm a novice but from what I read in an GNU emacs manual ESC and Meta are
initially the same. So maybe you mapped something to ESC and emacs is
confused?
-- 
Alexander
7D9C597B

[-- Attachment #2: Type: text/html, Size: 1418 bytes --]

RE: M-x mystery

[Drew Adams]

> in trying to debug some bad interactions between vip.el (VI emulation)
> and icicles, I've come against an obstable I can't surmount: M-x
> sometimes ceases to be M-x and becomes instead ESC-x.  
> 
> Minimal example: I run "emacs -q -l foo.el" where foo.el 
> contains this code:
> (defun doit ()
>   "run test"
>   (interactive)
>   (defvar vip-mode-map (make-keymap))
>   (define-key vip-mode-map "\e" 'kill-line)
>   (use-local-map vip-mode-map))
> 
> Then I into a scratch buffer and type M-x doit<RET> -- so far so good.
> After that, when I type M-x the behavior, it is as if I had typed ESC
> and x: the current line is killed and an x is inserted.
> Any ideas about what I'm doing wrong?

`M-x' is typically the same as `ESC x' (it is translated from `ESC x').  If you
redefine `ESC' as a command (`kill-line') then it can no longer continue to act
as a prefix key.  So both `M-x' and `ESC x' are then undefined.

(Also, why are you doing `make-keymap' here?  Why is that necessary?  Why not
just invoke `doit' after you have loaded Vip, so the keymap will be already
defined?)

FYI: You can customize the key that Icicles uses to invoke
`icicle-execute-extended-command' (by default it is `M-x').  Just customize
option `icicle-top-level-key-bindings', changing the entry for that command.

Anyway, I tried to follow your recipe.  It seems incomplete.  You start with
`emacs -q`, which is good, but you do not show where/when you load vip.el[c] or
Icicles, and you do not show where/when you turn on Vip mode or Icicles mode.
Sometimes the load or mode-invocation order can matter (but it should not).

When I do `emacs -Q', then load vip.el, then evaluate the code above and do `M-x
doit', then load Icicles and turn on Icicle mode, I do not see the behavior you
describe.  If I then turn on Vip mode (using `M-x vip-mode' and answering `n' to
inhibiting startup msg), I still do not see that behavior.  If I toggle Icicle
mode off then on again I still do not see that behavior.  And I don't see any
difference if I turn on vip-mode before Icicle mode, etc.  The order does not
seem to matter.

`C-h k' says `M-x' and `ESC x' are undefined.  But they both seem to work
normally, invoking `icicle-execute-extended-command' in Icicle mode and
`execute-extended-command' when I turn off Icicle mode!  Yet looking at
`vip-mode-map' I see that it has `ESC' defined as `kill-line'.  But `C-h w
kill-line' does not list `ESC' as one of the bindings.  I get the impression
that I am not really in Vip mode (?).  And yet I see `Vi:' in the mode line.
But `C-h m' does not mention Vip at all (?).

I tried toggling Vip mode using `M-x vip-mode', but I still see `Vi:' in the
mode line. I know nothing about Vip, so I'm probably not turning it off and on
correctly.  I took a quick look at the Vip doc, but it wasn't very helpful,
IMHO.

Please send me a more complete recipe, mentioning also your Emacs version. We
can take this off list. But if someone else familiar with Vip can help, please
post.

Re: M-x mystery

[Silvio Levy]

Hi Drew, I'll answer this question first:

> (Also, why are you doing `make-keymap' here?  Why is that necessary?  

It is vip that does make-keymap. I simply boiled down wha vip.el is
doing to the minimum code that causes the problem. If I just reported
"icicle messes up vi emulation mode" I couldn't expect someone to look
at the vi emulation code. So I distilled out of that the problem I'm
reporting. I can reproduce it in 6 lines. Let me rewrite them here
without mentioning vip at all:

(defun doit ()
  "run test"
  (interactive)
  (defvar my-map (make-keymap))
  (define-key my-map "\e" 'kill-line)
  (use-local-map my-map))

Now let me address this:
> `M-x' is typically the same as `ESC x' (it is translated from `ESC x')

I should have written alt-x,  Surely that's distinct from ESC x. 

When I start a fresh emacs -q, if I type <help> k <alt>x, emacs
reports that M-x is bound to execute-extended-command. Now suppose I
go into a buffer containing the code above, and do C-x C-e at the end
of it. Then after M-x doit<RET>, it seems that <alt>x is not even
recognized anymore: when I type <help> k <alt>x, emacs reports that
Escape is bounded to kill-line,

And that's what I don't understand. It's one thing for <ESC> x to be
treated as M-x because some people lack that alt key, but for emacs to
*convert* the keycode <alt>x to <ESC> x seems to me like poor design.

Why is it poor design? This is where I need to explain my motivation
for rebinding <ESC>.

Vi makes a distinction between insert mode and command mode. The fact
that (standard) emacs treats undecorated keys as self-insertion is in
my view its only major design flaw, and one that is neatly remedied by
vi emulation. In vi, undecorated keys are used for all sorts of
purposes, like moving around the file, which avoid having to reach for
the control key for every damn thing. Naturally, when you're adding
text you want these same keys to self-insert; ergo, there is an insert
mode. And in insert mode, ESC means "get me out of insert mode".

So that's why vip rebinds ESC (when in insert mode). Icicle disables
that. So I was trying to rebind vip-change-to-vi-mode (get out of
insert mode) away from ESC to, say, <F1>, which is close to ESC on the
keyboard -- close enough that I can live with it. But nothing I did
worked.

And in trying to figure out why, I'm frustrated by the fact that
<alt>x is disabled by vip's rebinding of ESC. And **that's** the
problem I'm trying to solve first. I have a hard time imagining that
emacs cannot be persuaded to recognize <alt>x separately from <ESC>.

Best,

Silvio


> > in trying to debug some bad interactions between vip.el (VI emulation)
> > and icicles, I've come against an obstable I can't surmount: M-x
> > sometimes ceases to be M-x and becomes instead ESC-x.  
> > 
> > Minimal example: I run "emacs -q -l foo.el" where foo.el 
> > contains this code:
> > (defun doit ()
> >   "run test"
> >   (interactive)
> >   (defvar vip-mode-map (make-keymap))
> >   (define-key vip-mode-map "\e" 'kill-line)
> >   (use-local-map vip-mode-map))
> > 
> > Then I into a scratch buffer and type M-x doit<RET> -- so far so good.
> > After that, when I type M-x the behavior, it is as if I had typed ESC
> > and x: the current line is killed and an x is inserted.
> > Any ideas about what I'm doing wrong?
> 
> `M-x' is typically the same as `ESC x' (it is translated from `ESC x').  If you
> redefine `ESC' as a command (`kill-line') then it can no longer continue to act
> as a prefix key.  So both `M-x' and `ESC x' are then undefined.
> 
> (Also, why are you doing `make-keymap' here?  Why is that necessary?  Why not
> just invoke `doit' after you have loaded Vip, so the keymap will be already
> defined?)
> 
> FYI: You can customize the key that Icicles uses to invoke
> `icicle-execute-extended-command' (by default it is `M-x').  Just customize
> option `icicle-top-level-key-bindings', changing the entry for that command.
> 
> Anyway, I tried to follow your recipe.  It seems incomplete.  You start with
> `emacs -q`, which is good, but you do not show where/when you load vip.el[c] or
> Icicles, and you do not show where/when you turn on Vip mode or Icicles mode.
> Sometimes the load or mode-invocation order can matter (but it should not).
> 
> When I do `emacs -Q', then load vip.el, then evaluate the code above and do `M-x
> doit', then load Icicles and turn on Icicle mode, I do not see the behavior you
> describe.  If I then turn on Vip mode (using `M-x vip-mode' and answering `n' to
> inhibiting startup msg), I still do not see that behavior.  If I toggle Icicle
> mode off then on again I still do not see that behavior.  And I don't see any
> difference if I turn on vip-mode before Icicle mode, etc.  The order does not
> seem to matter.
> 
> `C-h k' says `M-x' and `ESC x' are undefined.  But they both seem to work
> normally, invoking `icicle-execute-extended-command' in Icicle mode and
> `execute-extended-command' when I turn off Icicle mode!  Yet looking at
> `vip-mode-map' I see that it has `ESC' defined as `kill-line'.  But `C-h w
> kill-line' does not list `ESC' as one of the bindings.  I get the impression
> that I am not really in Vip mode (?).  And yet I see `Vi:' in the mode line.
> But `C-h m' does not mention Vip at all (?).
> 
> I tried toggling Vip mode using `M-x vip-mode', but I still see `Vi:' in the
> mode line. I know nothing about Vip, so I'm probably not turning it off and on
> correctly.  I took a quick look at the Vip doc, but it wasn't very helpful,
> IMHO.
> 
> Please send me a more complete recipe, mentioning also your Emacs version. We
> can take this off list. But if someone else familiar with Vip can help, please
> post.
> 

RE: M-x mystery

[Drew Adams]

> And that's what I don't understand. It's one thing for <ESC> x to be
> treated as M-x because some people lack that alt key, but for emacs to
> *convert* the keycode <alt>x to <ESC> x seems to me like poor design.

That's just what Emacs does, and it always has.  (This much has nothing to do
with Icicles.)

See user option `meta-prefix-char', which is bound by default to ESC (character
value 27).  See also the Elisp manual, node `Functions for Key Lookup'.  (You
might also look at node `Other Char Bits', about the Alt key.)

I'm no expert on Emacs's `Meta' key and the `Alt' key.  No doubt others could
inform you better about this.  In particular, perhaps a Vip or Viper user can
help you here.

> Why is it poor design? This is where I need to explain my motivation
> for rebinding <ESC>.
> 
> Vi makes a distinction between insert mode and command mode. The fact
> that (standard) emacs treats undecorated keys as self-insertion is in
> my view its only major design flaw, and one that is neatly remedied by
> vi emulation. In vi, undecorated keys are used for all sorts of
> purposes, like moving around the file, which avoid having to reach for
> the control key for every damn thing. Naturally, when you're adding
> text you want these same keys to self-insert; ergo, there is an insert
> mode. And in insert mode, ESC means "get me out of insert mode".
> 
> So that's why vip rebinds ESC (when in insert mode). Icicle disables
> that.

I wonder how.  Are you sure it does?  I believe there are some people who use
Icicles with Viper (dunno about Vip).

By default (see my previous mail about customizing this), Icicles remaps command
`execute-extended-command' to `icicle-execute-extended-command' when in Icicle
mode.  If you use Emacs 22+ (you still haven't said what Emacs version you are
using), then this is done using `remap' with `define-key'.  If you are using
Emacs < 22 then this is done using `substitute-key-definition'.

In effect, this rebinds `M-x' to the Icicles version of the command.  And AFAIK
this remapping means that all keys normally bound to `execute-extended-command'
are rebound - and that includes `ESC x'.

But I'm not aware where Icicles binds `ESC' - it remains a prefix key, but I
don't believe that Icicles redefines it as a prefix key if you have undefined it
as such by binding it to a command (e.g. `kill-line').  In any case, you should
be able to customize the binding of `icicle-execute-extended-command', as I
mentioned.

As I said, my attempt to repro your problem failed - I guess I need a more
detailed recipe.  When I tried to follow your recipe, `M-x' still behaved
normally, invoking `(icicle-)execute-extended-command' in and out of Icicle
mode.  AFAICT, I was never really in Vip mode (?).

> So I was trying to rebind vip-change-to-vi-mode (get out of
> insert mode) away from ESC to, say, <F1>, which is close to ESC on the
> keyboard -- close enough that I can live with it. But nothing I did
> worked.

It might be easier to leave Vip alone and customize Icicles.  Follow up off
list, if you like.  Let me know in detail (in order) what goes wrong when you
make no changes to either, and I can perhaps tell you how to customize Icicles
to help the two get along well.  Or perhaps a Vipian or Viperian on the list can
help.

> And in trying to figure out why, I'm frustrated by the fact that
> <alt>x is disabled by vip's rebinding of ESC. And **that's** the
> problem I'm trying to solve first. I have a hard time imagining that
> emacs cannot be persuaded to recognize <alt>x separately from <ESC>.

See above.  See option `meta-prefix-char'.

And can you please give me a more detailed recipe that will let me repro the
problem?  When and how do you activate Vip and Icicles etc.  See my previous
reply.  Thx - Drew.

Re: M-x mystery

[Teemu Likonen]

* 2012-01-02T09:37:15-08:00 * Silvio Levy wrote:

> And that's what I don't understand. It's one thing for <ESC> x to be
> treated as M-x because some people lack that alt key, but for emacs to
> *convert* the keycode <alt>x to <ESC> x seems to me like poor design.

That's a historical Unix text terminal thing. All input must have a
"slot" in the current character set. That is, everything must be a
character (code). There is this concept of Meta key which is produced by
Esc in today's keyboards. Its Ascii code is 27 so it's a character (kind
of).

If we want to use Alt key in text terminals we must decide which
character (code) it produces. Well, the choice has already been made for
us. Alt+x was chosen to insert Meta x so that it's easier to type those
Meta key combinations.

Emacs in Unix text terminals is tied to this design of text terminals
(remember, you can use them through a modem/serial line). Emacs's
graphical user interface does not _need_ to be as limited but apparently
developers want it to be as compatible as possible anyway - and I agree
with them.

Re: M-x mystery

[Silvio Levy]

Thank Teemu. I understand there are historical constraints. But note
that I'm not using the emacs GUI at all. I'm just talking about the
keyboard, and I thought alt-x is treated by keyboards in a way that's
exactly parallel to ctrl-x. 

I do see that saying (define-key global-map "\M-x" 'kill-line) after
I've redefined ESC generates the complaint "Key sequence M-x starts
with non-prefix key".  

I'm studying the references Drew sent in the Elisp manual, `Functions
for Key Lookup' and `Other Char Bits'. Obviously there are many
subtleties I wasn't aware of.

Best wishes,

Silvio


> From: Teemu Likonen <tlikonen@iki.fi>
> To: Silvio Levy <levy@msri.org>
> Cc: Drew Adams <drew.adams@oracle.com>, help-gnu-emacs@gnu.org
> Subject: Re: M-x mystery
> References: <20120102173715.D8C94180B85@neo.msri.org>
> Date: Mon, 02 Jan 2012 21:40:57 +0200
> In-Reply-To: <20120102173715.D8C94180B85@neo.msri.org> (Silvio Levy's message
> 	of "Mon, 02 Jan 2012 09:37:15 -0800")
> 
> * 2012-01-02T09:37:15-08:00 * Silvio Levy wrote:
> 
> > And that's what I don't understand. It's one thing for <ESC> x to be
> > treated as M-x because some people lack that alt key, but for emacs to
> > *convert* the keycode <alt>x to <ESC> x seems to me like poor design.
> 
> That's a historical Unix text terminal thing. All input must have a
> "slot" in the current character set. That is, everything must be a
> character (code). There is this concept of Meta key which is produced by
> Esc in today's keyboards. Its Ascii code is 27 so it's a character (kind
> of).
> 
> If we want to use Alt key in text terminals we must decide which
> character (code) it produces. Well, the choice has already been made for
> us. Alt+x was chosen to insert Meta x so that it's easier to type those
> Meta key combinations.
> 
> Emacs in Unix text terminals is tied to this design of text terminals
> (remember, you can use them through a modem/serial line). Emacs's
> graphical user interface does not _need_ to be as limited but apparently
> developers want it to be as compatible as possible anyway - and I agree
> with them.

Re: M-x mystery

[Silvio Levy]

Hi,

Sorry to bother you all, but the info in 
http://www.gnu.org/software/emacs/manual/html_node/elisp/Functions-for-Key-Lookup.html
doesn't seem to correspond to my experience.  I wonder if it
corresponds to yours.

First, let me tell you what does work as described. My ALT key does
seem to be interpreted by Emacs as Meta, in principle. The page above says

        M-<F1>, a function key, is not converted into <ESC> <F1>. 

and indeed, if I type ALT-F5 the response is "<M-f5> undefined", as
expected. (I'm using F5 because my F1 is intercepted by the window
manager - can't figure out how.)

And, ALT-x is interpreted as ESC x, also as described above. 

The problem is the variable meta-prefix-char, which is supposed to
alter that relationship.

If I load a file containing only the command (setq meta-prefix-char [f5])
and check the value of the variable, the answer is [f5].  

******** Yet, after that, ALT-x is still interpreted as ESC x. ********  

You might say that [f5] is not a valid prefix key. But [f2] is (it's
bound to 2C-mode-map in two-column.el). After I set meta-prefix-char
to [f2], typing ALT-x is still interpreted as ESC x, not [f2] x.

In fact, I can't even duplicate the example near the bottom of the
"Functions for Key Lookup" page (link above). Namely, setting
meta-prefix-char to 24 **should** make M-x behave like C-x, according
to that page. But that does not happen; even with meta-prefix-char=24,
ALT-x is interpreted as ESC x.

What have I misunderstood?

This is all in emacs 23.1.50.1 called with -q.

Silvio

RE: M-x mystery

[Drew Adams]

> The problem is the variable meta-prefix-char, which is supposed to
> alter that relationship.
> 
> If I load a file containing only the command (setq 
> meta-prefix-char [f5]) and check the value of the variable,
> the answer is [f5].  
> 
> ******** Yet, after that, ALT-x is still interpreted as ESC 
> x. ********  
> 
> You might say that [f5] is not a valid prefix key. But [f2] is (it's
> bound to 2C-mode-map in two-column.el). After I set meta-prefix-char
> to [f2], typing ALT-x is still interpreted as ESC x, not [f2] x.

I'm really no expert on these things.  No doubt someone will clarify better.
But [f2] is not a _character_.  (It is a vector.)  A character is a non-negative
integer (in a certain range) - see the Elisp manual, node `Character Codes'.

`meta-prefix-char' is a user option. Try using Customize to change its value.
Customize does type-checking, and it will not even allow you to set the variable
to [f2].  It raises this error: 

"This field should contain a single character"

(It should say "must", not "should".)

> In fact, I can't even duplicate the example near the bottom of the
> "Functions for Key Lookup" page (link above). Namely, setting
> meta-prefix-char to 24 **should** make M-x behave like C-x, according
> to that page. But that does not happen; 

I think it does.  Try `C-h k M-s' or `C-h k C-M-s'.  They correspond to `C-x s'
and `C-x C-x', respectively.

> even with meta-prefix-char=24,
> ALT-x is interpreted as ESC x.

I think you're doing things backwards.  Setting `meta-prefix-char' to the `C-x'
character (24) makes what used to be treated as `C-x SOMETHING' be treated as
`M-SOMETHING'.  If `C-x x' were defined as a command, then `M-x' would then
invoke that command (and not `execute-extended-command').  Since `C-x x' is
undefined by default, after setting `meta-prefix-char' to 24, `C-h k M-x' says
it is undefined.

In any case, doing that in no way affects the treatment of Meta as ESC, AFAIK.

> What have I misunderstood?

1. The value of `meta-prefix-char' needs to be a character.

2. I think the treatment of Meta as ESC happens at a different level.  Node
`Format of Keymaps' says this:

"Keymaps do not directly record bindings for the meta characters.
Instead, meta characters are regarded for purposes of key lookup as
sequences of two characters, the first of which is <ESC> (or whatever
is currently the value of `meta-prefix-char').  Thus, the key `M-a' is
internally represented as `<ESC> a', and its global binding is found at
the slot for `a' in `esc-map' (*note Prefix Keys::).

This conversion applies only to characters, not to function keys or
other input events; thus, `M-<end>' has nothing to do with `<ESC>
<end>'."

It seems that a Meta character is treated as a sequence of two chars, the first
of which is the value of `meta-prefix-char'.  AFAIK, that's just the way it is.
Someone will correct me if I misunderstand.