23.0.60; kbd returns wrong value

23.0.60; kbd returns wrong value

[Drew Adams]

This is a menu item in menu-bar-help-menu:
<describe> <describe-language-environment> <European> <Brazilian Portuguese>
 
Evaluating this:
(kbd "<describe> <describe-language-environment> <European> <Brazilian
Portuguese>")
 
produces the following incorrect result:
 
[describe describe-language-environment European
 60 66 114 97 122 105 108 105 97 110 80 111 114
 116 117 103 117 101 115 101 62]
 
After tracing edmacro-parse-keys, the problem seems to be here:
 
(while (and (< pos (length string))
  (string-match "[^ \t\n\f]+" string pos))
      (let ((word (substring string (match-beginning 0) (match-end 0)))
 
The sexp (substring "<describe> <describe-language-environment> <European>
<Brazilian Portuguese>" 54 64) returns "<Brazilian", which is only half of the
entry.
 
IOW, the code is not expecting a space char. Which is the problem: the
edmacro-parse-keys code or the definition of the key itself,
<Brazilian Portuguese>, which includes a space char?

This problem is not new with Emacs 23, BTW.
 

In GNU Emacs 23.0.60.1 (i386-mingw-nt5.1.2600)
 of 2008-05-04 on LENNART-69DE564
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4) --no-opt --cflags -Ic:/g/include
-fno-crossjumping'
 

RE: 23.0.60; kbd returns wrong value

[Drew Adams]

This problem is more widespread than I thought. Menu items can of course contain
spaces: <My Foobar>, and when used in submenus the same problem arises as for
<Brazilian Portuguese> in a <Describe> submenu.

An there are other keys that have spaces and so could make `edmacro-parse-keys'
choke in the same way (e.g. if in a submenu). In my `global-map', for instance,
I see these keys that contain spaces:

<RHP of Latin-1>
<RHP of Latin-2>
<RHP of Latin-3>
<RHP of Latin-4>
<RHP of TIS620>
<RHP of ISO8859/7>
<RHP of ISO8859/6>
<RHP of ISO8859/8>
<JISX0201 Katakana>
<JISX0201 Roman>
<RHP of ISO8859/5>
<RHP of Latin-5>
<RHP of Latin-9>
<RHP of Latin-8>
<Big5 (Level-1)>
<Big5 (Level-2)>
<VISCII lower>
<VISCII upper>
<Arabic digit>
<Arabic 1-col>
<rev ASCII>
<Arabic 2-col>
<IS 13194>
<Indian glyph>
<Tibetan 1-col>
<Unicode subset 2>
<Unicode subset 3>
<Unicode subset>
<Indian 2-col>
<Tibetan 2-col>

The bug is thus with `edmacro-parse-keys', not with the fact of having keys with
spaces.

> From: Drew Adams Sent: Tuesday, May 13, 2008 1:02 PM
> This is a menu item in menu-bar-help-menu:
> <describe> <describe-language-environment> <European> 
> <Brazilian Portuguese>
>  
> Evaluating this:
> (kbd "<describe> <describe-language-environment> <European> <Brazilian
> Portuguese>")
>  
> produces the following incorrect result:
>  
> [describe describe-language-environment European
>  60 66 114 97 122 105 108 105 97 110 80 111 114
>  116 117 103 117 101 115 101 62]
>  
> After tracing edmacro-parse-keys, the problem seems to be here:
>  
> (while (and (< pos (length string))
>   (string-match "[^ \t\n\f]+" string pos))
>       (let ((word (substring string (match-beginning 0) 
> (match-end 0)))
>  
> The sexp (substring "<describe> 
> <describe-language-environment> <European>
> <Brazilian Portuguese>" 54 64) returns "<Brazilian", which is 
> only half of the
> entry.
>  
> IOW, the code is not expecting a space char. Which is the problem: the
> edmacro-parse-keys code or the definition of the key itself,
> <Brazilian Portuguese>, which includes a space char?
> 
> This problem is not new with Emacs 23, BTW.
>  
> 
> In GNU Emacs 23.0.60.1 (i386-mingw-nt5.1.2600)
>  of 2008-05-04 on LENNART-69DE564
> Windowing system distributor `Microsoft Corp.', version 5.1.2600
> configured using `configure --with-gcc (3.4) --no-opt 
> --cflags -Ic:/g/include
> -fno-crossjumping'
>  
> 
> 
> 
> 

Re: 23.0.60; kbd returns wrong value

[David Kastrup]

"Drew Adams" <drew.adams@ORACLE.COM> writes:

> This problem is more widespread than I thought. Menu items can of course contain
> spaces: <My Foobar>, and when used in submenus the same problem arises as for
> <Brazilian Portuguese> in a <Describe> submenu.
>
> An there are other keys that have spaces and so could make `edmacro-parse-keys'
> choke in the same way (e.g. if in a submenu). In my `global-map', for instance,
> I see these keys that contain spaces:
>
> <RHP of Latin-1>
> <RHP of Latin-2>
> <RHP of Latin-3>
> <RHP of Latin-4>

> The bug is thus with `edmacro-parse-keys', not with the fact of having
> keys with spaces.

So how about using

>> After tracing edmacro-parse-keys, the problem seems to be here:
>>  
>> (while (and (< pos (length string))
>>   (string-match "[^ \t\n\f]+" string pos))
>>       (let ((word (substring string (match-beginning 0) 
>> (match-end 0)))

(string-match "[^ \t\n\f<]+\\|<[^>]+>" ...

instead?


-- 
David Kastrup

RE: 23.0.60; kbd returns wrong value

[Drew Adams]

> >> After tracing edmacro-parse-keys, the problem seems to be here:
> >>  
> >> (while (and (< pos (length string))
> >>   (string-match "[^ \t\n\f]+" string pos))
> >>       (let ((word (substring string (match-beginning 0) 
> >> (match-end 0)))
> 
> (string-match "[^ \t\n\f<]+\\|<[^>]+>" ...
> instead?

Yes, please install that fix; it seems to do the job. Thanks.

RE: 23.0.60; kbd returns wrong value

[Drew Adams]

> > >> After tracing edmacro-parse-keys, the problem seems to be here:
> > >>  
> > >> (while (and (< pos (length string))
> > >>   (string-match "[^ \t\n\f]+" string pos))
> > >>       (let ((word (substring string (match-beginning 0) 
> > >> (match-end 0)))
> > 
> > (string-match "[^ \t\n\f<]+\\|<[^>]+>" ... instead?
> 
> Yes, please install that fix; it seems to do the job. Thanks.

Actually, no, that breaks it for "C-x <", producing only "^X" (control-x
character).

The following regexp fixes that particular breakage: "[^
\t\n\f<]+\\|<[^>]+>\\|<+"

But that breaks other things. With input of "C-x <<>>", the output is [24 < 62].
With input of "C-x << >>", the output is [24 <\ 62]. In the vanilla code, e.g.,
the first produces [24 <>], and the second produces "^X<<>>" (control-x char),
which are presumably correct.

I have no idea what that is all about - there are no comments in the code wrt
what it is supposed to do with which inputs.

It would be good if someone who really understands `edmacro-parse-keys' would
please take a look and fix it as needed. And please add a doc string too.

RE: 23.0.60; kbd returns wrong value

[Drew Adams]

> > > >> After tracing edmacro-parse-keys, the problem seems to be here:
> > > >>  
> > > >> (while (and (< pos (length string))
> > > >>   (string-match "[^ \t\n\f]+" string pos))
> > > >>       (let ((word (substring string (match-beginning 0) 
> > > >> (match-end 0)))
> > > 
> > > (string-match "[^ \t\n\f<]+\\|<[^>]+>" ... instead?
> > 
> > Yes, please install that fix; it seems to do the job. Thanks.
> 
> Actually, no, that breaks it for "C-x <", producing only "^X" 
> (control-x character).
> 
> The following regexp fixes that particular breakage:
> "[^ \t\n\f<]+\\|<[^>]+>\\|<+"
> 
> But that breaks other things. With input of "C-x <<>>", the 
> output is [24 < 62]. With input of "C-x << >>", the output is
> [24 <\ 62]. In the vanilla code, e.g., the first produces
> [24 <>], and the second produces "^X<<>>" (control-x char),
> which are presumably correct.
> 
> I have no idea what that is all about - there are no comments 
> in the code wrt what it is supposed to do with which inputs.
> 
> It would be good if someone who really understands 
> `edmacro-parse-keys' would please take a look and fix it as
> needed. And please add a doc string too.

FYI - some more info about use of "[^ \t\n\f<]+\\|<[^>]+>\\|<+" as the regexp:

Input of "M-<" produces "M-<", but it produces [134217788] with vanilla code
(which is correct).

Re: 23.0.60; kbd returns wrong value

[Stefan Monnier]

> (string-match "[^ \t\n\f<]+\\|<[^>]+>" ...

To avoid problems with C-x < and such, I'd recomment

> (string-match "[^ \t\n\f<]+\\|<[^ \n>]\\(?:[^>]*[^ >\n]\\)?>" ...

So there can be spaces in symbols, but not as first or last char.

The handling of "C-x <<" and such shouldn't matter because these aren't
valid anyway.


        Stefan

RE: 23.0.60; kbd returns wrong value

[Drew Adams]

> > (string-match "[^ \t\n\f<]+\\|<[^>]+>" ...
> 
> To avoid problems with C-x < and such, I'd recomment
> (string-match "[^ \t\n\f<]+\\|<[^ \n>]\\(?:[^>]*[^ >\n]\\)?>" ...
> So there can be spaces in symbols, but not as first or last char.

See my previous message in reply to David, and my followup to that. Like David's
suggestion for the regexp, yours fails for "C-x <":

(edmacro-parse-keys "C-x <") gives "^X" (control-x character).

> The handling of "C-x <<" and such shouldn't matter because 
> these aren't valid anyway.

Then what is that part of the code for? Addin some comments to the code would
help understanding.

More key strangeness (was: 23.0.60; kbd returns wrong value)

[David Kastrup]

"Drew Adams" <drew.adams@oracle.com> writes:

>> > (string-match "[^ \t\n\f<]+\\|<[^>]+>" ...
>> 
>> To avoid problems with C-x < and such, I'd recomment
>> (string-match "[^ \t\n\f<]+\\|<[^ \n>]\\(?:[^>]*[^ >\n]\\)?>" ...
>> So there can be spaces in symbols, but not as first or last char.
>
> See my previous message in reply to David, and my followup to that. Like David's
> suggestion for the regexp, yours fails for "C-x <":
>
> (edmacro-parse-keys "C-x <") gives "^X" (control-x character).
>
>> The handling of "C-x <<" and such shouldn't matter because 
>> these aren't valid anyway.
>
> Then what is that part of the code for? Addin some comments to the code would
> help understanding.

Typing C-h k C-x <menu-bar> <options> <highlight-paren-mode>
gives the resulting error message
C-x <menu-bar> is undefined
in the *Messages* buffer, but
<highlight-paren-mode> is undefined
in the echo area.

Wow.  Where does that discrepancy come from?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: More key strangeness

[Stefan Monnier]

> Typing C-h k C-x <menu-bar> <options> <highlight-paren-mode>
> gives the resulting error message
> C-x <menu-bar> is undefined
> in the *Messages* buffer, but
> <highlight-paren-mode> is undefined
> in the echo area.

> Wow.  Where does that discrepancy come from?

I believe there's no discrepancy: What happens is that after selecting
the menu, you end up with 4 events:

   C-x <menu-bar> <options> <highlight-paren-mode>

and when reading them to find the corresponding key-sequence and
command, the first key-sequence is

   C-x <menu-bar>

and is unbound, so it generates the message "C-x <menu-bar> is
undefined".  Then the next event probably generates "<options> is undefined",
and so does <highlight-paren-mode>.  So you should see 3 messages
in total.


        Stefan