bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Drew Adams]

I have a library, lacarte.el, that lets you use the keyboard to navigate
the menu-bar menus.  The aim is thus similar to that of tmm, but you can
use completion against the full "path" to a menu item (submenus etc.).
Completion candidates look like this: `Buffers > Frames > foo.el' (item
`foo.el' in submenu `Frames' of menu-bar menu `Buffers'.

When you have multiple frames, the dynamically created `Buffers'
menu-bar menu has a (dynamically created) submenu `Frames'.

Starting with Emacs 23, instead of seeing candidates like `Buffers >
Frames > foo.el' in LaCarte, you see only a pseudo candidate for the
submenu itself: `Buffers > Frames', and if you choose that candidate you
get an error saying that there is no such command.  That's because the
LaCarte code assumes that the menu data structure corresponds to the
documented menu structures.

It seems that the Emacs code now uses an undocumented menu structure
here. The code in menu-bar.el that creates the `Buffers' menu (and its
`Frames' submenu) changed in Emacs 23 to use a vector of buffer (and a
vector of frame) entries instead of a list of them.

I don't see anything in the manual that mentions that a menu can take
this form.  Dunno whether I am not reading it well enough or the doc is
incomplete.

And I see nothing in the Emacs 23 NEWS about such a new menu structure.
(I do not understand how someone can make such a fundamental change and
not mention it in NEWS.  New menu structures are not something that
Emacs adds everyday.)

1. So at a minimum this is a DOC bug report: I would like the doc to
describe all of the possible forms of menus.  Apparently it no longer
does that.

2. Beyond that, using vectors here is a PITA for Lisp code.  It makes
code that traverses such code difficult, if not impossible.  Without
this change to vectors, a simple recursion on a list cdr is all that is
needed.  No doubt I'll find a fix, once I know the actual possible menu
structures available.  But using vectors here does not seem very lispy.

Could you perhaps consider changing the code back to using lists?


Here are some details - see function `menu-bar-update-buffers' in
`menu-bar.el.  This is a snippet from that function - the part that
creates the `Frames' submenu:

Emacs 22 (and prior is similar, but a little different (but still uses a
list, not a vector)):

;; Make a Frames menu if we have more than one frame.
(when (cdr frames)
  (let ((frames-menu
         (cons 'keymap
               (cons "Select Frame"
                     (mapcar
                      (lambda (frame)
                        (nconc
                         (list (frame-parameter frame 'name)
                               (frame-parameter frame 'name)
                               (cons nil nil))
                         'menu-bar-select-frame))
                      frames)))))
    ;; Put it after the normal buffers
    (setq buffers-menu
          (nconc buffers-menu
                 `((frames-separator "--")
                   (frames menu-item "Frames" ,frames-menu))))))

Emacs 23 - uses a vector (why?):

;; Make a Frames menu if we have more than one frame.
(when (cdr frames)
  (let* ((frames-vec (make-vector (length frames) nil)) ; <============
         (frames-menu (cons 'keymap (list "Select Frame" frames-vec)))
         (i 0))
    (dolist (frame frames)
      (aset frames-vec i
            (nconc
             (list
              (frame-parameter frame 'name)
              (cons nil nil))
             `(lambda ()
                (interactive) (menu-bar-select-frame ,frame))))
      (setq i (1+ i)))
    ;; Put it after the normal buffers
    (setq buffers-menu
          (nconc buffers-menu
                 `((frames-separator "--")
                   (frames menu-item "Frames" ,frames-menu))))))

I assume (hope) this change to using vectors was not gratuitous.  But is
it necessary?  The cost is added difficulty analyzing and traversing the
data structure, a priori.  What is the benefit?

As a heads-up, can you tell me where else, besides these two dynamically
created menus (`Buffers' and `Frames'), Emacs use vectors in menu data
structures?




In GNU Emacs 24.3.50.1 (i686-pc-mingw32)
 of 2013-07-01 on LEG570
Bzr revision: 113246 lekktu@gmail.com-20130701165437-ea20s94hqwp3ttaj
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --prefix=/c/usr --enable-checking CFLAGS='-O0 -g3'
 CPPFLAGS='-DGLYPH_DEBUG=1 -I/c/usr/include''

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Stefan Monnier]

> I don't see anything in the manual that mentions that a menu can take
> this form.  Dunno whether I am not reading it well enough or the doc is
> incomplete.

Indeed, vectors are an old format that is deprecated but
still supported.  They are similar to char-tables (which are supported,
on the other hand).

> (I do not understand how someone can make such a fundamental change and
> not mention it in NEWS.  New menu structures are not something that
> Emacs adds everyday.)

That's because it's a new menu structure, on the contrary.

> 2. Beyond that, using vectors here is a PITA for Lisp code.  It makes
> code that traverses such code difficult, if not impossible.  Without
> this change to vectors, a simple recursion on a list cdr is all that is
> needed.

You should probably use `map-keymap' instead.


        Stefan

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Drew Adams]

> > I don't see anything in the manual that mentions that a menu can take
> > this form.  Dunno whether I am not reading it well enough or the doc is
> > incomplete.
> 
> Indeed, vectors are an old format that is deprecated but
> still supported.  They are similar to char-tables (which are supported,
> on the other hand).

Really? In what Emacs version was that old format documented?  I do not
see it, going back to Emacs 20, and I do not remember ever seeing it
before that.

And why would we suddently make a code change in Emacs 24.4 to revert back
to using a deprecated menu structure?  And without adding any comment in
the code as to (a) the fact that we deliberately use a deprecated and
undocumented structure here and (b) why we do so.

What was gained by this?

Note too that deprecation of a feature (which as you note generally does
not imply its desupport) generally does not mean removing all doc for it.
As long as something is supported it is typically documented - at least
reference doc.  Support implies doc, generally.

This "old format" has not been documented since at least Emacs 20.
If the Emacs code is going to start using it again then it should be
documented.

> > (I do not understand how someone can make such a fundamental change and
> > not mention it in NEWS.  New menu structures are not something that
> > Emacs adds everyday.)
> 
> That's because it's a new menu structure, on the contrary.

Is there a "not" missing there, perhaps?  If not, I do not follow you.

AFAICT, it IS a new menu structure, at least in terms of documentation.
Where was it documented for users before, if in fact it was really a
supported menu structure?

> > 2. Beyond that, using vectors here is a PITA for Lisp code.  It makes
> > code that traverses such code difficult, if not impossible.  Without
> > this change to vectors, a simple recursion on a list cdr is all that is
> > needed.
> 
> You should probably use `map-keymap' instead.

Yes, eventually I will perhaps do that.  My code was originally based
on similar code in `substitute-key-definition', whose code evolved to use
`map-keymap' when that function was introduced.

But that is really beside the point here.  Why introduce an undocumented,
at best deprecated, menu structure in Emacs 24.4?  Was something useful
gained wrt the list structure that was used before?  Why move TOWARD
something that has (at best) been deprecated for >15 years and is less
lispy?

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Stefan Monnier]

> But that is really beside the point here.  Why introduce an undocumented,
> at best deprecated, menu structure in Emacs 24.4?

To see if you'd notice,


        Stefan

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Drew Adams]

> > But that is really beside the point here.  Why introduce an undocumented,
> > at best deprecated, menu structure in Emacs 24.4?
> 
> To see if you'd notice, Stefan

Clever.

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Drew Adams]

Could this bug please be fixed?  It is as simple as this:

A menu keymap (and perhaps other keymap?) can apparently have a
vector of menu entries as its third element.  This is the case for
`global-buffers-menu-map', for instance.

I do not see this form of keymap documented anywhere in the Elisp
manual.  We bother to document the `keymap-prompt', which is hardly
ever used in practice, but this alternative structure of a keymap is
completely undocumented, AFAICT.

Please document it.  Just what is the form that is allowed?  This
belongs in the Elisp manual, as part of the keymap structure
description.

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[npostavs]

[-- Attachment #1: Type: text/plain, Size: 182 bytes --]

tags 14797 patch
quit

Drew Adams <drew.adams@oracle.com> writes:

> What was gained by this?

I guess it's for efficiency?

Anyway, it might as well be documented, how about this:


[-- Attachment #2: patch --]
[-- Type: text/plain, Size: 1218 bytes --]

From 6dd01ff617b3ee8ef8ad307a7776c2142f7f02ed Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 6 Aug 2016 09:05:05 -0400
Subject: [PATCH v1] Document use of vectors in keymaps

* doc/lispref/keymaps.texi (Format of Keymaps): Mention vector
format (Bug #14797).
---
 doc/lispref/keymaps.texi | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/doc/lispref/keymaps.texi b/doc/lispref/keymaps.texi
index f5d3811..a47d790 100644
--- a/doc/lispref/keymaps.texi
+++ b/doc/lispref/keymaps.texi
@@ -199,6 +199,14 @@ Format of Keymaps
 bindings.  A keymap with such a char-table is called a @dfn{full
 keymap}.  Other keymaps are called @dfn{sparse keymaps}.
 
+@item @var{vector}
+This kind of element is similar to a char-table: element @var{n} is
+the binding for the character with code @var{n}.  Since the range of
+characters that can be bound this way is limited by the vector size,
+and vector creation allocates space for all character codes from 0 up,
+this format should not be used except for creating menu keymaps
+(@pxref{Menu Keymaps}), where the bindings themselves don't matter.
+
 @item @var{string}
 @cindex keymap prompt string
 @cindex overall prompt string
-- 
2.8.0

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Eli Zaretskii]

> From: npostavs@users.sourceforge.net
> Date: Sat, 06 Aug 2016 09:09:50 -0400
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, 14797@debbugs.gnu.org
> 
> --- a/doc/lispref/keymaps.texi
> +++ b/doc/lispref/keymaps.texi
> @@ -199,6 +199,14 @@ Format of Keymaps
>  bindings.  A keymap with such a char-table is called a @dfn{full
>  keymap}.  Other keymaps are called @dfn{sparse keymaps}.
>  
> +@item @var{vector}
> +This kind of element is similar to a char-table: element @var{n} is
> +the binding for the character with code @var{n}.  Since the range of
> +characters that can be bound this way is limited by the vector size,
> +and vector creation allocates space for all character codes from 0 up,
> +this format should not be used except for creating menu keymaps
> +(@pxref{Menu Keymaps}), where the bindings themselves don't matter.
> +
>  @item @var{string}
>  @cindex keymap prompt string
>  @cindex overall prompt string

LGTM, except that "the character with code N" could use some better
wording.  How about

  ... element whose index is @var{c} is the binding for the character
  @var{c}.

instead?

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Noam Postavsky]

On Sat, Aug 6, 2016 at 10:42 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: npostavs@users.sourceforge.net
>> Date: Sat, 06 Aug 2016 09:09:50 -0400
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, 14797@debbugs.gnu.org
>>
>> --- a/doc/lispref/keymaps.texi
>> +++ b/doc/lispref/keymaps.texi
>> @@ -199,6 +199,14 @@ Format of Keymaps
>>  bindings.  A keymap with such a char-table is called a @dfn{full
>>  keymap}.  Other keymaps are called @dfn{sparse keymaps}.
>>
>> +@item @var{vector}
>> +This kind of element is similar to a char-table: element @var{n} is
>> +the binding for the character with code @var{n}.  Since the range of
>> +characters that can be bound this way is limited by the vector size,
>> +and vector creation allocates space for all character codes from 0 up,
>> +this format should not be used except for creating menu keymaps
>> +(@pxref{Menu Keymaps}), where the bindings themselves don't matter.
>> +
>>  @item @var{string}
>>  @cindex keymap prompt string
>>  @cindex overall prompt string
>
> LGTM, except that "the character with code N" could use some better
> wording.  How about
>
>   ... element whose index is @var{c} is the binding for the character
>   @var{c}.
>
> instead?

Sure. I guess the char-table paragraph should use the same wording, right?

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Drew Adams]

> > +@item @var{vector}
> > +This kind of element is similar to a char-table: element @var{n} is
> > +the binding for the character with code @var{n}.  Since the range of
> > +characters that can be bound this way is limited by the vector size,
> > +and vector creation allocates space for all character codes from 0 up,
> > +this format should not be used except for creating menu keymaps
> > +(@pxref{Menu Keymaps}), where the bindings themselves don't matter.
> > +
> >  @item @var{string}
> >  @cindex keymap prompt string
> >  @cindex overall prompt string
> 
> LGTM, except that "the character with code N" could use some better
> wording.  How about
>   ... element whose index is @var{c} is the binding for the character
>   @var{c}.
> instead?

Both look OK to me.  And adding such doc is a definite
improvement.

It's not very clear to me from this doc what the relation
is between a key binding and a "binding for the character".
Keys and characters are different animals.  Clearly there
is a relation between them here, but it doesn't seem to
be made explicit.

But again, unless you want to try to clarify that, the
proposed doc is OK, and adding it improves things.

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[npostavs]

Drew Adams <drew.adams@oracle.com> writes:
> It's not very clear to me from this doc what the relation
> is between a key binding and a "binding for the character".
> Keys and characters are different animals.  Clearly there
> is a relation between them here, but it doesn't seem to
> be made explicit.

I think it's clear from the text in `(elisp) Key Sequences':

    A "key sequence", or "key" for short, is a sequence of one or more
    input events that form a unit.  Input events include characters,

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Drew Adams]

> > It's not very clear to me from this doc what the relation
> > is between a key binding and a "binding for the character".
> > Keys and characters are different animals.  Clearly there
> > is a relation between them here, but it doesn't seem to
> > be made explicit.
> 
> I think it's clear from the text in `(elisp) Key Sequences':
> 
>     A "key sequence", or "key" for short, is a sequence of one or more
>     input events that form a unit.  Input events include characters,

Maybe.  (But "character" is also something other than an input
event, even for Emacs.)

But it is still "not very clear to me from _this_ doc".

A few words clarifying that might help.  Or a cross-ref to
the doc that covers it.

> > But again, unless you want to try to clarify that, the
> > proposed doc is OK, and adding it improves things.

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Sat, 6 Aug 2016 11:42:14 -0400
> Cc: Drew Adams <drew.adams@oracle.com>, Stefan Monnier <monnier@iro.umontreal.ca>, 
> 	14797@debbugs.gnu.org
> 
> >   ... element whose index is @var{c} is the binding for the character
> >   @var{c}.
> >
> > instead?
> 
> Sure. I guess the char-table paragraph should use the same wording, right?

Right, thanks.

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[Eli Zaretskii]

> From: npostavs@users.sourceforge.net
> Cc: Eli Zaretskii <eliz@gnu.org>,  14797@debbugs.gnu.org,  monnier@iro.umontreal.ca
> Date: Sat, 06 Aug 2016 12:35:10 -0400
> 
> Drew Adams <drew.adams@oracle.com> writes:
> > It's not very clear to me from this doc what the relation
> > is between a key binding and a "binding for the character".
> > Keys and characters are different animals.  Clearly there
> > is a relation between them here, but it doesn't seem to
> > be made explicit.
> 
> I think it's clear from the text in `(elisp) Key Sequences':
> 
>     A "key sequence", or "key" for short, is a sequence of one or more
>     input events that form a unit.  Input events include characters,

Indeed.  So each character is a key.

bug#14797: 24.3.50; new, undocumented menu structure using VECTORS?

[npostavs]

tags 14797 fixed
close 14797 25.1
quit

Pushed as 3c9cb57c