bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Drew Adams]

The doc for :filter says this about the FILTER-FN:

   Emacs can call this function at any time that it does redisplay or
   operates on menu data structures, so you should write it so it can
   safely be called at any time.

Is this true in general, or only when the extended menu item is put on a
menu?

A common idiom is to make use of a `menu-item' construct with a :filter
to create a conditional _keyboard_ key binding.  In such a case, the
`menu-item' construct is not a real menu item - it is not placed on any
menu.

I'm guessing that in such a case this doc paragraph does not apply.  If
this guess is correct then please correct the paragraph, so that it says
something like "If an extended menu item that uses :filter is placed on
a menu then Emacs can call FILTER-FN when...".

Also, is it really the case that FILTER-FN can be called anytime Emacs
does redisplay?  Shouldn't the doc say only that it can be called
anytime Emacs "operates on menu data structures"?  Does it get called by
redisplay other than when redisplay operates on menu data structures?
In the case mentioned above (binding to a keyboard key), would FILTER-FN
ever be called during redisplay?  I'm guessing that it would not.

In GNU Emacs 24.5.1 (i686-pc-mingw32)
 of 2015-04-11 on LEG570
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --prefix=/c/usr --host=i686-pc-mingw32'

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Lars Ingebrigtsen]

Drew Adams <drew.adams@oracle.com> writes:

> The doc for :filter says this about the FILTER-FN:
>
>    Emacs can call this function at any time that it does redisplay or
>    operates on menu data structures, so you should write it so it can
>    safely be called at any time.
>
> Is this true in general, or only when the extended menu item is put on a
> menu?
>
> A common idiom is to make use of a `menu-item' construct with a :filter
> to create a conditional _keyboard_ key binding.  In such a case, the
> `menu-item' construct is not a real menu item - it is not placed on any
> menu.
>
> I'm guessing that in such a case this doc paragraph does not apply.  If
> this guess is correct then please correct the paragraph, so that it says
> something like "If an extended menu item that uses :filter is placed on
> a menu then Emacs can call FILTER-FN when...".

I think making the documentation more specific here serves no purpose.
The statement as is should be true: You should always write these filter
functions as if they are called at any time.

> Also, is it really the case that FILTER-FN can be called anytime Emacs
> does redisplay?  Shouldn't the doc say only that it can be called
> anytime Emacs "operates on menu data structures"?  Does it get called by
> redisplay other than when redisplay operates on menu data structures?
> In the case mentioned above (binding to a keyboard key), would FILTER-FN
> ever be called during redisplay?  I'm guessing that it would not.

I don't see why we should specify that at all.

Closing.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Drew Adams]

> > The doc for :filter says this about the FILTER-FN:
> >
> >    Emacs can call this function at any time that it does redisplay or
> >    operates on menu data structures, so you should write it so it can
> >    safely be called at any time.
> >
> > Is this true in general, or only when the extended menu item is put on a
> > menu?
> >
> > A common idiom is to make use of a `menu-item' construct with a :filter
> > to create a conditional _keyboard_ key binding.  In such a case, the
> > `menu-item' construct is not a real menu item - it is not placed on any
> > menu.
> >
> > I'm guessing that in such a case this doc paragraph does not apply.  If
> > this guess is correct then please correct the paragraph, so that it says
> > something like "If an extended menu item that uses :filter is placed on
> > a menu then Emacs can call FILTER-FN when...".
> 
> I think making the documentation more specific here serves no purpose.
> The statement as is should be true: You should always write these filter
> functions as if they are called at any time.
> 
> > Also, is it really the case that FILTER-FN can be called anytime Emacs
> > does redisplay?  Shouldn't the doc say only that it can be called
> > anytime Emacs "operates on menu data structures"?  Does it get called by
> > redisplay other than when redisplay operates on menu data structures?
> > In the case mentioned above (binding to a keyboard key), would FILTER-FN
> > ever be called during redisplay?  I'm guessing that it would not.
> 
> I don't see why we should specify that at all.

The Elisp manual is not only for readers reading as
end users.  This information is needed by people
writing Lisp code that uses extended menu items.

And as the bug report points out, a very different
use case does not involve use of a menu at all.

Now perhaps Emacs should provide some alternative
to using an extended menu item, specifically for
this very different use case.  But until and unless
it does that, the behavior for this particular use
case should be covered in the doc.  In particular,
if the caveat that is written does NOT apply for
this use case, then that should be pointed out.

IF the guess is correct that this caveat does NOT
in fact apply to this use case then no, it is
NOT the case that "You should always write these
filter functions as if they are called at any time."
If that guess is correct then what you say as the
reason for not fixing this doc bug simply isn't
relevant.

This is not some obscure detail.  It goes to the
heart of the behavior and use of extended menu
items.  If you instead want to create a new Lisp
construct to achieve such filtering for non-menu
uses, great - that would take care of this lack in
a different, equally (maybe even more) acceptable
way.

Just ignoring this isn't TRT.  I, for one, would
like to know what the case really is wrt the
display engine problem when you use an extended
menu item without a menu.

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Eli Zaretskii]

> Date: Sat, 12 Dec 2020 12:45:17 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 24237@debbugs.gnu.org
> 
> IF the guess is correct that this caveat does NOT
> in fact apply to this use case then no, it is
> NOT the case that "You should always write these
> filter functions as if they are called at any time."
> If that guess is correct then what you say as the
> reason for not fixing this doc bug simply isn't
> relevant.

I don't think I understand what "this case" is, but in general menu
functions could be called whenever the display engine needs to
recalculate the contents and the dimensions of the menu, and that
could basically be every redisplay cycle, depending on circumstances.

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Drew Adams]

> > IF the guess is correct that this caveat does NOT
> > in fact apply to this use case then no, it is
> > NOT the case that "You should always write these
> > filter functions as if they are called at any time."
> > If that guess is correct then what you say as the
> > reason for not fixing this doc bug simply isn't
> > relevant.
> 
> I don't think I understand what "this case" is, 

As was said in the original bug report and has
been repeated in the thread, this case is the

 "use of a `menu-item' construct with a :filter
  to create a conditional _keyboard_ key binding.
  In such a case, the `menu-item' construct is not
  a real menu item - it is not placed on any menu."

> but in general menu functions

Define "menu function", please.  Does this apply
to the case being discussed: an extended menu
item that's bound only to a keyboard key, i.e.,
that's not used in any menu?

> could be called whenever the display engine needs to
> recalculate the contents and the dimensions of the menu, and that
> could basically be every redisplay cycle, depending on circumstances.

See above.  There's no menu involved in the case
being discussed (at least none that's visible to
users, AFAIK).  So there should be no need or
possibility of recalculating menu contents and
dimensions.

This is (only) about the hack/idiom of using an
extended menu item for a keyboard key binding
(only).  And the use case for doing that is to
take advantage of the :filter handling.

I don't know where/when the :filter handling is
done.  Perhaps it is done as an integral part of
redisplay.  If that's the case, does it mean
that the warning in the doc is applicable in
this case?

If it is thus applicable in this case, does it
need to be?  IOW, could handling of :filter be
separated out, at least in the case where no menu
is involved?

Again, though, I'm not really arguing that using
a `menu-item' just to get :filter behavior for a
keyboard key binding is the best approach.

The point is that currently it is the _only_
approach, hence the idiom.  If Emacs provides a
more direct way to do this, that's uncoupled from
`menu-item' and its redisplay handling, great.

Lacking that, this bug is about having the doc
address the use case of the idiom.

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Eli Zaretskii]

> Date: Sun, 13 Dec 2020 09:24:52 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: larsi@gnus.org, 24237@debbugs.gnu.org
> 
> > I don't think I understand what "this case" is, 
> 
> As was said in the original bug report and has
> been repeated in the thread, this case is the
> 
>  "use of a `menu-item' construct with a :filter
>   to create a conditional _keyboard_ key binding.
>   In such a case, the `menu-item' construct is not
>   a real menu item - it is not placed on any menu."

That doesn't explain anything to me, sorry.

> > but in general menu functions
> 
> Define "menu function", please.  Does this apply
> to the case being discussed: an extended menu
> item that's bound only to a keyboard key, i.e.,
> that's not used in any menu?

I still don't understand what that means in practice.

> > could be called whenever the display engine needs to
> > recalculate the contents and the dimensions of the menu, and that
> > could basically be every redisplay cycle, depending on circumstances.
> 
> See above.  There's no menu involved in the case
> being discussed (at least none that's visible to
> users, AFAIK).  So there should be no need or
> possibility of recalculating menu contents and
> dimensions.

Then you already know the answer and don't need any help, it seems.
But if you do, please explain more about the specific use case you
worry about.

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Drew Adams]

> >  "use of a `menu-item' construct with a :filter
> >   to create a conditional _keyboard_ key binding.
> >   In such a case, the `menu-item' construct is not
> >   a real menu item - it is not placed on any menu."
> 
> That doesn't explain anything to me, sorry.

Sorry.  Does an example help?

(global-set-key
 "\C-o" 
 '`(menu-item
   "" forward-char
   :filter (lambda (cmd)
             (if (integerp prefix-arg) nil cmd))))

Stefan (cc'd), in particular, mentions this case
from time to time.  Here are some examples from him:

In emacs-devel thread "Proposed patch for lookup-key"
(2017-12-14):

  "bindings which are made dynamically conditional
   using a :filter function"

In help-gnu-emacs thread "How to define minor-mode
keybindings conditional on buffer's major-mode?"
(2013-10-04):

 (define-key minor-mode-map
   "<<key-binding1>>"
   `(menu-item
     "" minor-mode-command
     :filter ,(lambda (cmd)
                (if (eq major-mode 'xyz-mode) cmd))))
 (define-key minor-mode-map
   "<<key-binding2>>"
   `(menu-item
     "" minor-mode-command
     :filter ,(lambda (cmd)
                (if (not (eq major-mode 'xyz-mode)) cmd))))

In help-gnu-emacs thread "Commands with more than
one keybinding in menus" (2013-08-14):

 You can use dynamic key-bindings, i.e. instead of
 binding your key to
 (lambda () (interactive) (if foo (CMD))),
 bind it to
 (menu-item "" CMD :filter (lambda (cmd) (if foo cmd))).

In help-gnu-emacs thread "Shor-circuiting keybindings"
(2010-12-09):

 You can create conditional key bindings:
  (define-key map
    [?\C-b]
    '(menu-item "dummy" <command>
      :filter (lambda (binding)
                (if <condition> binding))))
                                      
 The filter function will receive <command> as
 argument (this is so that the same filter function
 can be used for several bindings) and can opt to
 return it or not or to return anything else (so
 you can create dynamic bindings, which is
 typically used to construct dynamic menus, such as
 the Buffers menu).
 The <command> you put as the "static binding" that
 gets passed to the filter is only used in cases
 such as where-is.

In help-gnu-emacs thread "HELP: One Bindkey for Two
Different Commands" (2007-11-30):

 How 'bout some neat hack like:
  (global-set-key "\M-o"
    '(menu-item "Foo" bar
      :filter (lambda (x)
                (if (cdr (window-list))
                    'other-window
                  'switch-to-buffer))))

> > Define "menu function", please.

?  (I don't think you answered that.  If you still
want to make a point about it, maybe let me know
what you mean by it.)

> > the case being discussed: an extended menu
> > item that's bound only to a keyboard key, i.e.,
> > that's not used in any menu?
> 
> I still don't understand what that means in practice.

Do the examples above help?  It's about using the
:filter mechanism designed for menu items with a
keyboard key instead, to create a key binding
that's "dynamic", i.e., conditional at the time
you use the key.

> > > could be called whenever the display engine needs to
> > > recalculate the contents and the dimensions of the menu, and that
> > > could basically be every redisplay cycle, depending on circumstances.
> >
> > See above.  There's no menu involved in the case
> > being discussed (at least none that's visible to
> > users, AFAIK).  So there should be no need or
> > possibility of recalculating menu contents and
> > dimensions.
> 
> Then you already know the answer and don't need any help, it seems.
> But if you do, please explain more about the specific use case you
> worry about.

Please reread what I said.  I'm not _aware_ of any
actual menu being involved in the keyboard-key case.
IF none is used, then I'm GUESSING that recalculating
menu contents (what menu?) doesn't happen.

But my point is only about having the doc address
this specific use case - use of :filter when only a
keyboard binding is used.  IF that in fact doesn't
involve any menus then shouldn't that paragraph be
clarified to exclude that case in its caveat?

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Eli Zaretskii]

> Date: Sun, 13 Dec 2020 11:52:53 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: larsi@gnus.org, 24237@debbugs.gnu.org,
>         Stefan Monnier
>  <monnier@iro.umontreal.ca>
> 
> > >  "use of a `menu-item' construct with a :filter
> > >   to create a conditional _keyboard_ key binding.
> > >   In such a case, the `menu-item' construct is not
> > >   a real menu item - it is not placed on any menu."
> > 
> > That doesn't explain anything to me, sorry.
> 
> Sorry.  Does an example help?
> 
> (global-set-key
>  "\C-o" 
>  '`(menu-item
>    "" forward-char
>    :filter (lambda (cmd)
>              (if (integerp prefix-arg) nil cmd))))

If this is never placed on the menu bar or on any pop-down menu, then
what the manual says is not relevant, because redisplay only considers
the menus that need to be displayed.

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Drew Adams]

> > > >  "use of a `menu-item' construct with a :filter
> > > >   to create a conditional _keyboard_ key binding.
> > > >   In such a case, the `menu-item' construct is not
> > > >   a real menu item - it is not placed on any menu."
> 
> If this is never placed on the menu bar or on any pop-down menu, then
> what the manual says is not relevant, because redisplay only considers
> the menus that need to be displayed.

1. The manual's description of :filter is entirely
relevant to this use case - _except_ for that last
paragraph, which apparently erroneously assumes that
the only possible use of an extended menu item with
:filter is in a menu.

Extended menu items are not just for menus (in spite
of the name).

2. What you've added here is what needs to be added
there: a qualification to that caveat, that _when
used in a menu_, "Emacs can call ... redisplay ...
You should write it so it can safely be called at
any time".

3. And because this use case is important and not
at all obvious, the doc there should explicitly
point out this use case.

   a. You can bind an extended menu item that uses
      :filter to a keyboard key, to give it a
      dynamic binding (explaining what that means
      with an example).

   b. If the only bindings for that extended menu
      item are keyboard bindings then the admonition
      about redisplay (see #2) does not apply.
      Don't assume that readers will understand
      that if not placed on a menu then it doesn't
      apply.  The admonition is currently too
      blanket.

Wrt #3: An example for this use case should be
presented somewhere in the Elisp manual.  If you
can think of a better place than in the description
of :filter, fine.  If not, then that's the place.

4. As I said in the bug report, having to use an
extended menu item for this feature is a hack or
workaround.  There's no other possibility out of
the box.

Better than just documenting that you can use an
extended menu item with :filter to get this behavior
would be to provide the behavior in another way.
IOW, ideally it wouldn't be "hidden" under the
misleading notion of a menu.  :filter itself is not
about menus.

More generally, `menu-item' is not just about menus
and menu items.  This is not obvious.  And in
particular, it's not obvious that you can make good
use of :filter with a keyboard key.

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Eli Zaretskii]

> Date: Sat, 19 Dec 2020 10:54:06 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: larsi@gnus.org, 24237@debbugs.gnu.org, monnier@iro.umontreal.ca
> 
> > > > >  "use of a `menu-item' construct with a :filter
> > > > >   to create a conditional _keyboard_ key binding.
> > > > >   In such a case, the `menu-item' construct is not
> > > > >   a real menu item - it is not placed on any menu."
> > 
> > If this is never placed on the menu bar or on any pop-down menu, then
> > what the manual says is not relevant, because redisplay only considers
> > the menus that need to be displayed.
> 
> 1. The manual's description of :filter is entirely
> relevant to this use case - _except_ for that last
> paragraph, which apparently erroneously assumes that
> the only possible use of an extended menu item with
> :filter is in a menu.

Since that last paragraph explicitly talks about displaying or acting
on menus, I see no reason to change anything there.

>    a. You can bind an extended menu item that uses
>       :filter to a keyboard key, to give it a
>       dynamic binding (explaining what that means
>       with an example).

IMO this is a kludge that we don't need to document.

> More generally, `menu-item' is not just about menus
> and menu items.  This is not obvious.  And in
> particular, it's not obvious that you can make good
> use of :filter with a keyboard key.

menu-item is about menus, as its name says.  That it can be used for
other purposes is an implementation detail.

And with that, I'm closing this bug report.

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Drew Adams]

> > > > > >  "use of a `menu-item' construct with a :filter
> > > > > >   to create a conditional _keyboard_ key binding.
> > > > > >   In such a case, the `menu-item' construct is not
> > > > > >   a real menu item - it is not placed on any menu."
> > >
> > > If this is never placed on the menu bar or on any pop-down menu, then
> > > what the manual says is not relevant, because redisplay only considers
> > > the menus that need to be displayed.
> >
> > 1. The manual's description of :filter is entirely
> > relevant to this use case - _except_ for that last
> > paragraph, which apparently erroneously assumes that
> > the only possible use of an extended menu item with
> > :filter is in a menu.
> 
> Since that last paragraph explicitly talks about displaying or acting
> on menus, I see no reason to change anything there.
> 
> >    a. You can bind an extended menu item that uses
> >       :filter to a keyboard key, to give it a
> >       dynamic binding (explaining what that means
> >       with an example).
> 
> IMO this is a kludge that we don't need to document.

It is indeed a kludge/workaround/hack.

It is also quite useful.  If you refuse to elevate it
to doc-worthiness, then please change this bug to an
enhancement request to provide the same feature
_without_ having to use `menu-item' with :filter.

Otherwise, I will I guess submit such an enhancement
request.  Elisp users shouldn't have to resort to an
undocumented kludge to get this useful behavior.

> > More generally, `menu-item' is not just about menus
> > and menu items.  This is not obvious.  And in
> > particular, it's not obvious that you can make good
> > use of :filter with a keyboard key.
> 
> menu-item is about menus, as its name says.  That it can be used for
> other purposes is an implementation detail.
> 
> And with that, I'm closing this bug report.

Ta da.

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Stefan Kangas]

Drew Adams <drew.adams@oracle.com> writes:

>> >    a. You can bind an extended menu item that uses
>> >       :filter to a keyboard key, to give it a
>> >       dynamic binding (explaining what that means
>> >       with an example).
>>
>> IMO this is a kludge that we don't need to document.
>
> It is indeed a kludge/workaround/hack.

I agree that this kludge shouldn't be documented.  It is very strange to
bind keys to menu-bindings just for :filter, and if we document it we in
effect say that this is the preferred way of doing this stuff.

> It is also quite useful.  If you refuse to elevate it
> to doc-worthiness, then please change this bug to an
> enhancement request to provide the same feature
> _without_ having to use `menu-item' with :filter.
>
> Otherwise, I will I guess submit such an enhancement
> request.  Elisp users shouldn't have to resort to an
> undocumented kludge to get this useful behavior.

It makes sense to me to file such a feature request.

bug#24237: 24.5; (elisp)`Extended Menu Items', :filter warning

[Drew Adams]

> I agree that this kludge shouldn't be documented.  It is very strange to
> bind keys to menu-bindings just for :filter, 

It's not strange to do that for menu items.
What's "strange", aka unexpected/new, is to
do it for keyboard keys.

> and if we document it we in effect say that this is
> the preferred way of doing this stuff.

Preferred way?  No.  It's the _only_ way (AFAIK).

> > It is also quite useful.  If you refuse to elevate it
> > to doc-worthiness, then please change this bug to an
> > enhancement request to provide the same feature
> > _without_ having to use `menu-item' with :filter.
> >
> > Otherwise, I will I guess submit such an enhancement
> > request.  Elisp users shouldn't have to resort to an
> > undocumented kludge to get this useful behavior.
> 
> It makes sense to me to file such a feature request.

Done: bug #45329.