bug#45177: 27.1; Access to invoking top level command in minibuffer

bug#45177: 27.1; Access to invoking top level command in minibuffer

[clemera]

Hello,

For command based settings it would be nice to be able to have
access to the top level command from which the current minibuffer
session was invoked from. This should also work with multiple minibuffer
invokations during a command. Using `minibuffer-setup-hook' to save
`real-this-command' does not work, for example with:

```elisp
(defun example-command ()
   (interactive)
   (read-string "Example: ")
   (message "%s" real-this-command))
```

`real-this-command' will be `exit-minibuffer' after the `read-string' so 
any minibuffer invokation within that command afterwards will no longer 
know about `example-command'.

The described issue is problem for completion frameworks. The popular 
Ivy(https://github.com/abo-abo/swiper/) package from GNU ELPA does use 
the `:caller` argument passed to `ivy-read` to circumvent this. With 
Selectrum(https://github.com/raxod502/selectrum/) we are trying to find 
a built-in way to handle this.


     Clemens



In GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30, 
cairo version 1.15.10)
  of 2020-08-20 built on clemera
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.5 LTS

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
  'configure --with-modules --with-json --with-cairo'

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS JSON PDUMPER LCMS2
GMP

Important settings:
   value of $LC_MONETARY: de_DE.UTF-8
   value of $LC_NUMERIC: de_DE.UTF-8
   value of $LC_TIME: de_DE.UTF-8
   value of $LANG: en_US.UTF-8
   value of $XMODIFIERS: @im=ibus
   locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
   tooltip-mode: t
   global-eldoc-mode: t
   eldoc-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   line-number-mode: t
   transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 44091 12180)
  (symbols 48 6003 1)
  (strings 32 15452 1784)
  (string-bytes 1 506298)
  (vectors 16 9257)
  (vector-slots 8 124488 12266)
  (floats 8 19 43)
  (intervals 56 183 0)
  (buffers 1000 11))

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Drew Adams]

> For command based settings it would be nice to be able to have
> access to the top level command from which the current minibuffer
> session was invoked from. This should also work with multiple minibuffer
> invokations during a command. Using `minibuffer-setup-hook' to save
> `real-this-command' does not work, for example with:

FWIW, `icicle-mode' puts this on `pre-command-hook':

(defun icicle-top-level-prep ()
  "Do top-level stuff.  Used in `pre-command-hook'."
  (unless (> (minibuffer-depth) 0)
    ;; ... <other stuff>
    (unless (memq this-command 
                  '(minibuffer-complete-and-exit
                    icicle-minibuffer-complete-and-exit
                    exit-minibuffer
                    icicle-exit-minibuffer))
      (setq icicle-last-top-level-command  this-command))
    ;; ... <other stuff>
    ))

(defvar icicle-last-top-level-command nil
  "Last top-level command used.")

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Lars Ingebrigtsen]

clemera@posteo.net writes:

> For command based settings it would be nice to be able to have
> access to the top level command from which the current minibuffer
> session was invoked from. This should also work with multiple minibuffer
> invokations during a command. Using `minibuffer-setup-hook' to save
> `real-this-command' does not work, for example with:
>
> ```elisp
> (defun example-command ()
>   (interactive)
>   (read-string "Example: ")
>   (message "%s" real-this-command))
> ```
>
> `real-this-command' will be `exit-minibuffer' after the `read-string'
> so any minibuffer invokation within that command afterwards will no
> longer know about `example-command'.

Hm...  I'm not quite sure "the top level command" is a well-defined
concept?  You can enter a number of nested recursive edits, and I think
what you probably want is the innermost command that invoked a recursive
edit?

So perhaps it would make sense for Frecursive_edit (or some other handy
function when entering the minibuffer) to let-bind a new variable (say,
`this-recursive-command'?) to the value of `real-this-command'?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Clemens]

> (defun icicle-top-level-prep ()
>    "Do top-level stuff.  Used in `pre-command-hook'."
>    (unless (> (minibuffer-depth) 0)
>      ;; ... <other stuff>
>      (unless (memq this-command
>                    '(minibuffer-complete-and-exit
>                      icicle-minibuffer-complete-and-exit
>                      exit-minibuffer
>                      icicle-exit-minibuffer))
>        (setq icicle-last-top-level-command  this-command))
>      ;; ... <other stuff>
>      ))
> 
> (defvar icicle-last-top-level-command nil
>    "Last top-level command used.")
> 


Thanks Drew! Looks like a nice workaround!

bug#45177: 27.1; Access to invoking top level command in minibuffer

[clemera]

> Hm...  I'm not quite sure "the top level command" is a well-defined
> concept?  You can enter a number of nested recursive edits, and I think
> what you probably want is the innermost command that invoked a recursive
> edit?

Yes, that would be ideal.

> So perhaps it would make sense for Frecursive_edit (or some other handy
> function when entering the minibuffer) to let-bind a new variable (say,
> `this-recursive-command'?) to the value of `real-this-command'?

I don't know anything about the C side of Emacs so I can't be of any 
help here.

bug#45177: 27.1; Access to invoking top level command in minibuffer

[clemera]

> (defun icicle-top-level-prep ()
>    "Do top-level stuff.  Used in `pre-command-hook'."
>    (unless (> (minibuffer-depth) 0)
>      ;; ... <other stuff>
>      (unless (memq this-command
>                    '(minibuffer-complete-and-exit
>                      icicle-minibuffer-complete-and-exit
>                      exit-minibuffer
>                      icicle-exit-minibuffer))
>        (setq icicle-last-top-level-command  this-command))
>      ;; ... <other stuff>
>      ))
> 
> (defvar icicle-last-top-level-command nil
>    "Last top-level command used.")

Note that ideally I'm looking for something which allows me to
get the "innermost command that invoked a recursive
edit" as Lars described it.

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Juri Linkov]

>> For command based settings it would be nice to be able to have
>> access to the top level command from which the current minibuffer
>> session was invoked from. This should also work with multiple minibuffer
>> invokations during a command. Using `minibuffer-setup-hook' to save
>> `real-this-command' does not work, for example with:
>>
>> ```elisp
>> (defun example-command ()
>>   (interactive)
>>   (read-string "Example: ")
>>   (message "%s" real-this-command))
>> ```
>>
>> `real-this-command' will be `exit-minibuffer' after the `read-string'
>> so any minibuffer invokation within that command afterwards will no
>> longer know about `example-command'.
>
> Hm...  I'm not quite sure "the top level command" is a well-defined
> concept?  You can enter a number of nested recursive edits, and I think
> what you probably want is the innermost command that invoked a recursive
> edit?
>
> So perhaps it would make sense for Frecursive_edit (or some other handy
> function when entering the minibuffer) to let-bind a new variable (say,
> `this-recursive-command'?) to the value of `real-this-command'?

Or set a (mini)buffer-local variable in minibuffer-setup-hook.

bug#45177: 27.1; Access to invoking top level command in minibuffer

[clemera]

> Or set a (mini)buffer-local variable in minibuffer-setup-hook.

But wouldn't this still have the problem that multiple minibuffer 
invocations in the same command would change real-this-command as in my 
initial example?

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Juri Linkov]

>> Or set a (mini)buffer-local variable in minibuffer-setup-hook.
>
> But wouldn't this still have the problem that multiple minibuffer
> invocations in the same command would change real-this-command as in my
> initial example?

This is not a problem because multiple minibuffer invocations
create separate buffers with different names

  #<buffer  *Minibuf-1*>
  #<buffer  *Minibuf-2*>
  #<buffer  *Minibuf-3*>
 ...

bug#45177: 27.1; Access to invoking top level command in minibuffer

[clemera]

>> But wouldn't this still have the problem that multiple minibuffer
>> invocations in the same command would change real-this-command as in my
>> initial example?
> 
> This is not a problem because multiple minibuffer invocations
> create separate buffers with different names
> 
>    #<buffer  *Minibuf-1*>
>    #<buffer  *Minibuf-2*>
>    #<buffer  *Minibuf-3*>
>   ...

I meant sequential invocation the local variables are gone when you exit 
a session (a nice feature that we also rely on in Selectrum):

```elisp
(defvar saved-command nil)

(defun save-command ()
   (message "Before: %s" saved-command)
   (setq-local saved-command real-this-command)
   (message "After: %s" saved-command))

(defun example-command ()
   (interactive)
   (read-string "Example1 : ")
   (read-string "Example2 : "))
```

Calling example-command above gives :

     Before: nil
     After: example-command

     Before: nil
     After: exit-minibuffer

bug#45177: 27.1; Access to invoking top level command in minibuffer

[clemera]

I forgot to include the setup hook in my code example, here is the full 
version:

```elisp

(defvar saved-command nil)

(defun save-command ()
   (message "Before: %s" saved-command)
   (setq-local saved-command real-this-command)
   (message "After: %s" saved-command))

(defun example-command ()
   (interactive)
   (read-string "Example1 : ")
   (read-string "Example2 : "))

(add-hook 'minibuffer-setup-hook #'save-command)
```

Calling example-command above gives :

     Before: nil
     After: example-command

     Before: nil
     After: exit-minibuffer

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Lars Ingebrigtsen]

clemera@posteo.net writes:

> I don't know anything about the C side of Emacs so I can't be of any
> help here.

I'm not quite sure where to bind the variable...  Does the following
work for you?

diff --git a/src/callint.c b/src/callint.c
index f80436f3d9..a01338dfe1 100644
--- a/src/callint.c
+++ b/src/callint.c
@@ -283,6 +283,8 @@ DEFUN ("call-interactively", Fcall_interactively, Scall_interactively, 1, 3, 0,
   Lisp_Object save_real_this_command = Vreal_this_command;
   Lisp_Object save_last_command = KVAR (current_kboard, Vlast_command);
 
+  specbind (Qrecursive_this_command, Vreal_this_command);
+
   if (NILP (keys))
     keys = this_command_keys, key_count = this_command_key_count;
   else
diff --git a/src/keyboard.c b/src/keyboard.c
index dbca5be91e..ce2b7f1ef4 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -11830,6 +11830,11 @@ syms_of_keyboard (void)
 	       doc: /* This is like `this-command', except that commands should never modify it.  */);
   Vreal_this_command = Qnil;
 
+  DEFSYM (Qrecursive_this_command, "recursive-this-command");
+  DEFVAR_LISP ("recursive-this-command", Vrecursive_this_command,
+	       doc: /* This is like `real-this-command', but bound recursively in `call-interactively.  */);
+  Vrecursive_this_command = Qnil;
+
   DEFVAR_LISP ("this-command-keys-shift-translated",
 	       Vthis_command_keys_shift_translated,
 	       doc: /* Non-nil if the key sequence activating this command was shift-translated.


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#45177: 27.1; Access to invoking top level command in minibuffer

[clemera]

> I'm not quite sure where to bind the variable...  Does the following
> work for you?
> 
> diff --git a/src/callint.c b/src/callint.c
> index f80436f3d9..a01338dfe1 100644
> --- a/src/callint.c
> +++ b/src/callint.c
> @@ -283,6 +283,8 @@ DEFUN ("call-interactively", Fcall_interactively, Scall_interactively, 1, 3, 0,
>     Lisp_Object save_real_this_command = Vreal_this_command;
>     Lisp_Object save_last_command = KVAR (current_kboard, Vlast_command);
>   
> +  specbind (Qrecursive_this_command, Vreal_this_command);
> +
>     if (NILP (keys))
>       keys = this_command_keys, key_count = this_command_key_count;
>     else
> diff --git a/src/keyboard.c b/src/keyboard.c
> index dbca5be91e..ce2b7f1ef4 100644
> --- a/src/keyboard.c
> +++ b/src/keyboard.c
> @@ -11830,6 +11830,11 @@ syms_of_keyboard (void)
>   	       doc: /* This is like `this-command', except that commands should never modify it.  */);
>     Vreal_this_command = Qnil;
>   
> +  DEFSYM (Qrecursive_this_command, "recursive-this-command");
> +  DEFVAR_LISP ("recursive-this-command", Vrecursive_this_command,
> +	       doc: /* This is like `real-this-command', but bound recursively in `call-interactively.  */);
> +  Vrecursive_this_command = Qnil;
> +
>     DEFVAR_LISP ("this-command-keys-shift-translated",
>   	       Vthis_command_keys_shift_translated,
>   	       doc: /* Non-nil if the key sequence activating this command was shift-translated.
> 

Thank you! I recompiled Emacs with this and tested with various nested 
sequential and recursive calls and this correctly reports the top level 
command the current minibuffer session was entered from, which is 
exactly what we are looking for :)

bug#45177: 27.1; Access to invoking top level command in minibuffer

[clemera]

 > the top level command the current minibuffer session was entered from

I rather meant the command that was current when entering the current 
minibuffer session.

bug#45177: 27.1; Access to invoking top level command in minibuffer

[clemera]

If you are going to include something like this would it make sense
to have an `real-recursive-this-command` and `recursive-this-command`?

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Lars Ingebrigtsen]

clemera@posteo.net writes:

> Thank you! I recompiled Emacs with this and tested with various nested
> sequential and recursive calls and this correctly reports the top
> level command the current minibuffer session was entered from, which
> is exactly what we are looking for :)

Thanks for testing -- I've now pushed this to Emacs 28, but I renamed
the variable to `current-minibuffer-command'.  (And if somebody wants to
rename that, please go ahead.)

With that, I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Clemens]

> Thanks for testing -- I've now pushed this to Emacs 28, but I renamed
> the variable to `current-minibuffer-command'.  (And if somebody wants to
> rename that, please go ahead.)


Thank you very much! I also think it was a good decision to make it 
point to `this-command` that makes more sense after all.

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Clemens]

Oh I just noticed it still points to real-this-command right? I was
mislead by the docs ;) My thought why `this-command` might have been
a good idea was that people might want to call commands from Elisp like this

(let ((this-command 'my-cmd))
     (call-interactively 'my-cmd))

and then expect the configuration of the minibuffer applied by Selectrum 
to apply. To make this work with real-this-command they would need to 
bind that but it is discouraged I think? This is mostly a consideration 
about what might happen in user code later.

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Lars Ingebrigtsen]

Clemens <clemera@posteo.net> writes:

> Oh I just noticed it still points to real-this-command right? I was
> mislead by the docs ;) My thought why `this-command` might have been
> a good idea was that people might want to call commands from Elisp like this
>
> (let ((this-command 'my-cmd))
>     (call-interactively 'my-cmd))
>
> and then expect the configuration of the minibuffer applied by
> Selectrum to apply. To make this work with real-this-command they
> would need to bind that but it is discouraged I think? This is mostly
> a consideration about what might happen in user code later.

Good catch; I meant to change it to this-command, but I forgot to change
it in the actual binding.  Fixed now.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Clemens]

 > Good catch; I meant to change it to this-command, but I forgot to change
 > it in the actual binding.  Fixed now.

Thank you!

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Clemens]

After some experiments, this isn't sufficient for our purposes. What we 
would really need is identifying the call which invokes the minibuffer 
session. Having access to the surrounding function call would help, in 
combination with the prompt text and in some cases completion table data 
this should work. I guess we need to use advice or would it be possible 
to expose the surrounding function call of the current minibuffer session?

I think having the new variable is still an improvement as one just 
hadn't access to this sate before and it can be used for other purposes 
as minibuffer session configuration, too.

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[ Resending because of that damn archiving misfeature.  ]

clemera@posteo.net [2020-12-11 15:20:34] wrote:
> For command based settings it would be nice to be able to have
> access to the top level command from which the current minibuffer
> session was invoked from. This should also work with multiple minibuffer
> invokations during a command. Using `minibuffer-setup-hook' to save
> `real-this-command' does not work, for example with:
>
> ```elisp
> (defun example-command ()
>   (interactive)
>   (read-string "Example: ")
>   (message "%s" real-this-command))
> ```
>
> `real-this-command' will be `exit-minibuffer' after the `read-string' so any
> minibuffer invokation within that command afterwards will no longer know
> about `example-command'.

If you invoke `example-command` from an alias, you won't get "the right"
answer either anyway.

> The described issue is problem for completion frameworks. The popular
> Ivy(https://github.com/abo-abo/swiper/) package from GNU ELPA does use the
> `:caller` argument passed to `ivy-read` to circumvent this. With
> Selectrum(https://github.com/raxod502/selectrum/) we are trying to find
> a built-in way to handle this.

I think this is an XY problem.

Changing the behavior based on the caller's name is a very bad idea.
It's a bit like `called-interactively-p`, except it's worse because
you're looking for even more ill-defined data than just a boolean.

The standard UI has introduced the `category` metadata for that kind
of problems.  I don't claim it's a perfect solution, but whichever
"right" solution we come up with it should not be based on the name of
the caller but instead it should be based on data provided by
the caller.


        Stefan

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[ Resending because of that damn archiving misfeature.  ]

> Thanks for testing -- I've now pushed this to Emacs 28, but I renamed
> the variable to `current-minibuffer-command'.  (And if somebody wants to
> rename that, please go ahead.)
> With that, I'm closing this bug report.

Using such let-binding is exactly what we're trying to move away from;
we should bind it buffer-locally in the minibuffer instead (as we've
seen with `minibuffer-local-map`).


        Stefan

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Using such let-binding is exactly what we're trying to move away from;
> we should bind it buffer-locally in the minibuffer instead (as we've
> seen with `minibuffer-local-map`).

Sure -- I guess the results will be pretty much the same when using a
recursive minibuffer?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#45177: 27.1; Access to invoking top level command in minibuffer

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Changing the behavior based on the caller's name is a very bad idea.
> It's a bit like `called-interactively-p`, except it's worse because
> you're looking for even more ill-defined data than just a boolean.

It's not ideal to look at the name, of course, but I think the user
should have a mechanism to do so anyway -- it's an easy and practical
way for users to tweak stuff.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no