bug#51143: 29.0.50; Long delay after M-x commandname

bug#51143: 29.0.50; Long delay after M-x commandname

[Eduardo Ochs]

I noticed that when I was invoking commands with M-x commandname and
using tab for completing parts of the name then sometimes there would
be a long delay - sometimes as long as 10 seconds, but my laptop is
quite old - between the RET and the execution of the command. This
delay was big on emacs28 and practically imperceptible on emacs27, and
it seemed that it was getting worse in more recent versions of
emacs28/29.

I discussed this on help-gnu-emacs before submitting a bug report. The
main links are these ones:

https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00232.html Edrx
https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00259.html MH
https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00261.html Edrx 2
https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00265.html MH 2

Let me summarize. The part of "M-x commandname" that takes a long time
is the call to `execute-extended-command--shorter' - like this:

  (execute-extended-command--shorter "find-eev-quick-intro" "find-eev-qui")

Michael Heerdegen was able to reproduce the problem and profile it.
See this message:

https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00265.html MH 2

Cheers,
  Eduardo Ochs
  http://angg.twu.net/#eev



In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
3.24.24, cairo version 1.16.0)
 of 2021-10-09 built on scylla
Repository revision: cc202c713be60355dbcf4e5ea87603c5e16a92db
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Debian GNU/Linux 11 (bullseye)

Configured using:
 'configure --with-modules'

(etc etc)

bug#51143: 29.0.50; Long delay after M-x commandname

[Lars Ingebrigtsen]

Eduardo Ochs <eduardoochs@gmail.com> writes:

> Let me summarize. The part of "M-x commandname" that takes a long time
> is the call to `execute-extended-command--shorter' - like this:
>
>   (execute-extended-command--shorter "find-eev-quick-intro" "find-eev-qui")

But it's run like this:

(while-no-input (execute-extended-command--shorter "find-eev-quick-intro" "find-eev-qui"))

So typing any key should abort it.

I'm not saying we shouldn't make the function shorter, but what's the
use case where this matters to the user?  You mention it affecting
`M-x TAB', but as far as I can see, that function is never called when
doing that?

I tried

M-x debug-on-entry RET execute-extended-command--shorter RET
M-x TAB

and it wasn't called.  Do you have a complete recipe, starting from
"emacs -Q", that demonstrates the problem?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#51143: 29.0.50; Long delay after M-x commandname

[Eli Zaretskii]

> From: Eduardo Ochs <eduardoochs@gmail.com>
> Date: Mon, 11 Oct 2021 23:44:57 -0300
> 
> I noticed that when I was invoking commands with M-x commandname and
> using tab for completing parts of the name then sometimes there would
> be a long delay - sometimes as long as 10 seconds, but my laptop is
> quite old - between the RET and the execution of the command. This
> delay was big on emacs28 and practically imperceptible on emacs27, and
> it seemed that it was getting worse in more recent versions of
> emacs28/29.
> 
> I discussed this on help-gnu-emacs before submitting a bug report. The
> main links are these ones:
> 
> https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00232.html Edrx
> https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00259.html MH
> https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00261.html Edrx 2
> https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00265.html MH 2
> 
> Let me summarize. The part of "M-x commandname" that takes a long time
> is the call to `execute-extended-command--shorter' - like this:
> 
>   (execute-extended-command--shorter "find-eev-quick-intro" "find-eev-qui")
> 
> Michael Heerdegen was able to reproduce the problem and profile it.
> See this message:
> 
> https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00265.html MH 2

Stefan, any advice?

bug#51143: 29.0.50; Long delay after M-x commandname

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00265.html MH 2
> Stefan, any advice?

I already gave some advice in that gnu.emacs.help thread, I don't think
I have much more to add.  Michael's ideas for optimizations are
definitely worth exploring (my intuition suggests the current approach
(shorter-to-longer) is better than the other way around, but I never
bothered to confirm it).


        Stefan

bug#51143: 29.0.50; Long delay after M-x commandname

[Eduardo Ochs]

On Tue, 12 Oct 2021 at 09:56, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>
> But it's run like this:
>
> (while-no-input (execute-extended-command--shorter "find-eev-quick-intro" "find-eev-qui"))
>
> So typing any key should abort it.
>
> I'm not saying we shouldn't make the function shorter, but what's the
> use case where this matters to the user?  You mention it affecting
> `M-x TAB', but as far as I can see, that function is never called when
> doing that?
>
> I tried
>
> M-x debug-on-entry RET execute-extended-command--shorter RET
> M-x TAB
>
> and it wasn't called.  Do you have a complete recipe, starting from
> "emacs -Q", that demonstrates the problem?


Hi Lars,

I commit many mistakes when I type, so I prefer recipes that can be
executed by just typing `C-x C-e' at the right places.

The delay is much longer on emacses with more packages loaded and more
functions defined.

If I try to run `M-x find-eev-quick-intro' by typing

  M-x f i n d - e e v - q TAB RET

in an Emacs28 without -Q then the code in `execute-extended-command'
takes 10 seconds to display the message

  You can run the command `find-eev-quick-intro' with `M-x -q-i'

and _ONLY THEN_ it shows the temporary buffer that
find-eev-quick-intro produces. You're right about this being
interruptible by typing a key - I didn't know that trick - but I'm
used to only choose what to type in those buffers after seeing their
contents, so I'll have retrain myself to type, say, <down>, on such
delays, and for me it's easier to just execute

  (setq extended-command-suggest-shorter nil)

when I start to get bothered by such delays than to retrain myself, so
here's one user - me - and one use case - the one above - in which
this matters... =S

There is a recipe here:

  https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00261.html

its main line (plus something that I will explain soon) is:

  (execute-extended-command--shorter "find-eev-quick-intro" "find-eev-qui")
  (execute-extended-command--shorter "find-eev-quick-intro" "123456789012")

`execute-extended-command--shorter' only cares about the length of its
second argument, so using "find-eev-qui" in the second argument is
equivalent to using "123456789012".

The discussion about that bug is happening part here and part in
help-gnu-emacs. See:

  https://lists.gnu.org/archive/html/help-gnu-emacs/2021-10/msg00280.html

Cheers,
  E.

bug#51143: 29.0.50; Long delay after M-x commandname

[Lars Ingebrigtsen]

Eduardo Ochs <eduardoochs@gmail.com> writes:

> If I try to run `M-x find-eev-quick-intro' by typing
>
>   M-x f i n d - e e v - q TAB RET
>
> in an Emacs28 without -Q then the code in `execute-extended-command'
> takes 10 seconds to display the message
>
>   You can run the command `find-eev-quick-intro' with `M-x -q-i'

So TAB doesn't take ten seconds?

> and _ONLY THEN_ it shows the temporary buffer that
> find-eev-quick-intro produces.

That's really weird, because `execute-extended-command--shorter' is run
from a timer after the command has finished executed -- so it shouldn't
get in the way of executing the command, no matter how long it takes.

Try to

M-x debug-on-entry RET execute-extended-command--shorter RET
M-x find-eev-quick-intro RET

and post the resulting backtrace.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#51143: 29.0.50; Long delay after M-x commandname

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Try to
>
> M-x debug-on-entry RET execute-extended-command--shorter RET
> M-x find-eev-quick-intro RET
>
> and post the resulting backtrace.

More information was requested, but no response was given within a
month, so I'm closing this bug report.  If the problem still exists,
please respond to this email and we'll reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#51143: 29.0.50; Long delay after M-x commandname

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> More information was requested, but no response was given within a
> month, so I'm closing this bug report.  If the problem still exists,
> please respond to this email and we'll reopen the bug report.

I could reproduce the issue and I had some ideas how to improve the
situation, please reopen.

Michael.

bug#51143: 29.0.50; Long delay after M-x commandname

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> I could reproduce the issue and I had some ideas how to improve the
> situation, please reopen.

OK; reopened.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#51143: 29.0.50; Long delay after M-x commandname

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

>> More information was requested, but no response was given within a
>> month, so I'm closing this bug report.  If the problem still exists,
>> please respond to this email and we'll reopen the bug report.
>
> I could reproduce the issue and I had some ideas how to improve the
> situation, please reopen.

This was almost a year ago -- do you have a recipe to reproduce the
problem?

bug#51143: 29.0.50; Long delay after M-x commandname

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> > I could reproduce the issue and I had some ideas how to improve the
> > situation, please reopen.
>
> This was almost a year ago -- do you have a recipe to reproduce the
> problem?

AFAIR the originally posted recipe was complete, you only had to have
enough stuff defined so that `execute-extended-command--shorter' gets
slow.  It had been my goal to improve the speed of that function.

I remember what I had discussed with Stefan.  The uncontroversial part
was to factor out the `commandp' tests out of
`completion-try-completion'.

I'll have a look.

Michael.

bug#51143: 29.0.50; Long delay after M-x commandname

[Michael Heerdegen]

[-- Attachment #1: Type: text/plain, Size: 251 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> I remember what I had discussed with Stefan.  The uncontroversial part
> was to factor out the `commandp' tests out of
> `completion-try-completion'.
>
> I'll have a look.

This is what I have:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Try-to-speed-up-extended-command-shorthand-computati.patch --]
[-- Type: text/x-diff, Size: 1763 bytes --]

From 6bc974197ff3e191971f933cc7f6fd952cdbba86 Mon Sep 17 00:00:00 2001
From: Michael Heerdegen <michael_heerdegen@web.de>
Date: Sun, 18 Sep 2022 12:26:53 +0200
Subject: [PATCH] Try to speed up extended command shorthand computation

* lisp/simple.el (execute-extended-command--shorter): Compute a
complete list of `commandp' symbols once.  This significantly speeds
up complicated cases while the slowdown of simple cases is still
accetable.
---
 lisp/simple.el | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/lisp/simple.el b/lisp/simple.el
index 10a610e0c6..6724f3d6fc 100644
--- a/lisp/simple.el
+++ b/lisp/simple.el
@@ -2465,9 +2465,13 @@ execute-extended-command--shorter-1

 (defun execute-extended-command--shorter (name typed)
   (let ((candidates '())
+        commands
         (max (length typed))
         (len 1)
         binding)
+    ;; Precompute a list of commands once to avoid repeated `commandp' testing
+    ;; of symbols in the `completion-try-completion' call inside the loop below
+    (mapatoms (lambda (s) (when (commandp s) (push s commands))))
     (while (and (not binding)
                 (progn
                   (unless candidates
@@ -2480,8 +2484,8 @@ execute-extended-command--shorter
       (input-pending-p)    ;Dummy call to trigger input-processing, bug#23002.
       (let ((candidate (pop candidates)))
         (when (equal name
-                       (car-safe (completion-try-completion
-                                  candidate obarray 'commandp len)))
+                     (car-safe (completion-try-completion
+                                candidate commands nil len)))
           (setq binding candidate))))
     binding))

--
2.30.2


[-- Attachment #3: Type: text/plain, Size: 374 bytes --]


This speeds up computations of extended command shortcuts by a factor of
around 4 in the complicated cases, while the slowdown for the easy cases
is hardly noticeable.  Maybe Eduardo can give that a try?

Further improvements are probably not that low-hanging fruits.  The
calculation is also always interruptible so I don't think more work is
worth the trouble.

Michael.

bug#51143: 29.0.50; Long delay after M-x commandname

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Subject: [PATCH] Try to speed up extended command shorthand computation

Makes sense to me.

bug#51143: 29.0.50; Long delay after M-x commandname

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> > Subject: [PATCH] Try to speed up extended command shorthand computation
>
> Makes sense to me.

Ok, thanks.  I installed that patch and close the bug again.

Michael.