bug#63754: 29.0.91; tmm breaks when tmm-completion-prompt is nil

bug#63754: 29.0.91; tmm breaks when tmm-completion-prompt is nil

[Thiago Melo]

Issue doesn't happen with Emacs 28.

* How to reproduce

run:
emacs -Q

eval:
(setq tmm-completion-prompt nil)
(tmm-menubar) ; or `tmm-prompt' with any menu/keymap

* Backtrace

Debugger entered--Lisp error: (search-failed "Possible completions are:\n")
  search-forward("Possible completions are:\n")
  tmm-completion-delete-prompt()
  run-hooks(completion-setup-hook)
  display-completion-list((#("f==>File" 0 1 (face
(completions-first-difference))) #("e==>Edit" 0 1 (face
(completions-first-difference))) #("o==>Options" 0 1 (face
(completions-first-difference))) #("b==>Buffers" 0 1 (face
(completions-first-difference))) #("t==>Tools" 0 1 (face
(completions-first-difference))) #("l==>Lisp-Interaction" 0 1 (face
(completions-first-difference))) #("h==>Help" 0 1 (face
(completions-first-difference)))) nil nil)
  #f(compiled-function (window) #<bytecode
0x1c5839fada789551>)(#<window 9 on *Completions*>)
  window--display-buffer(#<buffer *Completions*> #<window 9 on
*Completions*> window ((window-height .
completions--fit-window-to-buffer) nil (body-function .
#f(compiled-function (window) #<bytecode 0x1c5839fada789551>))))
  display-buffer-at-bottom(#<buffer *Completions*> ((window-height .
completions--fit-window-to-buffer) nil (body-function .
#f(compiled-function (window) #<bytecode 0x1c5839fada789551>))))
  display-buffer(#<buffer *Completions*>
((display-buffer--maybe-same-window display-buffer-reuse-window
display-buffer--maybe-pop-up-frame display-buffer-at-bottom)
(window-height . completions--fit-window-to-buffer) nil (body-function
. #f(compiled-function (window) #<bytecode 0x1c5839fada789551>))))
  temp-buffer-window-show(#<buffer *Completions*>
((display-buffer--maybe-same-window display-buffer-reuse-window
display-buffer--maybe-pop-up-frame display-buffer-at-bottom)
(window-height . completions--fit-window-to-buffer) nil (body-function
. #f(compiled-function (window) #<bytecode 0x1c5839fada789551>))))
  minibuffer-completion-help()
  tmm-add-prompt()
  minibuffer-setup()
  read-from-minibuffer("Menu bar (up/down to change, PgUp to menu): "
nil (keymap (10 . minibuffer-complete-and-exit) (13 .
minibuffer-complete-and-exit) keymap (menu-bar keymap (minibuf
"Minibuf" keymap (tab menu-item "Complete" minibuffer-complete :help
"Complete as far as possible") (space menu-item "Complete Word"
minibuffer-complete-word :help "Complete at most one word") (63
menu-item "List Completions" minibuffer-completion-help :help "Display
all possible completions") "Minibuf")) (M-down .
minibuffer-next-completion) (M-up . minibuffer-previous-completion)
(27 keymap (13 . minibuffer-choose-completion) (103 keymap (27 keymap
(99 . switch-to-completions))) (118 . switch-to-completions)) (prior .
switch-to-completions) (63 . minibuffer-completion-help) (32 .
minibuffer-complete-word) (backtab . minibuffer-complete) (9 .
minibuffer-complete) keymap (menu-bar keymap (minibuf "Minibuf" keymap
(previous menu-item "Previous History Item" previous-history-element
:help "Put previous minibuffer history element in the min...") (next
menu-item "Next History Item" next-history-element :help "Put next
minibuffer history element in the minibuf...") (isearch-backward
menu-item "Isearch History Backward" isearch-backward :help
"Incrementally search minibuffer history backward") (isearch-forward
menu-item "Isearch History Forward" isearch-forward :help
"Incrementally search minibuffer history forward") (return menu-item
"Enter" exit-minibuffer :key-sequence "\15" :help "Terminate input and
exit minibuffer") (quit menu-item "Quit" abort-recursive-edit :help
"Abort input and exit minibuffer") "Minibuf")) (24 keymap (down .
minibuffer-complete-defaults) (up . minibuffer-complete-history)) (13
. exit-minibuffer) (10 . exit-minibuffer) (7 . abort-minibuffers)
(C-tab . file-cache-minibuffer-complete) (9 . self-insert-command)
(XF86Back . previous-history-element) (up .
previous-line-or-history-element) (prior . previous-history-element)
(XF86Forward . next-history-element) (down .
next-line-or-history-element) (next . next-history-element) (27 keymap
(60 . minibuffer-beginning-of-buffer) (114 .
previous-matching-history-element) (115 .
next-matching-history-element) (112 . previous-history-element) (110 .
next-history-element))) nil tmm--history ("f==>File" "e==>Edit"
"o==>Options" "b==>Buffers" "t==>Tools" "l==>Lisp-Interaction"
"h==>Help") nil)
  completing-read-default("Menu bar (up/down to change, PgUp to menu):
" #f(compiled-function (string pred action) #<bytecode
-0x81ee482af888ac8>) nil t nil tmm--history ("f==>File" "e==>Edit"
"o==>Options" "b==>Buffers" "t==>Tools" "l==>Lisp-Interaction"
"h==>Help"))
  tmm-prompt((keymap ... ... ... ... ... ... ... ...) nil nil)
  tmm-menubar()
  eval((tmm-menubar) t)
  eval-expression((tmm-menubar) nil nil 127)
  funcall-interactively(eval-expression (tmm-menubar) nil nil 127)
  call-interactively(eval-expression nil nil)
  command-execute(eval-expression)


* System information:

In GNU Emacs 29.0.91 (build 1, x86_64-pc-linux-gnu, cairo version
 1.16.0) of 2023-05-26 built on debian-x250
Repository revision: 5e7c826bfa5cb7459f5b162b498af1c57c4578e6
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Debian GNU/Linux 11 (bullseye)

Configured using:
 'configure --prefix=/home/user/apps/emacs-29.0.91-5e7c826
 --with-x-toolkit=no --with-mailutils --with-modules
 --without-compress-install --without-toolkit-scroll-bars
 --with-native-compilation=no --with-tree-sitter'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY OLDXMENU PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM ZLIB

Important settings:
  value of $LANG: en_DK.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
help-fns radix-tree cl-print byte-opt gv bytecomp byte-compile debug
backtrace help-mode find-func cl-loaddefs cl-lib tmm rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq
simple cl-generic indonesian philippine cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo xinput2 x multi-tty make-network-process emacs)

Memory information:
((conses 16 58330 6430)
 (symbols 48 6229 0)
 (strings 32 16451 1591)
 (string-bytes 1 489144)
 (vectors 16 12732)
 (vector-slots 8 171485 10082)
 (floats 8 27 43)
 (intervals 56 1188 0)
 (buffers 984 13))

bug#63754: 29.0.91; tmm breaks when tmm-completion-prompt is nil

[Eli Zaretskii]

> From: Thiago Melo <tmdmelo@gmail.com>
> Date: Sat, 27 May 2023 14:53:59 +0000
> 
> Issue doesn't happen with Emacs 28.
> 
> * How to reproduce
> 
> run:
> emacs -Q
> 
> eval:
> (setq tmm-completion-prompt nil)
> (tmm-menubar) ; or `tmm-prompt' with any menu/keymap
> 
> * Backtrace
> 
> Debugger entered--Lisp error: (search-failed "Possible completions are:\n")
>   search-forward("Possible completions are:\n")
>   tmm-completion-delete-prompt()
>   run-hooks(completion-setup-hook)
>   display-completion-list((#("f==>File" 0 1 (face

Thanks.  This was caused by a change in the completion heading line in
Emacs 29: it is now a customizable format string, and can be nil.  So
the method used by tmm.el for finding the prompt (so it could delete
it) is no longer workable, and must be replaced by a different method.

Should be fixed now on the emacs-29 branch.  (I also fixed it to work
when 'tmm-mid-prompt' is nil, something that seems to have been broken
since about forever -- I found that bug while testing the patch,
because the patch has to work also when 'tmm-mid-prompt' is nil, and
there are no menu shortcuts shown.)

bug#63754: 29.0.91; tmm breaks when tmm-completion-prompt is nil

[Thiago Melo]

On Sat, May 27, 2023 at 5:14 PM Eli Zaretskii <eliz@gnu.org> wrote:
> Thanks.  This was caused by a change in the completion heading line in
> Emacs 29: it is now a customizable format string, and can be nil.  So
> the method used by tmm.el for finding the prompt (so it could delete
> it) is no longer workable, and must be replaced by a different method.
>
> Should be fixed now on the emacs-29 branch.

Cool.  I've tested, it's working nicely.  Thanks for the fix.

> (I also fixed it to work
> when 'tmm-mid-prompt' is nil, something that seems to have been broken
> since about forever -- I found that bug while testing the patch,
> because the patch has to work also when 'tmm-mid-prompt' is nil, and
> there are no menu shortcuts shown.)

Sorry if my comment is out of place, but I also gave a try to
`tmm-mid-prompt' = nil.  Since it disables the tmm shortcuts, maybe it
would be more intuitive to let users input keys and do completion at
the minibuffer in this situation? (unless I'm missing some unwanted
side effect here)

With this little change, for example:

--- tmm.el.orig    2023-05-27 21:26:44.594743965 +0200
+++ tmm.el    2023-05-27 21:27:13.571769823 +0200
@@ -305,7 +305,7 @@
 ;; This returns the old map.
 (defun tmm-define-keys (minibuffer)
   (let ((map (make-sparse-keymap)))
-    (suppress-keymap map t)
+    (when tmm-mid-prompt (suppress-keymap map t))
     (dolist (c tmm-short-cuts)
       (if (listp tmm-shortcut-style)
           (define-key map (char-to-string c) 'tmm-shortcut)

bug#63754: 29.0.91; tmm breaks when tmm-completion-prompt is nil

[Eli Zaretskii]

> From: Thiago Melo <tmdmelo@gmail.com>
> Date: Sat, 27 May 2023 19:48:54 +0000
> Cc: 63754@debbugs.gnu.org
> 
> On Sat, May 27, 2023 at 5:14 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > Thanks.  This was caused by a change in the completion heading line in
> > Emacs 29: it is now a customizable format string, and can be nil.  So
> > the method used by tmm.el for finding the prompt (so it could delete
> > it) is no longer workable, and must be replaced by a different method.
> >
> > Should be fixed now on the emacs-29 branch.
> 
> Cool.  I've tested, it's working nicely.  Thanks for the fix.

Thanks for testing, I'm therefore closing this bug.

> > (I also fixed it to work
> > when 'tmm-mid-prompt' is nil, something that seems to have been broken
> > since about forever -- I found that bug while testing the patch,
> > because the patch has to work also when 'tmm-mid-prompt' is nil, and
> > there are no menu shortcuts shown.)
> 
> Sorry if my comment is out of place, but I also gave a try to
> `tmm-mid-prompt' = nil.  Since it disables the tmm shortcuts, maybe it
> would be more intuitive to let users input keys and do completion at
> the minibuffer in this situation? (unless I'm missing some unwanted
> side effect here)
> 
> With this little change, for example:

I don't know.  The original completion works on the shortcuts, AFAIU,
and thus makes no sense when tmm-mid-prompt is nil.  When that
variable is nil, the user needs to use the arrow keys to select the
menu items.  If someone wants to add a completion feature to that, I
don't mind, but I don't know enough about tmm to tell, and such a
change is definitely not for the release branch.

If someone here has an opinion on this last suggestion, please speak
up.

bug#63754: 29.0.91; tmm breaks when tmm-completion-prompt is nil

[Juri Linkov]

>> > Should be fixed now on the emacs-29 branch.
>>
>> Cool.  I've tested, it's working nicely.  Thanks for the fix.
>
> Thanks for testing, I'm therefore closing this bug.

There is one remaining case where it fails:

  (setq completions-header-format nil)
  (setq tmm-completion-prompt nil)
  (tmm-menubar) ; or `tmm-prompt' with any menu/keymap

Then the first item is deleted.  The patch below fixes this.

>> Sorry if my comment is out of place, but I also gave a try to
>> `tmm-mid-prompt' = nil.  Since it disables the tmm shortcuts, maybe it
>> would be more intuitive to let users input keys and do completion at
>> the minibuffer in this situation? (unless I'm missing some unwanted
>> side effect here)
>>
>> With this little change, for example:
>
> I don't know.  The original completion works on the shortcuts, AFAIU,
> and thus makes no sense when tmm-mid-prompt is nil.  When that
> variable is nil, the user needs to use the arrow keys to select the
> menu items.  If someone wants to add a completion feature to that, I
> don't mind, but I don't know enough about tmm to tell, and such a
> change is definitely not for the release branch.
>
> If someone here has an opinion on this last suggestion, please speak
> up.

AFAICS, completion works without problems in tmm.  The artificial restriction
that removes self-inserting keys was needed to allow only character shortcuts
when tmm-mid-prompt is nil.

diff --git a/lisp/tmm.el b/lisp/tmm.el
index 88254f0d1b8..a4058594622 100644
--- a/lisp/tmm.el
+++ b/lisp/tmm.el
@@ -329,7 +329,8 @@ tmm-completion-delete-prompt
   (with-current-buffer standard-output
     (goto-char (point-min))
     (let* (;; First candidate: first string with mouse-face
-           (menu-start-1 (next-single-char-property-change (point) 'mouse-face))
+           (menu-start-1 (or (and (get-text-property (point) 'mouse-face) (point))
+                             (next-single-char-property-change (point) 'mouse-face)))
            ;; Second candidate: an inactive menu item with tmm-inactive face
            (tps-result (save-excursion
                          (text-property-search-forward 'face 'tmm-inactive t)))

bug#63754: 29.0.91; tmm breaks when tmm-completion-prompt is nil

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: eliz@gnu.org,  tmdmelo@gmail.com
> Date: Mon, 29 May 2023 19:37:58 +0300
> 
> There is one remaining case where it fails:
> 
>   (setq completions-header-format nil)
>   (setq tmm-completion-prompt nil)
>   (tmm-menubar) ; or `tmm-prompt' with any menu/keymap
> 
> Then the first item is deleted.  The patch below fixes this.

Thanks, please install on emacs-29.