bug#62709: 29.0.60; quail-minibuffer-message prevents set-mark-command from activating region in minibuffer

bug#62709: 29.0.60; quail-minibuffer-message prevents set-mark-command from activating region in minibuffer

[Visuwesh]

To reproduce the issue, one must activate an input method that
eventually calls `quail-minibuffer-message' to insert the guidance
string and the "title" of the input method at the end of the minibuffer.

- To observe the problem, first define a dummy Quail input method

    (quail-define-package "none" "asdf" "none")

- Then activate recursive minibuffers

    M-: (setq enable-recursive-minibuffers t) RET

- Now open a minibuffer (C-x C-f is a good one) then activate the "none"
  input method.

- Now type M-x set-mark-command RET and note how the region is not
  active.

This problem goes away if you redefine `quail-minibuffer-message' to
simply echo the string by saying

    (defalias 'quail-minibuffer-message #'message)

Furthermore, the comment above `quail-minibuffer-message' says

    ;; Quail specific version of minibuffer-message.  It displays STRING
    ;; with timeout 1000000 seconds instead of two seconds.

but even with that redefinition, the guidance string is printed just
fine (minus the extra square brackets that message adds).

AFAICT, the guidance string contains text properties but the title of
the input method doesn't.

In GNU Emacs 29.0.60 (build 2, x86_64-pc-linux-gnu, X toolkit, Xaw
 scroll bars) of 2023-03-23 built on astatine
Repository revision: be8147c53f9dd2d896c27d5b32bf4f38b0af9b3f
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12101004
System Description: Debian GNU/Linux bookworm/sid

Configured using:
 'configure --with-sound=alsa --with-x-toolkit=lucid --with-json
 --without-xaw3d --without-gconf --without-libsystemd --without-cairo'
Configured features:
ACL DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON
LIBOTF LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS WEBP X11 XDBE XFT
XIM XINPUT2 XPM LUCID ZLIB
Important settings:
  value of $LC_MONETARY: ta_IN.UTF-8
  value of $LC_NUMERIC: ta_IN.UTF-8
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix

Major mode: ELisp/l

Minor modes in effect:
  csv-field-index-mode: t
  bug-reference-prog-mode: t
  recentf-mode: t
  shell-dirtrack-mode: t
  server-mode: t
  paredit-mode: t
  eros-mode: t
  flymake-mode: t
  pdf-occur-global-minor-mode: t
  minibuffer-depth-indicate-mode: t
  repeat-mode: t
  display-time-mode: t
  display-battery-mode: t
  delete-selection-mode: t
  xterm-mouse-mode: t
  straight-use-package-mode: t
  straight-package-neutering-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  use-hard-newlines: t
  tab-bar-history-mode: t
  tab-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  undelete-frame-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  abbrev-mode: t

Load-path shadows:
None found.

Features:
(shadow emacsbug edebug shr-color ind-util csv-mode csv-mode-autoloads
ement-taxy svg-lib pcmpl-gnu url-http url-gw org-pdftools org-noter
org-num oc-basic org-element org-persist avl-tree calcalg2 calc-poly
calc-units calc-prog calc-trail calc-undo calc-arith calc-misc calc-vec
calc-yank calccomp calc-bin calc-alg calc-menu calc-aent calc-ext calc
calc-loaddefs calc-macs conf-mode icomplete ffap net-utils view
autorevert image-crop rfc2104 debug backtrace etags fileloop generator
imenu-xref tabify man skeleton pdf-sync pdf-annot facemenu pdf-outline
pdf-links ob-gnuplot ob-fortran ob-python python compat ob-julia ob-C
cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs ob-shell ob-racket async ob-async cdlatex texmathp
org-attach org-id org-refile ol-eww ol-rmail ol-mhe ol-irc ol-info
ol-gnus nnselect ol-docview doc-view ol-bibtex bibtex ol-bbdb ol-w3m
ol-doi org-link-doi org-tempo tempo ol-man org ob ob-tangle ob-ref
ob-lob ob-table ob-exp org-macro org-src ob-comint org-pcomplete
org-list org-footnote org-faces org-entities ob-emacs-lisp ob-core
ob-eval org-cycle org-table ol org-fold org-fold-core org-keys oc
org-loaddefs org-version org-compat org-macs pdf-history bug-reference
misc wdired dabbrev url-cache url-auth eww xdg url-queue mm-url reveal
noutline outline vc-annotate vc-filewise gnuplot-context gnuplot
info-look pulse xref mule-util typo msb rect ediff ediff-merg ediff-mult
ediff-wind ediff-diff ediff-help ediff-init ediff-util shortdoc log-edit
add-log vc-dir f90 cus-start avy tramp-cmds cl-print help-fns radix-tree
image-file image-converter pcmpl-unix display-line-numbers
ement-room-list ement ement-notify notifications ement-room ement-lib
ement-api ement-structs plz color ement-macros taxy-magit-section
magit-section taxy ewoc dns misearch multi-isearch dired-aux gnus-dired
tramp-cache time-stamp tramp-sh tramp tramp-loaddefs trampver
tramp-integration tramp-compat ls-lisp recentf tree-widget flyspell
ispell ecomplete smerge-mode flow-fill mm-archive sort gnus-cite
mail-extr textsec uni-scripts idna-mapping ucs-normalize uni-confusable
textsec-check qp gnus-async gnus-bcklg gnus-ml network-stream nsm
nndraft nnmh nnmaildir nnagent nnml vc-backup log-view pcvs-util vc diff
vc-git diff-mode vc-dispatcher nnnil gnus-agent gnus-srvr gnus-score
score-mode nnvirtual gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime
smime gnutls dig nntp gnus-cache gnus-sum shr pixel-fill kinsoku
url-file svg gnus-group gnus-undo gnus-start gnus-dbus gnus-cloud nnimap
nnmail mail-source utf7 nnoo parse-time iso8601 gnus-spec gnus-int
gnus-range message sendmail yank-media puny rfc822 mml mml-sec epa epg
rfc6068 epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus
nnheader gnus-util mail-utils range mm-util mail-prsvr sh-script smie
treesit executable files-x shell-command+ cursor-sensor shell pcomplete
face-remap server paredit edmacro kmacro eros time-date checkdoc
lisp-mnt flymake-proc flymake project thingatpt wordel-autoloads
sokoban-autoloads ement-autoloads svg-lib-autoloads
taxy-magit-section-autoloads magit-section-autoloads dash-autoloads
taxy-autoloads plz-autoloads nov-autoloads esxml-autoloads kv-autoloads
transmission-autoloads lua-mode-autoloads nix-mode-autoloads
gnuplot-autoloads devdocs-autoloads julia-mode-autoloads
racket-mode-autoloads eros-autoloads writegood-mode-autoloads
siege-mode-autoloads paredit-autoloads puni-autoloads
expand-region-autoloads filladapt-autoloads compose scroll-other-window
org-pdftools-autoloads org-noter-autoloads change-env-autoloads
math-delimiters-autoloads doct-autoloads ob-async-autoloads
async-autoloads emacs-ob-racket-autoloads valign-autoloads
cdlatex-autoloads auctex-autoloads tex-site pdf-occur ibuf-ext ibuffer
ibuffer-loaddefs tablist advice tablist-filter semantic/wisent/comp
semantic/wisent semantic/wisent/wisent semantic/util-modes semantic/util
semantic semantic/tag semantic/lex semantic/fw mode-local find-func
cedet pdf-isearch let-alist pdf-misc imenu pdf-tools package browse-url
url url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source eieio eieio-core json map byte-opt url-vars compile comint
ansi-osc ansi-color ring cus-edit wid-edit pdf-view password-cache
jka-compr pdf-cache pdf-info tq pdf-util pdf-macs image-mode dired-x
dired dired-loaddefs exif pdf-tools-autoloads tablist-autoloads
typo-autoloads mb-depth repeat visual-fill-autoloads olivetti-autoloads
time format-spec battery dbus filenotify xml dom tamil99 quail
disp-table lacarte-autoloads shell-command-plus-autoloads delsel
easy-mmode xt-mouse cus-load avy-autoloads finder-inf icalendar
diary-lib diary-loaddefs cal-menu calendar cal-loaddefs warnings icons
filecache imenu-xref-autoloads derived chemtable-autoloads
molar-mass-autoloads vc-backup-autoloads compat-autoloads
saveplace-pdf-view saveplace bookmark text-property-search pp
saveplace-pdf-view-autoloads rx pcase inspector-autoloads xr-autoloads
straight-autoloads cl-seq info cl-extra help-mode straight subr-x
cl-macs gv cl-loaddefs cl-lib bytecomp byte-compile vz-nh-theme
vz-options-theme rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting x-toolkit
xinput2 x multi-tty make-network-process emacs)

Memory information:
((conses 16 3239675 543854)
 (symbols 48 53579 557)
 (strings 32 367026 33804)
 (string-bytes 1 38672819)
 (vectors 16 177851)
 (vector-slots 8 3247188 386527)
 (floats 8 10625 2907)
 (intervals 56 280522 15666)
 (buffers 976 266))

bug#62709: 29.0.60; quail-minibuffer-message prevents set-mark-command from activating region in minibuffer

[Eli Zaretskii]

> From: Visuwesh <visuweshm@gmail.com>
> Date: Fri, 07 Apr 2023 16:24:55 +0530
> 
> - To observe the problem, first define a dummy Quail input method
> 
>     (quail-define-package "none" "asdf" "none")
> 
> - Then activate recursive minibuffers
> 
>     M-: (setq enable-recursive-minibuffers t) RET
> 
> - Now open a minibuffer (C-x C-f is a good one) then activate the "none"
>   input method.
> 
> - Now type M-x set-mark-command RET and note how the region is not
>   active.
> 
> This problem goes away if you redefine `quail-minibuffer-message' to
> simply echo the string by saying
> 
>     (defalias 'quail-minibuffer-message #'message)
> 
> Furthermore, the comment above `quail-minibuffer-message' says
> 
>     ;; Quail specific version of minibuffer-message.  It displays STRING
>     ;; with timeout 1000000 seconds instead of two seconds.
> 
> but even with that redefinition, the guidance string is printed just
> fine (minus the extra square brackets that message adds).

I'd prefer to solve the problem that prevents the region from becoming
active, without changing how Quail shows its messages.  Can you try
finding such a solution, or at least explaining why
quail-minibuffer-message prevents the region from becoming active in
this case?

Thanks.

bug#62709: 29.0.60; quail-minibuffer-message prevents set-mark-command from activating region in minibuffer

[Visuwesh]

[வெள்ளி ஏப்ரல் 07, 2023] Eli Zaretskii wrote:

>> - To observe the problem, first define a dummy Quail input method
>> 
>>     (quail-define-package "none" "asdf" "none")
>> 
>> - Then activate recursive minibuffers
>> 
>>     M-: (setq enable-recursive-minibuffers t) RET
>> 
>> - Now open a minibuffer (C-x C-f is a good one) then activate the "none"
>>   input method.
>> 
>> - Now type M-x set-mark-command RET and note how the region is not
>>   active.
>> 
>> This problem goes away if you redefine `quail-minibuffer-message' to
>> simply echo the string by saying
>> 
>>     (defalias 'quail-minibuffer-message #'message)
>> 
>> Furthermore, the comment above `quail-minibuffer-message' says
>> 
>>     ;; Quail specific version of minibuffer-message.  It displays STRING
>>     ;; with timeout 1000000 seconds instead of two seconds.
>> 
>> but even with that redefinition, the guidance string is printed just
>> fine (minus the extra square brackets that message adds).
>
> I'd prefer to solve the problem that prevents the region from becoming
> active, without changing how Quail shows its messages.  Can you try
> finding such a solution, 

One solution would be to use an overlay to show the guidance string,
like what `minibuffer-depth-mode' does.

> or at least explaining why quail-minibuffer-message prevents the
> region from becoming active in this case?

Unfortunately, that is well beyond my abilities.  If I simply comment
out the part where Quail `insert's the guidance string, I see no issue
but if I leave it and check the return value of `use-region-p' and value
of `mark-active' before and after the `insert' call, I see nil before
and after the insert call...

I investigated insert and delete-region because they deactivate an
active region in a "normal buffer" when called.

> Thanks.

bug#62709: 29.0.60; quail-minibuffer-message prevents set-mark-command from activating region in minibuffer

[Visuwesh]

[வெள்ளி ஏப்ரல் 07, 2023] Visuwesh wrote:

>> I'd prefer to solve the problem that prevents the region from becoming
>> active, without changing how Quail shows its messages.  Can you try
>> finding such a solution, 
>
> One solution would be to use an overlay to show the guidance string,
> like what `minibuffer-depth-mode' does.

I came close to solving the issue with an overlay but I hit a roadblock
with the cursor property.
I currently create an overlay at minibuffer's point-max but the cursor
is shown _after_ the overlay string.  I tried to use the cursor text
property to move the cursor before the string but it does not do it if
the string contains a newline character _anywhere_, which is
contradictory to what the manual says

     One subtlety of this property is that it doesn’t work to put this
     property on a newline character that is part of a display or
              ^^^^^^^^^^^^^^^^^^^^^^
     overlay string.  That’s because the newline doesn’t have a graphic
     representation on the screen for Emacs to find when it looks for a
     character on display with that ‘cursor’ property.

but I do _not_ put the property on a newline character, I put it in on a
space character.

To spell things out, the overlay's after-string is something like
"  [none]
Guidance string here"

I put the cursor=t text-property on the first space character, but the
cursor is not displayed there.  If I remove the newline so the
after-string becomes

"  [none] Guidance string here"

then the cursor is just before the space character i.e., |  [none]...
where | is the cursor.

I'm attaching my work so far as a diff

diff --git a/lisp/international/quail.el b/lisp/international/quail.el
index 2ffe339233..20eba9c1d4 100644
--- a/lisp/international/quail.el
+++ b/lisp/international/quail.el
@@ -85,6 +85,9 @@ quail-overlay
 (defvar-local quail-conv-overlay nil
   "Overlay which covers the text to be converted in Quail mode.")
 
+(defvar-local quail-minibuf-guidance-overlay nil
+  "Overlay used to display the guidance string when in minibuffer.")
+
 (defvar-local quail-current-key nil
   "Current key for translation in Quail mode.")
 
@@ -571,18 +574,22 @@ quail-activate
     (setq deactivate-current-input-method-function #'quail-deactivate)
     (setq describe-current-input-method-function #'quail-help)
     (quail-delete-overlays)
-    (setq quail-guidance-str "")
-    (quail-show-guidance)
     ;; If we are in minibuffer, turn off the current input method
     ;; before exiting.
     (when (eq (selected-window) (minibuffer-window))
+      (quail-minibuf-setup-guidance-overlay)
       (add-hook 'minibuffer-exit-hook #'quail-exit-from-minibuffer)
       (add-hook 'post-command-hook #'quail-show-guidance nil t))
+    (setq quail-guidance-str "")
+    (quail-show-guidance)
     (run-hooks 'quail-activate-hook)
     (setq-local input-method-function #'quail-input-method)))
 
 (defun quail-exit-from-minibuffer ()
   (deactivate-input-method)
+  (when (and (overlayp quail-minibuf-guidance-overlay)
+             (overlay-start quail-minibuf-guidance-overlay))
+    (delete-overlay quail-minibuf-guidance-overlay))
   (if (<= (minibuffer-depth) 1)
       (remove-hook 'minibuffer-exit-hook 'quail-exit-from-minibuffer)))
 
@@ -1993,17 +2000,33 @@ quail-require-guidance-buf
 ;; with timeout 1000000 seconds instead of two seconds.
 
 (defun quail-minibuffer-message (string)
-  (message nil)
-  (let ((point-max (point-max))
-	(inhibit-quit t))
-    (save-excursion
-      (goto-char point-max)
-      (insert string))
-    (sit-for 1000000)
-    (delete-region point-max (point-max))
-    (when quit-flag
-      (setq quit-flag nil)
-      (quail-add-unread-command-events 7 t))))
+  (put-text-property 0 1 'cursor t string)
+  (overlay-put quail-minibuf-guidance-overlay
+               'after-string string))
+
+;; (defun quail-minibuffer-message (string)
+;;   (message nil)
+;;   (let ((point-max (point-max))
+;; 	(inhibit-quit t))
+;;     (save-excursion
+;;       (goto-char point-max)
+;;       (insert string))
+;;     (sit-for 1000000)
+;;     (delete-region point-max (point-max))
+;;     (when quit-flag
+;;       (setq quit-flag nil)
+;;       (quail-add-unread-command-events 7 t))))
+
+(defun quail-minibuf-move-guidance-overlay (ov afterp &rest _)
+  (when afterp
+    (move-overlay ov (point-max) (point-max))))
+
+(defun quail-minibuf-setup-guidance-overlay ()
+  (setq quail-minibuf-guidance-overlay
+        (make-overlay (point-max) (point-max)))
+  (overlay-put quail-minibuf-guidance-overlay 'insert-behind-hooks
+               '(quail-minibuf-move-guidance-overlay)))
+
 
 (defun quail-show-guidance ()
   "Display a guidance for Quail input method in some window.

bug#62709: 29.0.60; quail-minibuffer-message prevents set-mark-command from activating region in minibuffer

[Eli Zaretskii]

> From: Visuwesh <visuweshm@gmail.com>
> Cc: 62709@debbugs.gnu.org
> Date: Fri, 07 Apr 2023 20:09:13 +0530
> 
> [வெள்ளி ஏப்ரல் 07, 2023] Visuwesh wrote:
> 
> >> I'd prefer to solve the problem that prevents the region from becoming
> >> active, without changing how Quail shows its messages.  Can you try
> >> finding such a solution, 
> >
> > One solution would be to use an overlay to show the guidance string,
> > like what `minibuffer-depth-mode' does.
> 
> I came close to solving the issue with an overlay but I hit a roadblock
> with the cursor property.

Thanks, but I think this is an unnecessary complication.

Does the patch below fix the problems you saw, without introducing any
regressions?

diff --git a/lisp/international/quail.el b/lisp/international/quail.el
index 2ffe339..317ea84 100644
--- a/lisp/international/quail.el
+++ b/lisp/international/quail.el
@@ -1995,7 +1995,8 @@ quail-require-guidance-buf
 (defun quail-minibuffer-message (string)
   (message nil)
   (let ((point-max (point-max))
-	(inhibit-quit t))
+	(inhibit-quit t)
+        (deactivate-mark nil))
     (save-excursion
       (goto-char point-max)
       (insert string))

bug#62709: 29.0.60; quail-minibuffer-message prevents set-mark-command from activating region in minibuffer

[Visuwesh]

[வெள்ளி ஏப்ரல் 07, 2023] Eli Zaretskii wrote:

>> From: Visuwesh <visuweshm@gmail.com>
>> Cc: 62709@debbugs.gnu.org
>> Date: Fri, 07 Apr 2023 20:09:13 +0530
>> 
>> [வெள்ளி ஏப்ரல் 07, 2023] Visuwesh wrote:
>> 
>> >> I'd prefer to solve the problem that prevents the region from becoming
>> >> active, without changing how Quail shows its messages.  Can you try
>> >> finding such a solution, 
>> >
>> > One solution would be to use an overlay to show the guidance string,
>> > like what `minibuffer-depth-mode' does.
>> 
>> I came close to solving the issue with an overlay but I hit a roadblock
>> with the cursor property.
>
> Thanks, but I think this is an unnecessary complication.
>
> Does the patch below fix the problems you saw, without introducing any
> regressions?

Yes, it fixes the issue.  Thanks for the quick fix!

> diff --git a/lisp/international/quail.el b/lisp/international/quail.el
> index 2ffe339..317ea84 100644
> --- a/lisp/international/quail.el
> +++ b/lisp/international/quail.el
> @@ -1995,7 +1995,8 @@ quail-require-guidance-buf
>  (defun quail-minibuffer-message (string)
>    (message nil)
>    (let ((point-max (point-max))
> -	(inhibit-quit t))
> +	(inhibit-quit t)
> +        (deactivate-mark nil))
>      (save-excursion
>        (goto-char point-max)
>        (insert string))

bug#62709: 29.0.60; quail-minibuffer-message prevents set-mark-command from activating region in minibuffer

[Eli Zaretskii]

> From: Visuwesh <visuweshm@gmail.com>
> Cc: 62709@debbugs.gnu.org
> Date: Fri, 07 Apr 2023 20:39:25 +0530
> 
> [வெள்ளி ஏப்ரல் 07, 2023] Eli Zaretskii wrote:
> 
> > Does the patch below fix the problems you saw, without introducing any
> > regressions?
> 
> Yes, it fixes the issue.  Thanks for the quick fix!

Thanks, installed on the emacs-29 branch, and closing the bug.