bug#74833: 31.0.50; Copy to OS clipboard doesn't work in macOS Terminal.app with xterm-mouse-mode enabled

[Jared Finder via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 844 bytes --]

On 2024-12-28 23:13, Eli Zaretskii wrote:
>> Date: Sat, 28 Dec 2024 21:16:28 -0800
>> From: Jared Finder <jared@finder.org>
>> Cc: shipmints@gmail.com, rms@gnu.org, gerd.moellmann@gmail.com,
>>  fgunbin@fastmail.fm, 74833@debbugs.gnu.org
>> 
>> I'm proposing to add a single regexp that matches against the terminal
>> name and version string. If there's a match, automatically enable
>> xterm-mouse-mode. For terminals that aren't supported or don't support
>> "\e[>0q", leave xterm-mouse-mode as is. No other complexity is needed. 
>> A
>> user can always customize xterm-mouse-mode (it's a user option) if 
>> they
>> want to enable it anyways.
> 
> Sounds okay, but can you post a patch to try?

Patch attached.

I also noticed outdated text in the docstring for xterm-mouse-mode and 
attached a second patch to delete that text.

   -- MJF

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Don-t-always-enable-xterm-mouse-mode-bug-74833.patch --]
[-- Type: text/x-diff; name=0001-Don-t-always-enable-xterm-mouse-mode-bug-74833.patch, Size: 9601 bytes --]

From 60d9ab0b4cb9e70718feebc1ec37e51eb1d96ab0 Mon Sep 17 00:00:00 2001
From: Jared Finder <jared@finder.org>
Date: Wed, 1 Jan 2025 22:36:25 -0800
Subject: [PATCH 1/2] Don't always enable xterm-mouse-mode (bug#74833)

Many terminals set the environment variable TERM to "xterm" even when they
don't support all functionality in xterm.  This means that enabling
xterm-mouse-mode can break critical editing workflows like copy/paste.  This
adds checks for the specific terminal Emacs is run in and only enables
xterm-mouse-mode on terminals knows to support all critical editing workflows.

* etc/NEWS: Update announcement
* lisp/term/xterm.el (xterm--auto-xt-mouse-allowed-names)
(xterm--auto-xt-mouse-allowed-types): New variables to control
what terminals automatically enable xterm-mouse-mode.
(xterm--report-background-handler, xterm--version-handler): Use
xterm--read-string.
(xterm--read-string, xterm--query-name-and-version): New
function.
(xterm--init): Check what terminal is running and if
xterm-mouse-mode was manually called.
* lisp/xt-mouse.el (xterm-mouse-mode-called): New variable.
(xterm-mouse-mode): Set xterm-mouse-mode-called.
---
 etc/NEWS           | 14 ++++---
 lisp/term/xterm.el | 96 ++++++++++++++++++++++++++++++++++++++--------
 lisp/xt-mouse.el   |  7 +++-
 3 files changed, 95 insertions(+), 22 deletions(-)

diff --git a/etc/NEWS b/etc/NEWS
index 85213cbaa6f..4b5bc9ea3d7 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -34,11 +34,15 @@ incorrectly in rare cases.
 \f
 * Startup Changes in Emacs 31.1
 
-** When run inside xterm, 'xterm-mouse-mode' is turned on by default.
-This means that the mouse will work by default inside xterm terminals.
-If your terminal does not behave properly with xterm mouse tracking
-enabled, you can disable mouse tracking by putting '(xterm-mouse-mode
--1)' in your init file.
+** In compatible terminals, 'xterm-mouse-mode' is turned on by default.
+For these terminals the mouse will work by default.  A compatible
+terminal is one that supports Emacs seting and getting the OS selection
+data (a.k.a. the clipboard) and mouse button and motion events.  With
+xterm-mouse-mode enabled, you must use Emacs keybindings to copy to the
+OS selection instead of terminal-specific keybindings.
+
+You can keep the old behavior by putting `(xterm-mouse-mode -1)' in your
+init file.
 
 \f
 * Changes in Emacs 31.1
diff --git a/lisp/term/xterm.el b/lisp/term/xterm.el
index c4f33cd0faa..a8e88c79acd 100644
--- a/lisp/term/xterm.el
+++ b/lisp/term/xterm.el
@@ -83,6 +83,39 @@ xterm-store-paste-on-kill-ring
 (defconst xterm-paste-ending-sequence "\e[201~"
   "Characters sent by the terminal to end a bracketed paste.")
 
+(defconst xterm--auto-xt-mouse-allowed-names
+  (mapconcat (lambda (s) (concat "^" s "\\>"))
+             '("Konsole"
+               "WezTerm"
+               ;; "XTerm"   ;Disabled because OSC52 support is opt-in only.
+               "iTerm2"     ;OSC52 support has opt-in/out UI on first usage
+               "kitty")
+             "\\|")
+  "Regexp for terminals that automatically enable `xterm-mouse-mode' at startup.
+This will get matched against the terminal's XTVERSION string.
+
+It is expected that any matching terminal supports the following
+functionality:
+
+\"Set selection data\" (OSC52): Allows Emacs to set the OS clipboard.
+\"Get selection data\" (OSC52 or bracketed paste): Allows Emacs to get
+    the contents of the OS clipboard.
+\"Basic mouse mode\" (DECSET1000): Allows Emacs to get events on mouse
+    clicks.
+\"Mouse motion mode\" (DECSET1003): Allows Emacs to get event on mouse
+    motion.
+
+Also see `xterm--auto-xt-mouse-allowed-types' which mtches against the
+value of TERM instead.")
+
+(defconst xterm--auto-xt-mouse-allowed-types
+  (mapconcat (lambda (s) (concat "^" s "$"))
+             '("alacritty"
+               "contour")
+             "\\|")
+  "Like `xterm--auto-xt-mouse-allowed-names', but for the terminal's type.
+This will get matched against the environment variable \"TERM\".")
+
 (defun xterm--pasted-text ()
   "Handle the rest of a terminal paste operation.
 Return the pasted text as a string."
@@ -707,11 +740,8 @@ xterm-standard-colors
   "Names of 16 standard xterm/aixterm colors, their numbers, and RGB values.")
 
 (defun xterm--report-background-handler ()
-  (let ((str "")
-        chr)
-    ;; The reply should be: \e ] 11 ; rgb: NUMBER1 / NUMBER2 / NUMBER3 \e \\
-    (while (and (setq chr (xterm--read-event-for-query)) (not (equal chr ?\\)))
-      (setq str (concat str (string chr))))
+  ;; The reply should be: \e ] 11 ; rgb: NUMBER1 / NUMBER2 / NUMBER3 \e \\
+  (let ((str (xterm--read-string ?\e ?\\)))
     (when (string-match
            "rgb:\\([a-f0-9]+\\)/\\([a-f0-9]+\\)/\\([a-f0-9]+\\)" str)
       (let ((recompute-faces
@@ -730,16 +760,13 @@ xterm--report-background-handler
           (tty-set-up-initial-frame-faces))))))
 
 (defun xterm--version-handler ()
-  (let ((str "")
-        chr)
-    ;; The reply should be: \e [ > NUMBER1 ; NUMBER2 ; NUMBER3 c
-    ;; If the timeout is completely removed for read-event, this
-    ;; might hang for terminals that pretend to be xterm, but don't
-    ;; respond to this escape sequence.  RMS' opinion was to remove
-    ;; it completely.  That might be right, but let's first try to
-    ;; see if by using a longer timeout we get rid of most issues.
-    (while (and (setq chr (xterm--read-event-for-query)) (not (equal chr ?c)))
-      (setq str (concat str (string chr))))
+  ;; The reply should be: \e [ > NUMBER1 ; NUMBER2 ; NUMBER3 c
+  ;; If the timeout is completely removed for read-event, this
+  ;; might hang for terminals that pretend to be xterm, but don't
+  ;; respond to this escape sequence.  RMS' opinion was to remove
+  ;; it completely.  That might be right, but let's first try to
+  ;; see if by using a longer timeout we get rid of most issues.
+  (let ((str (xterm--read-string ?c)))
     ;; Since xterm-280, the terminal type (NUMBER1) is now 41 instead of 0.
     (when (string-match "\\([0-9]+\\);\\([0-9]+\\);[01]" str)
       (let ((version (string-to-number (match-string 2 str))))
@@ -810,6 +837,21 @@ xterm--read-event-for-query
 				  xterm-query-timeout
 				  (time-since start-time)))))))))
 
+(defun xterm--read-string (term1 &optional term2)
+  "Read a string with terminating characters.
+This uses `xterm--read-event-for-query' internally."
+  (let ((str "")
+        chr last)
+    (while (and (setq last chr
+                      chr (xterm--read-event-for-query))
+                (if term2
+                    (not (and (equal last term1) (equal chr term2)))
+                  (not (equal chr term1))))
+      (setq str (concat str (string chr))))
+    (if term2
+        (substring str 0 -1)
+      str)))
+
 (defun xterm--query (query handlers &optional no-async)
   "Send QUERY string to the terminal and watch for a response.
 HANDLERS is an alist with elements of the form (STRING . FUNCTION).
@@ -860,6 +902,20 @@ xterm--query
               (push (aref (car handler) (setq i (1- i)))
                     unread-command-events))))))))
 
+(defun xterm--query-name-and-version ()
+  "Get the terminal name and version string (XTVERSION)."
+  ;; The default timeout time causes a noticeable startup delay on
+  ;; terminals that ignore the query.
+  (let ((xterm-query-timeout 0.1))
+    (catch 'result
+      (xterm--query
+       "\e[>0q"
+       '(("\eP>|" . (lambda ()
+                      ;; The reply should be: \e P > | STRING \e \\
+                      (let ((str (xterm--read-string ?\e ?\\)))
+                        (throw 'result str))))))
+      nil)))
+
 (defun xterm--push-map (map basemap)
   ;; Use inheritance to let the main keymaps override those defaults.
   ;; This way we don't override terminfo-derived settings or settings
@@ -907,7 +963,15 @@ xterm--init
 
   (when xterm-set-window-title
     (xterm--init-frame-title))
-  (when xterm-mouse-mode
+  (when (and (not xterm-mouse-mode-called)
+             ;; Only automatically enable xterm mouse on terminals
+             ;; confirmed to still support all critical editing
+             ;; workflows (bug#74833).
+             (or (string-match-p xterm--auto-xt-mouse-allowed-types
+                                 (tty-type (selected-frame)))
+                 (and-let* ((name-and-version (xterm--query-name-and-version)))
+                   (string-match-p xterm--auto-xt-mouse-allowed-names
+                                   name-and-version))))
     (xterm-mouse-mode 1))
   ;; Unconditionally enable bracketed paste mode: terminals that don't
   ;; support it just ignore the sequence.
diff --git a/lisp/xt-mouse.el b/lisp/xt-mouse.el
index 2ba60ded899..c390788ecd3 100644
--- a/lisp/xt-mouse.el
+++ b/lisp/xt-mouse.el
@@ -362,6 +362,11 @@ xterm-mouse-event
         (set-terminal-parameter nil 'xterm-mouse-frame frame)
         (setq last-input-event event)))))
 
+;;;###autoload
+(defvar xterm-mouse-mode-called nil
+  "If `xterm-mouse-mode' has been called already.
+This can be used to detect if xterm-mouse-mode was explicitly set.")
+
 ;;;###autoload
 (define-minor-mode xterm-mouse-mode
   "Toggle XTerm mouse mode.
@@ -373,8 +378,8 @@ xterm-mouse-mode
 mouse functionality for such clicks is still available by holding
 down the SHIFT key while pressing the mouse button."
   :global t :group 'mouse
-  :init-value t
   :version "31.1"
+  (setq xterm-mouse-mode-called t)
   (funcall (if xterm-mouse-mode 'add-hook 'remove-hook)
            'terminal-init-xterm-hook
            'turn-on-xterm-mouse-tracking-on-terminal)
-- 
2.39.5


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-Modifier-keys-and-double-clicks-work-correctly.patch --]
[-- Type: text/x-diff; name=0002-Modifier-keys-and-double-clicks-work-correctly.patch, Size: 1386 bytes --]

From f1c7858c5794a13b19b36884bb5ac5db019005f7 Mon Sep 17 00:00:00 2001
From: Jared Finder <jared@finder.org>
Date: Wed, 1 Jan 2025 23:04:23 -0800
Subject: [PATCH 2/2] Modifier keys and double clicks work correctly

* lisp/xt-mouse.el (xterm-mouse-mode): Update comment.
---
 lisp/xt-mouse.el | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/lisp/xt-mouse.el b/lisp/xt-mouse.el
index c390788ecd3..b9288bde6b6 100644
--- a/lisp/xt-mouse.el
+++ b/lisp/xt-mouse.el
@@ -371,12 +371,10 @@ xterm-mouse-mode-called
 (define-minor-mode xterm-mouse-mode
   "Toggle XTerm mouse mode.
 
-Turn it on to use Emacs mouse commands, and off to use xterm mouse commands.
-This works in terminal emulators compatible with xterm.  It only
-works for simple uses of the mouse.  Basically, only non-modified
-single clicks are supported.  When turned on, the normal xterm
-mouse functionality for such clicks is still available by holding
-down the SHIFT key while pressing the mouse button."
+Turn it on to use Emacs mouse commands, and off to use xterm mouse
+commands.  This works in terminal emulators compatible with xterm.  When
+turned on, the normal xterm mouse functionality for such clicks is still
+available by holding down the SHIFT key while pressing the mouse button."
   :global t :group 'mouse
   :version "31.1"
   (setq xterm-mouse-mode-called t)
-- 
2.39.5