bug#61022: 28.2.50; Mouse tracking of high coordinates not working in rxvt-unicode

bug#61022: 28.2.50; Mouse tracking of high coordinates not working in rxvt-unicode

[git]

Hi,

For some reason, xt-mouse with xterm-mouse-utf-8 on isn't working for me
under rxvt-unicode. I assume it worked at some point, but it's not
working now.

I investigated it a little and it looks like the (read-char nil nil 0.1)
call is no longer reading an entire UTF-8 character unless the
inherit-input-method argument is non-nil. I don't know why that is.
But, in any case, this patch fixes it for me:

----------------------------------------------------------------------
From 32e4b5ab67006ed08c853396686315ed373691be Mon Sep 17 00:00:00 2001
From: Vladimir Panteleev <git@cy.md>
Date: Sun, 22 Jan 2023 18:54:05 +0000
Subject: [PATCH] Fix xterm mouse tracking of high coordinates with
 xterm-mouse-utf-8

read-char now needs inherit-input-method to be non-nil for it to read
a whole UTF-8 code point, even with set-keyboard-coding-system.

* lisp/xt-mouse.el (xterm-mouse--read-coordinate): Call read-char with
inherit-input-method as t.
---
 lisp/xt-mouse.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/xt-mouse.el b/lisp/xt-mouse.el
index adfa480bc0f..2b6b448305e 100644
--- a/lisp/xt-mouse.el
+++ b/lisp/xt-mouse.el
@@ -160,7 +160,7 @@ xterm-mouse--read-coordinate
              'no-conversion))
           ;; Wait only a little; we assume that the entire escape sequence
           ;; has already been sent when this function is called.
-          (read-char nil nil 0.1))
+          (read-char nil t 0.1))
       (set-keyboard-coding-system previous-keyboard-coding-system))))
 
 ;; In default mode, each numeric parameter of XTerm's mouse report is
-- 
2.38.1
----------------------------------------------------------------------

Reproducer:

- Run urxvt
- In it, run emacs -Q -nw
- M-: (require 'xt-mouse)
- M-: (setq xterm-mouse-utf-8 t)
- M-x xterm-mouse-mode
- Make sure the window is wider than 96 lines or columns
- Move the mouse around past the 96th line or column

Hope this helps.



In GNU Emacs 28.2.50 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2022-10-03 built on home.thecybershadow.net
Repository revision: 992611b10a2ef4621b5c936d80cf31644ca3653d
Repository branch: makepkg
System Description: Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --mandir=/usr/share/man --with-gameuser=:games
 --with-modules --without-libotf --without-m17n-flt --without-gconf
 --without-gsettings --enable-link-time-optimization --with-xinput2
 --with-native-compilation --with-x-toolkit=lucid --with-xft
 --with-xaw3d --without-cairo --with-sound=alsa
 --without-compress-install
 '--program-transform-name=s/\([ec]tags\)/\1.emacs/'
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions
 -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security
 -fstack-clash-protection -fcf-protection -fuse-ld=gold -fuse-ld=gold'
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
ACL DBUS FREETYPE GIF GLIB GMP GNUTLS GPM HARFBUZZ JPEG JSON LCMS2
LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XAW3D XDBE XFT XIM
XPM LUCID ZLIB

Important settings:
  value of $LC_TIME: en_DK.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Magit

Minor modes in effect:
  diff-hl-margin-mode: t
  xterm-mouse-mode: t
  which-key-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  term-title-mode: t
  term-keys-mode: t
  term-cursor-color-mode: t
  save-place-mode: t
  savehist-mode: t
  projectile-mode: t
  delete-selection-mode: t
  cua-mode: t
  helm-mode: t
  helm-minibuffer-history-mode: t
  recentf-mode: t
  async-bytecomp-package-mode: t
  global-diff-hl-mode: t
  global-company-mode: t
  company-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  shell-dirtrack-mode: t
  global-flycheck-mode: t
  override-global-mode: t
  straight-use-package-mode: t
  straight-package-neutering-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/vladimir/.emacs.d/straight/build/cmake-mode/cmake-mode hides /usr/share/emacs/site-lisp/cmake-mode
/home/vladimir/work/extern/notmuch/emacs/notmuch hides /usr/share/emacs/site-lisp/notmuch
/home/vladimir/work/extern/notmuch/emacs/notmuch-wash hides /usr/share/emacs/site-lisp/notmuch-wash
/home/vladimir/work/extern/notmuch/emacs/notmuch-version hides /usr/share/emacs/site-lisp/notmuch-version
/home/vladimir/work/extern/notmuch/emacs/notmuch-tree hides /usr/share/emacs/site-lisp/notmuch-tree
/home/vladimir/work/extern/notmuch/emacs/notmuch-tag hides /usr/share/emacs/site-lisp/notmuch-tag
/home/vladimir/work/extern/notmuch/emacs/notmuch-show hides /usr/share/emacs/site-lisp/notmuch-show
/home/vladimir/work/extern/notmuch/emacs/notmuch-query hides /usr/share/emacs/site-lisp/notmuch-query
/home/vladimir/work/extern/notmuch/emacs/notmuch-print hides /usr/share/emacs/site-lisp/notmuch-print
/home/vladimir/work/extern/notmuch/emacs/notmuch-parser hides /usr/share/emacs/site-lisp/notmuch-parser
/home/vladimir/work/extern/notmuch/emacs/notmuch-mua hides /usr/share/emacs/site-lisp/notmuch-mua
/home/vladimir/work/extern/notmuch/emacs/notmuch-message hides /usr/share/emacs/site-lisp/notmuch-message
/home/vladimir/work/extern/notmuch/emacs/notmuch-maildir-fcc hides /usr/share/emacs/site-lisp/notmuch-maildir-fcc
/home/vladimir/work/extern/notmuch/emacs/notmuch-lib hides /usr/share/emacs/site-lisp/notmuch-lib
/home/vladimir/work/extern/notmuch/emacs/notmuch-jump hides /usr/share/emacs/site-lisp/notmuch-jump
/home/vladimir/work/extern/notmuch/emacs/notmuch-hello hides /usr/share/emacs/site-lisp/notmuch-hello
/home/vladimir/work/extern/notmuch/emacs/notmuch-draft hides /usr/share/emacs/site-lisp/notmuch-draft
/home/vladimir/work/extern/notmuch/emacs/notmuch-crypto hides /usr/share/emacs/site-lisp/notmuch-crypto
/home/vladimir/work/extern/notmuch/emacs/notmuch-compat hides /usr/share/emacs/site-lisp/notmuch-compat
/home/vladimir/work/extern/notmuch/emacs/notmuch-company hides /usr/share/emacs/site-lisp/notmuch-company
/home/vladimir/work/extern/notmuch/emacs/notmuch-address hides /usr/share/emacs/site-lisp/notmuch-address
/home/vladimir/work/extern/notmuch/emacs/coolj hides /usr/share/emacs/site-lisp/coolj
/home/vladimir/.emacs.d/straight/build/transient/transient hides /usr/share/emacs/28.2.50/lisp/transient
/home/vladimir/.emacs.d/straight/build/jsonrpc/jsonrpc hides /usr/share/emacs/28.2.50/lisp/jsonrpc
/home/vladimir/.emacs.d/straight/build/xref/xref hides /usr/share/emacs/28.2.50/lisp/progmodes/xref
/home/vladimir/.emacs.d/straight/build/project/project hides /usr/share/emacs/28.2.50/lisp/progmodes/project
/home/vladimir/.emacs.d/straight/build/flymake/flymake hides /usr/share/emacs/28.2.50/lisp/progmodes/flymake
/home/vladimir/.emacs.d/straight/build/let-alist/let-alist hides /usr/share/emacs/28.2.50/lisp/emacs-lisp/let-alist
/home/vladimir/.emacs.d/straight/build/eldoc/eldoc hides /usr/share/emacs/28.2.50/lisp/emacs-lisp/eldoc

Memory information:
((conses 16 752116 488420)
 (symbols 48 78701 92)
 (strings 32 328620 30949)
 (string-bytes 1 8451232)
 (vectors 16 105736)
 (vector-slots 8 3130938 240432)
 (floats 8 458 3116)
 (intervals 56 3490 2630)
 (buffers 992 30))

bug#61022: 28.2.50; Mouse tracking of high coordinates not working in rxvt-unicode

[Eli Zaretskii]

> From: git@vladimir.panteleev.md
> Date: Sun, 22 Jan 2023 19:36:50 +0000
> 
> For some reason, xt-mouse with xterm-mouse-utf-8 on isn't working for me
> under rxvt-unicode. I assume it worked at some point, but it's not
> working now.
> 
> I investigated it a little and it looks like the (read-char nil nil 0.1)
> call is no longer reading an entire UTF-8 character unless the
> inherit-input-method argument is non-nil. I don't know why that is.
> But, in any case, this patch fixes it for me:
> 
> ----------------------------------------------------------------------
> >From 32e4b5ab67006ed08c853396686315ed373691be Mon Sep 17 00:00:00 2001
> From: Vladimir Panteleev <git@cy.md>
> Date: Sun, 22 Jan 2023 18:54:05 +0000
> Subject: [PATCH] Fix xterm mouse tracking of high coordinates with
>  xterm-mouse-utf-8
> 
> read-char now needs inherit-input-method to be non-nil for it to read
> a whole UTF-8 code point, even with set-keyboard-coding-system.
> 
> * lisp/xt-mouse.el (xterm-mouse--read-coordinate): Call read-char with
> inherit-input-method as t.
> ---
>  lisp/xt-mouse.el | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/lisp/xt-mouse.el b/lisp/xt-mouse.el
> index adfa480bc0f..2b6b448305e 100644
> --- a/lisp/xt-mouse.el
> +++ b/lisp/xt-mouse.el
> @@ -160,7 +160,7 @@ xterm-mouse--read-coordinate
>               'no-conversion))
>            ;; Wait only a little; we assume that the entire escape sequence
>            ;; has already been sent when this function is called.
> -          (read-char nil nil 0.1))
> +          (read-char nil t 0.1))
>        (set-keyboard-coding-system previous-keyboard-coding-system))))
>  
>  ;; In default mode, each numeric parameter of XTerm's mouse report is
> -- 
> 2.38.1
> ----------------------------------------------------------------------
> 
> Reproducer:
> 
> - Run urxvt
> - In it, run emacs -Q -nw
> - M-: (require 'xt-mouse)
> - M-: (setq xterm-mouse-utf-8 t)
> - M-x xterm-mouse-mode
> - Make sure the window is wider than 96 lines or columns
> - Move the mouse around past the 96th line or column
> 
> Hope this helps.

Jared, any comments?

Thanks.

bug#61022: 28.2.50; Mouse tracking of high coordinates not working in rxvt-unicode

[Jared Finder via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 2023-01-23 5:15 am, Eli Zaretskii wrote:
>> From: git@vladimir.panteleev.md
>> Date: Sun, 22 Jan 2023 19:36:50 +0000
>> 
>> For some reason, xt-mouse with xterm-mouse-utf-8 on isn't working for 
>> me
>> under rxvt-unicode. I assume it worked at some point, but it's not
>> working now.
>> 
...
>> Reproducer:
>> 
>> - Run urxvt
>> - In it, run emacs -Q -nw
>> - M-: (require 'xt-mouse)
>> - M-: (setq xterm-mouse-utf-8 t)
>> - M-x xterm-mouse-mode
>> - Make sure the window is wider than 96 lines or columns
>> - Move the mouse around past the 96th line or column
>> 
>> Hope this helps.
> 
> Jared, any comments?
> 
> Thanks.

The change mostly works as inherit-input-method also causes UTF-8 
decoding to happen deep in read_char at the C level. (Is this 
intentional? I assume so because read-char just reads single bytes 
normally.) However, I think the following change is more appropriate:

-          (read-char nil nil 0.1))
+          ;; Read a character with input method conversion enabled
+          ;; but no conversion to force read-char to decode UTF-8
+          ;; byte sequences.
+          (let ((input-method-function nil))
+            (read-char nil t 0.1)))

This way we don't apply an actual input method conversion to characters. 
For example, without this additional change, if the 'british input 
method was active, the # ==> £ conversion would
happen, causing mouse events with X=2 to instead have X=131.

I verified this works for me locally.

   -- MJF

bug#61022: 28.2.50; Mouse tracking of high coordinates not working in rxvt-unicode

[Eli Zaretskii]

> Date: Mon, 23 Jan 2023 22:56:09 -0800
> From: Jared Finder <jared@finder.org>
> Cc: git@vladimir.panteleev.md, 61022@debbugs.gnu.org
> 
> The change mostly works as inherit-input-method also causes UTF-8 
> decoding to happen deep in read_char at the C level. (Is this 
> intentional? I assume so because read-char just reads single bytes 
> normally.)

Yes, that's how we decode keyboard input using keyboard-coding-system.

> However, I think the following change is more appropriate:
> 
> -          (read-char nil nil 0.1))
> +          ;; Read a character with input method conversion enabled
> +          ;; but no conversion to force read-char to decode UTF-8
> +          ;; byte sequences.
> +          (let ((input-method-function nil))
> +            (read-char nil t 0.1)))
> 
> This way we don't apply an actual input method conversion to characters. 
> For example, without this additional change, if the 'british input 
> method was active, the # ==> £ conversion would
> happen, causing mouse events with X=2 to instead have X=131.

OK, but shouldn't we also use INHERIT-INPUT-METHOD = t in the call to
read-char only when xterm-mouse-utf-8 option is set?  Otherwise, we
rely on read-char to not perform any conversions, but why rely on that
if we already know we don't want any conversions in that case?  Using
nil when xterm-mouse-utf-8 is unset sounds like a more future-proof
change, no?

Thanks.

bug#61022: 28.2.50; Mouse tracking of high coordinates not working in rxvt-unicode

[Jared Finder via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On 2023-01-24 4:24 am, Eli Zaretskii wrote:
>> Date: Mon, 23 Jan 2023 22:56:09 -0800
>> From: Jared Finder <jared@finder.org>
>> Cc: git@vladimir.panteleev.md, 61022@debbugs.gnu.org
>> 
>> The change mostly works as inherit-input-method also causes UTF-8
>> decoding to happen deep in read_char at the C level. (Is this
>> intentional? I assume so because read-char just reads single bytes
>> normally.)
> 
> Yes, that's how we decode keyboard input using keyboard-coding-system.
> 
>> However, I think the following change is more appropriate:
>> 
>> -          (read-char nil nil 0.1))
>> +          ;; Read a character with input method conversion enabled
>> +          ;; but no conversion to force read-char to decode UTF-8
>> +          ;; byte sequences.
>> +          (let ((input-method-function nil))
>> +            (read-char nil t 0.1)))
>> 
>> This way we don't apply an actual input method conversion to 
>> characters.
>> For example, without this additional change, if the 'british input
>> method was active, the # ==> £ conversion would
>> happen, causing mouse events with X=2 to instead have X=131.
> 
> OK, but shouldn't we also use INHERIT-INPUT-METHOD = t in the call to
> read-char only when xterm-mouse-utf-8 option is set?  Otherwise, we
> rely on read-char to not perform any conversions, but why rely on that
> if we already know we don't want any conversions in that case?  Using
> nil when xterm-mouse-utf-8 is unset sounds like a more future-proof
> change, no?

I think that's not just future-proof, it's more correct.

   -- MJF

bug#61022: 28.2.50; Mouse tracking of high coordinates not working in rxvt-unicode

[Eli Zaretskii]

> Date: Tue, 24 Jan 2023 21:09:03 -0800
> From: Jared Finder <jared@finder.org>
> Cc: git@vladimir.panteleev.md, 61022@debbugs.gnu.org
> 
> On 2023-01-24 4:24 am, Eli Zaretskii wrote:
> >> -          (read-char nil nil 0.1))
> >> +          ;; Read a character with input method conversion enabled
> >> +          ;; but no conversion to force read-char to decode UTF-8
> >> +          ;; byte sequences.
> >> +          (let ((input-method-function nil))
> >> +            (read-char nil t 0.1)))
> >> 
> >> This way we don't apply an actual input method conversion to 
> >> characters.
> >> For example, without this additional change, if the 'british input
> >> method was active, the # ==> £ conversion would
> >> happen, causing mouse events with X=2 to instead have X=131.
> > 
> > OK, but shouldn't we also use INHERIT-INPUT-METHOD = t in the call to
> > read-char only when xterm-mouse-utf-8 option is set?  Otherwise, we
> > rely on read-char to not perform any conversions, but why rely on that
> > if we already know we don't want any conversions in that case?  Using
> > nil when xterm-mouse-utf-8 is unset sounds like a more future-proof
> > change, no?
> 
> I think that's not just future-proof, it's more correct.

Thanks.  So I've installed such a change on the emacs-29 branch, and
I'm closing this bug.