bug#53219: 27.2; shell-mode output is stolen by python-eldoc

bug#53219: 27.2; shell-mode output is stolen by python-eldoc

[awrhygty]

Evaluate the form below and type 'C-p' or 'C-n' repeatedly to display
python eldoc messages.
After 20-30 key input, an error occurs.
Then look at the *shell* buffer, some numbers in output lines are
absent.
If setting debug-on-error to 't, *Backtrace* buffer popups,
but the buffer and its window is deleted with any key input
and I cannot copy text in the buffer.
The command in the shell-mode buffer need not be python command.
'ping -n 200 localhost' will be alternative, but its output lines are
perhaps identical and difficult to know about absent lines.

(let ((buf (generate-new-buffer "tmp")))
  (shell)
  (insert "python -c \"import time;[(print('output',i,flush=True),time.sleep(1)) for i in range(200)]\"")
  (comint-send-input)
  (run-python)
  (switch-to-buffer buf)
  (insert "len\nstr")
  (python-mode))

Here is a copied text from *Message* buffer.

error in process filter: replace-regexp-in-string: Wrong type argument: arrayp, nil
error in process filter: Wrong type argument: arrayp, nil
error in process filter: replace-regexp-in-string: Wrong type argument: arrayp, nil
error in process filter: Wrong type argument: arrayp, nil
ElDoc echo-area display muted in this buffer, see ‘python-eldoc-function’
error in process filter: replace-regexp-in-string: Wrong type argument: arrayp, nil
error in process filter: Wrong type argument: arrayp, nil
error in process filter: replace-regexp-in-string: Wrong type argument: arrayp, nil
error in process filter: Wrong type argument: arrayp, nil
ElDoc echo-area display muted in this buffer, see ‘python-eldoc-function’



In GNU Emacs 27.2 (build 1, x86_64-w64-mingw32)
 of 2021-03-26 built on CIRROCUMULUS
Repository revision: deef5efafb70f4b171265b896505b92b6eef24e6
Repository branch: HEAD
Windowing system distributor 'Microsoft Corp.', version 10.0.19043
System Description: Microsoft Windows 10 Pro (v10.0.2009.19043.1415)

Recent messages:

Configured using:
 'configure --without-dbus --host=x86_64-w64-mingw32
 --without-compress-install 'CFLAGS=-O2 -static''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY W32NOTIFY ACL GNUTLS LIBXML2
HARFBUZZ ZLIB TOOLKIT_SCROLL_BARS MODULES THREADS JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: JPN
  locale-coding-system: cp932

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(thingatpt compile cl-extra python tramp-sh tramp tramp-loaddefs
trampver tramp-integration files-x tramp-compat parse-time iso8601 shell
pcomplete comint ansi-color ring misearch multi-isearch gnutls
network-stream nsm mailalias smtpmail auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs json map tabify help-mode pp shadow
sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs text-property-search time-date
subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs
cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
term/bobcat japan-util tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win
w32-win w32-vars term/common-win tool-bar dnd fontset image regexp-opt
fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads w32notify w32 lcms2 multi-tty make-network-process
emacs)

Memory information:
((conses 16 91541 12697)
 (symbols 48 9330 0)
 (strings 32 29585 1136)
 (string-bytes 1 1056848)
 (vectors 16 16818)
 (vector-slots 8 280556 15602)
 (floats 8 50 279)
 (intervals 56 1609 0)
 (buffers 1000 21))

bug#53219: 27.2; shell-mode output is stolen by python-eldoc

[Lars Ingebrigtsen]

awrhygty@outlook.com writes:

> Evaluate the form below and type 'C-p' or 'C-n' repeatedly to display
> python eldoc messages.
> After 20-30 key input, an error occurs.
> Then look at the *shell* buffer, some numbers in output lines are
> absent.
> If setting debug-on-error to 't, *Backtrace* buffer popups,

That sounds familiar -- I think I recall something like this being fixed
in eldoc...

Yes, I can reproduce this problem in Emacs 27.2, but not in Emacs 29.
So I guess this has been fixed by now, and I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#53219: 27.2; shell-mode output is stolen by python-eldoc

[miha--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1.1: Type: text/plain, Size: 1091 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> awrhygty@outlook.com writes:
>
>> Evaluate the form below and type 'C-p' or 'C-n' repeatedly to display
>> python eldoc messages.
>> After 20-30 key input, an error occurs.
>> Then look at the *shell* buffer, some numbers in output lines are
>> absent.
>> If setting debug-on-error to 't, *Backtrace* buffer popups,
>
> That sounds familiar -- I think I recall something like this being fixed
> in eldoc...
>
> Yes, I can reproduce this problem in Emacs 27.2, but not in Emacs 29.
> So I guess this has been fixed by now, and I'm closing this bug report.

Not quite yet. I found a more reliable reproducer for Emacs 29, which
could reproduce the bug on my machine with emacs -q.

(progn
  (setq debug-on-error t)
  (shell)
  (insert " i=0; while sleep 0.05; do i=$((i + 1)); echo $i; done")
  (comint-send-input)
  (run-python)
  (switch-to-buffer (generate-new-buffer "tmp"))
  (insert "len\nstr")
  (python-mode))

If it doesn't work bug immediately, try lowering the "0.05" number.

Please consider applying the following patch to fix this.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-Fix-python-eldoc-affecting-unrelated-comint-processe.patch --]
[-- Type: text/x-patch, Size: 1624 bytes --]

From 45c345f0b170c3ffc41f186b3c2b70dbccaa23fd Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miha=20Rihtar=C5=A1i=C4=8D?= <miha@kamnitnik.top>
Date: Thu, 13 Jan 2022 18:20:13 +0100
Subject: [PATCH] Fix python-eldoc affecting unrelated comint processes

* lisp/progmodes/python.el (python-shell-send-string-no-output): Don't
let-bind comint-preoutput-filter-functions globally for all comint
processes.  Modify the behaviour of only the current python
process (bug#53219).
---
 lisp/progmodes/python.el | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index edd3139a7a..5889f2ab67 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -3216,11 +3216,13 @@ python-shell-output-filter
 (defun python-shell-send-string-no-output (string &optional process)
   "Send STRING to PROCESS and inhibit output.
 Return the output."
-  (let ((process (or process (python-shell-get-process-or-error)))
-        (comint-preoutput-filter-functions
-         '(python-shell-output-filter))
-        (python-shell-output-filter-in-progress t)
-        (inhibit-quit t))
+  (or process (setq process (python-shell-get-process-or-error)))
+  (cl-letf (((process-filter process)
+             (lambda (_proc str)
+               (with-current-buffer (process-buffer process)
+                 (python-shell-output-filter str))))
+            (python-shell-output-filter-in-progress t)
+            (inhibit-quit t))
     (or
      (with-local-quit
        (python-shell-send-string string process)
-- 
2.34.1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

bug#53219: 27.2; shell-mode output is stolen by python-eldoc

[Lars Ingebrigtsen]

miha@kamnitnik.top writes:

> Please consider applying the following patch to fix this.

Thanks; applied to Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no