bug#55119: 27.2; Inferior Python/ M-x run-python problem with Tk on Windows

bug#55119: 27.2; Inferior Python/ M-x run-python problem with Tk on Windows

[Robert Thorpe]

I have a problem with M-x run-python on Windows 10 enterprise.

The python code I'm working on uses Tk.  Unfortunately, after Tk is used
the inferior python no longer responds to inputs.  I see one more
prompt, then nothing at all.

Type the following into an inferior python process.  Or alternatively,
put it in a file and import it.

import tkinter as tk
from tkinter import ttk
window = tk.Tk()

This pops up a Tk window with nothing in it.  It also gives another
python prompt.  But after that communication between Emacs and the
inferior python process seems to be lost.

It seems that someone had the same problem 20 years ago.  I can't find
anyone having this problem recently though.
https://python-list.python.narkive.com/miSHgA8U/problems-with-tkinter-based-scripts-under-emacs

If it helps I'm using Python 3.9.11 64-bit and Tcl/Tk 8.6.

(I am sending this from Emacs 27.2 on Arch but I've copied over the
debug text below from the PC with the issue).

BR,
Robert Thorpe


In GNU Emacs 28.1 (build 52, x86_64-w64-mingw32)
 of 2022-04-04 built on AVALON
Repository revision: 5a223c7f2ef4c31abbd46367b6ea83cd19d30aa7
Repository branch: heads/emacs-28.1
Windowing system distributor 'Microsoft Corp.', version 10.0.19042
System Description: Microsoft Windows 10 Enterprise (v10.0.2009.19042.1586)

Configured using:
 'configure --without-dbus --with-native-compilation
 --without-compress-install CFLAGS=-O2'

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
XPM ZLIB

(NATIVE_COMP present but libgccjit not available)

Important settings:
  value of $LANG: ENI
  locale-coding-system: cp1252

Major mode: Inferior Python

Minor modes in effect:
  compilation-shell-minor-mode: t
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny rfc822 mml mml-sec epa
derived epg rfc6068 epg-config gnus-util rmail rmail-loaddefs mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
mule-util compile text-property-search python tramp-sh tramp
tramp-loaddefs trampver tramp-integration files-x tramp-compat shell
pcomplete parse-time iso8601 time-date format-spec auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
comint ring ansi-color cl-extra seq byte-opt gv bytecomp byte-compile
cconv help-mode novice dired-aux cl-loaddefs cl-lib dired dired-loaddefs
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel dos-w32 ls-lisp disp-table
term/w32-win w32-win w32-vars term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice
button loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads w32notify w32 lcms2 multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 79959 6409)
 (symbols 48 9143 1)
 (strings 32 29136 1864)
 (string-bytes 1 996338)
 (vectors 16 18266)
 (vector-slots 8 255188 8970)
 (floats 8 46 170)
 (intervals 56 562 0)
 (buffers 992 17))

bug#55119: 27.2; Inferior Python/ M-x run-python problem with Tk on Windows

[Eli Zaretskii]

> From: Robert Thorpe <rt@robertthorpeconsulting.com>
> Date: Tue, 26 Apr 2022 05:03:58 +0100
> 
> I have a problem with M-x run-python on Windows 10 enterprise.
> 
> The python code I'm working on uses Tk.  Unfortunately, after Tk is used
> the inferior python no longer responds to inputs.  I see one more
> prompt, then nothing at all.
> 
> Type the following into an inferior python process.  Or alternatively,
> put it in a file and import it.
> 
> import tkinter as tk
> from tkinter import ttk
> window = tk.Tk()
> 
> This pops up a Tk window with nothing in it.  It also gives another
> python prompt.  But after that communication between Emacs and the
> inferior python process seems to be lost.

Unless there's someone here who uses this configuration on Windows,
knows the details, and can dig into the problem, you'll have to tell
more about what the above does, from the technical POV.  It is likely
some snafu with I/O between Emacs and Python when it uses Tcl/Tk, but
what exactly and whether it's at all solvable, is anybody's guess at
this point.  All I can tell you is that Emacs on Windows uses a
bidirectional pipe to communicate with subprocesses, so if the above
fiddles with standard I/O streams in some non-trivial ways, or assumes
it can separate input device from output device, or redirect them
separately to different places, or even that one or both of them are
or can be made unbuffered -- it will simply fail to work on Windows.
You have a single bidirectional pipe which is buffered by the OS, and
at least Python (or any other external program) will see that its
standard I/O is not connected to a console device, and will
(mis)behave accordingly.

The buffering issue you could perhaps deal with, by telling Python to
use unbuffered I/O.  But I'm not even sure this is the root cause of
what you see.

That is all I can tell without more detail, sorry.

bug#55119: 27.2; Inferior Python/ M-x run-python problem with Tk on Windows

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> The buffering issue you could perhaps deal with, by telling Python to
> use unbuffered I/O.  But I'm not even sure this is the root cause of
> what you see.
>
> That is all I can tell without more detail, sorry.

More information was requested, but no response was given within a few
months, so I'm closing this bug report.  If the problem still exists,
please respond to this email and we'll reopen the bug report.

bug#55119: 27.2; Inferior Python/ M-x run-python problem with Tk on Windows

[Robert Thorpe]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> The buffering issue you could perhaps deal with, by telling Python to
>> use unbuffered I/O.  But I'm not even sure this is the root cause of
>> what you see.
>>
>> That is all I can tell without more detail, sorry.
>
> More information was requested, but no response was given within a few
> months, so I'm closing this bug report.  If the problem still exists,
> please respond to this email and we'll reopen the bug report.

I have a little more information.

* 1. Ipython.

This bug does not appear with ipython.  So I have simply switched to
using ipython.

* 2. Tell Python to use Unbuffered IO.

I tried this by setting the PYTHONUNBUFFERED environment variable, it
doesn't help, the same error occurs

* 3. Other Investigation.

A user called Bombastry on Reddit did more digging into this problem.
Bombastry found that it happens with the Qt widget library too.

On the other hand, I don't think that any of this is important enough to
reopen the bug.  I'm really just writing it so that hopefully it appears
in a web-search for anyone who has this problem in the future.

BR,
Rob