bug#49822: master e32c7d2: Change Python eval to send directly instead of using temporary files

[Augusto Stoffel <arstoffel@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1628 bytes --]

On Mon,  6 Sep 2021 at 13:53, Andreas Schwab <schwab@linux-m68k.org> wrote:

> On Sep 06 2021, Michael Albinus wrote:
>
>> Andreas Schwab <schwab@linux-m68k.org> writes:
>>
>>>>> Okay, so is the TTY line length limit of the OS available in Lisp
>>>>> somewhere?  Otherwise, can we at least trust that 1024 is the universal
>>>>> lower bound?
>>>>
>>>> On systems with a POSIX shell:
>>>>
>>>> (let ((default-directory "/"))
>>>>   (read (shell-command-to-string "getconf LINE_MAX")))
>>>
>>> LINE_MAX has nothing to do with the tty line discipline.
>>
>> I'll never learn it :-(
>>
>> We've discussed this issue some years ago already. IIRC, the outcome was
>> to use "getconf PIPE_BUF /" instead.
>
> Neither has PIPE_BUF.
>
> Andreas.

Okay then. Since there seem to be no better alternatives, I have
attached a new patch reducing the limit to a hard-coded 1024 bytes.  If
some day someone adds a variable specifying a more precise limit, then
we can change this.

I have also rearranged things a bit so that the setup code is sent to
the inferior process just once, rather than of on every call to
`python-shell-send-string'.  This way, the smaller line length limit
doesn't increase too much the use of temp files, which, as I mentioned,
is slow over ssh.

It would be great if someone with access to a BSD-like OS could test
this.  I can only test locally on Linux and over ssh between Linux
machines.

PS: I have some more suggestions around the Python shell.  Is the ideal
workflow to keep creating bugs with a small patch to each improvement,
or do you prefer to review a larger collection of changes bundled
together?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Reduce-limit-of-line-length-sent-to-the-Python-infer.patch --]
[-- Type: text/x-patch, Size: 3680 bytes --]

From 05947ad82219af3f4bcb228f076995f181a7255d Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Mon, 6 Sep 2021 23:34:48 +0200
Subject: [PATCH] Reduce limit of line length sent to the Python inferior

* lisp/progmodes/python.el
(python-shell-comint-watch-for-first-prompt-output-filter): Send setup
code to the inferior process only once and at this stage.
(python-shell-send-string-no-output): Revert changes of e32c7d2a8d
(python-shell-send-string): Assume a smaller TTY line length limit,
as required by some OSes.
(python-shell-send-file): Ditto.
---
 lisp/progmodes/python.el | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index d8ec032402..347c3ffe00 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -2826,6 +2826,15 @@ python-shell-comint-watch-for-first-prompt-output-filter
           (setq python-shell--first-prompt-received-output-buffer nil)
         (setq-local python-shell--first-prompt-received t)
         (setq python-shell--first-prompt-received-output-buffer nil)
+        (cl-letf (((symbol-function 'python-shell-send-string)
+                   (lambda (string process)
+                     (comint-send-string
+                      process
+                      (format "exec(%s)\n" (python-shell--encode-string string))))))
+          ;; Bootstrap: the normal definition of `python-shell-send-string'
+          ;; depends on the Python code sent here.
+          (python-shell-send-string-no-output python-shell-eval-setup-code)
+          (python-shell-send-string-no-output python-shell-eval-file-setup-code))
         (with-current-buffer (current-buffer)
           (let ((inhibit-quit nil))
             (run-hooks 'python-shell-first-prompt-hook))))))
@@ -3128,12 +3137,11 @@ python-shell-send-string
   (interactive
    (list (read-string "Python command: ") nil t))
   (let ((process (or process (python-shell-get-process-or-error msg)))
-        (code (format "exec(%s);__PYTHON_EL_eval(%s, %s)\n"
-                      (python-shell--encode-string python-shell-eval-setup-code)
+        (code (format "__PYTHON_EL_eval(%s, %s)\n"
                       (python-shell--encode-string string)
                       (python-shell--encode-string (or (buffer-file-name)
                                                        "<string>")))))
-    (if (<= (string-bytes code) 4096)
+    (if (<= (string-bytes code) 1024)
         (comint-send-string process code)
       (let* ((temp-file-name (with-current-buffer (process-buffer process)
                                (python-shell--save-temp-file string)))
@@ -3180,8 +3188,7 @@ python-shell-send-string-no-output
         (inhibit-quit t))
     (or
      (with-local-quit
-       (comint-send-string
-        process (format "exec(%s)\n" (python-shell--encode-string string)))
+       (python-shell-send-string string process)
        (while python-shell-output-filter-in-progress
          ;; `python-shell-output-filter' takes care of setting
          ;; `python-shell-output-filter-in-progress' to NIL after it
@@ -3419,9 +3426,7 @@ python-shell-send-file
     (comint-send-string
      process
      (format
-      "exec(%s);exec(%s);__PYTHON_EL_eval_file(%s, %s, %s, %s)\n"
-      (python-shell--encode-string python-shell-eval-setup-code)
-      (python-shell--encode-string python-shell-eval-file-setup-code)
+      "__PYTHON_EL_eval_file(%s, %s, %s, %s)\n"
       (python-shell--encode-string file-name)
       (python-shell--encode-string (or temp-file-name ""))
       (python-shell--encode-string (symbol-name encoding))
-- 
2.31.1