From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Augusto Stoffel Newsgroups: gmane.emacs.bugs Subject: bug#49822: [RFC PATCH] python-shell-send functions show no output Date: Sat, 28 Aug 2021 11:28:16 +0200 Message-ID: <87sfytex5b.fsf_-_@gmail.com> References: <855f9eaf1b1d39bb9325d0a88e7f66c0ba0b45d0.camel@gmail.com> <87r1f9fzj4.fsf@gnus.org> <87im0a6789.fsf@gmail.com> <8735rdha6y.fsf@gnus.org> <69e713da-5186-780a-f0b7-8099ce6cde3d@easy-emacs.de> <87h7fsnzmo.fsf@gmail.com> <51fa5c12-53b4-ad92-505f-1896d955b68c@easy-emacs.de> <87k0k8wqyo.fsf@gmail.com> <87h7fc1enm.fsf@gnus.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="17250"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) Cc: 49822@debbugs.gnu.org To: Lars Ingebrigtsen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Aug 28 11:29:10 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mJueE-0004DX-92 for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 28 Aug 2021 11:29:10 +0200 Original-Received: from localhost ([::1]:36870 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mJueC-0002f5-Gf for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 28 Aug 2021 05:29:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:45056) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mJue6-0002ev-CM for bug-gnu-emacs@gnu.org; Sat, 28 Aug 2021 05:29:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:41857) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mJue6-0001Fz-4m for bug-gnu-emacs@gnu.org; Sat, 28 Aug 2021 05:29:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mJue6-0000sh-1x for bug-gnu-emacs@gnu.org; Sat, 28 Aug 2021 05:29:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Augusto Stoffel Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 28 Aug 2021 09:29:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49822 X-GNU-PR-Package: emacs Original-Received: via spool by 49822-submit@debbugs.gnu.org id=B49822.16301429133354 (code B ref 49822); Sat, 28 Aug 2021 09:29:02 +0000 Original-Received: (at 49822) by debbugs.gnu.org; 28 Aug 2021 09:28:33 +0000 Original-Received: from localhost ([127.0.0.1]:53403 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mJudZ-0000ry-VS for submit@debbugs.gnu.org; Sat, 28 Aug 2021 05:28:33 -0400 Original-Received: from mail-wm1-f50.google.com ([209.85.128.50]:41796) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mJudU-0000rf-De for 49822@debbugs.gnu.org; Sat, 28 Aug 2021 05:28:28 -0400 Original-Received: by mail-wm1-f50.google.com with SMTP id c129-20020a1c35870000b02902e6b6135279so5914765wma.0 for <49822@debbugs.gnu.org>; Sat, 28 Aug 2021 02:28:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=tzcuqu9p7uiCs9WIH8VpDAD/fzbMSSHmbUkCEx6ctYo=; b=kTNVjjpeRaKoB0AcOK6xPASBqL8JbsGEu6o0zT/HHV/ymnMZ1KwxOO+Gqm9ubRfUYI xIcR4+pJ6G6HH60PR8nK3mfA3fYY10Gkm+u4Awd4yn2YkCdnnt12WfToDbQDNwcL3Nk2 qlq+m/L4YjJzjzJky3VL8ZtlkXig+hdMbCg3/axyOqO9Dbuuk9m4goP8Jx9IgELO5tpc TuovXZGKuzwmjcY89p88uEn7nVNESRKnKP3oQZXf9zRKZ1zuQeBFLuY2XYPrkYf+rmNP oQX+P+TZaowycVIqObjfjX1Gko3/Sw2kAkyO6m/ILI+CWXXx69a51Y+r7QGdYo5Ypwd8 t82g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=tzcuqu9p7uiCs9WIH8VpDAD/fzbMSSHmbUkCEx6ctYo=; b=kbEi4NwYh6QOsEJR4UakbWXC2oyAYy1VLC+u7bCg12ix6MhPxi5WYlF3552opWtLvH EHrvHzr5tp5/+Iw+s1u+DvZLu5EKfMsR6GijF5SACTwOP2dLDZccUHJmZCAFcXffYqdi ZS/3D5AVLSh7qC2PJsgT1W2SkENHCokqbT1msOJrf/PcOb9v6in3EizgNSTvN8te5j+m Cj56gtb4tUsJYrNRBjsJpbpsSGeKjkXrbgepp3P+PZ5iNU9og3286J4AvfLt9xeh/cgX sFW/QEMphDts0FIy4efSC5NO67v5sVTTeicM8Z9pIxRu9/t5ycy3a5tuRqIXSTWYL9lx rpIw== X-Gm-Message-State: AOAM531lc8MsJL6Sqfrse/V5jU2nG0ZiEJFHNL9AzwjymQelvPgHCe06 L2EmKDDt4y3WHe/ZVRGPulOx4ddkFsI= X-Google-Smtp-Source: ABdhPJyUetolMmyD8FfZfxO2cvExuRUkmgNpk9WobTZNgl/wWax9+OP1JeQ9IhmrtmM6hALP73O1EQ== X-Received: by 2002:a05:600c:3656:: with SMTP id y22mr23160303wmq.58.1630142897924; Sat, 28 Aug 2021 02:28:17 -0700 (PDT) Original-Received: from ars3 ([2a02:8109:8ac0:56d0::ae3f]) by smtp.gmail.com with ESMTPSA id d9sm11043917wrb.36.2021.08.28.02.28.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 28 Aug 2021 02:28:17 -0700 (PDT) In-Reply-To: <87h7fc1enm.fsf@gnus.org> (Lars Ingebrigtsen's message of "Thu, 26 Aug 2021 16:09:17 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:212861 Archived-At: --=-=-= Content-Type: text/plain On Thu, 26 Aug 2021 at 16:09, Lars Ingebrigtsen wrote: > If you could fix this (i.e., not use temporary files), that would be > very nice. All right, I'm attaching a proposal for 'python-shell-send-string' (affecting also all other 'python-shell-send-' functions except 'python-shell-send-buffer'). The main user-visible change is that now a result is printed in the shell in all possible cases, i.e.: 1. If the string being evaluated is a one-line expression. 2. If the string being evaluated is a multiline expression such as "(1+\n1)". 3. The string being evaluated is contains multiple statements, the last of which is an expression. E.g., "x=1\nx". Currently, only case 1 prints a result (this is the actual topic of this bug). By avoiding to use 'python-shell-send-file', the following additional issues are fixed: 1. The plumbing code used to execute the user's code could crop up in a stack trace (Bug#32042). This is now almost completely fixed (the only exception I see is in the plain python3 interpreter, where a not-too-annoying mention to __PYTHON_EL_eval still appears.) 2. Sending temporary files over Tramp has a noticeable delay (and lots of echo area messages). In particular, completion is now faster on remote Python shells. 3. Some global symbols (os, codecs, etc.) used to be redefined; some others (compile, etc.) used to be assumed to not have been redefined by the user. We now only assume this is the case for 'exec' (because it's a keyword in Python 2). Note, however, that 'python-shell-send-file' remains unchanged. This is a reasonably sensitive change and it would be good to collect feedback from others before merging it (a properly formatted patch will then follow). In particular, since I'm not an expert in Python's internals, there might be some possible refinements to the __PYTHON_EL_eval function. Is this perhaps a better topic for the emacs-devel list? --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-Improve-python-shell-send-functions.patch >From 10b474c9f82f13b3ee9c451a6aea0d3660041095 Mon Sep 17 00:00:00 2001 From: Augusto Stoffel Date: Sat, 28 Aug 2021 11:16:40 +0200 Subject: [PATCH] Improve python-shell-send functions --- lisp/progmodes/python.el | 64 +++++++++++++++++++++++++++++++--------- 1 file changed, 50 insertions(+), 14 deletions(-) diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el index d5209d8d2f..bed709bb65 100644 --- a/lisp/progmodes/python.el +++ b/lisp/progmodes/python.el @@ -3081,6 +3081,45 @@ python-shell--save-temp-file (delete-trailing-whitespace)) temp-file-name)) +(defvar python-shell-eval-setup-code + "\ +def __PYTHON_EL_eval(source, filename): + import ast, sys + if sys.version_info[0] == 2: + from __builtin__ import compile, eval, globals + else: + from builtins import compile, eval, globals + sys.stdout.write('\\n') + try: + p, e = ast.parse(source, filename), None + except SyntaxError: + t, v, tb = sys.exc_info() + sys.excepthook(t, v, tb.tb_next) + return + if p.body and isinstance(p.body[-1], ast.Expr): + e = p.body.pop() + try: + g = globals() + exec(compile(p, filename, 'exec'), g, g) + if e: + return eval(compile(ast.Expression(e.value), filename, 'eval'), g, g) + except Exception: + t, v, tb = sys.exc_info() + sys.excepthook(t, v, tb.tb_next)" + "Code used to evaluate statements in inferior Python processes.") + +(defalias 'python-shell--encode-string + (let ((fun (if (and (fboundp 'json-serialize) + (>= emacs-major-version 28)) + 'json-serialize + (require 'json) + 'json-encode-string))) + (lambda (text) + (if (stringp text) + (funcall fun text) + (signal 'wrong-type-argument (list 'stringp text))))) + "Encode TEXT as a valid Python string.") + (defun python-shell-send-string (string &optional process msg) "Send STRING to inferior Python PROCESS. When optional argument MSG is non-nil, forces display of a @@ -3088,16 +3127,12 @@ python-shell-send-string t when called interactively." (interactive (list (read-string "Python command: ") nil t)) - (let ((process (or process (python-shell-get-process-or-error msg)))) - (if (string-match ".\n+." string) ;Multiline. - (let* ((temp-file-name (with-current-buffer (process-buffer process) - (python-shell--save-temp-file string))) - (file-name (or (buffer-file-name) temp-file-name))) - (python-shell-send-file file-name process temp-file-name t)) - (comint-send-string process string) - (when (or (not (string-match "\n\\'" string)) - (string-match "\n[ \t].*\n?\\'" string)) - (comint-send-string process "\n"))))) + (comint-send-string + (or process (python-shell-get-process-or-error msg)) + (format "exec(%s);__PYTHON_EL_eval(%s, %s)\n" + (python-shell--encode-string python-shell-eval-setup-code) + (python-shell--encode-string string) + (python-shell--encode-string (or (buffer-file-name) ""))))) (defvar python-shell-output-filter-in-progress nil) (defvar python-shell-output-filter-buffer nil) @@ -3139,7 +3174,8 @@ python-shell-send-string-no-output (inhibit-quit t)) (or (with-local-quit - (python-shell-send-string string process) + (comint-send-string process (format "exec(%s)\n" + (python-shell--encode-string string))) (while python-shell-output-filter-in-progress ;; `python-shell-output-filter' takes care of setting ;; `python-shell-output-filter-in-progress' to NIL after it @@ -3362,7 +3398,8 @@ python-shell-send-file (temp-file-name (when temp-file-name (file-local-name (expand-file-name temp-file-name))))) - (python-shell-send-string + (comint-send-string + process (format (concat "import codecs, os;" @@ -3372,8 +3409,7 @@ python-shell-send-file (when (and delete temp-file-name) (format "os.remove('''%s''');" temp-file-name)) "exec(compile(__code, '''%s''', 'exec'));") - (or temp-file-name file-name) encoding encoding file-name) - process))) + (or temp-file-name file-name) encoding encoding file-name)))) (defun python-shell-switch-to-shell (&optional msg) "Switch to inferior Python process buffer. -- 2.31.1 --=-=-=--