From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Augusto Stoffel Newsgroups: gmane.emacs.bugs Subject: bug#49822: master e32c7d2: Change Python eval to send directly instead of using temporary files Date: Wed, 08 Sep 2021 16:05:11 +0200 Message-ID: <87zgsnp3hk.fsf__46555.3250674688$1631110070$gmane$org@gmail.com> References: <20210903122828.16890.65271@vcs0.savannah.gnu.org> <20210903122829.EAAC220B71@vcs0.savannah.gnu.org> <87r1e4eklw.fsf@gmail.com> <87v93fnfmm.fsf@igel.home> <87lf4aooge.fsf@gmail.com> <878s0a409g.fsf@gmx.de> <87ilzef65i.fsf@igel.home> <878s0aeyna.fsf@gmx.de> <87r1e1ex7z.fsf@igel.home> <875yvcmgm2.fsf@gmail.com> <877dfr5ww1.fsf@gnus.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="12518"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: Andy Moreton , "Barton, Mark" , emacs-devel@gnu.org, Andreas Schwab , Stefan Monnier , 49822@debbugs.gnu.org, Michael Albinus To: Lars Ingebrigtsen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Sep 08 16:07:42 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mNyEn-00033S-N9 for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 08 Sep 2021 16:07:41 +0200 Original-Received: from localhost ([::1]:55092 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mNyEm-0003VE-KO for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 08 Sep 2021 10:07:40 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:40826) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mNyDC-0002Nw-Kd for bug-gnu-emacs@gnu.org; Wed, 08 Sep 2021 10:06:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:49903) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mNyDC-0003bv-Cs for bug-gnu-emacs@gnu.org; Wed, 08 Sep 2021 10:06:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mNyDC-0006gh-5K for bug-gnu-emacs@gnu.org; Wed, 08 Sep 2021 10:06:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Augusto Stoffel Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 08 Sep 2021 14:06:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49822 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 49822-submit@debbugs.gnu.org id=B49822.163110992525662 (code B ref 49822); Wed, 08 Sep 2021 14:06:02 +0000 Original-Received: (at 49822) by debbugs.gnu.org; 8 Sep 2021 14:05:25 +0000 Original-Received: from localhost ([127.0.0.1]:33216 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mNyCa-0006fq-Ub for submit@debbugs.gnu.org; Wed, 08 Sep 2021 10:05:25 -0400 Original-Received: from mail-wr1-f44.google.com ([209.85.221.44]:43633) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mNyCV-0006fZ-Dj for 49822@debbugs.gnu.org; Wed, 08 Sep 2021 10:05:23 -0400 Original-Received: by mail-wr1-f44.google.com with SMTP id b6so3463960wrh.10 for <49822@debbugs.gnu.org>; Wed, 08 Sep 2021 07:05:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=sYW7XlJZ7MLuf3aqanS/27POetbwdkpqv5vsvdK7huI=; b=DezgbP9CLH7FSCc2RvpiPXy0813pUsndrhiMrbL6mGVhdPCjCs5HubjbPDkRVUBTBH rSEwbaD/zaNVtPGJgfTrQnt7L7SvwdrRA/f2ZSPyOOVupaGkS1z24FVpPVJJaBLylpMf ZUXaOqIL4d/p09Z6WCPSJGksBk21/hGTssI6S+yEPL0YpCcpGZlYU0MCy9Xda2VrwAbf 3JX3rSV/qrxyC8j7eki4Xy8ggpHEw/kBJ3auMANvqgeKSupus2DR+lAbR2vW1hQChjxm Cd4a5LT8cRFiKwT+HvNvHYeDm907Wdo8bbiWZtxX2x7ut5AZCh4pL73NDwofwbZQ/Ebq IovQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=sYW7XlJZ7MLuf3aqanS/27POetbwdkpqv5vsvdK7huI=; b=3EZzLU9AVTiyjfpuQmQdtPY5e0ZfBZzciNb9kZqH8i2BwSIuu4VVUY3DMTsqI4EPRF M3c/A6zqlSHTEfpaCl8iGvGQM5h/iUTlFNHfPcKJIOdu5cZ2iMTZb3LUP2ZcsRVR/Pf0 RXf9vgd1GFboHz3DBdkhbcrFpFFJqKsQyQM9gmOE6bDz//uCr4kv2aYwn3gUQ/hgt6jR c4IvLgLxoO/TGp34xSZtF+MgK/j4bIahFCaTn4dB8ovQ8eO47kBeXBqIWncTLm3f690D HK5FCY85yifBWLBJXrj5SwriR2KvxwDEsbDsND2yE+Fjg0ZdZhcvJBh4yP0ecmfctNeV t/UA== X-Gm-Message-State: AOAM532fxSskoNayJtPjaFZmF8mPJ/Amc4AUNBWbfPZXHfH2uWvn5geF A7IX56FSEdpW67tlg1sLcFc= X-Google-Smtp-Source: ABdhPJwhZQMopoSaLu2iqLVXV8YGXHj11mb7l3CrlmzQS8phCrKTSbDkGR9hHlrmWP5ueiPtzBdjtw== X-Received: by 2002:adf:de8f:: with SMTP id w15mr4409036wrl.277.1631109913518; Wed, 08 Sep 2021 07:05:13 -0700 (PDT) Original-Received: from ars3 ([2a02:8109:8ac0:56d0::b1d]) by smtp.gmail.com with ESMTPSA id i11sm2292249wrb.48.2021.09.08.07.05.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 07:05:12 -0700 (PDT) In-Reply-To: <877dfr5ww1.fsf@gnus.org> (Lars Ingebrigtsen's message of "Wed, 08 Sep 2021 09:50:22 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:213835 Archived-At: --=-=-= Content-Type: text/plain On Wed, 8 Sep 2021 at 09:50, Lars Ingebrigtsen wrote: > Augusto Stoffel writes: > >> I have also rearranged things a bit so that the setup code is sent to >> the inferior process just once, rather than of on every call to >> `python-shell-send-string'. This way, the smaller line length limit >> doesn't increase too much the use of temp files, which, as I mentioned, >> is slow over ssh. > > This leads to compilation warnings about > python-shell-eval-file-setup-code etc. > > Can you adjust the patch to fix that, and also add the defconst for > 1024, like Eli discussed? > >> PS: I have some more suggestions around the Python shell. Is the ideal >> workflow to keep creating bugs with a small patch to each improvement, >> or do you prefer to review a larger collection of changes bundled >> together? > > Smaller patches are fine by me, and you can submit them with `M-x > submit-emacs-patch'. Hi Lars, I've attached a new patch addressing these 2 points. I've added the max line length defconst to comint.el, since this something general about PTYs. The value reflects my knowledge of the situation (4096 on Linux, possibly 1024 on other OSes). Hopefully other people will fill in more cases eventually. FWIW, when I read the man page for termios, it seems pretty clear that MAX_CANON is the name of this parameter we're discussing here. But if I do 'getconf MAX_CANON /', I get 255, which is definitely not the right value for my system. --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Better-treatment-of-line-length-limits-for-the-Pytho.patch >From 3eab275464f4a53bf86153f5f3693e522a240b28 Mon Sep 17 00:00:00 2001 From: Augusto Stoffel Date: Mon, 6 Sep 2021 23:34:48 +0200 Subject: [PATCH] Better treatment of line length limits for the Python inferior * lisp/comint.el (comint-max-line-length): new constant reflecting a safe maximum line size that can be sent to an inferior process. * lisp/progmodes/python.el (python-shell-comint-watch-for-first-prompt-output-filter): Send setup code to the inferior process only once and at this stage. (python-shell-eval-setup-code, python-shell-eval-file-setup-code): Move, unchanged, to an earlier point to avoid byte-compiler warnings. (python-shell-send-string-no-output): Revert changes of e32c7d2a8d (python-shell-send-string): Use 'comint-max-line-length' to decide when to resort to temp files. (python-shell-send-string, python-shell-send-file): Don't send setup code each time. --- lisp/comint.el | 6 +++ lisp/progmodes/python.el | 98 +++++++++++++++++++++------------------- 2 files changed, 57 insertions(+), 47 deletions(-) diff --git a/lisp/comint.el b/lisp/comint.el index e058e6b8cf..c2e528a5d5 100644 --- a/lisp/comint.el +++ b/lisp/comint.el @@ -479,6 +479,12 @@ comint-terminfo-terminal :group 'comint :version "26.1") +(defconst comint-max-line-length + (pcase system-type + ('gnu/linux 4096) + (_ 1024)) + "Maximum line length, in bytes, accepted by the inferior process.") + (defvar comint-mode-map (let ((map (make-sparse-keymap))) ;; Keys: diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el index d8ec032402..4f222b4cf5 100644 --- a/lisp/progmodes/python.el +++ b/lisp/progmodes/python.el @@ -2811,6 +2811,44 @@ python-shell-first-prompt-hook :type 'hook :group 'python) +(defconst python-shell-eval-setup-code + "\ +def __PYTHON_EL_eval(source, filename): + import ast, sys + if sys.version_info[0] == 2: + from __builtin__ import compile, eval, globals + else: + from builtins import compile, eval, globals + sys.stdout.write('\\n') + try: + p, e = ast.parse(source, filename), None + except SyntaxError: + t, v, tb = sys.exc_info() + sys.excepthook(t, v, tb.tb_next) + return + if p.body and isinstance(p.body[-1], ast.Expr): + e = p.body.pop() + try: + g = globals() + exec(compile(p, filename, 'exec'), g, g) + if e: + return eval(compile(ast.Expression(e.value), filename, 'eval'), g, g) + except Exception: + t, v, tb = sys.exc_info() + sys.excepthook(t, v, tb.tb_next)" + "Code used to evaluate statements in inferior Python processes.") + +(defconst python-shell-eval-file-setup-code + "\ +def __PYTHON_EL_eval_file(filename, tempname, encoding, delete): + import codecs, os + with codecs.open(tempname or filename, encoding=encoding) as file: + source = file.read().encode(encoding) + if delete and tempname: + os.remove(tempname) + return __PYTHON_EL_eval(source, filename)" + "Code used to evaluate files in inferior Python processes.") + (defun python-shell-comint-watch-for-first-prompt-output-filter (output) "Run `python-shell-first-prompt-hook' when first prompt is found in OUTPUT." (when (not python-shell--first-prompt-received) @@ -2826,6 +2864,15 @@ python-shell-comint-watch-for-first-prompt-output-filter (setq python-shell--first-prompt-received-output-buffer nil) (setq-local python-shell--first-prompt-received t) (setq python-shell--first-prompt-received-output-buffer nil) + (cl-letf (((symbol-function 'python-shell-send-string) + (lambda (string process) + (comint-send-string + process + (format "exec(%s)\n" (python-shell--encode-string string)))))) + ;; Bootstrap: the normal definition of `python-shell-send-string' + ;; depends on the Python code sent here. + (python-shell-send-string-no-output python-shell-eval-setup-code) + (python-shell-send-string-no-output python-shell-eval-file-setup-code)) (with-current-buffer (current-buffer) (let ((inhibit-quit nil)) (run-hooks 'python-shell-first-prompt-hook)))))) @@ -3081,33 +3128,6 @@ python-shell--save-temp-file (delete-trailing-whitespace)) temp-file-name)) -(defconst python-shell-eval-setup-code - "\ -def __PYTHON_EL_eval(source, filename): - import ast, sys - if sys.version_info[0] == 2: - from __builtin__ import compile, eval, globals - else: - from builtins import compile, eval, globals - sys.stdout.write('\\n') - try: - p, e = ast.parse(source, filename), None - except SyntaxError: - t, v, tb = sys.exc_info() - sys.excepthook(t, v, tb.tb_next) - return - if p.body and isinstance(p.body[-1], ast.Expr): - e = p.body.pop() - try: - g = globals() - exec(compile(p, filename, 'exec'), g, g) - if e: - return eval(compile(ast.Expression(e.value), filename, 'eval'), g, g) - except Exception: - t, v, tb = sys.exc_info() - sys.excepthook(t, v, tb.tb_next)" - "Code used to evaluate statements in inferior Python processes.") - (defalias 'python-shell--encode-string (let ((fun (if (and (fboundp 'json-serialize) (>= emacs-major-version 28)) @@ -3128,12 +3148,11 @@ python-shell-send-string (interactive (list (read-string "Python command: ") nil t)) (let ((process (or process (python-shell-get-process-or-error msg))) - (code (format "exec(%s);__PYTHON_EL_eval(%s, %s)\n" - (python-shell--encode-string python-shell-eval-setup-code) + (code (format "__PYTHON_EL_eval(%s, %s)\n" (python-shell--encode-string string) (python-shell--encode-string (or (buffer-file-name) ""))))) - (if (<= (string-bytes code) 4096) + (if (<= (string-bytes code) comint-max-line-length) (comint-send-string process code) (let* ((temp-file-name (with-current-buffer (process-buffer process) (python-shell--save-temp-file string))) @@ -3180,8 +3199,7 @@ python-shell-send-string-no-output (inhibit-quit t)) (or (with-local-quit - (comint-send-string - process (format "exec(%s)\n" (python-shell--encode-string string))) + (python-shell-send-string string process) (while python-shell-output-filter-in-progress ;; `python-shell-output-filter' takes care of setting ;; `python-shell-output-filter-in-progress' to NIL after it @@ -3378,18 +3396,6 @@ python-shell-send-defun nil ;; noop msg)))) - -(defconst python-shell-eval-file-setup-code - "\ -def __PYTHON_EL_eval_file(filename, tempname, encoding, delete): - import codecs, os - with codecs.open(tempname or filename, encoding=encoding) as file: - source = file.read().encode(encoding) - if delete and tempname: - os.remove(tempname) - return __PYTHON_EL_eval(source, filename)" - "Code used to evaluate files in inferior Python processes.") - (defun python-shell-send-file (file-name &optional process temp-file-name delete msg) "Send FILE-NAME to inferior Python PROCESS. @@ -3419,9 +3425,7 @@ python-shell-send-file (comint-send-string process (format - "exec(%s);exec(%s);__PYTHON_EL_eval_file(%s, %s, %s, %s)\n" - (python-shell--encode-string python-shell-eval-setup-code) - (python-shell--encode-string python-shell-eval-file-setup-code) + "__PYTHON_EL_eval_file(%s, %s, %s, %s)\n" (python-shell--encode-string file-name) (python-shell--encode-string (or temp-file-name "")) (python-shell--encode-string (symbol-name encoding)) -- 2.31.1 --=-=-=--