From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: kobarity Newsgroups: gmane.emacs.bugs Subject: bug#71440: Python Inferior Mode =?UTF-8?Q?Can=E2=80=99t?= Recognize My Prompt Date: Wed, 12 Jun 2024 01:24:39 +0900 Message-ID: References: <86sexnbckv.fsf@gnu.org> <86r0d7bawc.fsf@gnu.org> <86ikyjb7ky.fsf@gnu.org> <86h6e2buul.fsf@gnu.org> <8634pmbefs.fsf@gnu.org> <86y17e9y5p.fsf@gnu.org> Mime-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: multipart/mixed; boundary="Multipart_Wed_Jun_12_01:24:39_2024-1" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33761"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?Q?Goj=C5=8D?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.0.50 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) Cc: 71440@debbugs.gnu.org To: "shynur ." , Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Jun 11 18:26:18 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sH4K9-0008aX-Kb for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 11 Jun 2024 18:26:17 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sH4Jy-0005ay-O4; Tue, 11 Jun 2024 12:26:07 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sH4Jw-0005XZ-Ob for bug-gnu-emacs@gnu.org; Tue, 11 Jun 2024 12:26:04 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sH4Jw-000271-0x for bug-gnu-emacs@gnu.org; Tue, 11 Jun 2024 12:26:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sH4Ju-0001l1-BU for bug-gnu-emacs@gnu.org; Tue, 11 Jun 2024 12:26:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: kobarity Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 11 Jun 2024 16:26:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 71440 X-GNU-PR-Package: emacs Original-Received: via spool by 71440-submit@debbugs.gnu.org id=B71440.17181231516725 (code B ref 71440); Tue, 11 Jun 2024 16:26:02 +0000 Original-Received: (at 71440) by debbugs.gnu.org; 11 Jun 2024 16:25:51 +0000 Original-Received: from localhost ([127.0.0.1]:35996 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sH4Ji-0001kP-Gv for submit@debbugs.gnu.org; Tue, 11 Jun 2024 12:25:50 -0400 Original-Received: from mail-pl1-f182.google.com ([209.85.214.182]:51348) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sH4Jf-0001jw-UF for 71440@debbugs.gnu.org; Tue, 11 Jun 2024 12:25:48 -0400 Original-Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1f6da06ba24so34374685ad.2 for <71440@debbugs.gnu.org>; Tue, 11 Jun 2024 09:25:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718123084; x=1718727884; darn=debbugs.gnu.org; h=mime-version:user-agent:references:in-reply-to:subject:cc:to:from :message-id:date:from:to:cc:subject:date:message-id:reply-to; bh=3eE5jTxTs3jV0xpx2/PLHrab4sfWyBOnInmjOOHNPpQ=; b=AtGk3YdCDSZJ7MsHjbgBqiT+CS3Mv1RHrAtFiwK/xJowDy0+sVaXL4a42bNGplfonD i18NyGQglPRAeeggJwOKIDlfbe2UZgCoCULxM67irYCacykerlX9iINvWRWYnY9PN5Gj qNQtyKZ9KNH+o5hFdNxjG/rOh4lNGe7ufb1HZqF+FKJfT/zCwPVBKwEpliRdocC1Sd6l Ex44iXsIKgzUNMOrsKDlDZlEd10ydSDrgLBAnkZ1llEFWay/jX4VkhwHN4v7rbQXkJPb a0QNmVCTIuW1zj5bNrZTGiCv5cKnB/BVOynBDBNQW2bXBgnp04HpmYBilJkOk9j4HjHR mgvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718123084; x=1718727884; h=mime-version:user-agent:references:in-reply-to:subject:cc:to:from :message-id:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=3eE5jTxTs3jV0xpx2/PLHrab4sfWyBOnInmjOOHNPpQ=; b=PutZwDq1EcfnsfXiH/4mUTRcTJcsQEdaxY68eE4RZduUSZJ+zJjrL4eTD5OuXdoBy9 4xRoCMS8mAf3fUAJpKn+z6sO9Feq2NfA2awaX2lpusWE+FvZ/+pGzOKPhrDvZu3wKKxH nR5zZ+69jP+Mff40W3WpSiyWqN9mR89twvzYCosTwkpyYrUTApss53BRQnTXf3OwM4UN SUJZfzdx08AUPcktM54jDuOnpGgV1OAvYTCF0KW1H52Huk704FFnuZBB6fzznyMJUC7B fJT43C+HyGzdmuUqJVtux47PpIaHm7BvW2pV5umKoq3VGqFl5MRyno/PvOHJH1ELdkXH 8rag== X-Gm-Message-State: AOJu0YyiQPbZd+zNz3ti2I0ESy6CfQDFrUOfsST2wV1oDbYOjYPWkGAp G92wJ6lzBIfWkjCbhHxtwF+Rq0/HJyG/y6+cpvNdmSzYfZOTzQDH X-Google-Smtp-Source: AGHT+IGgh9kXx3Kf+TZYVYkRWjtf8zM+sBfpOTlsjNwHqhEp7zPziiIlhWuU3sF8yKkIXL69WUqs9w== X-Received: by 2002:a17:902:ec8b:b0:1f7:124:b820 with SMTP id d9443c01a7336-1f70124ba4bmr82335355ad.50.1718123083593; Tue, 11 Jun 2024 09:24:43 -0700 (PDT) Original-Received: from localhost (58x12x133x161.ap58.ftth.ucom.ne.jp. [58.12.133.161]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f741790ca9sm4442795ad.85.2024.06.11.09.24.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 09:24:42 -0700 (PDT) In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:287091 Archived-At: --Multipart_Wed_Jun_12_01:24:39_2024-1 Content-Type: text/plain; charset=ISO-8859-7 Content-Transfer-Encoding: quoted-printable "shynur ." wrote: > It works perfectly this time, except there=A2s a > Warning (python): Python shell prompts cannot be detected. Thanks for testing. Could you please try the improved patch I've attached, which I believe will eliminate this warning as well? > Eli Zaretskii wrote: > > > > I wonder how PS1 and PS2 are at all relevant when using=20 > > > > python-shell-send-buffer? That function sends the buffer > > > > text to Python, so where do PS1 and PS2 come into play, > > > > and why does Python say "__PYTHON_EL_eval is not defined"=20 > > > > just because you have PS1 and PS2 customized? PS1 and PS2 are set to `comint-prompt-regexp', and used to identify execution completion. The prompts must also be identified to determine if the command can be sent. Python REPL cannot accept the Python code as is. For example, try pasting the following code to the REPL: if True: print("True") print("Hi") You should see the message "SyntaxError: invalid syntax". This is because the REPL requires an empty line to recognize the completion of a block. For such reasons, `python-shell-send-*' sends a string as the argument to __PYTHON_EL_eval instead of sending the code as is. __PYTHON_EL_eval is defined during the initialization of inferior Python, but if the prompt is not recognized, its definition cannot be done either. The prompts are recognized by `python-shell-prompt-detect'. A small Python code sends PS1, etc. to Emacs as an array of JSON strings. However, the JSON strings are hand-crafted for compatibility, as noted in the comments below. ;; JSON is built manually for compatibility The json package was added in Python 2.6, so I assume it is to support Python 2.5 and earlier. This is fine for prompts consisting only of ordinary characters, but will not result in a correct JSON string if it contains escape sequences. The attached improved patch uses the json package if available, so it can handle escape sequences; without the json package, it works as before. It means that prompts containing escape sequences can be recognized in Python 2.6 or later. I have also added an ERT to check this. In the Inferior Python buffer, the escape sequences are removed using `ansi-color-filter-apply'. Therefore, I used the same function to remove escape sequences at the end of `python-shell-prompt-detect' to make the prompts match. Another approach would be to remove the escape sequences on the Python side. It seems to be possible to do this using regular expressions, but there does not seem to be a dedicated function for this. --Multipart_Wed_Jun_12_01:24:39_2024-1 Content-Type: application/octet-stream; type=patch; name="0001-Allow-escape-sequences-in-Python-prompts.patch" Content-Disposition: attachment; filename="0001-Allow-escape-sequences-in-Python-prompts.patch" Content-Transfer-Encoding: 7bit >From 3554c6d80134b6048f09b9c175c893968b8ddbc2 Mon Sep 17 00:00:00 2001 From: kobarity Date: Wed, 12 Jun 2024 01:09:21 +0900 Subject: [PATCH] Allow escape sequences in Python prompts * lisp/progmodes/python.el (python-shell-prompt-detect): Use Python's json package if available, and remove escape sequences in prompts. * test/lisp/progmodes/python-tests.el (python-tests-interpreter-2-6-higher-p): New predicate function. (python-shell-prompt-detect-7): New test. (Bug#71440) --- lisp/progmodes/python.el | 9 +++++++-- test/lisp/progmodes/python-tests.el | 28 ++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+), 2 deletions(-) diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el index 2664d71d358..ca5ecfab6ea 100644 --- a/lisp/progmodes/python.el +++ b/lisp/progmodes/python.el @@ -3104,8 +3104,13 @@ python-shell-prompt-detect (let* ((code (concat "import sys\n" "ps = [getattr(sys, 'ps%s' % i, '') for i in range(1,4)]\n" + "try:\n" + " import json\n" + " ps_json = '\\n' + json.dumps(ps)\n" + "except ImportError:\n" ;; JSON is built manually for compatibility - "ps_json = '\\n[\"%s\", \"%s\", \"%s\"]\\n' % tuple(ps)\n" + " ps_json = '\\n[\"%s\", \"%s\", \"%s\"]\\n' % tuple(ps)\n" + "\n" "print (ps_json)\n" "sys.exit(0)\n")) (interpreter python-shell-interpreter) @@ -3168,7 +3173,7 @@ python-shell-prompt-detect "Or alternatively in:\n" " + `python-shell-prompt-input-regexps'\n" " + `python-shell-prompt-output-regexps'"))) - prompts)))) + (mapcar #'ansi-color-filter-apply prompts))))) (defun python-shell-prompt-validate-regexps () "Validate all user provided regexps for prompts. diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el index ce103921454..04b9b448ec2 100644 --- a/test/lisp/progmodes/python-tests.el +++ b/test/lisp/progmodes/python-tests.el @@ -3820,6 +3820,17 @@ python-tests-interpreter-3-p (when (string= (car (split-string (cdr info) "\\.")) "3") (car info))) +(defun python-tests-interpreter-2-6-higher-p (info) + "Check if the interpreter major version in INFO is 2.6 or higher. +This function is intended to be used as the PRED argument of +`python-tests-get-shell-interpreter'." + (let* ((version (split-string (cdr info) "\\.")) + (major (string-to-number (car version))) + (minor (string-to-number (cadr version)))) + (when (or (>= major 3) + (and (= major 2) (>= minor 6))) + (car info)))) + (ert-deftest python-shell-get-process-name-1 () "Check process name calculation sans `buffer-file-name'." (python-tests-with-temp-buffer @@ -4353,6 +4364,23 @@ python-shell-prompt-detect-6 (should (not (get-buffer "*Warnings*")))) (ignore-errors (delete-file startup-file)))))) +(ert-deftest python-shell-prompt-detect-7 () + "Check prompt autodetection with escape sequences. Bug#71440." + (python-tests-with-shell-interpreter + #'python-tests-interpreter-2-6-higher-p + (let* ((process-environment process-environment) + (startup-code (concat "import sys\n" + "sys.ps1 = '\033[32mpy> \033[0m'\n" + "sys.ps2 = '\033[32m..> \033[0m'\n" + "sys.ps3 = '\033[32mout \033[0m'\n")) + (startup-file (python-shell--save-temp-file startup-code))) + (unwind-protect + (progn + (setenv "PYTHONSTARTUP" startup-file) + (should python-shell-prompt-detect-enabled) + (should (equal (python-shell-prompt-detect) '("py> " "..> " "out ")))) + (ignore-errors (delete-file startup-file)))))) + (ert-deftest python-shell-prompt-validate-regexps-1 () "Check `python-shell-prompt-input-regexps' are validated." (let* ((python-shell-prompt-input-regexps '("\\(")) -- 2.34.1 --Multipart_Wed_Jun_12_01:24:39_2024-1--