From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: kobarity Newsgroups: gmane.emacs.bugs Subject: bug#58780: python.el infinite loop in info-current-defun Date: Sun, 05 Mar 2023 17:19:43 +0900 Message-ID: References: Mime-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: multipart/mixed; boundary="Multipart_Sun_Mar__5_17:19:43_2023-1" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33985"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?Q?Goj=C5=8D?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/30.0.50 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) To: JD Smith , Ryan B , Tom Gillespie , 58780@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sun Mar 05 09:21:15 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pYjcH-0008bi-Gc for geb-bug-gnu-emacs@m.gmane-mx.org; Sun, 05 Mar 2023 09:21:13 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pYjc7-00029a-Pp; Sun, 05 Mar 2023 03:21:03 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pYjc6-00029R-Hy for bug-gnu-emacs@gnu.org; Sun, 05 Mar 2023 03:21:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pYjc6-0004bU-9S for bug-gnu-emacs@gnu.org; Sun, 05 Mar 2023 03:21:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pYjc5-0006lf-P1 for bug-gnu-emacs@gnu.org; Sun, 05 Mar 2023 03:21:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: kobarity Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 05 Mar 2023 08:21:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 58780 X-GNU-PR-Package: emacs Original-Received: via spool by 58780-submit@debbugs.gnu.org id=B58780.167800440825927 (code B ref 58780); Sun, 05 Mar 2023 08:21:01 +0000 Original-Received: (at 58780) by debbugs.gnu.org; 5 Mar 2023 08:20:08 +0000 Original-Received: from localhost ([127.0.0.1]:38367 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pYjbE-0006k6-87 for submit@debbugs.gnu.org; Sun, 05 Mar 2023 03:20:08 -0500 Original-Received: from mail-pl1-f180.google.com ([209.85.214.180]:42769) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pYjbC-0006jS-90 for 58780@debbugs.gnu.org; Sun, 05 Mar 2023 03:20:06 -0500 Original-Received: by mail-pl1-f180.google.com with SMTP id i10so7023979plr.9 for <58780@debbugs.gnu.org>; Sun, 05 Mar 2023 00:20:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1678004400; h=mime-version:user-agent:references:in-reply-to:subject:to:from :message-id:date:from:to:cc:subject:date:message-id:reply-to; bh=gB6nWlLFlbou1bJP2JtbDv1HdHl73YxK9vhjgI+88ps=; b=Zr9huuiGPRmsFjf2NJ74DMA1uXiWxSA8ORI4R33KZsMe7gJFyh2YwUs8DKpj2dlgcC 8BKjkN0SIsDANh/FWLXAK9oROASuFlpaJ0VwbumtTJMCRPMRPZnHhWyrWW5GbG0JZAgd QLl/mvnoHFBtk+F/zPgtrhlQTvmieQ+Z2OWYvoWB7QQ41yGLEM7bxs8wmj5N8B0EPCVZ Pnv5pq6TnmtmKXUQhowQbPaLu39S3syQnwJoQUPtEVplBXqTH/JN/owrkK9/6+bYjmtD uN2XlsmXKaEbGlRsgZLamnDEADxMdSmd3S8Ccx0JcdB7MJzdowBW394cyZpJFlEilO8m Y8VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678004400; h=mime-version:user-agent:references:in-reply-to:subject:to:from :message-id:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=gB6nWlLFlbou1bJP2JtbDv1HdHl73YxK9vhjgI+88ps=; b=khkm/nduDz33+Jq8KhgWyL+jfxcg8op5Gft0oJgF8QniZSEr+0NNjz41+Ila1dYR6G zG9/Fqrv56FLhuH1ozIQtID3raHLrkfH0ebxgRft9CfcJOvJPgKD+Y6F8pxoplkZQZ+W 6dvSWzy/pyXMJP5KDFLVMQFMvtLh2UmA3DFDD/esEZko0PhsJFM4Uetc2p99OBdOIYfB IZfmdK8Oouqjuuy5ksNw601wmtp3neeqCDcpF3sArl45wZE/pBsL6rk/pNUaLltjAY96 gSYEiIhiUrIbH339d/kUhLXZk3IS1JazkZNY5D5Ci1vZE6hBbovCC30E1wsrJ1agH7EK eD/Q== X-Gm-Message-State: AO0yUKUnfbDn6rHsUgMdrx547ICorm/i0WJuLidd3c8K8c+TQBWWbN4n TtfAeuU8QzGhzbho1+cxMRc= X-Google-Smtp-Source: AK7set+rn8thAosoiFzvkcbH791RkQAsgY1ijjZQvq9aEl4xDyPUolaSjPRvPL3H/ZmTCBPWehXvNw== X-Received: by 2002:a17:902:ee89:b0:19a:841f:56 with SMTP id a9-20020a170902ee8900b0019a841f0056mr6998187pld.20.1678004400057; Sun, 05 Mar 2023 00:20:00 -0800 (PST) Original-Received: from localhost (58x12x133x161.ap58.ftth.ucom.ne.jp. [58.12.133.161]) by smtp.gmail.com with ESMTPSA id j17-20020a170903025100b00189ac5a2340sm4357418plh.124.2023.03.05.00.19.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 05 Mar 2023 00:19:58 -0800 (PST) In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:257340 Archived-At: --Multipart_Sun_Mar__5_17:19:43_2023-1 Content-Type: text/plain; charset=US-ASCII As JD Smith pointed out, this problem is caused by `python-nav-end-of-statement' moving point backward. A simple example illustrating the problem of `python-nav-end-of-statement' is as follows: #+begin_src python ' """ a = 1 #+end_src When point is placed on the second line and `python-nav-end-of-statement' is executed, point will move to the end of the first line. Here is why this happens. `python-nav-end-of-statement' moves point to the end of the second line, and checks if it is in the middle of the string using: (python-syntax-context 'string) It tells that the point is within the string started at the single-quote in the first line, and the variable `string-start' is set to the start of the string (single-quote in the first line). Then, point is moved to the start of the string, and the end of the string is searched for using: (re-search-forward (rx (syntax string-delimiter)) nil t) It finds the triple double-quotes and moves point immediately after them. The variable `last-string-end' is set to this position. In the second iteration of the while loop, the variable `string-start' is set to the single-quote in the first line again. This results in exiting the loop because the following condition is not met. (>= string-start last-string-end) There are several issues here: 1. Although `python-syntax-context' detects both string-quote (single quote) and string-delimiter (triple quotes) as the start of string, only string-delimiter is searched for as the end of string. 2. The triple double-quotes is marked as string-delimiter even though they are within the string begins with the single-quote. 3. As the end of the first line is not escaped using a backslash, the string begins with the single-quote should not be continued on the second line. I would like to discuss the second and third issues separately. The first issue is the main cause of Bug#58780. This issue also causes another type of problem, illustrated by the following example. #+begin_src python abc = 'a\ b\ c' d = '''d''' #+end_src When `python-nav-end-of-statement' is executed with point located at the second line, point is moved to the end of the fourth line instead of the third line. This is a wrong point move for a correct Python program. The attached 0001-Fix-searching-for-end-of-string-in-python-nav-end-of.patch is my proposal to fix this issue with some ERTs. `forward-sexp' (in fact `python-nav--lisp-forward-sexp') is added to search for the end of the string if the string begins with single single/double-quote (' or "). I think this 0001 patch also fixes Bug#56271. At least, it passes the ERT python-nav-end-of-block-2 which is introduced in Bug#56271, even the workaround introduced in Bug#56271 is reverted. The attached 0002-Revert-workaround-introduced-in-Bug-56271.patch is a patch to revert the workaround. If the 0001 patch is accepted, the 0002 patch can also be applied. But if we want to be safer, we can leave the workaround as it is by not applying the 0002 patch. I look forward to your comments. --Multipart_Sun_Mar__5_17:19:43_2023-1 Content-Type: application/octet-stream; type=patch; name="0001-Fix-searching-for-end-of-string-in-python-nav-end-of.patch" Content-Disposition: attachment; filename="0001-Fix-searching-for-end-of-string-in-python-nav-end-of.patch" Content-Transfer-Encoding: 7bit >From 62cfa24a89fdbf90cbe866ad88ca635327eb1f49 Mon Sep 17 00:00:00 2001 From: kobarity Date: Sun, 5 Mar 2023 17:06:26 +0900 Subject: [PATCH 1/2] Fix searching for end of string in python-nav-end-of-statement * lisp/progmodes/python.el (python-nav-end-of-statement): Add searching for corresponding string-quote. * test/lisp/progmodes/python-tests.el (python-nav-end-of-statement-3) (python-nav-end-of-statement-4, python-info-current-defun-4): New tests. (Bug#58780) --- lisp/progmodes/python.el | 14 ++++++--- test/lisp/progmodes/python-tests.el | 44 +++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+), 4 deletions(-) diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el index 1f970633bfc..cc4ece1669c 100644 --- a/lisp/progmodes/python.el +++ b/lisp/progmodes/python.el @@ -2076,10 +2076,16 @@ python-nav-end-of-statement (goto-char (+ (point) (python-syntax-count-quotes (char-after (point)) (point)))) - (setq last-string-end - (or (re-search-forward - (rx (syntax string-delimiter)) nil t) - (goto-char (point-max))))))) + (setq + last-string-end + (or (if (eq t (nth 3 (syntax-ppss))) + (re-search-forward + (rx (syntax string-delimiter)) nil t) + (ignore-error scan-error + (goto-char string-start) + (python-nav--lisp-forward-sexp) + (point))) + (goto-char (point-max))))))) ((python-syntax-context 'paren) ;; The statement won't end before we've escaped ;; at least one level of parenthesis. diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el index 4f24c042c6a..e9df4a2c843 100644 --- a/test/lisp/progmodes/python-tests.el +++ b/test/lisp/progmodes/python-tests.el @@ -2943,6 +2943,36 @@ python-nav-end-of-statement-2 "'\n''\n" (python-nav-end-of-statement))) +(ert-deftest python-nav-end-of-statement-3 () + "Test unmatched quotes (Bug#58780)." + (python-tests-with-temp-buffer + " +' \"\"\" +v = 1 +" + (python-tests-look-at "v =") + (should (= (save-excursion + (python-nav-end-of-statement) + (point)) + (save-excursion + (point-max)))))) + +(ert-deftest python-nav-end-of-statement-4 () + (python-tests-with-temp-buffer + " +abc = 'a\\ +b\\ +c' +d = '''d''' +" + (python-tests-look-at "b\\") + (should (= (save-excursion + (python-nav-end-of-statement) + (point)) + (save-excursion + (python-tests-look-at "c'") + (pos-eol)))))) + (ert-deftest python-nav-forward-statement-1 () (python-tests-with-temp-buffer " @@ -5209,6 +5239,20 @@ python-info-current-defun-3 (should (string= (python-info-current-defun t) "def decoratorFunctionWithArguments")))) +(ert-deftest python-info-current-defun-4 () + "Ensure unmatched quotes do not cause hang (Bug#58780)." + (python-tests-with-temp-buffer + " +def func(): + ' \"\"\" + v = 1 +" + (python-tests-look-at "v = 1") + (should (string= (python-info-current-defun) + "func")) + (should (string= (python-info-current-defun t) + "def func")))) + (ert-deftest python-info-current-symbol-1 () (python-tests-with-temp-buffer " -- 2.34.1 --Multipart_Sun_Mar__5_17:19:43_2023-1 Content-Type: application/octet-stream; type=patch; name="0002-Revert-workaround-introduced-in-Bug-56271.patch" Content-Disposition: attachment; filename="0002-Revert-workaround-introduced-in-Bug-56271.patch" Content-Transfer-Encoding: 7bit >From f80cf3e53ceeaa940fa82a2e294dd43c6e74231b Mon Sep 17 00:00:00 2001 From: kobarity Date: Sun, 5 Mar 2023 17:07:17 +0900 Subject: [PATCH 2/2] Revert workaround introduced in Bug#56271 * lisp/progmodes/python.el (python-nav-end-of-statement) (python-nav-end-of-block): Revert workaround introduced in Bug#56271 as the bug is fixedin Bug#58780. --- lisp/progmodes/python.el | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el index cc4ece1669c..aa0a043a275 100644 --- a/lisp/progmodes/python.el +++ b/lisp/progmodes/python.el @@ -2062,10 +2062,6 @@ python-nav-end-of-statement ;; are somehow out of whack. This has been ;; observed when using `syntax-ppss' during ;; narrowing. - ;; It can also fail in cases where the buffer is in - ;; the process of being modified, e.g. when creating - ;; a string with `electric-pair-mode' disabled such - ;; that there can be an unmatched single quote (when (>= string-start last-string-end) (goto-char string-start) (if (python-syntax-context 'paren) @@ -2154,10 +2150,7 @@ python-nav-end-of-block (while (and (forward-line 1) (not (eobp)) (or (and (> (current-indentation) block-indentation) - (let ((start (point))) - (python-nav-end-of-statement) - ;; must move forward otherwise infinite loop - (> (point) start))) + (or (python-nav-end-of-statement) t)) (python-info-current-line-comment-p) (python-info-current-line-empty-p)))) (python-util-forward-comment -1) -- 2.34.1 --Multipart_Sun_Mar__5_17:19:43_2023-1--