bug#58023: 29.0.50; end-of-defun in python-mode does not work with nested function with multi-line arguments

bug#58023: 29.0.50; end-of-defun in python-mode does not work with nested function with multi-line arguments

[kobarity]

[-- Attachment #1: Type: text/plain, Size: 4092 bytes --]


Hi,

The following steps will place the point at the end of the class C,
not the end of the method a.

1. emacs -Q
2. Load the following Python file using M-x find-file

#+begin_src python
class C:
    def a(self
          ):
        pass

    def b(self):
        pass
#+end_src

3. C-n (M-x forward-line)
4. C-M-e (M-x end-of-defun)

This happens when the method (nested function) has multi-line
arguments.

This is caused by the wrong behavior of
`python-nav-beginning-of-defun'.  When the point is inside of the
parens of the nested function, it fails to detect the beginning of the
nested function and moves the point to the beginning of the outer
class/function.

The root cause of this problem exists in
`python-info-looking-at-beginning-of-defun'.  It checks syntax context
to detect correct keywords, but it checks at the current position
although keywords are searched for at the beginning of the line.  If
the point is inside of the parens of the function definition, syntax
context is "paren" and `python-info-looking-at-beginning-of-defun'
returns nil, which is not expected.  The syntax context should be
checked after moving the point to the beginning of the line.

Attached is a patch to fix this issue.

Best Regards,

--

In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.16.0, Xaw scroll bars) of 2022-09-23 built on ubuntu
Repository revision: 8238d87e15e2b2b730b6290965d14787650a15af
Repository branch: master
System Description: Ubuntu 22.04.1 LTS

Configured using:
 'configure --without-libgmp'

Configured features:
ACL CAIRO FREETYPE GIF GLIB GNUTLS GSETTINGS HARFBUZZ JPEG LIBSELINUX
LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG SECCOMP SOUND SQLITE3 THREADS
TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM LUCID ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils term/screen term/xterm xterm byte-opt gv bytecomp
byte-compile cconv rmc iso-transl tooltip eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads inotify
dynamic-setting system-font-setting font-render-setting cairo x-toolkit
x multi-tty make-network-process emacs)

Memory information:
((conses 16 45893 5646)
 (symbols 48 5782 0)
 (strings 32 15032 1212)
 (string-bytes 1 425800)
 (vectors 16 8334)
 (vector-slots 8 104450 6830)
 (floats 8 24 229)
 (intervals 56 200 0)
 (buffers 1000 10))

[-- Attachment #2: 0001-Fix-syntax-check-in-python-info-looking-at-beginning.patch --]
[-- Type: application/octet-stream, Size: 3796 bytes --]

From f5ed0ff20639e344c4309b67f2279a3ac80f7529 Mon Sep 17 00:00:00 2001
From: kobarity <kobarity@gmail.com>
Date: Fri, 23 Sep 2022 15:46:22 +0900
Subject: [PATCH] Fix syntax check in python-info-looking-at-beginning-of-defun

* lisp/progmodes/python.el
(python-info-looking-at-beginning-of-defun): Check syntax after moving
to the beginning of line.
* test/lisp/progmodes/python-tests.el (python-nav-beginning-of-defun-6)
(python-end-of-defun-1, python-info-looking-at-beginning-of-defun-3):
New tests.
---
 lisp/progmodes/python.el            | 10 +++---
 test/lisp/progmodes/python-tests.el | 48 +++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+), 5 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index d5ff059cc8..80c5b31b6e 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -5505,11 +5505,11 @@ python-info-looking-at-beginning-of-defun
   "Check if point is at `beginning-of-defun' using SYNTAX-PPSS.
 When CHECK-STATEMENT is non-nil, the current statement is checked
 instead of the current physical line."
-  (and (not (python-syntax-context-type (or syntax-ppss (syntax-ppss))))
-       (save-excursion
-         (when check-statement
-           (python-nav-beginning-of-statement))
-         (beginning-of-line 1)
+  (save-excursion
+    (when check-statement
+      (python-nav-beginning-of-statement))
+    (beginning-of-line 1)
+    (and (not (python-syntax-context-type (or syntax-ppss (syntax-ppss))))
          (looking-at python-nav-beginning-of-defun-regexp))))
 
 (defun python-info-looking-at-beginning-of-block ()
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 20a7a0132a..fdaedb5fd7 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -2342,6 +2342,21 @@ python-nav-beginning-of-defun-5
                 (beginning-of-line)
                 (point))))))
 
+(ert-deftest python-nav-beginning-of-defun-6 ()
+  (python-tests-with-temp-buffer
+   "
+class C:
+    def foo(self):
+        pass
+"
+   (python-tests-look-at "self")
+   (should (= (save-excursion
+                (python-nav-beginning-of-defun)
+                (point))
+              (save-excursion
+                (beginning-of-line)
+                (point))))))
+
 (ert-deftest python-nav-end-of-defun-1 ()
   (python-tests-with-temp-buffer
    "
@@ -2472,6 +2487,26 @@ python-nav-end-of-defun-3
               (save-excursion
                 (point-max))))))
 
+(ert-deftest python-end-of-defun-1 ()
+  (python-tests-with-temp-buffer
+   "
+class C:
+    def a(self
+          ):
+        pass
+
+    def b(self):
+        pass
+"
+   (should (= (save-excursion
+                (python-tests-look-at "def a")
+                (end-of-defun)
+                (point))
+              (save-excursion
+                (python-tests-look-at "def b")
+                (forward-line -1)
+                (point))))))
+
 (ert-deftest python-nav-backward-defun-1 ()
   (python-tests-with-temp-buffer
    "
@@ -5734,6 +5769,19 @@ python-info-looking-at-beginning-of-defun-2
    (should (not (python-info-looking-at-beginning-of-defun)))
    (should (not (python-info-looking-at-beginning-of-defun nil t)))))
 
+(ert-deftest python-info-looking-at-beginning-of-defun-3 ()
+  (python-tests-with-temp-buffer
+   "
+def foo(arg=\"default\"):  # Comment
+    pass
+"
+   (python-tests-look-at "arg")
+   (should (python-info-looking-at-beginning-of-defun))
+   (python-tests-look-at "default")
+   (should (python-info-looking-at-beginning-of-defun))
+   (python-tests-look-at "Comment")
+   (should (python-info-looking-at-beginning-of-defun))))
+
 (ert-deftest python-info-looking-at-beginning-of-block-1 ()
   (python-tests-with-temp-buffer
    "
-- 
2.34.1

bug#58023: 29.0.50; end-of-defun in python-mode does not work with nested function with multi-line arguments

[Andreas Röhler]

Am 23.09.22 um 14:01 schrieb kobarity:
> Hi,
>
> The following steps will place the point at the end of the class C,
> not the end of the method a.
>
> 1. emacs -Q
> 2. Load the following Python file using M-x find-file
>
> #+begin_src python
> class C:
>      def a(self
>            ):
>          pass
>
>      def b(self):
>          pass
> #+end_src
>
> 3. C-n (M-x forward-line)
> 4. C-M-e (M-x end-of-defun)
>
> This happens when the method (nested function) has multi-line
> arguments.


Probably not a big thing WRT editing purpose: For me method "a" starts 
with the "d" of def.

 From second line, BOL, reaching the end of class seems okay.

bug#58023: 29.0.50; end-of-defun in python-mode does not work with nested function with multi-line arguments

[Lars Ingebrigtsen]

kobarity <kobarity@gmail.com> writes:

> Attached is a patch to fix this issue.

Thanks; pushed to Emacs 29.