bug#37828: 26.3; python-shell-send-defun doesn't find the (whole) definition

bug#37828: 26.3; python-shell-send-defun doesn't find the (whole) definition

[Per Starbäck]

With emacs 26.3.

There are two related bugs which makes python-shell-send-defun
sometimes not send the right contents.

== reproduce the first one ==

$ emacs -Q /tmp/newfile.py
C-c C-p                         [start python process]
def foo(): RET                  [define a ...]
pass RET                        [... python function]
C-M-x                           [python-shell-send-defun]

In the echo area it says "Sent:    pass..."
because only that line was sent. Not the whole definition, and the
function is not defined in the inferior python.

A variant is to have a line "@property" before the "def foo():" line.
Then python-shell-send-defun will enter an infinite loop instead.

== reason ==

python-shell-send-defun moves lines backwards one at a time until it
is out of the defun, and then (forward-line 1) to go back into it.
When the file begins immediately with the first defun (which isn't that
common, since normally there'd be a shebang or other comment there)
it never goes outside of the defun so then (forward-line 1) is wrong.

== fix ==

======================================================================
$ diff -u python.el python-fixed.el 
--- python.el	2019-07-25 21:41:28.000000000 +0200
+++ python-fixed.el	2019-10-20 08:17:46.871142868 +0200
@@ -3151,9 +3151,10 @@
                        (beginning-of-line 1))
                    (> (current-indentation) 0)))
        (when (not arg)
-         (while (and (forward-line -1)
-                     (looking-at (python-rx decorator))))
-         (forward-line 1))
+         (let ((remains-to-move 0))
+           (while (and (zerop (setq remains-to-move (forward-line -1)))
+                       (looking-at (python-rx decorator))))
+           (forward-line (1+ remains-to-move))))
        (point-marker))
      (progn
        (or (python-nav-end-of-defun)
======================================================================

=== reproduce the second one ==

$ emacs -Q testfile.py
where testfile.py contains

----------------------
def foo():
    pass

@property
def bar():
    pass
----------------------

C-c C-p                         [run-python]
M->                             [end-of-buffer]
C-M-x                           [python-shell-send-defun]

In the echo area it echoes "Sent: ..." and "bar" is not defined in the
inferior python which it should be.

== reason ==

When python-shell-send-defun goes to the beginning of the defun it goes
to the line with "@property", but from there python-nav-end-of-defun
doesn't find the end of that defun.

== fix ==

This could be seen as a bug in python-nav-end-of-defun and be fixed
only there instead. I haven't done that.

This patches this for python-shell-send-defun (including
the patch one above), which might be a good idea anyway, making
python-shell-send-defun more robust by going to the end-of-defun from
the original position and not from where it ended up looking for the
beginning.

(The fix is really small except for the indentation changes.)

======================================================================
$ diff -u python.el python-fixed-more.el 
--- python.el	2019-07-25 21:41:28.000000000 +0200
+++ python-fixed-more.el	2019-10-20 08:16:20.799758867 +0200
@@ -3143,24 +3143,27 @@
 user-friendly message if there's no process running; defaults to
 t when called interactively."
   (interactive (list current-prefix-arg t))
-  (save-excursion
-    (python-shell-send-region
-     (progn
-       (end-of-line 1)
-       (while (and (or (python-nav-beginning-of-defun)
-                       (beginning-of-line 1))
-                   (> (current-indentation) 0)))
-       (when (not arg)
-         (while (and (forward-line -1)
-                     (looking-at (python-rx decorator))))
-         (forward-line 1))
-       (point-marker))
-     (progn
-       (or (python-nav-end-of-defun)
-           (end-of-line 1))
-       (point-marker))
-     nil  ;; noop
-     msg)))
+  (let ((starting-pos (point)))
+    (save-excursion
+      (python-shell-send-region
+       (progn
+	 (end-of-line 1)
+	 (while (and (or (python-nav-beginning-of-defun)
+			 (beginning-of-line 1))
+                     (> (current-indentation) 0)))
+	 (when (not arg)
+	   (let ((remains-to-move 0))
+             (while (and (zerop (setq remains-to-move (forward-line -1)))
+			 (looking-at (python-rx decorator))))
+             (forward-line (1+ remains-to-move))))
+	 (point-marker))
+       (progn
+	 (goto-char starting-pos)
+	 (or (python-nav-end-of-defun)
+             (end-of-line 1))
+	 (point-marker))
+       nil  ;; noop
+       msg))))
 
 (defun python-shell-send-file (file-name &optional process temp-file-name
                                          delete msg)
======================================================================

bug#37828: 26.3; python-shell-send-defun doesn't find the (whole) definition

[Tomas Nordin]

Hello Per

Per Starbäck <per@starback.se> writes:

> With emacs 26.3.
>
> There are two related bugs which makes python-shell-send-defun
> sometimes not send the right contents.
>
> == reproduce the first one ==
>
> $ emacs -Q /tmp/newfile.py
> C-c C-p                         [start python process]
> def foo(): RET                  [define a ...]
> pass RET                        [... python function]
> C-M-x                           [python-shell-send-defun]
>
> In the echo area it says "Sent:    pass..."
> because only that line was sent. Not the whole definition, and the
> function is not defined in the inferior python.

I think this is reported by Bug#30822 and fixed in master.

Best regards
--
Tomas

bug#37828: 26.3; python-shell-send-defun doesn't find the (whole) definition

[Per Starbäck]

[-- Attachment #1: Type: text/plain, Size: 267 bytes --]

> I think this is reported by Bug#30822 and fixed in master.

Thanks! Now I have had time to compare with the trunk. The first part
of the bug is indeed fixed, but not the second part:

Here is an updated patch made against the trunk. (There are only two new lines.)

[-- Attachment #2: patch.txt --]
[-- Type: text/plain, Size: 1819 bytes --]

$ diff -u python.el python-fixed.el
--- python.el	2019-10-30 21:42:17.569928936 +0100
+++ python-fixed.el	2019-10-31 07:23:50.087973158 +0100
@@ -3178,27 +3178,29 @@
 user-friendly message if there's no process running; defaults to
 t when called interactively."
   (interactive (list current-prefix-arg t))
-  (save-excursion
-    (python-shell-send-region
-     (progn
-       (end-of-line 1)
-       (while (and (or (python-nav-beginning-of-defun)
-                       (beginning-of-line 1))
-                   (> (current-indentation) 0)))
-       (when (not arg)
-         (while (and
-                 (eq (forward-line -1) 0)
-                 (if (looking-at (python-rx decorator))
-                     t
-                   (forward-line 1)
-                   nil))))
-       (point-marker))
-     (progn
-       (or (python-nav-end-of-defun)
-           (end-of-line 1))
-       (point-marker))
-     nil  ;; noop
-     msg)))
+  (let ((starting-pos (point)))
+    (save-excursion
+      (python-shell-send-region
+       (progn
+         (end-of-line 1)
+         (while (and (or (python-nav-beginning-of-defun)
+                         (beginning-of-line 1))
+                     (> (current-indentation) 0)))
+         (when (not arg)
+           (while (and
+                   (eq (forward-line -1) 0)
+                   (if (looking-at (python-rx decorator))
+                       t
+                     (forward-line 1)
+                     nil))))
+         (point-marker))
+       (progn
+         (goto-char starting-pos)
+         (or (python-nav-end-of-defun)
+             (end-of-line 1))
+         (point-marker))
+       nil ;; noop
+       msg))))
 
 (defun python-shell-send-file (file-name &optional process temp-file-name
                                          delete msg)

bug#37828: 26.3; python-shell-send-defun doesn't find the (whole) definition

[Lars Ingebrigtsen]

Per Starbäck <per@starback.se> writes:

> Thanks! Now I have had time to compare with the trunk. The first part
> of the bug is indeed fixed, but not the second part:
>
> Here is an updated patch made against the trunk. (There are only two
> new lines.)

Thanks; I tried the patch, and it did indeed fix the problem, so I've
now applied it to Emacs 28.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no