bug#30822: 25.3; python-shell-send-defun sends only one line

bug#30822: 25.3; python-shell-send-defun sends only one line

[David Liu]

[-- Attachment #1: Type: text/plain, Size: 4278 bytes --]

note: no other Python-related emacs packages are being used;
only the builtin one.

1. create new file "test.py" with following:

def f(x):
    return x+1

2. C-c C-p to run-python

3. place point immediately after "return x+1"

4. C-M-x for python-shell-send-defun

result:

    SyntaxError: 'return' outside function

it appears that only one line is being sent, instead of both lines.



In GNU Emacs 25.3.1 (x86_64-apple-darwin13.4.0, NS appkit-1265.21 Version
10.9.5 (Build 13F1911))
 of 2017-09-12 built on builder10-9.local
Windowing system distributor 'Apple', version 10.3.1561
Configured using:
 'configure --with-ns '--enable-locallisppath=/Library/Application
 Support/Emacs/${version}/site-lisp:/Library/Application
 Support/Emacs/site-lisp' --with-modules'

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Python

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
uncompressing python.el.gz...done
Making completion list...
t

delete-backward-char: Text is read-only
Quit

<C-tab> is undefined
mouse-2, RET: find function's definition
uncompressing python.el.gz...done

Load-path shadows:
/Users/daveliu/quicklisp/dists/quicklisp/software/slime-v2.20/slime-autoloads
hides /Users/daveliu/.emacs.d/elpa/slime-20180308.733/slime-autoloads
/Users/daveliu/quicklisp/dists/quicklisp/software/slime-v2.20/slime-tests
hides /Users/daveliu/.emacs.d/elpa/slime-20180308.733/slime-tests
/Users/daveliu/quicklisp/dists/quicklisp/software/slime-v2.20/slime hides
/Users/daveliu/.emacs.d/elpa/slime-20180308.733/slime

Features:
(shadow sort mail-extr emacsbug message dired rfc822 mml mml-sec epg
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils jka-compr
eieio-opt speedbar sb-image ezimage dframe find-func tramp-cache python
tramp-sh tramp tramp-compat auth-source gnus-util mm-util help-fns
mail-prsvr password-cache tramp-loaddefs trampver shell pcomplete
format-spec json map slime-fancy slime-trace-dialog slime-fontifying-fu
slime-package-fu slime-references slime-compiler-notes-tree
slime-scratch slime-presentations advice bridge slime-macrostep
macrostep slime-mdot-fu slime-enclosing-context slime-fuzzy
slime-fancy-trace slime-fancy-inspector slime-c-p-c
slime-editing-commands slime-autodoc slime-repl elp slime-parse slime
derived edmacro kmacro lisp-mnt gud apropos compile etags xref cl-seq
project eieio eieio-core arc-mode archive-mode noutline outline
easy-mmode pp comint ansi-color ring hyperspec cl-macs thingatpt
browse-url cl tango-dark-theme finder-inf tex-site info slime-autoloads
package epg-config seq byte-opt gv bytecomp byte-compile cl-extra
help-mode easymenu cconv cl-loaddefs pcase cl-lib time-date mule-util
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel ns-win ucs-normalize term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote kqueue cocoa ns multi-tty
make-network-process emacs)

Memory information:
((conses 16 337784 17797)
 (symbols 48 30156 0)
 (miscs 40 149 592)
 (strings 32 48025 8240)
 (string-bytes 1 1429236)
 (vectors 16 43362)
 (vector-slots 8 786245 7866)
 (floats 8 339 383)
 (intervals 56 649 397)
 (buffers 976 27))

[-- Attachment #2: Type: text/html, Size: 5659 bytes --]

bug#30822: 25.3; python-shell-send-defun sends only one line

[David Liu]

[-- Attachment #1: Type: text/plain, Size: 4777 bytes --]

regarding the possible bug with python-shell-send-defun, it seems to *only*
occurs if the python function starts on the very first line of the file.
Otherwise, python-shell-send-defun works perfectly.

On Wed, Mar 14, 2018 at 4:05 PM, David Liu <daveliu@alum.mit.edu> wrote:

> note: no other Python-related emacs packages are being used;
> only the builtin one.
>
> 1. create new file "test.py" with following:
>
> def f(x):
>     return x+1
>
> 2. C-c C-p to run-python
>
> 3. place point immediately after "return x+1"
>
> 4. C-M-x for python-shell-send-defun
>
> result:
>
>     SyntaxError: 'return' outside function
>
> it appears that only one line is being sent, instead of both lines.
>
>
>
> In GNU Emacs 25.3.1 (x86_64-apple-darwin13.4.0, NS appkit-1265.21 Version
> 10.9.5 (Build 13F1911))
>  of 2017-09-12 built on builder10-9.local
> Windowing system distributor 'Apple', version 10.3.1561
> Configured using:
>  'configure --with-ns '--enable-locallisppath=/Library/Application
>  Support/Emacs/${version}/site-lisp:/Library/Application
>  Support/Emacs/site-lisp' --with-modules'
>
> Configured features:
> NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES
>
> Important settings:
>   value of $LANG: en_US.UTF-8
>   locale-coding-system: utf-8-unix
>
> Major mode: Python
>
> Minor modes in effect:
>   shell-dirtrack-mode: t
>   tooltip-mode: t
>   global-eldoc-mode: t
>   electric-indent-mode: t
>   mouse-wheel-mode: t
>   file-name-shadow-mode: t
>   global-font-lock-mode: t
>   font-lock-mode: t
>   blink-cursor-mode: t
>   auto-composition-mode: t
>   auto-encryption-mode: t
>   auto-compression-mode: t
>   line-number-mode: t
>   transient-mark-mode: t
>
> Recent messages:
> uncompressing python.el.gz...done
> Making completion list...
> t
>
> delete-backward-char: Text is read-only
> Quit
>
> <C-tab> is undefined
> mouse-2, RET: find function's definition
> uncompressing python.el.gz...done
>
> Load-path shadows:
> /Users/daveliu/quicklisp/dists/quicklisp/software/slime-v2.20/slime-autoloads
> hides /Users/daveliu/.emacs.d/elpa/slime-20180308.733/slime-autoloads
> /Users/daveliu/quicklisp/dists/quicklisp/software/slime-v2.20/slime-tests
> hides /Users/daveliu/.emacs.d/elpa/slime-20180308.733/slime-tests
> /Users/daveliu/quicklisp/dists/quicklisp/software/slime-v2.20/slime hides
> /Users/daveliu/.emacs.d/elpa/slime-20180308.733/slime
>
> Features:
> (shadow sort mail-extr emacsbug message dired rfc822 mml mml-sec epg
> mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
> mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils jka-compr
> eieio-opt speedbar sb-image ezimage dframe find-func tramp-cache python
> tramp-sh tramp tramp-compat auth-source gnus-util mm-util help-fns
> mail-prsvr password-cache tramp-loaddefs trampver shell pcomplete
> format-spec json map slime-fancy slime-trace-dialog slime-fontifying-fu
> slime-package-fu slime-references slime-compiler-notes-tree
> slime-scratch slime-presentations advice bridge slime-macrostep
> macrostep slime-mdot-fu slime-enclosing-context slime-fuzzy
> slime-fancy-trace slime-fancy-inspector slime-c-p-c
> slime-editing-commands slime-autodoc slime-repl elp slime-parse slime
> derived edmacro kmacro lisp-mnt gud apropos compile etags xref cl-seq
> project eieio eieio-core arc-mode archive-mode noutline outline
> easy-mmode pp comint ansi-color ring hyperspec cl-macs thingatpt
> browse-url cl tango-dark-theme finder-inf tex-site info slime-autoloads
> package epg-config seq byte-opt gv bytecomp byte-compile cl-extra
> help-mode easymenu cconv cl-loaddefs pcase cl-lib time-date mule-util
> tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
> mwheel ns-win ucs-normalize term/common-win tool-bar dnd fontset image
> regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
> prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
> mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham
> georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
> korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
> european ethiopic indian cyrillic chinese charscript case-table epa-hook
> jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
> loaddefs button faces cus-face macroexp files text-properties overlay
> sha1 md5 base64 format env code-pages mule custom widget
> hashtable-print-readable backquote kqueue cocoa ns multi-tty
> make-network-process emacs)
>
> Memory information:
> ((conses 16 337784 17797)
>  (symbols 48 30156 0)
>  (miscs 40 149 592)
>  (strings 32 48025 8240)
>  (string-bytes 1 1429236)
>  (vectors 16 43362)
>  (vector-slots 8 786245 7866)
>  (floats 8 339 383)
>  (intervals 56 649 397)
>  (buffers 976 27))
>
>

[-- Attachment #2: Type: text/html, Size: 6294 bytes --]

bug#30822: 25.3; python-shell-send-defun sends only one line

[Noam Postavsky]

retitle 30822 python-shell-send-defun can't send first line of buffer
tags 30822 + confirmed
found 30822 26.1
severity 30822 minor
quit

David Liu <daveliu@alum.mit.edu> writes:

> regarding the possible bug with python-shell-send-defun, it seems to
> *only* occurs if the python function starts on the very first line of
> the file. Otherwise, python-shell-send-defun works perfectly.

Indeed, there is an unconditional call to `forward-line', which makes it
impossible to for python-shell-send-defun to send the first line of the
buffer:

(defun python-shell-send-defun (&optional arg msg)
  [...]
    (python-shell-send-region
     (progn
       (end-of-line 1)
       (while (and (or (python-nav-beginning-of-defun)
                       (beginning-of-line 1))
                   (> (current-indentation) 0)))
       (when (not arg)
         (while (and (forward-line -1)
                     (looking-at (python-rx decorator))))
         (forward-line 1)) ; <-----------------------------------
       (point-marker))
      [...]

bug#30822: 25.3; python-shell-send-defun sends only one line

[Nitish]

[-- Attachment #1: Type: text/plain, Size: 443 bytes --]

As Noam Postavsky mentioned, there was an unconditional (forward-line 1) expression.

The issue manifests itself in a different way too. In the example that David Liu gave, add a decorator to the function on the first line. Then `python-shell-send-defun` enters an infinite loop.

The attached patch fixes both these issues. `(forward-line 1)` should be called only when `(forward-line -1)` moves Point and `(looking-at..` returns false.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-30822-Fixed-infinite-loop-and-also-removed-an-extra-.patch --]
[-- Type: text/x-patch; name="0001-30822-Fixed-infinite-loop-and-also-removed-an-extra-.patch", Size: 1249 bytes --]

From 17b43cf36c58bb396bf269762fb081af1e08f736 Mon Sep 17 00:00:00 2001
From: nitishch <nitishchandrachinta@gmail.com>
Date: Fri, 6 Apr 2018 08:01:14 +0530
Subject: [PATCH] 30822: Fixed infinite loop and also removed an extra call to
 forward-line

---
 lisp/progmodes/python.el | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index afafd1b42c..3043428413 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -3163,9 +3163,13 @@ python-shell-send-defun
                        (beginning-of-line 1))
                    (> (current-indentation) 0)))
        (when (not arg)
-         (while (and (forward-line -1)
-                     (looking-at (python-rx decorator))))
-         (forward-line 1))
+         (while (and
+                 ;; Make sure forward-line actually moves Point
+                 (eq (forward-line -1) 0)
+                 ;; If Point is at a decorator, loop.
+                 ;; else, move one line down and exit the loop.
+                 (or (looking-at (python-rx decorator))
+                     (and (forward-line 1) nil)))))
        (point-marker))
      (progn
        (or (python-nav-end-of-defun)
-- 
2.16.2

bug#30822: 25.3; python-shell-send-defun sends only one line

[Noam Postavsky]

Nitish <nitishch@protonmail.com> writes:

>         (when (not arg)
> -         (while (and (forward-line -1)
> -                     (looking-at (python-rx decorator))))
> -         (forward-line 1))
> +         (while (and
> +                 ;; Make sure forward-line actually moves Point
> +                 (eq (forward-line -1) 0)
> +                 ;; If Point is at a decorator, loop.
> +                 ;; else, move one line down and exit the loop.
> +                 (or (looking-at (python-rx decorator))
> +                     (and (forward-line 1) nil)))))

Thanks.  You could drop the comments, since they just repeat what the
code is saying.  And, while I know it's fairly common to use `and' and
`or' like that to encode conditionals, perhaps using `if' directly would
minimize the temptation to rephrase it as an "if" statement in English.

bug#30822: 25.3; python-shell-send-defun sends only one line

[Nitish]

[-- Attachment #1: Type: text/plain, Size: 973 bytes --]

I will remove the comments.

To use if condition, we would need equivalent of a break. What is the elisp-y way of breaking out of a loop.

Sent from ProtonMail mobile

-------- Original Message --------
On Apr 6, 2018, 7:26 PM, Noam Postavsky wrote:

> Nitish  writes:
>
>> (when (not arg)
>> - (while (and (forward-line -1)
>> - (looking-at (python-rx decorator))))
>> - (forward-line 1))
>> + (while (and
>> + ;; Make sure forward-line actually moves Point
>> + (eq (forward-line -1) 0)
>> + ;; If Point is at a decorator, loop.
>> + ;; else, move one line down and exit the loop.
>> + (or (looking-at (python-rx decorator))
>> + (and (forward-line 1) nil)))))
>
> Thanks. You could drop the comments, since they just repeat what the
> code is saying. And, while I know it's fairly common to use `and' and
> `or' like that to encode conditionals, perhaps using `if' directly would
> minimize the temptation to rephrase it as an "if" statement in English. @protonmail.com>

[-- Attachment #2: Type: text/html, Size: 1402 bytes --]

bug#30822: 25.3; python-shell-send-defun sends only one line

[Noam Postavsky]

Nitish <nitishch@protonmail.com> writes:

> I will remove the comments.
>
> To use if condition, we would need equivalent of a break. What is the
> elisp-y way of breaking out of a loop.

I meant, instead of

    (or (looking-at (python-rx decorator))
        (and (forward-line 1) nil))

use

    (if (looking-at (python-rx decorator)) t
      (forward-line 1)
      nil)

Or at least use `progn' instead of that `and', since forward-line always
returns non-nil anyway:

    (or (looking-at (python-rx decorator))
        (progn (forward-line 1) nil))

bug#30822: 25.3; python-shell-send-defun sends only one line

[Nitish]

[-- Attachment #1: Type: text/plain, Size: 30 bytes --]

Attaching the updated patch.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-python-shell-send-defun-for-functions-starting-a.patch --]
[-- Type: text/x-patch; name="0001-Fix-python-shell-send-defun-for-functions-starting-a.patch", Size: 1096 bytes --]

From ee7aacb36671bf8321156d9e72d7f41282083a5e Mon Sep 17 00:00:00 2001
From: nitishch <nitishchandrachinta@gmail.com>
Date: Sat, 7 Apr 2018 08:45:03 +0530
Subject: [PATCH] Fix python-shell-send-defun for functions starting at
 beginning of file

---
 lisp/progmodes/python.el | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index afafd1b42c..adf7b33ccb 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -3163,9 +3163,12 @@ python-shell-send-defun
                        (beginning-of-line 1))
                    (> (current-indentation) 0)))
        (when (not arg)
-         (while (and (forward-line -1)
-                     (looking-at (python-rx decorator))))
-         (forward-line 1))
+         (while (and
+                 (eq (forward-line -1) 0)
+                 (if (looking-at (python-rx decorator))
+                     t
+                   (forward-line 1)
+                   nil))))
        (point-marker))
      (progn
        (or (python-nav-end-of-defun)
-- 
2.16.2

bug#30822: 25.3; python-shell-send-defun sends only one line

[Noam Postavsky]

Nitish <nitishch@protonmail.com> writes:

> Attaching the updated patch.

Thanks.  I see you have one previous tiny change, am I correct in
assuming your copyright status is unchanged?  (This patch should still
fit within the 15 line limit, though you are approaching it.)

bug#30822: 25.3; python-shell-send-defun sends only one line

[Nitish]

[-- Attachment #1: Type: text/plain, Size: 404 bytes --]

Yes. You are right. Will change it sometime :)

-------- Original Message --------
On Apr 8, 2018, 9:07 AM, Noam Postavsky wrote:

> Nitish  writes:
>
>> Attaching the updated patch.
>
> Thanks. I see you have one previous tiny change, am I correct in
> assuming your copyright status is unchanged? (This patch should still
> fit within the 15 line limit, though you are approaching it.) @protonmail.com>

[-- Attachment #2: Type: text/html, Size: 600 bytes --]

bug#30822: 25.3; python-shell-send-defun sends only one line

[Noam Postavsky]

tags 30822 fixed
close 30822 27.1
quit

Nitish <nitishch@protonmail.com> writes:

> Attaching the updated patch.

Pushed to master.

[1: 5fb34352f6]: 2018-04-08 12:36:00 -0400
  Fix python-shell-send-defun at start of buffer (Bug#30822)
  https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=5fb34352f6050ba225d14f2a51a6b99739826b1b

bug#30822: 25.3; python-shell-send-defun sends only one line

[Andreas Röhler]

On 08.04.2018 18:44, Noam Postavsky wrote:
> tags 30822 fixed
> close 30822 27.1
> quit
> 
> Nitish<nitishch@protonmail.com>  writes:
> 
>> Attaching the updated patch.
> Pushed to master.
> 
> [1: 5fb34352f6]: 2018-04-08 12:36:00 -0400
>    Fix python-shell-send-defun at start of buffer (Bug#30822)
>    https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=5fb34352f6050ba225d14f2a51a6b99739826b1b
> 
> 
> 

Please consider re-opening.
Isn't this a bug of python-nav-beginning-of-defun and the fix belonging 
there?

Cheers,
Andreas

bug#30822: 25.3; python-shell-send-defun sends only one line

[Noam Postavsky]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> Please consider re-opening.
> Isn't this a bug of python-nav-beginning-of-defun and the fix
> belonging there?

I don't see why.  If you have a bug to report against
python-nav-beginning-of-defun please just say what you mean.

bug#30822: 25.3; python-shell-send-defun sends only one line

[Andreas Röhler]

On 09.04.2018 13:56, Noam Postavsky wrote:
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
> 
>> Please consider re-opening.
>> Isn't this a bug of python-nav-beginning-of-defun and the fix
>> belonging there?
> 
> I don't see why.  If you have a bug to report against
> python-nav-beginning-of-defun please just say what you mean.
> 
> 


Well, if a function is decorated, shouldn't a command reaching the start 
of def now reach the start of decorator?

bug#30822: 25.3; python-shell-send-defun sends only one line

[Noam Postavsky]

On 11 April 2018 at 10:49, Andreas Röhler <andreas.roehler@easy-emacs.de> wrote:
> On 09.04.2018 13:56, Noam Postavsky wrote:
>>
>> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>>
>>> Please consider re-opening.
>>> Isn't this a bug of python-nav-beginning-of-defun and the fix
>>> belonging there?
>>
>> I don't see why.  If you have a bug to report against
>> python-nav-beginning-of-defun please just say what you mean.
>
> Well, if a function is decorated, shouldn't a command reaching the start of
> def now reach the start of decorator?

Changing python-nav-beginning-of-defun to behave that way sounds
reasonable, but it should be pursued in a separate (wishlist) bug, not
this one.

bug#30822: 25.3; python-shell-send-defun sends only one line

[Andreas Röhler]

On 11.04.2018 16:57, Noam Postavsky wrote:
> On 11 April 2018 at 10:49, Andreas Röhler <andreas.roehler@easy-emacs.de> wrote:
>> On 09.04.2018 13:56, Noam Postavsky wrote:
>>>
>>> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>>>
>>>> Please consider re-opening.
>>>> Isn't this a bug of python-nav-beginning-of-defun and the fix
>>>> belonging there?
>>>
>>> I don't see why.  If you have a bug to report against
>>> python-nav-beginning-of-defun please just say what you mean.
>>
>> Well, if a function is decorated, shouldn't a command reaching the start of
>> def now reach the start of decorator?
> 
> Changing python-nav-beginning-of-defun to behave that way sounds
> reasonable, but it should be pursued in a separate (wishlist) bug, not
> this one.
> 


Or fixing the bug here at the right place solves two at once?

bug#30822: 25.3; python-shell-send-defun sends only one line

[Noam Postavsky]

On 11 April 2018 at 13:29, Andreas Röhler <andreas.roehler@easy-emacs.de> wrote:

> Or fixing the bug here at the right place solves two at once?

Look, the bug reported here was fixed. If you want to start changing
the behaviour of python-nav-beginning-of-defun (possibly breaking
other code which relies on the current behaviour) it should be
discussed separately.
Bug numbers are not a scarce resource.