bug#22663: 25.0.91; python.el electric-indent misbehaviour with 'except:'

bug#22663: 25.0.91; python.el electric-indent misbehaviour with 'except:'

[Matthew Woodcraft]

If I have the following code and add the colon after the 'ValueError',
python.el's electric-indent helpfully dedents the 'except' line, but
also indents the line below.

def foo():
    try:
        a = int(s)
        except ValueError
    bar(a)

In practice this is never what I want to happen: if I'm adding
try/except around part of an existing function, the existing lines after
'except' are for the non-exceptional case.

(Seen in emacs 25.0.91; 24.5 is the same; checked with 'emacs -Q'.)


In GNU Emacs 25.0.91.1 (i586-pc-linux-gnu, GTK+ Version 3.14.5)
 of 2016-02-14, modified by Debian built on golux
Windowing system distributor 'The X.Org Foundation', version 11.0.11604000
System Description:	Debian GNU/Linux 8.3 (jessie)

Configured using:
 'configure --build i586-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs-snapshot:/etc/emacs:/usr/local/share/emacs/25.0.91/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/25.0.91/site-lisp:/usr/share/emacs/site-lisp
 --build i586-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs-snapshot:/etc/emacs:/usr/local/share/emacs/25.0.91/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/25.0.91/site-lisp:/usr/share/emacs/site-lisp
 --with-x=yes --with-x-toolkit=gtk3 --with-toolkit-scroll-bars
 'CFLAGS=-g -O2 -fstack-protector-strong -Wformat
 -Werror=format-security -Wall -fno-omit-frame-pointer'
 CPPFLAGS=-D_FORTIFY_SOURCE=2 LDFLAGS=-Wl,-z,relro'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
  value of $LC_CTYPE: en_GB.UTF-8
  locale-coding-system: utf-8-unix

bug#22663: python-mode: Fix electric indent bug after dedenting colon

[Joel Rosdahl]

[-- Attachment #1.1: Type: text/plain, Size: 488 bytes --]

This fixes bug #22663 ("25.0.91; python.el electric-indent misbehaviour
with 'except:'"). Note that the bug is not restricted to "except:" but
affects all statements that are dedented by an electric colon.

For instance, entering a colon after "else" in

    if do:
        something()
        else
    outside

results in

    if do:
        something()
    else:
        outside

before the bug fix but

    if do:
        something()
    else:
    outside

after the bug fix.

-- Joel

[-- Attachment #1.2: Type: text/html, Size: 895 bytes --]

[-- Attachment #2: 0001-python-mode-Fix-electric-indent-bug-after-dedenting-.patch --]
[-- Type: text/x-patch, Size: 2802 bytes --]

From 555fb47d0c3061346c2792e97206a0d64dd1e578 Mon Sep 17 00:00:00 2001
From: Joel Rosdahl <joel@rosdahl.net>
Date: Sun, 12 Feb 2017 16:10:47 +0100
Subject: [PATCH] python-mode: Fix electric indent bug after dedenting colon

This fixes bug #22663 ("25.0.91; python.el electric-indent misbehaviour
with 'except:'"). Note that the bug is not restricted to "except:" but
affects all statements that are dedented by an electric colon.

For instance, entering a colon after "else" in

    if do:
        something()
        else
    outside

results in

    if do:
        something()
    else:
        outside

before the bug fix but

    if do:
        something()
    else:
    outside

after the bug fix.
---
 lisp/progmodes/python.el            | 19 ++++++++++---------
 test/lisp/progmodes/python-tests.el |  3 +++
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 90b5e4e..467ac08 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -1286,16 +1286,17 @@ python-indent-post-self-insert-function
            (not (equal ?: (char-before (1- (point)))))
            (not (python-syntax-comment-or-string-p)))
       ;; Just re-indent dedenters
-      (let ((dedenter-pos (python-info-dedenter-statement-p))
-            (current-pos (point)))
+      (let ((dedenter-pos (python-info-dedenter-statement-p)))
         (when dedenter-pos
-          (save-excursion
-            (goto-char dedenter-pos)
-            (python-indent-line)
-            (unless (= (line-number-at-pos dedenter-pos)
-                       (line-number-at-pos current-pos))
-              ;; Reindent region if this is a multiline statement
-              (python-indent-region dedenter-pos current-pos)))))))))
+          (let ((start (copy-marker dedenter-pos))
+                (end (point-marker)))
+            (save-excursion
+              (goto-char start)
+              (python-indent-line)
+              (unless (= (line-number-at-pos start)
+                         (line-number-at-pos end))
+                ;; Reindent region if this is a multiline statement
+                (python-indent-region start end))))))))))
 
 \f
 ;;; Mark
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 1e6b867..e911921 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -1130,10 +1130,13 @@ python-tests-visible-string
 if do:
     something()
     else
+outside
 "
    (python-tests-look-at "else")
    (goto-char (line-end-position))
    (python-tests-self-insert ":")
+   (should (= (current-indentation) 0))
+   (python-tests-look-at "outside")
    (should (= (current-indentation) 0))))
 
 (ert-deftest python-indent-electric-colon-3 ()
-- 
2.7.4

bug#22663: python-mode: Fix electric indent bug after dedenting colon

[Joel Rosdahl]

[-- Attachment #1: Type: text/plain, Size: 371 bytes --]

It looks like the bug still exists in Emacs 26.1 and on latest master.

Attached is an updated patch with change log entries in the commit
message on suggested by
https://www.gnu.org/software/emacs/manual/html_node/emacs/Sending-Patches.html#Sending-Patches.
Please also note that the patch includes a test case update that
verifies that the bug has been fixed.

-- Joel

[-- Attachment #2: 0001-Fix-electric-indent-bug-in-python-mode-after-dedenti.patch --]
[-- Type: text/x-patch, Size: 2687 bytes --]

From 078168c86055d5b117dee4674038190821df674a Mon Sep 17 00:00:00 2001
From: Joel Rosdahl <joel@rosdahl.net>
Date: Thu, 27 Dec 2018 16:52:07 +0100
Subject: [PATCH] Fix electric indent bug in python-mode after dedenting colon

* list/progmodes/python.el (python-indent-post-self-insert-function):
  Use markers instead of positions when reindenting statement(s) after
  inserting electric colon to avoid reindenting too many
  statements (bug#22663).

* test/lisp/progmodes/python-tests.el (python-indent-electric-colon-2):
  Improve test case to also verify the fix of bug#22663.
---
 lisp/progmodes/python.el            | 19 ++++++++++---------
 test/lisp/progmodes/python-tests.el |  3 +++
 2 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 654a0d3aea..0bde1fb8db 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -1334,16 +1334,17 @@ the line will be re-indented automatically if needed."
            (not (equal ?: (char-before (1- (point)))))
            (not (python-syntax-comment-or-string-p)))
       ;; Just re-indent dedenters
-      (let ((dedenter-pos (python-info-dedenter-statement-p))
-            (current-pos (point)))
+      (let ((dedenter-pos (python-info-dedenter-statement-p)))
         (when dedenter-pos
-          (save-excursion
-            (goto-char dedenter-pos)
-            (python-indent-line)
-            (unless (= (line-number-at-pos dedenter-pos)
-                       (line-number-at-pos current-pos))
-              ;; Reindent region if this is a multiline statement
-              (python-indent-region dedenter-pos current-pos)))))))))
+          (let ((start (copy-marker dedenter-pos))
+                (end (point-marker)))
+            (save-excursion
+              (goto-char start)
+              (python-indent-line)
+              (unless (= (line-number-at-pos start)
+                         (line-number-at-pos end))
+                ;; Reindent region if this is a multiline statement
+                (python-indent-region start end))))))))))
 
 \f
 ;;; Mark
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 0b9f8484c1..3be3ef6cc7 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -1161,10 +1161,13 @@ def b()
 if do:
     something()
     else
+outside
 "
    (python-tests-look-at "else")
    (goto-char (line-end-position))
    (python-tests-self-insert ":")
+   (should (= (current-indentation) 0))
+   (python-tests-look-at "outside")
    (should (= (current-indentation) 0))))
 
 (ert-deftest python-indent-electric-colon-3 ()
-- 
2.17.1

bug#22663: python-mode: Fix electric indent bug after dedenting colon

[Eli Zaretskii]

> From: Joel Rosdahl <joel@rosdahl.net>
> Date: Thu, 27 Dec 2018 18:38:09 +0100
> 
> It looks like the bug still exists in Emacs 26.1 and on latest master.
> 
> Attached is an updated patch with change log entries in the commit
> message on suggested by
> https://www.gnu.org/software/emacs/manual/html_node/emacs/Sending-Patches.html#Sending-Patches.
> Please also note that the patch includes a test case update that
> verifies that the bug has been fixed.

Thanks, pushed to the master branch.