bug#62031: 29.0.60; python-mode indentation after re.match

bug#62031: 29.0.60; python-mode indentation after re.match

[Rob Moss]

Hi there,

Starting from 'emacs -Q' I do the following:

1. Visit a new Python file ("~/test.py", say);

2. Enter the following lines:

----------------------------------------
import re

def test_re(string):
    if re.match('^[a-c]+$', string):
        print('yes')
    else:
        print('no')
----------------------------------------

3. Place the cursor on the "else:" line and press <TAB>.

This has the following effect on the buffer contents:

----------------------------------------
import re

def test_re(string):
    if re.match('^[a-c]+$', string):
        print('yes')
else:
        print('no')
----------------------------------------

Pressing <TAB> repeatedly has no effect, this incorrect indentation is
the only indentation it allows.

I experience this issue with "python-mode" and "python-ts-mode", but
if I change "re.match" to "re.matches", the correct indentation is
preserved. So presumably the "match" in "re.match" is being treated as
a match statement.

All the best,
Rob

In GNU Emacs 29.0.60 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.20, cairo version 1.16.0) of 2023-02-24 built on
 5050l-202293-l.mobility.unimelb.net.au
Repository revision: 94e70ed4261dbfcef679697dfa0dc2348a90dbdc
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12013000
System Description: Ubuntu 20.04.5 LTS

Configured using:
 'configure --with-native-compilation --with-json --with-cairo
 --with-tree-sitter --with-sqlite3 --prefix=/opt/emacs29 CFLAGS=-O2'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP
NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_AU.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Python

Minor modes in effect:
  consult-org-roam-mode: t
  org-roam-db-autosync-mode: t
  flyspell-mode: t
  pdf-occur-global-minor-mode: t
  pyvenv-mode: t
  smartparens-global-mode: t
  smartparens-mode: t
  global-company-mode: t
  company-mode: t
  global-flycheck-mode: t
  flycheck-mode: t
  recentf-mode: t
  desktop-save-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  shell-dirtrack-mode: t
  global-diff-hl-mode: t
  diff-hl-mode: t
  vertico-multiform-mode: t
  vertico-mode: t
  savehist-mode: t
  marginalia-mode: t
  mood-line-mode: t
  which-key-mode: t
  global-hl-todo-mode: t
  hl-todo-mode: t
  olivetti-mode: t
  delete-selection-mode: t
  global-hl-line-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  context-menu-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  line-number-mode: t
  visual-line-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Features:
(shadow mail-extr emacsbug cl-print rst shortdoc help-fns radix-tree
pulse skeleton misearch multi-isearch latexenc oc-basic rng-xsd
xsd-regexp rng-cmpct rng-nxml rng-valid nxml-mode nxml-outln nxml-rap
sgml-mode facemenu make-mode cal-move ess-sp6-d tabify ucs-normalize
org-roam-boxes consult-org-entity embark-org embark-consult embark
consult-org consult-org-roam consult-org-roam-buffer f consult-vertico
consult compat-28 org-roam-migrate org-roam-log org-roam-mode
org-roam-capture org-roam-id org-roam-node org-roam-db org-roam-utils
org-roam-compat org-roam org-capture org-attach emacsql-sqlite url-http
url-auth url-gw novice vertico-directory mule-util conf-mode sh-script
smie cursor-sensor reftex-dcr reftex reftex-loaddefs reftex-vars
tex-mode mixed-pitch adaptive-wrap reveal org-superstar ox-beamer ffap
ol-eww eww url-queue mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus
nnselect gnus-art mm-uu mml2015 mm-view mml-smime smime dig gnus-sum
gnus-group gnus-undo gnus-start gnus-dbus dbus gnus-cloud nnimap nnmail
mail-source utf7 nnoo gnus-spec gnus-int gnus-range gnus-win gnus
nnheader range ol-docview doc-view ol-bibtex ol-bbdb ol-w3m ol-doi
org-link-doi outshine outshine-org-cmds outorg ox-jfm ox-gfm ox-md
ox-odt rng-loc rng-uri rng-parse rng-match rng-dt rng-util rng-pttrn
nxml-parse nxml-ns nxml-enc xmltok nxml-util ox-latex ox-icalendar
org-agenda ox-html table ox-ascii ox-publish ox org-element org-persist
xdg org-id org-refile avl-tree ob-shell ob-dot ob-R ob-python ob-haskell
org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-src
ob-comint org-pcomplete org-list org-footnote org-faces org-entities
ob-emacs-lisp ob-core ob-eval org-cycle org-table ol org-fold
org-fold-core org-keys oc org-loaddefs cal-menu calendar cal-loaddefs
org-version org-compat org-macs eglot external-completion array jsonrpc
ert debug backtrace python-blue python compat treesit init
sanityinc-tomorrow-night-theme executable vc-git whitespace face-remap
flyspell ispell sort gnutls nsm dns elpher pdf-occur ibuf-ext ibuffer
ibuffer-loaddefs pdf-isearch let-alist pdf-misc pdf-tools pdf-view
magit-bookmark bookmark jka-compr pdf-cache pdf-info tq pdf-util
pdf-macs image-mode exif tablist tablist-filter semantic/wisent/comp
semantic/wisent semantic/wisent/wisent semantic/util-modes semantic/util
semantic semantic/tag semantic/lex semantic/fw mode-local cedet
mastodon-alt mastodon-media url-cache mastodon-tl ts s mastodon shr
pixel-fill kinsoku url-file svg dom mastodon-toot mastodon-iso persist
mastodon-http request ffmpeg-elf yaml-mode emacsql-sqlite-builtin sqlite
emacsql-sqlite-common emacsql emacsql-compiler xml bibtex rust-mode
poly-R ess-r-mode ess-r-flymake flymake-proc flymake ess-r-xref ess-trns
ess-r-package ess-r-completion ess-roxy ess-r-syntax ess-rd hideshow
ess-s-lang ess-help ess-mode ess-inf ess-tracebug compile tramp
tramp-loaddefs trampver tramp-integration tramp-compat parse-time
iso8601 ls-lisp poly-noweb poly-markdown markdown-mode noutline outline
polymode poly-lock polymode-base polymode-weave polymode-export
polymode-compat polymode-methods polymode-core polymode-classes
eieio-custom eieio-base ess lisp-mnt ess-utils ess-custom pyvenv eshell
esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg esh-module esh-groups
esh-util cus-edit pp cus-load files-x driverr smartparens thingatpt
company-oddmuse company-keywords company-etags etags fileloop generator
xref project company-gtags company-dabbrev-code company-dabbrev
company-files company-capf company-cmake company-xcode company-clang
company-semantic company-eclim company-template company-bbdb company
flycheck find-func recentf tree-widget wid-edit desktop frameset
magit-submodule magit-obsolete magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log which-func imenu
magit-diff smerge-mode diff git-commit log-edit message sendmail
yank-media puny dired dired-loaddefs rfc822 mml mml-sec epa derived epg
rfc6068 epg-config gnus-util text-property-search time-date mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils mailheader
pcvs-util add-log magit-core magit-autorevert autorevert filenotify
magit-margin magit-transient magit-process with-editor shell pcomplete
comint ansi-osc ring server ansi-color magit-mode transient format-spec
magit-git magit-section magit-utils crm diff-hl vc-dir ewoc vc
vc-dispatcher diff-mode vertico-multiform vertico savehist orderless
marginalia edmacro kmacro emojify advice apropos tar-mode arc-mode
archive-mode pcase ht dash mood-line undo-fu hercules which-key hl-todo
color-theme-sanityinc-tomorrow color olivetti delsel hl-line comp
comp-cstr warnings icons cl-extra help-mode use-package
use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core
undo-fu-autoloads haskell-mode-autoloads consult-dir-autoloads
python-blue-autoloads emacsql-sqlite-builtin-autoloads
mastodon-autoloads persist-autoloads consult-org-entity-autoloads
pdf-tools-autoloads orderless-autoloads consult-org-roam-autoloads
which-key-autoloads olivetti-autoloads smartparens-autoloads rx
flycheck-autoloads org-roam-autoloads emacsql-sqlite-autoloads
emacsql-autoloads finder-inf color-theme-sanityinc-tomorrow-autoloads
citar-embark-autoloads citar-autoloads citeproc-autoloads
string-inflection-autoloads parsebib-autoloads queue-autoloads
embark-consult-autoloads embark-autoloads marginalia-autoloads
vertico-autoloads polymode-autoloads ess-autoloads consult-autoloads
compat-autoloads ts-autoloads info package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib
early-init rmc iso-transl tooltip cconv eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 1904084 978312)
 (symbols 48 62463 106)
 (strings 32 307966 100211)
 (string-bytes 1 10375851)
 (vectors 16 147394)
 (vector-slots 8 3144694 1170710)
 (floats 8 905 2495)
 (intervals 56 45653 40421)
 (buffers 984 67))

bug#62031: 29.0.60; python-mode indentation after re.match

[Dmitry Gutov]

Hi! Thanks for the report.

On 07/03/2023 06:44, Rob Moss wrote:
> Hi there,
> 
> Starting from 'emacs -Q' I do the following:
> 
> 1. Visit a new Python file ("~/test.py", say);
> 
> 2. Enter the following lines:
> 
> ----------------------------------------
> import re
> 
> def test_re(string):
>      if re.match('^[a-c]+$', string):
>          print('yes')
>      else:
>          print('no')
> ----------------------------------------
> 
> 3. Place the cursor on the "else:" line and press <TAB>.
> 
> This has the following effect on the buffer contents:
> 
> ----------------------------------------
> import re
> 
> def test_re(string):
>      if re.match('^[a-c]+$', string):
>          print('yes')
> else:
>          print('no')
> ----------------------------------------
> 
> Pressing <TAB> repeatedly has no effect, this incorrect indentation is
> the only indentation it allows.
> 
> I experience this issue with "python-mode" and "python-ts-mode", but
> if I change "re.match" to "re.matches", the correct indentation is
> preserved. So presumably the "match" in "re.match" is being treated as
> a match statement.

In python-mode, this is a regression from

   commit 35d0190b0b91c085c73bbe6c2b8e93ea8288b589
   Author: Lele Gaifax <lele@metapensiero.it>
   Date:   Sun May 22 10:44:31 2022 +0200

       Properly indent Python PEP634 match/case blocks <...>

The addition of "match" in python-rx has that effect.

How to fix this without reverting the feature, is not immediately 
obvious to me, so I'm Cc'ing the author of that commit.

python-ts-mode reuses indentation code from python-mode, so it's no 
coincidence that is shares the problem.

bug#62031: 29.0.60; python-mode indentation after re.match

[Lele Gaifax]

Thank you for the head up, I will try to understand the issue and
hopefully fix the problem as time permits.

ciao, lele.
-- 
nickname: Lele Gaifax | Dire che Emacs è "conveniente" è come
real: Emanuele Gaifas | etichettare l'ossigeno come "utile"
lele@etour.tn.it      |                           -- Rens Troost

bug#62031: 29.0.60; python-mode indentation after re.match

[Rob Moss]

Thank you both very much.

This issue is a minor inconvenience, I've only encountered this in a
single block of code. And I didn't realise that python-ts-mode reuses
the python-mode indentation code, I had expected it would use
tree-sitter instead.

All the best,
Rob

On Wed, 8 Mar 2023 at 01:57, Lele Gaifax <lele@metapensiero.it> wrote:
>
> Thank you for the head up, I will try to understand the issue and
> hopefully fix the problem as time permits.
>
> ciao, lele.
> --
> nickname: Lele Gaifax | Dire che Emacs è "conveniente" è come
> real: Emanuele Gaifas | etichettare l'ossigeno come "utile"
> lele@etour.tn.it      |                           -- Rens Troost

bug#62031: 29.0.60; python-mode indentation after re.match

[Lele Gaifax]

I had a quick glance at this, and while I can confirm the problem with
the following test

    diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
    index 4f24c042c6a..9926c4b002f 100644
    --- a/test/lisp/progmodes/python-tests.el
    +++ b/test/lisp/progmodes/python-tests.el
    @@ -1982,6 +1982,17 @@ python-virt-bin
        (should (eq (car (python-indent-context)) :after-block-start))
        (should (= (python-indent-calculate-indentation) 8))))

    +(ert-deftest python-indent-after-re-match ()
    +  "Test BUG 62031 regression."
    +  (python-tests-with-temp-buffer
    +   "
    +def test_re(string):
    +    if re.match('^[a-c]+$', string):
    +        print('yes')
    +    else:
    +"
    +   (python-tests-look-at "else:")
    +   (should (= (python-indent-calculate-indentation) 4))))

I could not figure out how this can be fixed, with my current knowledge
on python.el.

There are several places where the regexp produced by `(python-rx
block-start)' is used, many within a `(looking-at ...)' form and several
others in a `(re-search-forward ...)': given that block starting
"keywords" are required to be at the beginning of a line, possibly
preceded by whitespace, I think that the problem may arise from those
usages which do not explicitly enforce the constraint, but unfortunately
all my attempts to do that a) didn't make the test above green and b)
broke one or more of the other tests.

I will try again, but I'm afraid this will go beyond my current spare
time availability.

bye, lele.

bug#62031: 29.0.60; python-mode indentation after re.match

[Ruijie Yu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Lele Gaifax <lele@metapensiero.it> writes:

> [...]
>     +    if re.match('^[a-c]+$', string):
>     +        print('yes')
>     +    else:
> [...]

I think you might want to add a variation to this test, like this:

    from re import match
    if match(...): pass

--
Best,


RY

bug#62031: 29.0.60; python-mode indentation after re.match

[Dmitry Gutov]

On 08/03/2023 15:23, Ruijie Yu via Bug reports for GNU Emacs, the Swiss 
army knife of text editors wrote:
> Lele Gaifax<lele@metapensiero.it>  writes:
> 
>> [...]
>>      +    if re.match('^[a-c]+$', string):
>>      +        print('yes')
>>      +    else:
>> [...]
> I think you might want to add a variation to this test, like this:
> 
>      from re import match
>      if match(...): pass

I think this would be testing some different part of the logic (forward 
statement navigation?).

Anyway, if you see any problem with indentation (with that snippet or 
not), please report.

bug#62031: 29.0.60; python-mode indentation after re.match

[Rob Moss]

Hi everyone,

Thank you very much for fixing this issue. I've built Emacs 29 from
this commit (34c1443) and the indentation is now correct.

On Thu, 9 Mar 2023 at 09:46, Dmitry Gutov <dgutov@yandex.ru> wrote:
> > I think you might want to add a variation to this test, like this:
> >
> >      from re import match
> >      if match(...): pass
>
> I think this would be testing some different part of the logic (forward
> statement navigation?).
>
> Anyway, if you see any problem with indentation (with that snippet or
> not), please report.

Yes, the above example appears to depend on different logic, because
the else statement is not correctly indented in this example (results
shown after pressing <TAB>):

    from re import match
    def test_re(string):
        if match('^[a-c]+$', string):
            pass
    else:
            pass

Should I file this as a new bug?

Sincerely,
Rob

bug#62031: 29.0.60; python-mode indentation after re.match

[Eli Zaretskii]

> Cc: Rob Moss <robm.dev@gmail.com>, Dmitry Gutov <dgutov@yandex.ru>
> From: Lele Gaifax <lele@metapensiero.it>
> Date: Wed, 08 Mar 2023 08:53:19 +0100
> 
> I had a quick glance at this, and while I can confirm the problem with
> the following test
> 
>     diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
>     index 4f24c042c6a..9926c4b002f 100644
>     --- a/test/lisp/progmodes/python-tests.el
>     +++ b/test/lisp/progmodes/python-tests.el
>     @@ -1982,6 +1982,17 @@ python-virt-bin
>         (should (eq (car (python-indent-context)) :after-block-start))
>         (should (= (python-indent-calculate-indentation) 8))))
> 
>     +(ert-deftest python-indent-after-re-match ()
>     +  "Test BUG 62031 regression."
>     +  (python-tests-with-temp-buffer
>     +   "
>     +def test_re(string):
>     +    if re.match('^[a-c]+$', string):
>     +        print('yes')
>     +    else:
>     +"
>     +   (python-tests-look-at "else:")
>     +   (should (= (python-indent-calculate-indentation) 4))))
> 
> I could not figure out how this can be fixed, with my current knowledge
> on python.el.
> 
> There are several places where the regexp produced by `(python-rx
> block-start)' is used, many within a `(looking-at ...)' form and several
> others in a `(re-search-forward ...)': given that block starting
> "keywords" are required to be at the beginning of a line, possibly
> preceded by whitespace, I think that the problem may arise from those
> usages which do not explicitly enforce the constraint, but unfortunately
> all my attempts to do that a) didn't make the test above green and b)
> broke one or more of the other tests.
> 
> I will try again, but I'm afraid this will go beyond my current spare
> time availability.

Thank you for your efforts.

kobarity, any ideas or suggestions?

bug#62031: 29.0.60; python-mode indentation after re.match

[Dmitry Gutov]

On 08/03/2023 09:53, Lele Gaifax wrote:
> I had a quick glance at this, and while I can confirm the problem with
> the following test
> 
>      diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
>      index 4f24c042c6a..9926c4b002f 100644
>      --- a/test/lisp/progmodes/python-tests.el
>      +++ b/test/lisp/progmodes/python-tests.el
>      @@ -1982,6 +1982,17 @@ python-virt-bin
>          (should (eq (car (python-indent-context)) :after-block-start))
>          (should (= (python-indent-calculate-indentation) 8))))
> 
>      +(ert-deftest python-indent-after-re-match ()
>      +  "Test BUG 62031 regression."
>      +  (python-tests-with-temp-buffer
>      +   "
>      +def test_re(string):
>      +    if re.match('^[a-c]+$', string):
>      +        print('yes')
>      +    else:
>      +"
>      +   (python-tests-look-at "else:")
>      +   (should (= (python-indent-calculate-indentation) 4))))
> 
> I could not figure out how this can be fixed, with my current knowledge
> on python.el.
> 
> There are several places where the regexp produced by `(python-rx
> block-start)' is used, many within a `(looking-at ...)' form and several
> others in a `(re-search-forward ...)': given that block starting
> "keywords" are required to be at the beginning of a line, possibly
> preceded by whitespace, I think that the problem may arise from those
> usages which do not explicitly enforce the constraint, but unfortunately
> all my attempts to do that a) didn't make the test above green and b)
> broke one or more of the other tests.

Right, the problem is that we need some post-check for block-start searches.

python-rx (like rx in general) is just a syntax for Emacs regexp, and it 
doesn't support lookbehind or etc.

After some edebug-ing, the patch below seems to fix this case. Maybe 
other uses of (python-rx block-start) need this treatment as well, but 
I'd like to leave that to someone who comes later. And a lot of them 
happen after a (back-to-indentation) call, so those seem covered too.

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 1f970633bfc..aff8dc206b4 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -5792,7 +5792,9 @@ python-info-dedenter-opening-block-positions
            (catch 'exit
              (while (python-nav--syntactically
                      (lambda ()
-                      (re-search-backward (python-rx block-start) nil t))
+                      (cl-loop do (re-search-backward (python-rx 
block-start) nil t)
+                               until (memq (char-before) '(nil ?\s ?\t 
?\n))
+                               finally return (point)))
                      #'<)
                (let ((indentation (current-indentation)))
                  (when (and (not (memq indentation collected-indentations))

bug#62031: 29.0.60; python-mode indentation after re.match

[Lele Gaifax]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Right, the problem is that we need some post-check for block-start
> searches.

Thank you Dmitry!

> After some edebug-ing, the patch below seems to fix this case.

Are you going to commit that?

bye, lele.

bug#62031: 29.0.60; python-mode indentation after re.match

[Dmitry Gutov]

On 08/03/2023 20:10, Lele Gaifax wrote:
> Are you going to commit that?

With no further comments, I committed it with your test.

Pushed to emacs-29, thanks all, and closing!

bug#62031: 29.0.60; python-mode indentation after re.match

[Ruijie Yu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08/03/2023 20:10, Lele Gaifax wrote:
>> Are you going to commit that?
>
> With no further comments, I committed it with your test.
>
> Pushed to emacs-29, thanks all, and closing!

Hi Dmitry,

As I mentioned in the other subthread, there is a variation to this
issue that remains unfixed.  The following test case, a slightly
modified version of the test case in your commit, has failed (tested on
7e1012765c40):

--8<---------------cut here---------------start------------->8---
(ert-deftest python-indent-after-bare-match ()
  (python-tests-with-temp-buffer
   "
from re import match
def test_re(string):
    if match('^[a-c]+$', string):
        print('yes')
    else:
    "
   (python-tests-look-at "else:")
   (should (= (python-indent-calculate-indentation) 4))))
--8<---------------cut here---------------end--------------->8---

Does it belong to this bug report?  Or should I file a new bug instead?

--
Best,


RY

bug#62031: 29.0.60; python-mode indentation after re.match

[kobarity]

[-- Attachment #1: Type: text/plain, Size: 384 bytes --]


Ruijie Yu wrote:
> As I mentioned in the other subthread, there is a variation to this
> issue that remains unfixed.  The following test case, a slightly
> modified version of the test case in your commit, has failed (tested on
> 7e1012765c40):

I think it's better to check the block-start is located at the
beginning of the line.  Attached is a patch which can pass the test
case.

[-- Attachment #2: 0001-Don-t-misindent-else-after-if-match-in-Python.patch --]
[-- Type: application/octet-stream, Size: 2460 bytes --]

From 9d9f0d9af22f033a3471fd87f9e72d874bbb8b51 Mon Sep 17 00:00:00 2001
From: kobarity <kobarity@gmail.com>
Date: Fri, 10 Mar 2023 00:08:41 +0900
Subject: [PATCH] Don't misindent 'else:' after 'if match:' in Python

* lisp/progmodes/python.el
(python-info-dedenter-opening-block-positions): Check that the block
start is at the beginning of the line (Bug#62031).
* test/lisp/progmodes/python-tests.el (python-indent-after-bare-match):
New test.

Co-authored-by: Ruijie Yu <ruijie@netyu.xyz>
---
 lisp/progmodes/python.el            |  7 +++----
 test/lisp/progmodes/python-tests.el | 12 ++++++++++++
 2 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 630250c15c3..136bf6a81db 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -5792,12 +5792,11 @@ python-info-dedenter-opening-block-positions
           (catch 'exit
             (while (python-nav--syntactically
                     (lambda ()
-                      (cl-loop while (re-search-backward (python-rx block-start) nil t)
-                               if (memq (char-before) '(nil ?\s ?\t ?\n))
-                               return t))
+                      (re-search-backward (python-rx block-start) nil t))
                     #'<)
               (let ((indentation (current-indentation)))
-                (when (and (not (memq indentation collected-indentations))
+                (when (and (= indentation (- (point) (line-beginning-position)))
+                           (not (memq indentation collected-indentations))
                            (or (not collected-indentations)
                                (< indentation
                                   (apply #'min collected-indentations)))
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 6928e313dc4..4461921f142 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -1994,6 +1994,18 @@ python-indent-after-re-match
    (python-tests-look-at "else:")
    (should (= (python-indent-calculate-indentation) 4))))
 
+(ert-deftest python-indent-after-bare-match ()
+  (python-tests-with-temp-buffer
+   "
+from re import match
+def test_re(string):
+    if match('^[a-c]+$', string):
+        print('yes')
+    else:
+    "
+   (python-tests-look-at "else:")
+   (should (= (python-indent-calculate-indentation) 4))))
+
 \f
 ;;; Filling
 
-- 
2.34.1

bug#62031: 29.0.60; python-mode indentation after re.match

[Dmitry Gutov]

On 09/03/2023 17:29, kobarity wrote:
> Ruijie Yu wrote:
>> As I mentioned in the other subthread, there is a variation to this
>> issue that remains unfixed.  The following test case, a slightly
>> modified version of the test case in your commit, has failed (tested on
>> 7e1012765c40):
> I think it's better to check the block-start is located at the
> beginning of the line.  Attached is a patch which can pass the test
> case.

Thank you, I've pushed a different patch which checks for the same thing.

This is consistent with the search in python-nav-forward-block, for 
example. So it seems correct.

bug#62031: 29.0.60; python-mode indentation after re.match

[Lele Gaifax]

kobarity <kobarity@gmail.com> writes:

> Ruijie Yu wrote:
>> As I mentioned in the other subthread, there is a variation to this
>> issue that remains unfixed.  The following test case, a slightly
>> modified version of the test case in your commit, has failed (tested on
>> 7e1012765c40):
>
> I think it's better to check the block-start is located at the
> beginning of the line.  Attached is a patch which can pass the test
> case.

Yes, that's indeed what I initially tried to do, but in the wrong way
(that is, attempting to over-complicate the regexp) :-)

Thank you!

bye, lele.

bug#62031: 29.0.60; python-mode indentation after re.match

[Dmitry Gutov]

On 09/03/2023 05:04, Ruijie Yu wrote:
> As I mentioned in the other subthread, there is a variation to this
> issue that remains unfixed.  The following test case, a slightly
> modified version of the test case in your commit, has failed (tested on
> 7e1012765c40):
> 
> --8<---------------cut here---------------start------------->8---
> (ert-deftest python-indent-after-bare-match ()
>    (python-tests-with-temp-buffer
>     "
> from re import match
> def test_re(string):
>      if match('^[a-c]+$', string):
>          print('yes')
>      else:
>      "
>     (python-tests-look-at "else:")
>     (should (= (python-indent-calculate-indentation) 4))))
> --8<---------------cut here---------------end--------------->8---

That you for clarification with the full example

> Does it belong to this bug report?  Or should I file a new bug instead?

It's the same piece of logic, I've pushed a fix in 29228e24f20 that 
makes the check stricter.

bug#62031: 29.0.60; python-mode indentation after re.match

[Dmitry Gutov]

On 08/03/2023 19:57, Dmitry Gutov wrote:
> After some edebug-ing, the patch below seems to fix this case

Small correction for cases outside of any blocks:

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 1f970633bfc..81475f31f60 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -5792,7 +5792,9 @@ python-info-dedenter-opening-block-positions
            (catch 'exit
              (while (python-nav--syntactically
                      (lambda ()
-                      (re-search-backward (python-rx block-start) nil t))
+                      (cl-loop for pt = (re-search-backward (python-rx 
block-start) nil t)
+                               until (memq (char-before) '(nil ?\s ?\t 
?\n))
+                               finally return pt))
                      #'<)
                (let ((indentation (current-indentation)))
                  (when (and (not (memq indentation collected-indentations))