bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling

[kobarity <kobarity@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1682 bytes --]

rehan malak wrote:
> Deleting all the "#" in the f-string of the gef.py file removes the problem on my side.
>  sed -i -e 's/#0{align/0{align/g' gef.py
>  sed -i -e 's/#0{width/0{width/g' gef.py
>  sed -i -e 's/#07x}/07x}/g' gef.py
>  sed -i -e 's/#06x}/06x}/g' gef.py
>  sed -i -e 's/#04x}/04x}/g' gef.py
>  sed -i -e 's/#4x}/4x}/g' gef.py
>  sed -i -e 's/#8x}/8x}/g' gef.py
>  sed -i -e 's/#10x}/10x}/g' gef.py
>  sed -i -e 's/#x}/x}/g' gef.py
> 
> Your smaller example Eli also contains a f-string with the "#".
> 
> This python f-string format specifier is described here:
> https://docs.python.org/3/library/string.html#format-specification-mini-language
> 
> Is this format specifier supported from the beginning ?
> 
> For example :
> 
> value = 0xab
> print(f"{value:x} is a value")
> 
> => color syntax : is a value    has the color defined by font-lock-string-face
> 
> while
> 
> value = 0xab
> print(f"{value:#x} is a value")
> 
> => color are messed up... is has color defined by font-lock-keyword-face and a value has color defined by default face

The mechanism of the hang is still unclear, but I found the cause of
the font-lock malfunction: when using `up-list' to find the
corresponding closing brace for an opening brace in an f-string, if
the format specifier contains "#", the rest of the string is
considered a comment and the search fails.  Therefore, it can be
worked around by temporarily binding `parse-sexp-ignore-comments' to
nil, as in the attached patch.

Hangs on gef.py can also be avoided with this patch, but it assumes
that the braces are properly closed.  If you remove the closing braces
of expressions containing "#", it will eventually hang.

[-- Attachment #2: 0001-Fix-font-lock-of-Python-f-strings.patch --]
[-- Type: text/plain, Size: 1999 bytes --]

From 697f942d70e40bcd9bd32596313a9a371fffe35c Mon Sep 17 00:00:00 2001
From: kobarity <kobarity@gmail.com>
Date: Wed, 11 Dec 2024 23:21:04 +0900
Subject: [PATCH] Fix font-lock of Python f-strings

* lisp/progmodes/python.el (python--font-lock-f-strings): Bind
'parse-sexp-ignore-comments' to nil so that we can look for
closing braces even if a hash is used in the format specifier.
* test/lisp/progmodes/python-tests.el
(python-font-lock-f-string-1): New test.  (Bug#74738)
---
 lisp/progmodes/python.el            | 4 +++-
 test/lisp/progmodes/python-tests.el | 8 ++++++++
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index cfa3cc59568..827f437013f 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -631,7 +631,9 @@ python--font-lock-f-strings
               (forward-char 1)          ;Just skip over {{
             (let ((beg (match-beginning 0))
                   (end (condition-case nil
-                           (progn (up-list 1) (min send (point)))
+                           (let ((parse-sexp-ignore-comments))
+                             (up-list 1)
+                             (min send (point)))
                          (scan-error send))))
               (goto-char end)
               (put-text-property beg end 'face nil))))
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 4cc4040d0ff..ac17c5b1263 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -709,6 +709,14 @@ python-font-lock-string-literal-concatenation
      (17 . font-lock-operator-face) (18)
      (19 . font-lock-string-face))))
 
+(ert-deftest python-font-lock-f-string-1 ()
+  "Test for bug#74738."
+  (python-tests-assert-faces
+   "print(f\"{value:#x} is a value\")"
+   '((1 . font-lock-builtin-face) (6)
+     (8 . font-lock-string-face) (9)
+     (19 . font-lock-string-face) (31))))
+
 \f
 ;;; Indentation
 
-- 
2.43.0