bug#69357: [PATCH] Improve fontification of Python assignment statement with type hints

bug#69357: [PATCH] Improve fontification of Python assignment statement with type hints

[kobarity]

[-- Attachment #1: Type: text/plain, Size: 782 bytes --]


In python-mode (not python-ts-mode), "List" in the type hints of the
following code are fontified as variable names.

#+begin_src python
a: List[List[CustomInt], List[CustomInt]] = []
#+end_src

This is due to misinterpretation as a multiple assignment statement
such as:

#+begin_src python
a, b = 1, 2
#+end_src

To address this issue and to improve fontification, I suggest to
fontify type hints in the assignment statement as type names before
processing multiple assignment statements, as in the attached patch.

--
In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.16.0, Xaw scroll bars) of 2024-02-24 built on ubuntu
Repository revision: eeb89a5cb292bffe40ba7d0b0cf81f82f8452bf8
Repository branch: master
System Description: Ubuntu 22.04.4 LTS

[-- Attachment #2: 0001-Improve-fontification-of-Python-assignment-statement.patch --]
[-- Type: text/plain, Size: 5398 bytes --]

From 801e51e91d56606b3f017a3916519d26956353e7 Mon Sep 17 00:00:00 2001
From: kobarity <kobarity@gmail.com>
Date: Sat, 24 Feb 2024 23:11:02 +0900
Subject: [PATCH] Improve fontification of Python assignment statement with
 type hints

* lisp/progmodes/python.el (python-font-lock-keywords-maximum-decoration):
Fontify type hints of assignment statement.
* test/lisp/progmodes/python-tests.el
(python-font-lock-assignment-statement-11)
(python-font-lock-assignment-statement-12)
(python-font-lock-assignment-statement-13)
(python-font-lock-assignment-statement-18): Add fontification of type
hints.
(python-font-lock-assignment-statement-19): New test.
---
 lisp/progmodes/python.el            | 31 ++++++++++++++++++-----------
 test/lisp/progmodes/python-tests.el | 20 +++++++++++++++++++
 2 files changed, 39 insertions(+), 12 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index bedc61408ef..074d2e25dd1 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -790,6 +790,25 @@ python-font-lock-keywords-maximum-decoration
            )
           symbol-end)
      . font-lock-type-face)
+    ;; single assignment with/without type hints, e.g.
+    ;;   a: int = 5
+    ;;   b: Tuple[Optional[int], Union[Sequence[str], str]] = (None, 'foo')
+    ;;   c: Collection = {1, 2, 3}
+    ;;   d: Mapping[int, str] = {1: 'bar', 2: 'baz'}
+    (,(python-font-lock-assignment-matcher
+       (python-rx (or line-start ?\;) (* space)
+                  grouped-assignment-target (* space)
+                  (? ?: (* space) (group (+ not-simple-operator)) (* space))
+                  (group assignment-operator)))
+     (1 font-lock-variable-name-face)
+     (3 'font-lock-operator-face)
+     (,(python-rx symbol-name)
+      (progn
+        (when-let ((type-start (match-beginning 2)))
+          (goto-char type-start))
+        (match-end 0))
+      nil
+      (0 font-lock-type-face)))
     ;; multiple assignment
     ;; (note that type hints are not allowed for multiple assignments)
     ;;   a, b, c = 1, 2, 3
@@ -822,18 +841,6 @@ python-font-lock-keywords-maximum-decoration
         (match-beginning 2))            ; limit the search until the assignment
       nil
       (1 font-lock-variable-name-face)))
-    ;; single assignment with type hints, e.g.
-    ;;   a: int = 5
-    ;;   b: Tuple[Optional[int], Union[Sequence[str], str]] = (None, 'foo')
-    ;;   c: Collection = {1, 2, 3}
-    ;;   d: Mapping[int, str] = {1: 'bar', 2: 'baz'}
-    (,(python-font-lock-assignment-matcher
-       (python-rx (or line-start ?\;) (* space)
-                  grouped-assignment-target (* space)
-                  (? ?: (* space) (+ not-simple-operator) (* space))
-                  (group assignment-operator)))
-     (1 font-lock-variable-name-face)
-     (2 'font-lock-operator-face))
     ;; special cases
     ;;   (a) = 5
     ;;   [a] = 5,
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 6c6cd9eee2b..8c7cfe196d2 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -385,7 +385,11 @@ python-font-lock-assignment-statement-11
   (python-tests-assert-faces
    "b: Tuple[Optional[int], Union[Sequence[str], str]] = (None, 'foo')"
    '((1 . font-lock-variable-name-face) (2)
+     (4 . font-lock-type-face) (9)
+     (10 . font-lock-type-face) (18)
      (19 . font-lock-builtin-face) (22)
+     (25 . font-lock-type-face) (30)
+     (31 . font-lock-type-face) (39)
      (40 . font-lock-builtin-face) (43)
      (46 . font-lock-builtin-face) (49)
      (52 . font-lock-operator-face) (53)
@@ -396,12 +400,14 @@ python-font-lock-assignment-statement-12
   (python-tests-assert-faces
    "c: Collection = {1, 2, 3}"
    '((1 . font-lock-variable-name-face) (2)
+     (4 . font-lock-type-face) (14)
      (15 . font-lock-operator-face) (16))))
 
 (ert-deftest python-font-lock-assignment-statement-13 ()
   (python-tests-assert-faces
    "d: Mapping[int, str] = {1: 'bar', 2: 'baz'}"
    '((1 . font-lock-variable-name-face) (2)
+     (4 . font-lock-type-face) (11)
      (12 . font-lock-builtin-face) (15)
      (17 . font-lock-builtin-face) (20)
      (22 . font-lock-operator-face) (23)
@@ -466,14 +472,28 @@ python-font-lock-assignment-statement-18
      (58 . font-lock-operator-face) (59)
      (62 . font-lock-operator-face) (63)
      (70 . font-lock-variable-name-face) (72)
+     (74 . font-lock-type-face) (82)
+     (83 . font-lock-type-face) (92)
      (94 . font-lock-operator-face) (95)
      (102 . font-lock-operator-face) (103)
      (111 . font-lock-variable-name-face) (114)
+     (116 . font-lock-type-face) (125)
      (126 . font-lock-operator-face) (127)
      (128 . font-lock-builtin-face) (131)
      (136 . font-lock-operator-face) (137)
      (144 . font-lock-keyword-face) (150))))
 
+(ert-deftest python-font-lock-assignment-statement-19 ()
+  (python-tests-assert-faces
+   "a: List[List[CustomInt], List[CustomInt]] = []"
+   '((1 . font-lock-variable-name-face) (2)
+     (4 . font-lock-type-face) (8)
+     (9 . font-lock-type-face) (13)
+     (14 . font-lock-type-face) (23)
+     (26 . font-lock-type-face) (30)
+     (31 . font-lock-type-face) (40)
+     (43 . font-lock-operator-face) (44))))
+
 (ert-deftest python-font-lock-operator-1 ()
   (python-tests-assert-faces
    "1 << 2 ** 3 == +4%-5|~6&7^8%9"
-- 
2.34.1

bug#69357: [PATCH] Improve fontification of Python assignment statement with type hints

[Stefan Kangas]

close 69357 30.1
thanks

kobarity <kobarity@gmail.com> writes:

> In python-mode (not python-ts-mode), "List" in the type hints of the
> following code are fontified as variable names.
>
> #+begin_src python
> a: List[List[CustomInt], List[CustomInt]] = []
> #+end_src
>
> This is due to misinterpretation as a multiple assignment statement
> such as:
>
> #+begin_src python
> a, b = 1, 2
> #+end_src
>
> To address this issue and to improve fontification, I suggest to
> fontify type hints in the assignment statement as type names before
> processing multiple assignment statements, as in the attached patch.

Your analysis seems correct, and the fix works here.  It's very nice to
see tests for this too, as always.

Thus, I've now pushed your change to master (commit c97e7a2da2e).
I'm consequently closing this bug report.

Thanks for your contribution!