* bug#45341: [PATCH] Make python-mode fontify more assignment statements
@ 2020-12-20 16:25 Dario Gjorgjevski
2020-12-20 18:04 ` Basil L. Contovounesios
2020-12-21 4:41 ` Lars Ingebrigtsen
0 siblings, 2 replies; 4+ messages in thread
From: Dario Gjorgjevski @ 2020-12-20 16:25 UTC (permalink / raw)
To: 45341
[-- Attachment #1: Type: text/plain, Size: 454 bytes --]
The current implementation in python-mode fontifies only basic
assignment statements. It makes mistakes on or entirely misses more
complex ones such as
[a, b, c] = 1, 2, 3
a, *b, c = range(10)
inst.a, inst.b, inst.c = 'foo', 'bar', 'baz'
(a, b, *c, d) = x, *y = 5, 6, 7, 8, 9
The patch attached below extends the fontification to work correctly
with all these cases. I tested things as well as I could, but would
appreciate further testing.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Patch to make python-mode fontify more assignment statements --]
[-- Type: text/x-diff, Size: 6392 bytes --]
From 5881c2e98d9457e04f6502921ab1cbe7a8e74ef5 Mon Sep 17 00:00:00 2001
From: Dario Gjorgjevski <dario.gjorgjevski@gmail.com>
Date: Sun, 20 Dec 2020 17:12:01 +0100
Subject: [PATCH] Make python-mode fontify more assignment statements
The current implementation in python-mode fontifies only basic
assignment statements. It makes mistakes on or entirely misses more
complex ones such as
[a, b, c] = 1, 2, 3
a, *b, c = range(10)
inst.a, inst.b, inst.c = 'foo', 'bar', 'baz'
(a, b, *c, d) = x, *y = 5, 6, 7, 8, 9
This commit extends the fontification to work correctly with all these
cases.
* lisp/progmodes/python.el (python-font-lock-assignment-matcher): New
function to match assignment statements.
(python-rx): Add `assignment-target' and `grouped-assignment-target'.
(python-font-lock-keywords-maximum-decoration): Add new matchers.
---
lisp/progmodes/python.el | 96 +++++++++++++++++++++++++++++-----------
1 file changed, 69 insertions(+), 27 deletions(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index d58b32f3c3..50bb841111 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -394,6 +394,12 @@ python-rx
(any ?' ?\") "__main__" (any ?' ?\")
(* space) ?:))
(symbol-name (seq (any letter ?_) (* (any word ?_))))
+ (assignment-target (seq (? ?*)
+ (* symbol-name ?.) symbol-name
+ (? ?\[ (+ (not ?\])) ?\])))
+ (grouped-assignment-target (seq (? ?*)
+ (* symbol-name ?.) (group symbol-name)
+ (? ?\[ (+ (not ?\])) ?\])))
(open-paren (or "{" "[" "("))
(close-paren (or "}" "]" ")"))
(simple-operator (any ?+ ?- ?/ ?& ?^ ?~ ?| ?* ?< ?> ?= ?%))
@@ -605,6 +611,18 @@ python-font-lock-keywords-level-2
`python-font-lock-keywords-level-1', as well as keywords and
builtins.")
+(defun python-font-lock-assignment-matcher (regexp)
+ "Font lock matcher for assignments based on REGEXP.
+Return nil if REGEXP matched within a `paren' context (to avoid,
+e.g., default values for arguments or passing arguments by name
+being treated as assignments) or is followed by an '=' sign (to
+avoid '==' being treated as an assignment."
+ (lambda (limit)
+ (let ((res (re-search-forward regexp limit t)))
+ (unless (or (python-syntax-context 'paren)
+ (equal (char-after (point)) ?=))
+ res))))
+
(defvar python-font-lock-keywords-maximum-decoration
`((python--font-lock-f-strings)
,@python-font-lock-keywords-level-2
@@ -652,33 +670,57 @@ python-font-lock-keywords-maximum-decoration
)
symbol-end)
. font-lock-type-face)
- ;; assignments
- ;; support for a = b = c = 5
- (,(lambda (limit)
- (let ((re (python-rx (group symbol-name)
- ;; subscript, like "[5]"
- (? ?\[ (+ (not ?\])) ?\]) (* space)
- ;; type hint, like ": int" or ": Mapping[int, str]"
- (? ?: (* space) (+ not-simple-operator) (* space))
- assignment-operator))
- (res nil))
- (while (and (setq res (re-search-forward re limit t))
- (or (python-syntax-context 'paren)
- (equal (char-after (point)) ?=))))
- res))
- (1 font-lock-variable-name-face nil nil))
- ;; support for a, b, c = (1, 2, 3)
- (,(lambda (limit)
- (let ((re (python-rx (group symbol-name) (* space)
- (* ?, (* space) symbol-name (* space))
- ?, (* space) symbol-name (* space)
- assignment-operator))
- (res nil))
- (while (and (setq res (re-search-forward re limit t))
- (goto-char (match-end 1))
- (python-syntax-context 'paren)))
- res))
- (1 font-lock-variable-name-face nil nil)))
+ ;; multiple assignment
+ ;; (note that type hints are not allowed for multiple assignments)
+ ;; a, b, c = 1, 2, 3
+ ;; a, *b, c = 1, 2, 3, 4, 5
+ ;; [a, b] = (1, 2)
+ ;; (l[1], l[2]) = (10, 11)
+ ;; (a, b, c, *d) = *x, y = 5, 6, 7, 8, 9
+ ;; (a,) = 'foo'
+ ;; (*a,) = ['foo', 'bar', 'baz']
+ ;; d.x, d.y[0], *d.z = 'a', 'b', 'c', 'd', 'e'
+ ;; and variants thereof
+ ;; the cases
+ ;; (a) = 5
+ ;; [a] = 5
+ ;; [*a] = 5, 6
+ ;; are handled separately below
+ (,(python-font-lock-assignment-matcher
+ (python-rx (? (or "[" "(") (* space))
+ grouped-assignment-target (* space) ?, (* space)
+ (* assignment-target (* space) ?, (* space))
+ (? assignment-target (* space))
+ (? ?, (* space))
+ (? (or ")" "]") (* space))
+ (group assignment-operator)))
+ (1 font-lock-variable-name-face)
+ (,(python-rx grouped-assignment-target)
+ (progn
+ (goto-char (match-end 1)) ; go back after the first symbol
+ (match-beginning 2)) ; limit the search until the assignment
+ nil
+ (1 font-lock-variable-name-face)))
+ ;; single assignment with type hints, e.g.
+ ;; a: int = 5
+ ;; b: Tuple[Optional[int], Union[Sequence[str], str]] = (None, 'foo')
+ ;; c: Collection = {1, 2, 3}
+ ;; d: Mapping[int, str] = {1: 'bar', 2: 'baz'}
+ (,(python-font-lock-assignment-matcher
+ (python-rx grouped-assignment-target (* space)
+ (? ?: (* space) (+ not-simple-operator) (* space))
+ assignment-operator))
+ (1 font-lock-variable-name-face))
+ ;; special cases
+ ;; (a) = 5
+ ;; [a] = 5
+ ;; [*a] = 5, 6
+ (,(python-font-lock-assignment-matcher
+ (python-rx (or "[" "(") (* space)
+ grouped-assignment-target (* space)
+ (or ")" "]") (* space)
+ assignment-operator))
+ (1 font-lock-variable-name-face)))
"Font lock keywords to use in python-mode for maximum decoration.
This decoration level includes everything in
--
2.28.0
[-- Attachment #3: Type: text/plain, Size: 149 bytes --]
Best regards,
Dario
--
$ keyserver=hkps://hkps.pool.sks-keyservers.net
$ keyid=744A4F0B4F1C9371
$ gpg --keyserver $keyserver --search-keys $keyid
^ permalink raw reply related [flat|nested] 4+ messages in thread
* bug#45341: [PATCH] Make python-mode fontify more assignment statements
2020-12-20 16:25 bug#45341: [PATCH] Make python-mode fontify more assignment statements Dario Gjorgjevski
@ 2020-12-20 18:04 ` Basil L. Contovounesios
2020-12-21 4:43 ` Lars Ingebrigtsen
2020-12-21 4:41 ` Lars Ingebrigtsen
1 sibling, 1 reply; 4+ messages in thread
From: Basil L. Contovounesios @ 2020-12-20 18:04 UTC (permalink / raw)
To: Dario Gjorgjevski; +Cc: 45341
Dario Gjorgjevski <dario.gjorgjevski@gmail.com> writes:
> +(defun python-font-lock-assignment-matcher (regexp)
> + "Font lock matcher for assignments based on REGEXP.
> +Return nil if REGEXP matched within a `paren' context (to avoid,
> +e.g., default values for arguments or passing arguments by name
> +being treated as assignments) or is followed by an '=' sign (to
> +avoid '==' being treated as an assignment."
> + (lambda (limit)
> + (let ((res (re-search-forward regexp limit t)))
> + (unless (or (python-syntax-context 'paren)
> + (equal (char-after (point)) ?=))
Nit: char-after already defaults to looking at point, but you could also
use following-char which always returns a natnum.
--
Basil
^ permalink raw reply [flat|nested] 4+ messages in thread
* bug#45341: [PATCH] Make python-mode fontify more assignment statements
2020-12-20 16:25 bug#45341: [PATCH] Make python-mode fontify more assignment statements Dario Gjorgjevski
2020-12-20 18:04 ` Basil L. Contovounesios
@ 2020-12-21 4:41 ` Lars Ingebrigtsen
1 sibling, 0 replies; 4+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-21 4:41 UTC (permalink / raw)
To: Dario Gjorgjevski; +Cc: 45341
Dario Gjorgjevski <dario.gjorgjevski@gmail.com> writes:
> The patch attached below extends the fontification to work correctly
> with all these cases. I tested things as well as I could, but would
> appreciate further testing.
I did some cursory testing, and it seemed to work well for me, so I've
applied your patch to Emacs 28.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 4+ messages in thread
* bug#45341: [PATCH] Make python-mode fontify more assignment statements
2020-12-20 18:04 ` Basil L. Contovounesios
@ 2020-12-21 4:43 ` Lars Ingebrigtsen
0 siblings, 0 replies; 4+ messages in thread
From: Lars Ingebrigtsen @ 2020-12-21 4:43 UTC (permalink / raw)
To: Basil L. Contovounesios; +Cc: Dario Gjorgjevski, 45341
"Basil L. Contovounesios" <contovob@tcd.ie> writes:
>> + (let ((res (re-search-forward regexp limit t)))
>> + (unless (or (python-syntax-context 'paren)
>> + (equal (char-after (point)) ?=))
>
> Nit: char-after already defaults to looking at point, but you could also
> use following-char which always returns a natnum.
The old code also used this idiom, so I didn't rewrite it before
applying. python.el mostly uses (eq (char-after) ?x) throughout its
code, though.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-12-21 4:43 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-12-20 16:25 bug#45341: [PATCH] Make python-mode fontify more assignment statements Dario Gjorgjevski
2020-12-20 18:04 ` Basil L. Contovounesios
2020-12-21 4:43 ` Lars Ingebrigtsen
2020-12-21 4:41 ` Lars Ingebrigtsen
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).