From: Laurence Warne <laurencewarne@gmail.com>
To: Lars Ingebrigtsen <larsi@gnus.org>
Cc: 56757@debbugs.gnu.org
Subject: bug#56757: 29.0.50; Python f-string font-lock logic is incomplete
Date: Tue, 6 Sep 2022 11:08:32 +0100 [thread overview]
Message-ID: <CAE2oLqhhLRTONd_FPPWaEP_3jUdCSHG2RCEyvfaps4E-QBqUeQ@mail.gmail.com> (raw)
In-Reply-To: <875yi11vpc.fsf@gnus.org>
[-- Attachment #1.1: Type: text/plain, Size: 1338 bytes --]
Hi, I've attached a patch fixing up this test. The difference occurred with:
rf"""\x12 S \n \u1234 \U00010348 \N{Plus-Minus Sign}"""
(and equivalent variants: the same string but starting with fr""", fr''',
rf''').
The patch treats "{Plus-Minus Sign}" as an embedded expression within an
f-string, causing it to be fontified differently (previously it was
fontified the same as the rest of the string). This is invalid syntax
since "Plus-Minus Sign" is not a valid expression, so I think it's the
correct behaviour since it emphasises that "{Plus-Minus Sign}" will be
treated as an embedded expression even though it's invalid.
I hope that makes sense, thanks, Laurence
On Mon, Sep 5, 2022 at 8:13 PM Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Laurence Warne <laurencewarne@gmail.com> writes:
>
> > Hi, I've attached a patch which looks good to me locally using the above
> examples
> > and others.
> >
> > In addition to the above, mixed capitalizations are also covered:
> >
> https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals
> .
>
> I think your patch makes sense, but it leads to failures in:
>
> 1 unexpected results:
> FAILED python-font-lock-escape-sequence-multiline-string
>
> Could you have a look at that and possibly adjust the test too (if it's
> the test that's wrong here)?
>
>
[-- Attachment #1.2: Type: text/html, Size: 2079 bytes --]
[-- Attachment #2: 0001-Apply-syntax-highlighting-for-all-python-f-strings.patch --]
[-- Type: text/x-patch, Size: 3429 bytes --]
From 9244f15b1768f2f734f2728c65264e1f1107e391 Mon Sep 17 00:00:00 2001
From: Laurence Warne <laurencewarne@gmail.com>
Date: Sat, 30 Jul 2022 13:22:18 +0100
Subject: [PATCH] Apply syntax highlighting for all python f-strings
Apply syntax highlighting to all Python formatted string literals
(Bug#56757), including multiline strings and formatted strings also
having the raw string prefix ("r") either following or preceding "f".
* lisp/progmodes/python.el (python--f-string-p, python--font-lock-f-strings):
Edit functions to use a regular expression matching all f-strings.
---
lisp/progmodes/python.el | 17 ++++++++++++++---
test/lisp/progmodes/python-tests.el | 12 ++++++++----
2 files changed, 22 insertions(+), 7 deletions(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 147c5f248d..3247d7ad50 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -546,11 +546,22 @@ python-font-lock-syntactic-face-function
font-lock-string-face)
font-lock-comment-face))
+(defconst python--f-string-start-regexp
+ (rx bow
+ (or "f" "F" "fr" "Fr" "fR" "FR" "rf" "rF" "Rf" "RF")
+ (or "\"" "\"\"\"" "'" "'''"))
+ "A regular expression matching the beginning of an f-string.
+
+See URL `https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals'.")
+
(defun python--f-string-p (ppss)
"Return non-nil if the pos where PPSS was found is inside an f-string."
(and (nth 3 ppss)
- (let ((spos (1- (nth 8 ppss))))
- (and (memq (char-after spos) '(?f ?F))
+ (let* ((spos (1- (nth 8 ppss)))
+ (before-quote
+ (buffer-substring-no-properties (max (- spos 4) (point-min))
+ (min (+ spos 2) (point-max)))))
+ (and (string-match-p python--f-string-start-regexp before-quote)
(or (< (point-min) spos)
(not (memq (char-syntax (char-before spos)) '(?w ?_))))))))
@@ -569,7 +580,7 @@ python--font-lock-f-strings
(while
(progn
(while (and (not (python--f-string-p ppss))
- (re-search-forward "\\<f['\"]" limit 'move))
+ (re-search-forward python--f-string-start-regexp limit 'move))
(setq ppss (syntax-ppss)))
(< (point) limit))
(cl-assert (python--f-string-p ppss))
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 906f7eca7d..20a7a0132a 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -574,10 +574,14 @@ python-font-lock-escape-sequence-multiline-string
(195 . font-lock-string-face)
(196 . font-lock-constant-face)
(215 . font-lock-string-face) (218)
- (221 . font-lock-string-face) (274)
- (277 . font-lock-string-face) (330)
- (333 . font-lock-string-face) (386)
- (389 . font-lock-string-face) (442)
+ (221 . font-lock-string-face) (254)
+ (271 . font-lock-string-face) (274)
+ (277 . font-lock-string-face) (310)
+ (327 . font-lock-string-face) (330)
+ (333 . font-lock-string-face) (366)
+ (383 . font-lock-string-face) (386)
+ (389 . font-lock-string-face) (422)
+ (439 . font-lock-string-face) (442)
(444 . font-lock-string-face) (497)
(499 . font-lock-string-face) (552)
(555 . font-lock-string-face) (608)
--
2.30.2
next prev parent reply other threads:[~2022-09-06 10:08 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-25 9:06 bug#56757: 29.0.50; Python f-string font-lock logic is incomplete Dima Kogan
2022-07-30 14:58 ` bug#56757: " Laurence Warne
2022-07-30 15:51 ` bug#56757: 29.0.50; " Laurence Warne
2022-09-05 19:13 ` Lars Ingebrigtsen
2022-09-06 10:08 ` Laurence Warne [this message]
2022-09-06 10:28 ` Lars Ingebrigtsen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAE2oLqhhLRTONd_FPPWaEP_3jUdCSHG2RCEyvfaps4E-QBqUeQ@mail.gmail.com \
--to=laurencewarne@gmail.com \
--cc=56757@debbugs.gnu.org \
--cc=larsi@gnus.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.