bug#45897: 27.1; python mode font-lock confused by string concatenation

["Jakub Ječmínek" <jecminek.k@gmail.com>]

Tom Tromey <tom@tromey.com> writes:

> Consider this 2-line Python file:
>
> x = "hello"""
> y = "confused"
>
> If you put this into a .py file, Emacs will font-lock the second line
> incorrectly.  It appears that Emacs thinks the `"""' on the first line
> is an opening triple quote.  However, it is actually string
> concatenation, with the second string being empty.
>
> I tripped over a case like this in some real code.

Hi, thanks for the bug report and sorry for the late response! I confirm
that this is an issue.

I've prepared a patch which handles this special case:

From f34c84b8b4c629c51d547dd41d20222b32f838bb Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jakub=20Je=C4=8Dm=C3=ADnek?= <jecminek.k@gmail.com>
Date: Fri, 19 Jan 2024 16:38:21 +0100
Subject: [PATCH] Fix syntax highlighting after string literal concatenation in
 python-mode

* lisp/progmodes/python.el (python-syntax-stringify): Fix incorrect font-lock after string
literal concatenation.
---
 lisp/progmodes/python.el | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index e2f614f52c2..d6534a7ddae 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -908,7 +908,11 @@ is used to limit the scan."
 (defun python-syntax-stringify ()
   "Put `syntax-table' property correctly on single/triple quotes."
   (let* ((ppss (save-excursion (backward-char 3) (syntax-ppss)))
+        (line-ppss (save-excursion (backward-char 3) (parse-partial-sexp
+                                                      (line-beginning-position) (point))))
          (string-start (and (eq t (nth 3 ppss)) (nth 8 ppss)))
+        (string-literal-concat (and (null string-start)
+                           (and (not (null (nth 3 line-ppss))) (nth 8 line-ppss))))
          (quote-starting-pos (- (point) 3))
          (quote-ending-pos (point)))
     (cond ((or (nth 4 ppss)             ;Inside a comment
@@ -921,6 +925,8 @@ is used to limit the scan."
           ((nth 5 ppss)
            ;; The first quote is escaped, so it's not part of a triple quote!
            (goto-char (1+ quote-starting-pos)))
+          ;; Handle string literal concatenation (bug#45897)
+         (string-literal-concat nil)
           ((null string-start)
            ;; This set of quotes delimit the start of a string.  Put
            ;; string fence syntax on last quote. (bug#49518)
--
2.39.3 (Apple Git-145)


If the patch is good enough, I'll sign the paperwork (I've most likely
crossed the line limit).

Best,

Jakub Ječmínek