From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Laurence Warne Newsgroups: gmane.emacs.bugs Subject: bug#56757: 29.0.50; Python f-string font-lock logic is incomplete Date: Tue, 6 Sep 2022 11:08:32 +0100 Message-ID: References: <878roh5zoj.fsf@secretsauce.net> <875yi11vpc.fsf@gnus.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="00000000000099b27f05e7ff61e1" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33282"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 56757@debbugs.gnu.org To: Lars Ingebrigtsen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Sep 06 12:58:22 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oVWHd-0008N4-DH for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 06 Sep 2022 12:58:21 +0200 Original-Received: from localhost ([::1]:43082 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oVWHc-0002tc-9G for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 06 Sep 2022 06:58:20 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:41206) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oVVVu-0008KY-Nd for bug-gnu-emacs@gnu.org; Tue, 06 Sep 2022 06:09:04 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:60925) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oVVVu-0003Uq-Eo for bug-gnu-emacs@gnu.org; Tue, 06 Sep 2022 06:09:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1oVVVu-0008SM-8K for bug-gnu-emacs@gnu.org; Tue, 06 Sep 2022 06:09:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Laurence Warne Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 06 Sep 2022 10:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 56757 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch moreinfo Original-Received: via spool by 56757-submit@debbugs.gnu.org id=B56757.166245893232487 (code B ref 56757); Tue, 06 Sep 2022 10:09:02 +0000 Original-Received: (at 56757) by debbugs.gnu.org; 6 Sep 2022 10:08:52 +0000 Original-Received: from localhost ([127.0.0.1]:49624 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oVVVj-0008Rt-J1 for submit@debbugs.gnu.org; Tue, 06 Sep 2022 06:08:52 -0400 Original-Received: from mail-vs1-f45.google.com ([209.85.217.45]:40865) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oVVVh-0008RX-C3 for 56757@debbugs.gnu.org; Tue, 06 Sep 2022 06:08:49 -0400 Original-Received: by mail-vs1-f45.google.com with SMTP id 190so11117624vsz.7 for <56757@debbugs.gnu.org>; Tue, 06 Sep 2022 03:08:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=MphuRMgr6hoivfmWKiBp5ioYepKR6Cs7YOd6nnwRax8=; b=JwMy6AESdKhkFvZlAFQtRblvqMWpoStn7UoVZidP9GggBsdUtpaNeLnXtKBT06diJB +Hb1ERhAjRJFJAzYb9i4bloBdoTjBJZeg8cF91+4AMHDxgho71vMRzM5FNs+OoNBIzah Og8yifnLI527saBJ8PFjaj7eS4iLzyDoJ8qdPJW0w8/H4sj4nHkvRz1Gh1lbj45tMzGK iOe6H2BoPBdNc38B9AwHfwHT7S6OBFVUiegXzosqJl6aIQDIKyWJgu6VGIB0lR3GMC3H D/DTwpIdC7CeMJTd3HaJjsBbUUa5Ih9oKxn74m+UQJgY2VMvmyca1FBRSxvSYhGnDLXt TgkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=MphuRMgr6hoivfmWKiBp5ioYepKR6Cs7YOd6nnwRax8=; b=aXaLgKIEVRf4saeh/q4YM1SZWY28m8rUHCn8uNn2KKNQngP/l0KB86FV754DETSsL1 GdFhbsIFBA9M9PswRR8dcd8tby6CUIzjm8Xi7IKM42XXNrYgH4qFQ2jW6g+fzoXTdMtw r7jFB9N6p/QNFaI9AAznxa9DYAz/EdA/UDC8+R6w71sI08bgJLt7PS5RhwkkqB2IXFwp eTSLHGXIJ2zXkSusnlp0Jf4opPAiqgkjRL+IE+FZn3f4nL3NWBWXiez3nyIixy891BGY UkW7p7HctYiwhOf3XPeTS4SNtawpZt65wm9fLUaeXB2tKTKJU4V3ncMMTs1qP31CBlXa y/4g== X-Gm-Message-State: ACgBeo0h1ERjqptYVUu+HRAmHq7pMP5hMI45Ojrd3yw/xCOXUyyDPX0M WK8TANj9sxmWZoUbzEeIovclWAoHOC+THJIgqsg= X-Google-Smtp-Source: AA6agR5R0C5JT3dTIKMNGOM11KVY1nELB5bnEzsoqBP4Ou0wkbfh1U2hQzUHgpT8M/Xvjp5EfEL12IE3M6uUXKWkHvw= X-Received: by 2002:a05:6102:22d8:b0:391:518a:dc55 with SMTP id a24-20020a05610222d800b00391518adc55mr11693365vsh.67.1662458923686; Tue, 06 Sep 2022 03:08:43 -0700 (PDT) In-Reply-To: <875yi11vpc.fsf@gnus.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:241630 Archived-At: --00000000000099b27f05e7ff61e1 Content-Type: multipart/alternative; boundary="00000000000099b27d05e7ff61df" --00000000000099b27d05e7ff61df Content-Type: text/plain; charset="UTF-8" Hi, I've attached a patch fixing up this test. The difference occurred with: rf"""\x12 S \n \u1234 \U00010348 \N{Plus-Minus Sign}""" (and equivalent variants: the same string but starting with fr""", fr''', rf'''). The patch treats "{Plus-Minus Sign}" as an embedded expression within an f-string, causing it to be fontified differently (previously it was fontified the same as the rest of the string). This is invalid syntax since "Plus-Minus Sign" is not a valid expression, so I think it's the correct behaviour since it emphasises that "{Plus-Minus Sign}" will be treated as an embedded expression even though it's invalid. I hope that makes sense, thanks, Laurence On Mon, Sep 5, 2022 at 8:13 PM Lars Ingebrigtsen wrote: > Laurence Warne writes: > > > Hi, I've attached a patch which looks good to me locally using the above > examples > > and others. > > > > In addition to the above, mixed capitalizations are also covered: > > > https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals > . > > I think your patch makes sense, but it leads to failures in: > > 1 unexpected results: > FAILED python-font-lock-escape-sequence-multiline-string > > Could you have a look at that and possibly adjust the test too (if it's > the test that's wrong here)? > > --00000000000099b27d05e7ff61df Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi, I've attached a patch fixing up this test. Th= e difference occurred with:
rf"""\x12 S \n \u1234 = \U00010348 \N{Plus-Minus Sign}"""
(and equivalent = variants: the same string but starting with fr""", fr'&#= 39;', rf''').

The patch treats &qu= ot;{Plus-Minus Sign}" as an embedded expression within an f-string, ca= using it to be fontified differently (previously it was fontified the same = as the rest of the string).=C2=A0 This is invalid syntax since "Plus-M= inus Sign" is not a valid expression, so I think it's the correct = behaviour since it emphasises that "{Plus-Minus Sign}" will be tr= eated as an embedded expression even though it's invalid.
I hope that makes sense, thanks, Laurence

On Mon, Sep = 5, 2022 at 8:13 PM Lars Ingebrigtsen <= larsi@gnus.org> wrote:
Laurence Warne <laurencewarne@gmail.com> writes:

> Hi, I've attached a patch which looks good to me locally using the= above examples
> and others.
>
> In addition to the above, mixed capitalizations are also covered:
> https://docs= .python.org/3/reference/lexical_analysis.html#string-and-bytes-literals= .

I think your patch makes sense, but it leads to failures in:

1 unexpected results:
=C2=A0 =C2=A0FAILED=C2=A0 python-font-lock-escape-sequence-multiline-string=

Could you have a look at that and possibly adjust the test too (if it's=
the test that's wrong here)?

--00000000000099b27d05e7ff61df-- --00000000000099b27f05e7ff61e1 Content-Type: text/x-patch; charset="US-ASCII"; name="0001-Apply-syntax-highlighting-for-all-python-f-strings.patch" Content-Disposition: attachment; filename="0001-Apply-syntax-highlighting-for-all-python-f-strings.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_l7q11thb0 RnJvbSA5MjQ0ZjE1YjE3NjhmMmY3MzRmMjcyOGM2NTI2NGUxZjExMDdlMzkxIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBMYXVyZW5jZSBXYXJuZSA8bGF1cmVuY2V3YXJuZUBnbWFpbC5j b20+CkRhdGU6IFNhdCwgMzAgSnVsIDIwMjIgMTM6MjI6MTggKzAxMDAKU3ViamVjdDogW1BBVENI XSBBcHBseSBzeW50YXggaGlnaGxpZ2h0aW5nIGZvciBhbGwgcHl0aG9uIGYtc3RyaW5ncwoKQXBw bHkgc3ludGF4IGhpZ2hsaWdodGluZyB0byBhbGwgUHl0aG9uIGZvcm1hdHRlZCBzdHJpbmcgbGl0 ZXJhbHMKKEJ1ZyM1Njc1NyksIGluY2x1ZGluZyBtdWx0aWxpbmUgc3RyaW5ncyBhbmQgZm9ybWF0 dGVkIHN0cmluZ3MgYWxzbwpoYXZpbmcgdGhlIHJhdyBzdHJpbmcgcHJlZml4ICgiciIpIGVpdGhl ciBmb2xsb3dpbmcgb3IgcHJlY2VkaW5nICJmIi4KCiogbGlzcC9wcm9nbW9kZXMvcHl0aG9uLmVs IChweXRob24tLWYtc3RyaW5nLXAsIHB5dGhvbi0tZm9udC1sb2NrLWYtc3RyaW5ncyk6CkVkaXQg ZnVuY3Rpb25zIHRvIHVzZSBhIHJlZ3VsYXIgZXhwcmVzc2lvbiBtYXRjaGluZyBhbGwgZi1zdHJp bmdzLgotLS0KIGxpc3AvcHJvZ21vZGVzL3B5dGhvbi5lbCAgICAgICAgICAgIHwgMTcgKysrKysr KysrKysrKystLS0KIHRlc3QvbGlzcC9wcm9nbW9kZXMvcHl0aG9uLXRlc3RzLmVsIHwgMTIgKysr KysrKystLS0tCiAyIGZpbGVzIGNoYW5nZWQsIDIyIGluc2VydGlvbnMoKyksIDcgZGVsZXRpb25z KC0pCgpkaWZmIC0tZ2l0IGEvbGlzcC9wcm9nbW9kZXMvcHl0aG9uLmVsIGIvbGlzcC9wcm9nbW9k ZXMvcHl0aG9uLmVsCmluZGV4IDE0N2M1ZjI0OGQuLjMyNDdkN2FkNTAgMTAwNjQ0Ci0tLSBhL2xp c3AvcHJvZ21vZGVzL3B5dGhvbi5lbAorKysgYi9saXNwL3Byb2dtb2Rlcy9weXRob24uZWwKQEAg LTU0NiwxMSArNTQ2LDIyIEBAIHB5dGhvbi1mb250LWxvY2stc3ludGFjdGljLWZhY2UtZnVuY3Rp b24KICAgICAgICAgZm9udC1sb2NrLXN0cmluZy1mYWNlKQogICAgIGZvbnQtbG9jay1jb21tZW50 LWZhY2UpKQogCisoZGVmY29uc3QgcHl0aG9uLS1mLXN0cmluZy1zdGFydC1yZWdleHAKKyAgKHJ4 IGJvdworICAgICAgKG9yICJmIiAiRiIgImZyIiAiRnIiICJmUiIgIkZSIiAicmYiICJyRiIgIlJm IiAiUkYiKQorICAgICAgKG9yICJcIiIgIlwiXCJcIiIgIiciICInJyciKSkKKyAgIkEgcmVndWxh ciBleHByZXNzaW9uIG1hdGNoaW5nIHRoZSBiZWdpbm5pbmcgb2YgYW4gZi1zdHJpbmcuCisKK1Nl ZSBVUkwgYGh0dHBzOi8vZG9jcy5weXRob24ub3JnLzMvcmVmZXJlbmNlL2xleGljYWxfYW5hbHlz aXMuaHRtbCNzdHJpbmctYW5kLWJ5dGVzLWxpdGVyYWxzJy4iKQorCiAoZGVmdW4gcHl0aG9uLS1m LXN0cmluZy1wIChwcHNzKQogICAiUmV0dXJuIG5vbi1uaWwgaWYgdGhlIHBvcyB3aGVyZSBQUFNT IHdhcyBmb3VuZCBpcyBpbnNpZGUgYW4gZi1zdHJpbmcuIgogICAoYW5kIChudGggMyBwcHNzKQot ICAgICAgIChsZXQgKChzcG9zICgxLSAobnRoIDggcHBzcykpKSkKLSAgICAgICAgIChhbmQgKG1l bXEgKGNoYXItYWZ0ZXIgc3BvcykgJyg/ZiA/RikpCisgICAgICAgKGxldCogKChzcG9zICgxLSAo bnRoIDggcHBzcykpKQorICAgICAgICAgICAgICAoYmVmb3JlLXF1b3RlCisgICAgICAgICAgICAg ICAoYnVmZmVyLXN1YnN0cmluZy1uby1wcm9wZXJ0aWVzIChtYXggKC0gc3BvcyA0KSAocG9pbnQt bWluKSkKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKG1p biAoKyBzcG9zIDIpIChwb2ludC1tYXgpKSkpKQorICAgICAgICAgKGFuZCAoc3RyaW5nLW1hdGNo LXAgcHl0aG9uLS1mLXN0cmluZy1zdGFydC1yZWdleHAgYmVmb3JlLXF1b3RlKQogICAgICAgICAg ICAgICAob3IgKDwgKHBvaW50LW1pbikgc3BvcykKICAgICAgICAgICAgICAgICAgIChub3QgKG1l bXEgKGNoYXItc3ludGF4IChjaGFyLWJlZm9yZSBzcG9zKSkgJyg/dyA/XykpKSkpKSkpCiAKQEAg LTU2OSw3ICs1ODAsNyBAQCBweXRob24tLWZvbnQtbG9jay1mLXN0cmluZ3MKICAgICAod2hpbGUK ICAgICAgICAgKHByb2duCiAgICAgICAgICAgKHdoaWxlIChhbmQgKG5vdCAocHl0aG9uLS1mLXN0 cmluZy1wIHBwc3MpKQotICAgICAgICAgICAgICAgICAgICAgIChyZS1zZWFyY2gtZm9yd2FyZCAi XFw8ZlsnXCJdIiBsaW1pdCAnbW92ZSkpCisgICAgICAgICAgICAgICAgICAgICAgKHJlLXNlYXJj aC1mb3J3YXJkIHB5dGhvbi0tZi1zdHJpbmctc3RhcnQtcmVnZXhwIGxpbWl0ICdtb3ZlKSkKICAg ICAgICAgICAgIChzZXRxIHBwc3MgKHN5bnRheC1wcHNzKSkpCiAgICAgICAgICAgKDwgKHBvaW50 KSBsaW1pdCkpCiAgICAgICAoY2wtYXNzZXJ0IChweXRob24tLWYtc3RyaW5nLXAgcHBzcykpCmRp ZmYgLS1naXQgYS90ZXN0L2xpc3AvcHJvZ21vZGVzL3B5dGhvbi10ZXN0cy5lbCBiL3Rlc3QvbGlz cC9wcm9nbW9kZXMvcHl0aG9uLXRlc3RzLmVsCmluZGV4IDkwNmY3ZWNhN2QuLjIwYTdhMDEzMmEg MTAwNjQ0Ci0tLSBhL3Rlc3QvbGlzcC9wcm9nbW9kZXMvcHl0aG9uLXRlc3RzLmVsCisrKyBiL3Rl c3QvbGlzcC9wcm9nbW9kZXMvcHl0aG9uLXRlc3RzLmVsCkBAIC01NzQsMTAgKzU3NCwxNCBAQCBw eXRob24tZm9udC1sb2NrLWVzY2FwZS1zZXF1ZW5jZS1tdWx0aWxpbmUtc3RyaW5nCiAgICAgICgx OTUgLiBmb250LWxvY2stc3RyaW5nLWZhY2UpCiAgICAgICgxOTYgLiBmb250LWxvY2stY29uc3Rh bnQtZmFjZSkKICAgICAgKDIxNSAuIGZvbnQtbG9jay1zdHJpbmctZmFjZSkgKDIxOCkKLSAgICAg KDIyMSAuIGZvbnQtbG9jay1zdHJpbmctZmFjZSkgKDI3NCkKLSAgICAgKDI3NyAuIGZvbnQtbG9j ay1zdHJpbmctZmFjZSkgKDMzMCkKLSAgICAgKDMzMyAuIGZvbnQtbG9jay1zdHJpbmctZmFjZSkg KDM4NikKLSAgICAgKDM4OSAuIGZvbnQtbG9jay1zdHJpbmctZmFjZSkgKDQ0MikKKyAgICAgKDIy MSAuIGZvbnQtbG9jay1zdHJpbmctZmFjZSkgKDI1NCkKKyAgICAgKDI3MSAuIGZvbnQtbG9jay1z dHJpbmctZmFjZSkgKDI3NCkKKyAgICAgKDI3NyAuIGZvbnQtbG9jay1zdHJpbmctZmFjZSkgKDMx MCkKKyAgICAgKDMyNyAuIGZvbnQtbG9jay1zdHJpbmctZmFjZSkgKDMzMCkKKyAgICAgKDMzMyAu IGZvbnQtbG9jay1zdHJpbmctZmFjZSkgKDM2NikKKyAgICAgKDM4MyAuIGZvbnQtbG9jay1zdHJp bmctZmFjZSkgKDM4NikKKyAgICAgKDM4OSAuIGZvbnQtbG9jay1zdHJpbmctZmFjZSkgKDQyMikK KyAgICAgKDQzOSAuIGZvbnQtbG9jay1zdHJpbmctZmFjZSkgKDQ0MikKICAgICAgKDQ0NCAuIGZv bnQtbG9jay1zdHJpbmctZmFjZSkgKDQ5NykKICAgICAgKDQ5OSAuIGZvbnQtbG9jay1zdHJpbmct ZmFjZSkgKDU1MikKICAgICAgKDU1NSAuIGZvbnQtbG9jay1zdHJpbmctZmFjZSkgKDYwOCkKLS0g CjIuMzAuMgoK --00000000000099b27f05e7ff61e1--