From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Doug Davis Newsgroups: gmane.emacs.devel Subject: Revert recent buggy font-locking changes in python.el Date: Wed, 03 Mar 2021 16:28:45 -0500 Message-ID: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="38163"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (darwin) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Mar 03 22:30:12 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lHZ4N-0009ox-HJ for ged-emacs-devel@m.gmane-mx.org; Wed, 03 Mar 2021 22:30:11 +0100 Original-Received: from localhost ([::1]:36110 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lHZ4M-00046I-H8 for ged-emacs-devel@m.gmane-mx.org; Wed, 03 Mar 2021 16:30:10 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:39894) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lHZ36-00031s-SH for emacs-devel@gnu.org; Wed, 03 Mar 2021 16:28:52 -0500 Original-Received: from out4-smtp.messagingengine.com ([66.111.4.28]:42671) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lHZ34-000501-Ht for emacs-devel@gnu.org; Wed, 03 Mar 2021 16:28:52 -0500 Original-Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id E6BBC5C00E4 for ; Wed, 3 Mar 2021 16:28:47 -0500 (EST) Original-Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Wed, 03 Mar 2021 16:28:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ddavis.io; h= from:to:subject:date:message-id:mime-version:content-type; s= fm1; bh=2rw8gzzHwmbeOJUabGyi47f8dMy/gTaW/JUddwSP5Qg=; b=BMD+9sjM wZHEEdJI3RLKL2sbVTdof3/YbhuEn+u3xMcLz+fpnQ3yNoGeegAJZqToaTsHzVeq /cxcVq6wk875mwtwCxxSx4u3c3vWUBxPg6K0bKWic7xYK/+9jvvPF29oH3PbgTNB 9MKZJ3T46AXc0jBYj3eVA6ZJqQECXvXDIVh8+WsUXFlVIGHT7eBHg3UfNydqPpQX +JgcOy6pOm1/7yCn/qvbvjlLsBqSeef33FCoZ/ttrOBzOKdcZ8FM8vbKWbHnG6Hy moXfwJoWuMBd+OWw8/y3i2phT1AMQ5J2HzDBdRfQYibVXnihmGrH5SHPdpB4wK+R PUifeGktbFXh7Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; bh=2rw8gzzHwmbeOJUabGyi47f8dMy/g TaW/JUddwSP5Qg=; b=B7S9mBiwqOBMAJ/cU/fnhWFNzlfitiDpHM1QdsMKrc0WN hBXkQ4i/mIB814HCahhsE5ShieKessDTcm9yqM4bFDCJrLZX/sqVcjbl25At10PC CXfDtWTaQp5KaZUHKLChkXSMt+2j8VnJVyzojKjDFjlUL2IQ6/zSmYjN2MFpHva7 iRtxhc7sc/FIhxdwsqpZEVTiR1IzmkA2oNUiQDScji45XOk+09GleLHvZVC69AFT kRQjY6xFsLCUeu5iCDXSl3jTOcYmH60DY3GvaHcQvMSoxwV/WjXO1PRx5iqA8Ac7 vZLuNEhF3ABAHwO3FezLXM1wVrF1zPCkHvWy39uBQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledruddtvddgudegtdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhephffvufffkfgfgggtsehmtderre dtredtnecuhfhrohhmpeffohhughcuffgrvhhishcuoeguuggrvhhishesuggurghvihhs rdhioheqnecuggftrfgrthhtvghrnhepheekheduleeghedutedtueefhfduveffvdejte eigfdtkeeuvdduvdfggeffhfevnecukfhppedvgedrleekrdejiedrkedtnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepuggurghvihhsseguug grvhhishdrihho X-ME-Proxy: Original-Received: from localhost (c-24-98-76-80.hsd1.ga.comcast.net [24.98.76.80]) by mail.messagingengine.com (Postfix) with ESMTPA id EB8571080057 for ; Wed, 3 Mar 2021 16:28:46 -0500 (EST) Received-SPF: pass client-ip=66.111.4.28; envelope-from=ddavis@ddavis.io; helo=out4-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:265915 Archived-At: --=-=-= Content-Type: text/plain (Bug#45679) and (Bug#46233) report buggy font-locking in python.el. I was able to trace it back to b2ce94fa5e - but I haven't been able to figure out _what_ the bug is (it's easy to trigger the issue, i.e. trigger the wonky font-locking), but not obvious how it's happening). Would it be reasonable to revert those changes (the attached patch does it)? I would also be happy to help solve the underlying issue (by testing new patches or being pointed to any tips for debugging font locking issues). --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: attachment; filename=0001-Revert-python-font-locking-logic-that-introduced-bug.patch Content-Transfer-Encoding: quoted-printable >From 3915c45944daf63fed5275afd517ecdf186789bb Mon Sep 17 00:00:00 2001 From: Doug Davis Date: Wed, 3 Mar 2021 11:16:16 -0500 Subject: [PATCH] Revert python font-locking logic that introduced bugs. (Bug#45679) and (Bug#46233) are both caused by the code this commit reverts. Buggy code added in b2ce94fa5eecee0afd0e6237956cfb2b02b8bb0b * lisp/progmodes/python.el (python-rx): remove the additional assignment targets. (python-font-lock-assignment-matcher): remove new function that is only used in... (python-font-lock-keyword-maximum-decoration): remove new assignment font-locking logic that leads to the bugs quoted above. Bring back logic from before the commit quoted above. --- lisp/progmodes/python.el | 97 +++++++++++----------------------------- 1 file changed, 27 insertions(+), 70 deletions(-) diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el index afb96974b1..c3d77f15a8 100644 --- a/lisp/progmodes/python.el +++ b/lisp/progmodes/python.el @@ -4,7 +4,7 @@ =20 ;; Author: Fabi=C3=A1n E. Gallina ;; URL: https://github.com/fgallina/python.el -;; Version: 0.27.1 +;; Version: 0.27.2 ;; Package-Requires: ((emacs "24.2") (cl-lib "1.0")) ;; Maintainer: emacs-devel@gnu.org ;; Created: Jul 2010 @@ -394,12 +394,6 @@ python-rx (any ?' ?\") "__main__" (any ?' ?\") (* space) ?:)) (symbol-name (seq (any letter ?_) (* (any word ?_)))) - (assignment-target (seq (? ?*) - (* symbol-name ?.) symbol-name - (? ?\[ (+ (not ?\])) ?\]))) - (grouped-assignment-target (seq (? ?*) - (* symbol-name ?.) (group symb= ol-name) - (? ?\[ (+ (not ?\])) ?\]))) (open-paren (or "{" "[" "(")) (close-paren (or "}" "]" ")")) (simple-operator (any ?+ ?- ?/ ?& ?^ ?~ ?| ?* ?< ?> ?=3D ?%)) @@ -611,18 +605,6 @@ python-font-lock-keywords-level-2 `python-font-lock-keywords-level-1', as well as keywords and builtins.") =20 -(defun python-font-lock-assignment-matcher (regexp) - "Font lock matcher for assignments based on REGEXP. -Return nil if REGEXP matched within a `paren' context (to avoid, -e.g., default values for arguments or passing arguments by name -being treated as assignments) or is followed by an '=3D' sign (to -avoid '=3D=3D' being treated as an assignment." - (lambda (limit) - (let ((res (re-search-forward regexp limit t))) - (unless (or (python-syntax-context 'paren) - (equal (char-after (point)) ?=3D)) - res)))) - (defvar python-font-lock-keywords-maximum-decoration `((python--font-lock-f-strings) ,@python-font-lock-keywords-level-2 @@ -670,57 +652,32 @@ python-font-lock-keywords-maximum-decoration ) symbol-end) . font-lock-type-face) - ;; multiple assignment - ;; (note that type hints are not allowed for multiple assignments) - ;; a, b, c =3D 1, 2, 3 - ;; a, *b, c =3D 1, 2, 3, 4, 5 - ;; [a, b] =3D (1, 2) - ;; (l[1], l[2]) =3D (10, 11) - ;; (a, b, c, *d) =3D *x, y =3D 5, 6, 7, 8, 9 - ;; (a,) =3D 'foo' - ;; (*a,) =3D ['foo', 'bar', 'baz'] - ;; d.x, d.y[0], *d.z =3D 'a', 'b', 'c', 'd', 'e' - ;; and variants thereof - ;; the cases - ;; (a) =3D 5 - ;; [a] =3D 5 - ;; [*a] =3D 5, 6 - ;; are handled separately below - (,(python-font-lock-assignment-matcher - (python-rx (? (or "[" "(") (* space)) - grouped-assignment-target (* space) ?, (* space) - (* assignment-target (* space) ?, (* space)) - (? assignment-target (* space)) - (? ?, (* space)) - (? (or ")" "]") (* space)) - (group assignment-operator))) - (1 font-lock-variable-name-face) - (,(python-rx grouped-assignment-target) - (progn - (goto-char (match-end 1)) ; go back after the first symbol - (match-beginning 2)) ; limit the search until the assig= nment - nil - (1 font-lock-variable-name-face))) - ;; single assignment with type hints, e.g. - ;; a: int =3D 5 - ;; b: Tuple[Optional[int], Union[Sequence[str], str]] =3D (None, 'fo= o') - ;; c: Collection =3D {1, 2, 3} - ;; d: Mapping[int, str] =3D {1: 'bar', 2: 'baz'} - (,(python-font-lock-assignment-matcher - (python-rx grouped-assignment-target (* space) - (? ?: (* space) (+ not-simple-operator) (* space)) - assignment-operator)) - (1 font-lock-variable-name-face)) - ;; special cases - ;; (a) =3D 5 - ;; [a] =3D 5 - ;; [*a] =3D 5, 6 - (,(python-font-lock-assignment-matcher - (python-rx (or "[" "(") (* space) - grouped-assignment-target (* space) - (or ")" "]") (* space) - assignment-operator)) - (1 font-lock-variable-name-face))) + ;; assignments + ;; support for a =3D b =3D c =3D 5 + (,(lambda (limit) + (let ((re (python-rx (group (+ (any word ?. ?_))) + (? ?\[ (+ (not (any ?\]))) ?\]) (* space) + ;; A type, like " : int ". + (? ?: (* space) (+ (any word ?. ?_)) (* space= )) + assignment-operator)) + (res nil)) + (while (and (setq res (re-search-forward re limit t)) + (or (python-syntax-context 'paren) + (equal (char-after (point)) ?=3D)))) + res)) + (1 font-lock-variable-name-face nil nil)) + ;; support for a, b, c =3D (1, 2, 3) + (,(lambda (limit) + (let ((re (python-rx (group (+ (any word ?. ?_))) (* space) + (* ?, (* space) (+ (any word ?. ?_)) (* space= )) + ?, (* space) (+ (any word ?. ?_)) (* space) + assignment-operator)) + (res nil)) + (while (and (setq res (re-search-forward re limit t)) + (goto-char (match-end 1)) + (python-syntax-context 'paren))) + res)) + (1 font-lock-variable-name-face nil nil))) "Font lock keywords to use in python-mode for maximum decoration. =20 This decoration level includes everything in --=20 2.30.1 --=-=-=--