bug#51362: python font-lock-mode in emacs 28 seems broken

[Dario Gjorgjevski <dario.gjorgjevski@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1853 bytes --]

Hi,

Thanks a lot for the report and the reproducer, and sorry for the
regression that my patch introduced.
It has been a while since I wrote any Python, so I did not notice it.

What seems to be happening is that python-font-lock-assignment-matcher is
giving up too early.
Replacing its definition by
(defun python-font-lock-assignment-matcher (regexp)
  "Font lock matcher for assignments based on REGEXP.
Return nil if REGEXP matched within a `paren' context (to avoid,
e.g., default values for arguments or passing arguments by name
being treated as assignments) or is followed by an '=' sign (to
avoid '==' being treated as an assignment."
  (lambda (limit)
    (let ((res nil))
      (while (and (setq res (re-search-forward regexp limit t))
                  (or (python-syntax-context 'paren)
                      (equal (char-after (point)) ?=))))
      res)))
should fix the issue.
Would you be able to give it a try?

Unfortunately some issues still remain, all of which can be illustrated by
a file containing the following code:
CustomInt = int

def f(x: CustomInt) -> CustomInt:
    y = x + 1
    ys: Sequence[CustomInt] = [y, y + 1]
    res: CustomInt = sum(ys) + 1
    return res
When such a file is first opened:

   - The y in y = x + 1 is not highlighted — this is fixed by subsequent
   edits
   - The CustomInt in -> CustomInt is highlighted — this is fixed by
   subsequent edits
   - The CustomInt in Sequence[CustomInt] is highlighted — this is *not*
   fixed by subsequent edits

These are caused by the type hints (in the case of 1. — by the type hints
in the function's signature).
I am not sure how one would fix them.

Regarding reverting the patch: It can be done easily, but the previous code
had even more bugs happening around type hints.

Best regards,
Dario

[-- Attachment #2: Type: text/html, Size: 3101 bytes --]