bug#63622: lisp/progmodes/python.el: performance regression introduced by multiline font-lock

[Tom Gillespie <tgbugs@gmail.com>]

> If the problem is so severe, I wonder how come this comes up only now,
> 9 months after those changes were installed.  It probably means these
> cases are quite rare in practice.  Nevertheless, it would be good to
> solve them, of course.

I suspect it is because there are 3 factors that have to be just right
to notice.

1 the opening paren and the quote must be immediately adjacent to get
exceptionally bad behavior, there is still some performance degradation
when there is separation, but it would be harder to notice.

2 a user would have to directly edit a dictionary literal with enough lines
to notice the slowdown.

3 assigning the dictionary to a variable mitigates the issue, so only a
dict that is not assigned results in the full slowdown.

> FWIW, python-ts-mode doesn't show performance issues in the examples
> you posted.

I would imagine so. I've continued trying to hunt down the source of the
issue, and it is triggered by setting python-font-lock-extend-region as
the font-lock-extend-after-change-region-function function for python
this is true in old versions of Emacs (e.g. 28.2) as well.

As far as I can tell the existing implementation for python font locking
has some quadratic behavior that is revealed when a region is extended
inside a nested dictionary with multiple lines.

> Any chance of your posting some real-life Python code where the issue
> rears its head?  I mean, real-life code that makes sense, not just
> syntactically correct code invented to make a point?

Yep, basically any nested dictionary literal with more than 15 lines is
affected. With a note that the issue is masked if there is an equal sign
(=) before the opening paren, which is the common case.

An example of the particular file that caused me to spot the issue:
https://github.com/tgbugs/pyontutils/blob/master/pyontutils/auth-config.py

> P.S.  Tom, please don't change the Subject when posting followups,
> please use the same Subject for all your messages that discuss this
> bug.

Ack, apologies. I will keep it the same in the future.

> Also, for the record, please state which Emacs version are you using.
> You didn't use "M-x report-emacs-bug" to submit the bug report, so
> this and other important information is missing from your OP.

Ok, I wasn't sure how to handle it in this case since I was able to
reproduce the issue in multiple versions.

For the record:
The version I spotted it on was the emacs-29 branch at
3bc5efb87e5ac9b7068e71307466b2d0220e92fb but everything
on emacs-29 after 4915ca5dd4245a909c046e6691e8d4a1919890c8
is affected (according to git bisect results). So 29.0.90 and 29.0.91
should be affected as well.