* bug#15295: which-func-mode slow in long Python tuple
@ 2013-09-07 0:47 Dale
2013-10-26 9:16 ` bug#15295: python mode slow to unusable Alex V. Koval
2013-12-24 20:08 ` bug#15295: Fabián Ezequiel Gallina
0 siblings, 2 replies; 6+ messages in thread
From: Dale @ 2013-09-07 0:47 UTC (permalink / raw)
To: 15295
I happen to have a Python source file that has a relatively long tuple
at the module top level, i.e. a Python source file containing:
----------
foo = (
"item 1",
"item 2",
# ...and so on for ~500 lines
)
----------
I also use which-function-mode. If I go to the end of that tuple and
move the cursor in to it, Emacs becomes unusably slow. It will appear
to lock up and eat 100% CPU for 10-20 seconds each time I move the
cursor within the end of that tuple. Emacs remains responsive at the
top of the tuple.
I think this is happening because python-info-current-defun is slow
when dealing with long tuples. (Maybe lists, dicts, and other things
too; I only tested tuples.) Here's some elisp to produce a test case
and benchmark python-info-current-defun:
----------
(progn
(set-buffer (generate-new-buffer "*test*"))
(python-mode)
(insert "foo = (\n")
(dotimes (_ 500) (insert " \"bar\",\n"))
(insert ")\n")
(forward-line -2)
(message "%S" (benchmark-run (python-info-current-defun))))
----------
This makes a python-mode buffer named "*test*" containing only a
500-item Python tuple, as in my above example. On my hardware, the
above benchmark-run yields a result such as "(7.364507 131
0.9572049999999979)", i.e. 7.3 seconds to run.
Once that *test* buffer is created, feel free to turn on
which-function-mode in there and see that Emacs locks up every time you
move the cursor around in the end of that tuple. (which-function-mode
seems to be taking about twice the time reported by benchmark-run.
Perhaps it's calling python-info-current-defun twice?)
I have reproduced this behavior with "emacs -Q" using an Emacs I just
built from trunk, looks like revision 114162. (I get Emacs from Git,
where the master branch is 0f1532f2fe2.) I have also reproduced this
with python.el from the emacs-24 branch, looks like revision 111403.
Thanks to everyone who develops Emacs, an indispensable tool for me!
Dale
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#15295: python mode slow to unusable
2013-09-07 0:47 bug#15295: which-func-mode slow in long Python tuple Dale
@ 2013-10-26 9:16 ` Alex V. Koval
2013-10-26 11:39 ` Michael Heerdegen
2013-12-24 20:08 ` bug#15295: Fabián Ezequiel Gallina
1 sibling, 1 reply; 6+ messages in thread
From: Alex V. Koval @ 2013-10-26 9:16 UTC (permalink / raw)
To: 15295
For me this is happen as well. Emacs, starting from version 24.3 became
so slow in Python mode that I had to tell all developers at our company
to use version 24.2 until I sorted this out.
Sit today and started trying various emacs versions, and calling
different functions. The suggested test case from original author above,
runs with this benchmark:
(7.3956507 53 1.8788885930000063)
In fact, when I enable which-function-mode and just try to open
one of our project files, it reads it 62 seconds. *Same* file opened
with emacs 24.2 reads < 1second.
Same thing happens when I try to call 'help-imenu' - 46 seconds. In
emacs 24.2 - less then 1 second.
I have this bug in version 24.3 and 'bzr' current:
* Emacs branch: trunk
* Revision: 114814
* Emacs version number: 24.3.50
Please tell me what additional information should I provide.
Not very big expert in Lisp but may try to debug it more
to detail.
WBR, Alex
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#15295: python mode slow to unusable
2013-10-26 9:16 ` bug#15295: python mode slow to unusable Alex V. Koval
@ 2013-10-26 11:39 ` Michael Heerdegen
2013-10-27 4:23 ` Stefan Monnier
0 siblings, 1 reply; 6+ messages in thread
From: Michael Heerdegen @ 2013-10-26 11:39 UTC (permalink / raw)
To: Alex V. Koval; +Cc: 15295
Alex V. Koval <alex@ua2web.com> writes:
> For me this is happen as well. Emacs, starting from version 24.3 became
> so slow in Python mode that I had to tell all developers at our company
> to use version 24.2 until I sorted this out.
>
> Sit today and started trying various emacs versions, and calling
> different functions. The suggested test case from original author above,
> runs with this benchmark:
>
> (7.3956507 53 1.8788885930000063)
I profiled a bit, and, at least in this example, these two functions
seem to be extremely inefficient in combination:
(defun python-nav-beginning-of-statement ()
"Move to start of current statement."
(interactive "^")
(while (and (or (back-to-indentation) t)
(not (bobp))
(when (or
(save-excursion
(forward-line -1)
(python-info-line-ends-backslash-p))
(python-syntax-context 'string)
(python-syntax-context 'paren))
(forward-line -1))))
(point-marker))
(defun python-info-line-ends-backslash-p (&optional line-number)
"Return non-nil if current line ends with backslash.
With optional argument LINE-NUMBER, check that line instead."
(save-excursion
(save-restriction
(widen)
(when line-number
(python-util-goto-line line-number))
(while (and (not (eobp))
(goto-char (line-end-position))
(python-syntax-context 'paren)
(not (equal (char-before (point)) ?\\)))
(forward-line 1))
(when (equal (char-before) ?\\)
(point-marker)))))
They consume most of the time used. While the first function goes
backward, the second goes forward to the end in every loop cycle. This
makes the thing O(n^2), with n being the number of lines of the
expression.
I don't know Python, so I can't make any suggestions. Who can? At
least, changing the order of `or' expressions in
`python-nav-beginning-of-statement' seems to help in the example case:
(defun python-nav-beginning-of-statement ()
"Move to start of current statement."
(interactive "^")
(while (and (or (back-to-indentation) t)
(not (bobp))
(when (or
(python-syntax-context 'string)
(python-syntax-context 'paren)
(save-excursion
(forward-line -1)
(python-info-line-ends-backslash-p)))
(forward-line -1))))
(point-marker))
It's also not efficient how often `syntax-ppss' is called all the time.
Regards,
Michael.
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#15295: python mode slow to unusable
2013-10-26 11:39 ` Michael Heerdegen
@ 2013-10-27 4:23 ` Stefan Monnier
2013-10-27 8:01 ` Andreas Röhler
0 siblings, 1 reply; 6+ messages in thread
From: Stefan Monnier @ 2013-10-27 4:23 UTC (permalink / raw)
To: Fabián Ezequiel Gallina; +Cc: 15295
Could you take a look at this, as well?
Stefan
>>>>> "Michael" == Michael Heerdegen <michael_heerdegen@web.de> writes:
> Alex V. Koval <alex@ua2web.com> writes:
>> For me this is happen as well. Emacs, starting from version 24.3 became
>> so slow in Python mode that I had to tell all developers at our company
>> to use version 24.2 until I sorted this out.
>>
>> Sit today and started trying various emacs versions, and calling
>> different functions. The suggested test case from original author above,
>> runs with this benchmark:
>>
>> (7.3956507 53 1.8788885930000063)
> I profiled a bit, and, at least in this example, these two functions
> seem to be extremely inefficient in combination:
> (defun python-nav-beginning-of-statement ()
> "Move to start of current statement."
> (interactive "^")
> (while (and (or (back-to-indentation) t)
> (not (bobp))
> (when (or
> (save-excursion
> (forward-line -1)
> (python-info-line-ends-backslash-p))
> (python-syntax-context 'string)
> (python-syntax-context 'paren))
> (forward-line -1))))
> (point-marker))
> (defun python-info-line-ends-backslash-p (&optional line-number)
> "Return non-nil if current line ends with backslash.
> With optional argument LINE-NUMBER, check that line instead."
> (save-excursion
> (save-restriction
> (widen)
> (when line-number
> (python-util-goto-line line-number))
> (while (and (not (eobp))
> (goto-char (line-end-position))
> (python-syntax-context 'paren)
> (not (equal (char-before (point)) ?\\)))
> (forward-line 1))
> (when (equal (char-before) ?\\)
> (point-marker)))))
> They consume most of the time used. While the first function goes
> backward, the second goes forward to the end in every loop cycle. This
> makes the thing O(n^2), with n being the number of lines of the
> expression.
> I don't know Python, so I can't make any suggestions. Who can? At
> least, changing the order of `or' expressions in
> `python-nav-beginning-of-statement' seems to help in the example case:
> (defun python-nav-beginning-of-statement ()
> "Move to start of current statement."
> (interactive "^")
> (while (and (or (back-to-indentation) t)
> (not (bobp))
> (when (or
> (python-syntax-context 'string)
> (python-syntax-context 'paren)
> (save-excursion
> (forward-line -1)
> (python-info-line-ends-backslash-p)))
> (forward-line -1))))
> (point-marker))
> It's also not efficient how often `syntax-ppss' is called all the time.
> Regards,
> Michael.
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#15295: python mode slow to unusable
2013-10-27 4:23 ` Stefan Monnier
@ 2013-10-27 8:01 ` Andreas Röhler
0 siblings, 0 replies; 6+ messages in thread
From: Andreas Röhler @ 2013-10-27 8:01 UTC (permalink / raw)
To: 15295
Am 27.10.2013 05:23, schrieb Stefan Monnier:
> Could you take a look at this, as well?
>
>
> Stefan
>
>>>>>> "Michael" == Michael Heerdegen <michael_heerdegen@web.de> writes:
>
>> Alex V. Koval <alex@ua2web.com> writes:
>>> For me this is happen as well. Emacs, starting from version 24.3 became
>>> so slow in Python mode that I had to tell all developers at our company
>>> to use version 24.2 until I sorted this out.
>>>
>>> Sit today and started trying various emacs versions, and calling
>>> different functions. The suggested test case from original author above,
>>> runs with this benchmark:
>>>
>>> (7.3956507 53 1.8788885930000063)
>
>> I profiled a bit, and, at least in this example, these two functions
>> seem to be extremely inefficient in combination:
>
>> (defun python-nav-beginning-of-statement ()
>> "Move to start of current statement."
>> (interactive "^")
>> (while (and (or (back-to-indentation) t)
>> (not (bobp))
>> (when (or
>> (save-excursion
>> (forward-line -1)
>> (python-info-line-ends-backslash-p))
>> (python-syntax-context 'string)
>> (python-syntax-context 'paren))
>> (forward-line -1))))
>> (point-marker))
>
>> (defun python-info-line-ends-backslash-p (&optional line-number)
>> "Return non-nil if current line ends with backslash.
>> With optional argument LINE-NUMBER, check that line instead."
>> (save-excursion
>> (save-restriction
>> (widen)
>> (when line-number
>> (python-util-goto-line line-number))
>> (while (and (not (eobp))
>> (goto-char (line-end-position))
>> (python-syntax-context 'paren)
>> (not (equal (char-before (point)) ?\\)))
>> (forward-line 1))
>> (when (equal (char-before) ?\\)
>> (point-marker)))))
>
>> They consume most of the time used. While the first function goes
>> backward, the second goes forward to the end in every loop cycle. This
>> makes the thing O(n^2), with n being the number of lines of the
>> expression.
>
>> I don't know Python, so I can't make any suggestions. Who can? At
>> least, changing the order of `or' expressions in
>> `python-nav-beginning-of-statement' seems to help in the example case:
>
>> (defun python-nav-beginning-of-statement ()
>> "Move to start of current statement."
>> (interactive "^")
>> (while (and (or (back-to-indentation) t)
>> (not (bobp))
>> (when (or
>> (python-syntax-context 'string)
>> (python-syntax-context 'paren)
>> (save-excursion
>> (forward-line -1)
>> (python-info-line-ends-backslash-p)))
>> (forward-line -1))))
>> (point-marker))
>
>> It's also not efficient how often `syntax-ppss' is called all the time.
>
>
>> Regards,
>
>> Michael.
>
>
>
>
>
>
IMO it's a matter of coding style.
IIUC Emacs hackers should be warned somewhere in Elisp manual to code like
python-syntax-context
does. Python.el is not the only place where it's done like this.
It looks nice, but seems to port some dangers WRT speed.
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#15295:
2013-09-07 0:47 bug#15295: which-func-mode slow in long Python tuple Dale
2013-10-26 9:16 ` bug#15295: python mode slow to unusable Alex V. Koval
@ 2013-12-24 20:08 ` Fabián Ezequiel Gallina
1 sibling, 0 replies; 6+ messages in thread
From: Fabián Ezequiel Gallina @ 2013-12-24 20:08 UTC (permalink / raw)
To: 15295-done
Fixed in revno 115736.
Thanks Dale for such detailed recipe.
This patch banishes initial thoughts of `python-syntax-context' being a
bad idea. `python-syntax-context' is nothing than a thin semantic
wrapper over `syntax-ppss'. It makes code easier to grasp for newcomers
to Elisp and has almost no impact on itself, it's optional argument is a
`syntax-ppss' list which can be used instead to lower the amount of
calls to it (as it is happening in this new patch I've just committed).
The problem here was that `python-nav-beginning-of-statement' was coded
awfully (looking for the statement beginning line by line). Now it
should be extremely fast compared to that.
Using OP's suggested recipe, here are the elp results for when
which-func is triggered inside the big tuple:
python-info-current-defun 2 0.003719249 0.0018596245
python-nav-beginning-of-defun 2 0.0036946010 0.0018473005
python-nav--beginning-of-defun 2 0.003685751 0.0018428755
python-nav-backward-block 2 0.001836524 0.000918262
python-nav-forward-block 2 0.0018315750 0.0009157875
python-info-looking-at-beginning-of-defun 6 0.000889166 0.0001481943
python-nav-beginning-of-statement 4 0.000437251 0.0001093127
python-syntax-context-type 6 5.009e-06 8.348...e-07
And this is the benchmark-run result: (0.020715153 0 0.0)
Regards,
Fabián.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2013-12-24 20:08 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-09-07 0:47 bug#15295: which-func-mode slow in long Python tuple Dale
2013-10-26 9:16 ` bug#15295: python mode slow to unusable Alex V. Koval
2013-10-26 11:39 ` Michael Heerdegen
2013-10-27 4:23 ` Stefan Monnier
2013-10-27 8:01 ` Andreas Röhler
2013-12-24 20:08 ` bug#15295: Fabián Ezequiel Gallina
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).