CVS version: message mode + flyspell terribly slow

CVS version: message mode + flyspell terribly slow

[Klaus Zeitler]

It seems as if emacs 23 is under some circumstances a lot slower than 22.2.
I'm using CVS version under Solaris 5.8.
A good example is message-mode (from Gnus) together with flyspell.
1. start CVS version: emacs -Q
2. C-x C-f <message_file>
3. M-x message-mode
4. M-x flyspell-mode
Now editing, even cursor movement (forward-char, backward-char), is
unbearably slow. 

How can I figure out what's causing this?

BTW that is not the only case where emacs 23 is slower than 22.2, I have a
feeling that the CVS version is a bit slower in general.

Klaus

-- 
 --------------------------------------------------
|  Klaus Zeitler      Alcatel-Lucent               |
|  Email:             kzeitler@alcatel-lucent.com  |
 --------------------------------------------------
---
Military intelligence is a contradiction in terms. -- Groucho Marx

Re: CVS version: message mode + flyspell terribly slow

[Leo]

On 2008-11-24 12:44 +0000, Klaus Zeitler wrote:
[...]
> How can I figure out what's causing this?
>
> BTW that is not the only case where emacs 23 is slower than 22.2, I have a
> feeling that the CVS version is a bit slower in general.

I am on windows and I notice cursor movement is generally much slower
than say in GNU Linux.

Bye,
-- 
.:  Leo  :.  [ sdl.web AT gmail.com ]  .:  [ GPG Key: 9283AA3F ]  :.

        Use the better alternative -- http://www.openoffice.org/

Re: CVS version: message mode + flyspell terribly slow

[Chong Yidong]

Klaus Zeitler <kzeitler@alcatel-lucent.com> writes:

> It seems as if emacs 23 is under some circumstances a lot slower than 22.2.
> I'm using CVS version under Solaris 5.8.
> A good example is message-mode (from Gnus) together with flyspell.
> 1. start CVS version: emacs -Q
> 2. C-x C-f <message_file>
> 3. M-x message-mode
> 4. M-x flyspell-mode
> Now editing, even cursor movement (forward-char, backward-char), is
> unbearably slow. 
>
> How can I figure out what's causing this?

This may be related to bug#895, "slow processing of process output":

http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=895

If you can reproduce the slowness reliably, and have some time to spare,
could you please try rolling back CVS revisions to see when this problem
first appeared?

One possibility is that it is related to the Unicode changes, so I
suggest first trying a revision from early February, and comparing it to
a revision from late January.  (The Unicode merge happened on Feb 1.)

Re: CVS version: message mode + flyspell terribly slow

[Chong Yidong]

Klaus Zeitler <kzeitler@alcatel-lucent.com> writes:

> 1. start CVS version: emacs -Q
> 2. C-x C-f <message_file>
> 3. M-x message-mode
> 4. M-x flyspell-mode
> Now editing, even cursor movement (forward-char, backward-char), is
> unbearably slow. 

I can reproduce this.  From bisecting old revisions, I found that this
problem occurred between 2008-05-13 and 2008-05-15.  This is the same
time as the big checkin to make Emacs use only font-backends.

Re: CVS version: message mode + flyspell terribly slow

[Chong Yidong]

Chong Yidong <cyd@stupidchicken.com> writes:

> I can reproduce this.  From bisecting old revisions, I found that this
> problem occurred between 2008-05-13 and 2008-05-15.  This is the same
> time as the big checkin to make Emacs use only font-backends.

It turns out not to have to do with the font-backend checkin (which
would have been a surprising conclusion).  It's something to do with the
tool-bar; indeed, disabling the tool-bar removes this slowdown.  There
were some tool-bar changes around that time; I'll try to track down the
specific problem.

Re: CVS version: message mode + flyspell terribly slow

[Chong Yidong]

Klaus Zeitler <kzeitler@alcatel-lucent.com> writes:

> 1. start CVS version: emacs -Q
> 2. C-x C-f <message_file>
> 3. M-x message-mode
> 4. M-x flyspell-mode
> Now editing, even cursor movement (forward-char, backward-char), is
> unbearably slow. 

Here's the cause of the slowdown:

2008-05-07  Stefan Monnier  <monnier@iro.umontreal.ca>

	* tool-bar.el: Choose images dynamically.
	(tool-bar-make-keymap, tool-bar-find-image): New function.
	(tool-bar-find-image-cache): New var.
	(tool-bar-local-item, tool-bar-local-item-from-menu):
	Don't select the image yet, do it later in tool-bar-make-keymap.

Re: CVS version: message mode + flyspell terribly slow

[Chong Yidong]

Chong Yidong <cyd@stupidchicken.com> writes:

>> 1. start CVS version: emacs -Q
>> 2. C-x C-f <message_file>
>> 3. M-x message-mode
>> 4. M-x flyspell-mode
>> Now editing, even cursor movement (forward-char, backward-char), is
>> unbearably slow. 
>
> Here's the cause of the slowdown:
>
> 2008-05-07  Stefan Monnier  <monnier@iro.umontreal.ca>
>
> 	* tool-bar.el: Choose images dynamically.

To be precise, the change replaces the filter function for the
[tool-bar] key, which used to be a simple lambda function that returns
tool-bar-map.  Now, it calls the function tool-bar-make-keymap, which
maps over tool-bar-map performing some dynamic computation.

It turns out that tool-bar-make-keymap is rather expensive, leading to
slowdowns like the above when it is called repeatedly.  I'm not sure why
flyspell+message causes it to be called so often, though.  Maybe a gnus
hacker can enlighten us.

If we want to make tool-bar-make-keymap less expensive, one way is to
cache its computed value, in the same way that we cache tool-bar image
specs.  See attached patch, which seems to eliminate most of the
slowdown.  Stefan, WDYT?

(I think it's correct to use frame-terminal as I do here, as part of the
hash key.  But someone may want to check if this causes problems reaping
dead terminals.)

*** trunk/lisp/tool-bar.el.~1.21.~	2008-10-13 14:55:08.000000000 -0400
--- trunk/lisp/tool-bar.el	2008-11-26 16:20:14.000000000 -0500
***************
*** 92,101 ****
--- 92,108 ----
  
  (declare-function image-mask-p "image.c" (spec &optional frame))
  
+ (defconst tool-bar-keymap-cache (make-hash-table :weakness t :test 'equal))
+ 
  (defun tool-bar-make-keymap (&optional ignore)
    "Generate an actual keymap from `tool-bar-map'.
  Its main job is to figure out which images to use based on the display's
  color capability and based on the available image libraries."
+   (let ((key (cons (frame-terminal) tool-bar-map)))
+     (or (gethash key tool-bar-find-image-cache)
+ 	(puthash key (tool-bar-make-keymap-1) tool-bar-find-image-cache))))
+ 
+ (defun tool-bar-make-keymap-1 (&optional ignore)
    (mapcar (lambda (bind)
              (let (image-exp plist)
                (when (and (eq (car-safe (cdr-safe bind)) 'menu-item)

Re: CVS version: message mode + flyspell terribly slow

[Stefan Monnier]

> If we want to make tool-bar-make-keymap less expensive, one way is to
> cache its computed value, in the same way that we cache tool-bar image
> specs.  See attached patch, which seems to eliminate most of the
> slowdown.  Stefan, WDYT?

It looks OK.  But I wonder: doesn't it make image-spec-caching redundant?


        Stefan

Re: CVS version: message mode + flyspell terribly slow

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> It looks OK.  But I wonder: doesn't it make image-spec-caching
> redundant?

Yes.  I've checked it in, and removed the image-spec caching.

Performance seems to be improved, but still noticeably worse than when
the tool-bar is disabled.  For instance, the tool-bar seems to have an
impact on the slow scrolling that people have been discussing on and off
for the last couple of months.  To reproduce:

1. emacs -Q --eval "(setq scroll-conservatively 10)"
2. C-h n
2a. M-x tool-bar-mode RET
3. C-n (press and hold down, causing continuous scrolling)

When 2a is omitted, there are occasional pauses in scrolling.  When the
tool bar is disabled, scrolling is much smoother.

Re: CVS version: message mode + flyspell terribly slow

[Stefan Monnier]

>> It looks OK.  But I wonder: doesn't it make image-spec-caching
>> redundant?
> Yes.  I've checked it in, and removed the image-spec caching.

Good.

> Performance seems to be improved, but still noticeably worse than when
> the tool-bar is disabled.  For instance, the tool-bar seems to have an
> impact on the slow scrolling that people have been discussing on and off
> for the last couple of months.  To reproduce:

> 1. emacs -Q --eval "(setq scroll-conservatively 10)"
> 2. C-h n
> 2a. M-x tool-bar-mode RET
> 3. C-n (press and hold down, causing continuous scrolling)

> When 2a is omitted, there are occasional pauses in scrolling.  When the
> tool bar is disabled, scrolling is much smoother.

Is that linked to the "dynamic keymap" as well (even though you've sped
it up with caching) or is there some other reason?


        Stefan

Re: CVS version: message mode + flyspell terribly slow

[Klaus Zeitler]

>>>>> "Chong" == Chong Yidong <cyd@stupidchicken.com> writes:
    Chong> 
    Chong> It turns out that tool-bar-make-keymap is rather expensive, leading
    Chong> to slowdowns like the above when it is called repeatedly.  I'm not
    Chong> sure why flyspell+message causes it to be called so often, though.
    Chong> Maybe a gnus hacker can enlighten us.
    Chong> 
    Chong> If we want to make tool-bar-make-keymap less expensive, one way is
    Chong> to cache its computed value, in the same way that we cache tool-bar
    Chong> image specs.  See attached patch, which seems to eliminate most of
    Chong> the slowdown.

Thanks for the quick fix and especially for saving me a lot of time in tracing
this down (yesterday I started with a CVS version from 31.01.08 and I'm sure
it would have taken me a lot of time to find it).
I tried the newest version and it's a big improvement.
Maybe I should switch the tool-bar of anyway, since I hardly ever use it.
I mainly keep it so that my colleagues wont say that emacs is an old fashioned
editor :-).

Thanks again

Klaus


-- 
 --------------------------------------------------
|  Klaus Zeitler      Alcatel-Lucent               |
|  Email:             kzeitler@alcatel-lucent.com  |
 --------------------------------------------------
---
There are two ways to write error-free programs.  Only the third one works.

Re: CVS version: message mode + flyspell terribly slow

[Reiner Steib]

On Wed, Nov 26 2008, Chong Yidong wrote:

> It turns out that tool-bar-make-keymap is rather expensive, leading to
> slowdowns like the above when it is called repeatedly.  I'm not sure why
> flyspell+message causes it to be called so often, though.  Maybe a gnus
> hacker can enlighten us.

message-make-tool-bar only calls any tool-bar-* functions when
message-tool-bar-map is nil.  Thus it should only happen once per
session.  I don't see that it's called repeatedly.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: CVS version: message mode + flyspell terribly slow

[Chong Yidong]

> > Performance seems to be improved, but still noticeably worse than when
> > the tool-bar is disabled.
> >
> > 1. emacs -Q --eval "(setq scroll-conservatively 10)"
> > 2. C-h n
> > 2a. M-x tool-bar-mode RET
> > 3. C-n (press and hold down, causing continuous scrolling)
> >
> > When 2a is omitted, there are occasional pauses in scrolling.  When
> > the tool bar is disabled, scrolling is much smoother.
>
> Is that linked to the "dynamic keymap" as well (even though you've sped
> it up with caching) or is there some other reason?

I think it's likely to be related to the dynamic keymap.

The pauses in scrolling are due to garbage collection.  In the test case
described above (i.e. scrolling the NEWS buffer continuously with
scroll-conservatively on), garbage collection occurs once every three or
four seconds when the tool-bar is enabled.  When tool-bar is disabled,
it occurs once every thirty seconds.  My guess is that
tool-bar-make-keymap is called so often that the consing it performs
triggers frequent garbage collection.

One thing to note is that the hash tables we use for caching the
tool-bar keymaps are weak, so we have to recompute the keymaps after
each gc.  However, making the hash tables strong does not improve
performance significantly.