* Scrolling huge buffers and cc-mode
@ 2012-12-18 9:56 Dmitry Antipov
2012-12-18 17:30 ` Eli Zaretskii
2012-12-18 19:40 ` Stefan Monnier
0 siblings, 2 replies; 12+ messages in thread
From: Dmitry Antipov @ 2012-12-18 9:56 UTC (permalink / raw)
To: Emacs development discussions
[-- Attachment #1: Type: text/plain, Size: 586 bytes --]
Attached is a C++ source file to reproduce the painfully slow scrolling
under cc-mode. Test is simple: just visit it and press <page-down>, or <next>,
or whatever else bind to scroll-up-command, until buffer end is reached.
The file is ~900K; when scrolling reaches 50%, everything is stopped, may
be up to a few minutes (!), there is no response to any keyboard input
including C-g, and CPU consumption is ~80-90%. After that, scrolling is
resumed, but it's still _very_ slow. This looks like a C++ issue only
since xdisp.c of nearly the same size may be scrolled much faster.
Dmitry
[-- Attachment #2: test.cc.xz --]
[-- Type: application/x-xz, Size: 144492 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode
2012-12-18 9:56 Scrolling huge buffers and cc-mode Dmitry Antipov
@ 2012-12-18 17:30 ` Eli Zaretskii
2012-12-18 19:58 ` Stefan Monnier
2012-12-18 19:40 ` Stefan Monnier
1 sibling, 1 reply; 12+ messages in thread
From: Eli Zaretskii @ 2012-12-18 17:30 UTC (permalink / raw)
To: Dmitry Antipov, Alan Mackenzie; +Cc: emacs-devel
> Date: Tue, 18 Dec 2012 13:56:13 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
>
> Attached is a C++ source file to reproduce the painfully slow scrolling
> under cc-mode. Test is simple: just visit it and press <page-down>, or <next>,
> or whatever else bind to scroll-up-command, until buffer end is reached.
> The file is ~900K; when scrolling reaches 50%, everything is stopped, may
> be up to a few minutes (!), there is no response to any keyboard input
> including C-g, and CPU consumption is ~80-90%. After that, scrolling is
> resumed, but it's still _very_ slow. This looks like a C++ issue only
> since xdisp.c of nearly the same size may be scrolled much faster.
(Added Alan to the discussion.)
It looks like this is due to font-lock (invoked by JIT Lock each time
a new screenful is about to be displayed) that is specific to C++. To
see this, fontify the whole buffer, or turn on JIT Stealth and wait
for it to finish, and then try scrolling -- you will see a normal
scrolling speed.
While JIT Stealth was working, I noticed that one of my cores was busy
100%. So I turned on the profiler, and the result of running it for
about 1 minute is below. If I understand correctly, it blames
scan-sexps for most of the CPU load.
- apply 25518 84%
- jit-lock-stealth-fontify 25517 84%
- jit-lock-fontify-now 25517 84%
- byte-code 25517 84%
- run-hook-with-args 25517 84%
- font-lock-fontify-region 25517 84%
- c-font-lock-fontify-region 25517 84%
- font-lock-default-fontify-region 25495 84%
- font-lock-fontify-keywords-regio 25495 84%
- c-font-lock-enclosing-decls 25389 83%
- c-beginning-of-decl-1 25388 83%
- byte-code 25388 83%
- c-beginning-of-statement 25388 83%
- byte-code 25387 83%
- byte-code 25373 83%
scan-sexps 25372 83%
scan-lists 1 0%
- c-backward-sws 5 0%
looking-at 4 0%
previous-single-pr 1 0%
- c-looking-at-inexpr- 5 0%
looking-at 3 0%
c-backward-sws 2 0%
- c-beginning-of-macro 2 0%
beginning-of-line 2 0%
- c-crosses-statement- 2 0%
append 2 0%
- c-at-macro-vsemi-p 1 0%
- c-backward-sws 1 0%
looking-at 1 0%
+ c-parse-state 1 0%
+ c-font-lock-declarations 97 0%
+ #<compiled 0xea75d7> 5 0%
+ cwarn-font-lock-match-referenc 2 0%
+ c-font-lock-<>-arglists 2 0%
+ mapc 22 0%
+ auto-revert-buffers 1 0%
+ timer-event-handler 2496 8%
+ byte-code 1211 3%
+ redisplay_internal (C function) 392 1%
Automatic GC 294 0%
+ call-interactively 226 0%
+ jit-lock-stealth-fontify 95 0%
+ jit-lock-fontify-now 40 0%
+ run-hook-with-args 8 0%
+ font-lock-fontify-region 3 0%
+ profiler-calltree-walk 1 0%
tooltip-hide 1 0%
+ list 1 0%
+ input-pending-p 1 0%
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode
2012-12-18 9:56 Scrolling huge buffers and cc-mode Dmitry Antipov
2012-12-18 17:30 ` Eli Zaretskii
@ 2012-12-18 19:40 ` Stefan Monnier
1 sibling, 0 replies; 12+ messages in thread
From: Stefan Monnier @ 2012-12-18 19:40 UTC (permalink / raw)
To: Dmitry Antipov; +Cc: Emacs development discussions
> resumed, but it's still _very_ slow. This looks like a C++ issue only
> since xdisp.c of nearly the same size may be scrolled much faster.
M-x profiler-start RET would probably give more hints about where the
time is spent.
Stefan
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode
2012-12-18 17:30 ` Eli Zaretskii
@ 2012-12-18 19:58 ` Stefan Monnier
2012-12-19 6:04 ` Dmitry Antipov
0 siblings, 1 reply; 12+ messages in thread
From: Stefan Monnier @ 2012-12-18 19:58 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: Alan Mackenzie, Dmitry Antipov, emacs-devel
> about 1 minute is below. If I understand correctly, it blames
> scan-sexps for most of the CPU load.
Indeed, but running scan-sexps on the whole buffer doesn't take that
long, so the problem is not that scan-sexps is slow, but that it's
called too many times.
Stefan
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode
2012-12-18 19:58 ` Stefan Monnier
@ 2012-12-19 6:04 ` Dmitry Antipov
2012-12-19 7:20 ` Dmitry Antipov
0 siblings, 1 reply; 12+ messages in thread
From: Dmitry Antipov @ 2012-12-19 6:04 UTC (permalink / raw)
To: emacs-devel; +Cc: Alan Mackenzie, Eli Zaretskii, Stefan Monnier
On 12/18/2012 11:58 PM, Stefan Monnier wrote:
>> about 1 minute is below. If I understand correctly, it blames
>> scan-sexps for most of the CPU load.
>
> Indeed, but running scan-sexps on the whole buffer doesn't take that
> long, so the problem is not that scan-sexps is slow, but that it's
> called too many times.
I didn't try to profile elisp, but here is what I'm seeing at the lower level.
Since the percentage of exec_byte_code is just 1.21%, I suspect that elisp
is not a bottleneck here. Looking at enormous percentage of lookup_char_property
and next_interval, I would rather suspect an issue with interval trees.
Dmitry
# ========
# captured on: Tue Dec 18 19:30:31 2012
# hostname : localhost.localdomain
# os release : 3.6.10-2.fc17.x86_64
# perf version : 3.6.10-2.fc17.x86_64
# arch : x86_64
# nrcpus online : 4
# nrcpus avail : 4
# cpudesc : Intel(R) Core(TM) i5-2410M CPU @ 2.30GHz
# cpuid : GenuineIntel,6,42,7
# total memory : 3930292 kB
# cmdline : /usr/bin/perf record -e stalled-cycles-frontend -e stalled-cycles-backend -F 10000 ./src/emacs -Q ../../misc/huge.cc
# event : name = stalled-cycles-frontend, type = 0, config = 0x7, config1 = 0x0, config2 = 0x0, excl_usr = 0, excl_kern = 0, excl_host
# event : name = stalled-cycles-backend, type = 0, config = 0x8, config1 = 0x0, config2 = 0x0, excl_usr = 0, excl_kern = 0, excl_host
# HEADER_CPU_TOPOLOGY info available, use -I to display
# HEADER_NUMA_TOPOLOGY info available, use -I to display
# ========
#
# Samples: 3M of event 'stalled-cycles-frontend'
# Event count (approx.): 284964331659
#
# Overhead Command Shared Object Symbol
# ........ ........... ............................. ......................................................
#
22.65% emacs emacs [.] lookup_char_property
21.21% emacs emacs [.] scan_sexps_forward
17.65% emacs emacs [.] next_interval
8.29% emacs emacs [.] Fcdr
4.66% emacs emacs [.] update_syntax_table
2.15% emacs emacs [.] scan_lists
1.95% emacs emacs [.] mark_object
1.92% emacs emacs [.] update_interval
1.74% emacs emacs [.] Fassq
1.41% emacs emacs [.] back_comment
1.38% emacs emacs [.] forw_comment
1.21% emacs emacs [.] exec_byte_code
1.08% emacs emacs [.] char_quoted
1.05% emacs emacs [.] re_match_2_internal
0.70% emacs emacs [.] re_compile_pattern
0.67% emacs emacs [.] Fgarbage_collect
0.58% emacs emacs [.] find_interval
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode
2012-12-19 6:04 ` Dmitry Antipov
@ 2012-12-19 7:20 ` Dmitry Antipov
2012-12-19 22:39 ` Alan Mackenzie
2012-12-20 22:29 ` Alan Mackenzie
0 siblings, 2 replies; 12+ messages in thread
From: Dmitry Antipov @ 2012-12-19 7:20 UTC (permalink / raw)
To: emacs-devel; +Cc: Alan Mackenzie, Eli Zaretskii, Stefan Monnier
It looks like the reason of slow scrolling is how the C++ namespaces are handled.
Basically my test file is:
namespace X {
namespace Y {
/* 13K lines of code */
} }
/* middle-point */
namespace X {
namespace Y {
/* 13K lines of code */
} }
When scrolling reaches middle-point, everything hangs, most probably because
we need to scan huge regions (first and second namespace blocks). When I remove
all namespace definitions (with matched '}'), scrolling works much faster.
Corresponding profile is:
13.04% emacs emacs [.] mark_object
7.70% emacs emacs [.] exec_byte_code
6.79% emacs emacs [.] re_match_2_internal
6.48% emacs emacs [.] lookup_char_property
4.70% emacs emacs [.] scan_sexps_forward
4.67% emacs emacs [.] Fgarbage_collect
4.62% emacs emacs [.] re_compile_pattern
4.57% emacs emacs [.] next_interval
3.29% emacs emacs [.] find_interval
3.16% emacs emacs [.] mark_interval
2.15% emacs emacs [.] Fcdr
2.06% emacs emacs [.] update_syntax_table
1.83% emacs emacs [.] Flocal_variable_p
1.33% emacs emacs [.] Fassq
1.19% emacs emacs [.] balance_intervals_internal
1.13% emacs emacs [.] Ffuncall
1.00% emacs emacs [.] compile_pattern
This is very similar to what I'm seeing while scrolling over xdisp.c:
13.92% emacs emacs [.] mark_object
8.20% emacs emacs [.] exec_byte_code
4.91% emacs emacs [.] re_match_2_internal
4.78% emacs emacs [.] Fgarbage_collect
4.65% emacs emacs [.] lookup_char_property
3.60% emacs emacs [.] find_interval
3.55% emacs emacs [.] scan_sexps_forward
3.45% emacs emacs [.] re_compile_pattern
2.77% emacs emacs [.] Flocal_variable_p
2.71% emacs emacs [.] mark_interval
2.46% emacs emacs [.] next_interval
1.62% emacs emacs [.] update_syntax_table
1.52% emacs emacs [.] Fassq
1.45% emacs emacs [.] Fcdr
1.35% emacs emacs [.] Ffuncall
1.11% emacs emacs [.] compile_pattern
Dmitry
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode
2012-12-19 7:20 ` Dmitry Antipov
@ 2012-12-19 22:39 ` Alan Mackenzie
2012-12-20 0:52 ` Michael Welsh Duggan
2012-12-20 22:29 ` Alan Mackenzie
1 sibling, 1 reply; 12+ messages in thread
From: Alan Mackenzie @ 2012-12-19 22:39 UTC (permalink / raw)
To: Dmitry Antipov; +Cc: Eli Zaretskii, Stefan Monnier, emacs-devel
On Wed, Dec 19, 2012 at 11:20:55AM +0400, Dmitry Antipov wrote:
> It looks like the reason of slow scrolling is how the C++ namespaces are handled.
> Basically my test file is:
> namespace X {
> namespace Y {
> /* 13K lines of code */
> } }
> /* middle-point */
> namespace X {
> namespace Y {
> /* 13K lines of code */
> } }
> When scrolling reaches middle-point, everything hangs, most probably because
> we need to scan huge regions (first and second namespace blocks). When I remove
> all namespace definitions (with matched '}'), scrolling works much faster.
OK, I'll have a look at it in the next few days.
I have an idea for a solution, which would involve effectively masking
out enclosing namespaces' braces, but this might be quite complicated to
implement.
Am I right in thinking that a namespace declaration _cannot_ be enclosed
within any other type of declaration, such as a class?
> Dmitry
--
Alan Mackenzie (Nuremberg, Germany).
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode
2012-12-19 22:39 ` Alan Mackenzie
@ 2012-12-20 0:52 ` Michael Welsh Duggan
0 siblings, 0 replies; 12+ messages in thread
From: Michael Welsh Duggan @ 2012-12-20 0:52 UTC (permalink / raw)
To: Alan Mackenzie; +Cc: Eli Zaretskii, Dmitry Antipov, Stefan Monnier, emacs-devel
Alan Mackenzie <acm@muc.de> writes:
> Am I right in thinking that a namespace declaration _cannot_ be enclosed
> within any other type of declaration, such as a class?
It took a while to locate, but section 7.3.1 [namespace.def] states:
4 Every namespace-definition shall appear in the global scope or in a
namespace scope (3.3.5).
So, in essence, you are correct.
--
Michael Welsh Duggan
(md5i@md5i.com)
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode
2012-12-19 7:20 ` Dmitry Antipov
2012-12-19 22:39 ` Alan Mackenzie
@ 2012-12-20 22:29 ` Alan Mackenzie
2012-12-21 5:15 ` Dmitry Antipov
1 sibling, 1 reply; 12+ messages in thread
From: Alan Mackenzie @ 2012-12-20 22:29 UTC (permalink / raw)
To: Dmitry Antipov; +Cc: Eli Zaretskii, Stefan Monnier, emacs-devel
Hello, Dmitry,
On Wed, Dec 19, 2012 at 11:20:55AM +0400, Dmitry Antipov wrote:
> It looks like the reason of slow scrolling is how the C++ namespaces are handled.
> Basically my test file is:
> namespace X {
> namespace Y {
> /* 13K lines of code */
> } }
> /* middle-point */
> namespace X { <==============================================
> namespace Y {
> /* 13K lines of code */
> } }
> When scrolling reaches middle-point, everything hangs, most probably because
> we need to scan huge regions (first and second namespace blocks). When I remove
> all namespace definitions (with matched '}'), scrolling works much faster.
What is happening is that (c-beginning-of-statement-1 ...) is being
called repeatedly on the <===== line.
o - This moves to the beginning of "namespace X {",
o - then calls (scan-sexps (point) -1),
o - which has to scan over the 13k lines of code. This is slow.
The solution appears to be to give the backward search a search limit.
Could you try out the following patch please, and let me know if it works
OK:
diff -r ce04d3763229 cc-fonts.el
--- a/cc-fonts.el Thu Dec 20 20:11:22 2012 +0000
+++ b/cc-fonts.el Thu Dec 20 21:58:50 2012 +0000
@@ -1558,6 +1558,7 @@
;; prevent a repeat invocation. See elisp/lispref page "Search-based
;; Fontification".
(let* ((paren-state (c-parse-state))
+ (decl-search-lim (c-determine-limit 1000))
decl-context in-typedef ps-elt)
;; Are we in any nested struct/union/class/etc. braces?
(while paren-state
@@ -1566,7 +1567,7 @@
(when (and (atom ps-elt)
(eq (char-after ps-elt) ?\{))
(goto-char ps-elt)
- (setq decl-context (c-beginning-of-decl-1)
+ (setq decl-context (c-beginning-of-decl-1 decl-search-lim)
in-typedef (looking-at c-typedef-key))
(if in-typedef (c-forward-token-2))
(when (and c-opt-block-decls-with-vars-key
> Dmitry
--
Alan Mackenzie (Nuremberg, Germany).
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode
2012-12-20 22:29 ` Alan Mackenzie
@ 2012-12-21 5:15 ` Dmitry Antipov
2012-12-21 10:57 ` Alan Mackenzie
0 siblings, 1 reply; 12+ messages in thread
From: Dmitry Antipov @ 2012-12-21 5:15 UTC (permalink / raw)
To: Alan Mackenzie; +Cc: Eli Zaretskii, Stefan Monnier, emacs-devel
On 12/21/2012 02:29 AM, Alan Mackenzie wrote:
> The solution appears to be to give the backward search a search limit.
> Could you try out the following patch please, and let me know if it works
> OK:
Yes, that's much better. Thanks.
Dmitry
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode
2012-12-21 5:15 ` Dmitry Antipov
@ 2012-12-21 10:57 ` Alan Mackenzie
2012-12-22 3:28 ` Chong Yidong
0 siblings, 1 reply; 12+ messages in thread
From: Alan Mackenzie @ 2012-12-21 10:57 UTC (permalink / raw)
To: Stefan Monnier; +Cc: Eli Zaretskii, Dmitry Antipov, emacs-devel
Stefan,
should I commit this fix to the Emacs-24 branch?
On Fri, Dec 21, 2012 at 09:15:52AM +0400, Dmitry Antipov wrote:
> On 12/21/2012 02:29 AM, Alan Mackenzie wrote:
> > The solution appears to be to give the backward search a search limit.
> > Could you try out the following patch please, and let me know if it works
> > OK:
> Yes, that's much better. Thanks.
> Dmitry
--
Alan Mackenzie (Nuremberg, Germany).
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode
2012-12-21 10:57 ` Alan Mackenzie
@ 2012-12-22 3:28 ` Chong Yidong
0 siblings, 0 replies; 12+ messages in thread
From: Chong Yidong @ 2012-12-22 3:28 UTC (permalink / raw)
To: Alan Mackenzie; +Cc: Eli Zaretskii, Dmitry Antipov, Stefan Monnier, emacs-devel
Alan Mackenzie <acm@muc.de> writes:
> Stefan,
>
> should I commit this fix to the Emacs-24 branch?
Not Stefan, but I think we should take the risk of committing to
emacs-24; this bug would be very annoying for people using Emacs as an
editor for C++, and the fix seems safe enough.
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2012-12-22 3:28 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-12-18 9:56 Scrolling huge buffers and cc-mode Dmitry Antipov
2012-12-18 17:30 ` Eli Zaretskii
2012-12-18 19:58 ` Stefan Monnier
2012-12-19 6:04 ` Dmitry Antipov
2012-12-19 7:20 ` Dmitry Antipov
2012-12-19 22:39 ` Alan Mackenzie
2012-12-20 0:52 ` Michael Welsh Duggan
2012-12-20 22:29 ` Alan Mackenzie
2012-12-21 5:15 ` Dmitry Antipov
2012-12-21 10:57 ` Alan Mackenzie
2012-12-22 3:28 ` Chong Yidong
2012-12-18 19:40 ` Stefan Monnier
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.