* Scrolling huge buffers and cc-mode @ 2012-12-18 9:56 Dmitry Antipov 2012-12-18 17:30 ` Eli Zaretskii 2012-12-18 19:40 ` Stefan Monnier 0 siblings, 2 replies; 12+ messages in thread From: Dmitry Antipov @ 2012-12-18 9:56 UTC (permalink / raw) To: Emacs development discussions [-- Attachment #1: Type: text/plain, Size: 586 bytes --] Attached is a C++ source file to reproduce the painfully slow scrolling under cc-mode. Test is simple: just visit it and press <page-down>, or <next>, or whatever else bind to scroll-up-command, until buffer end is reached. The file is ~900K; when scrolling reaches 50%, everything is stopped, may be up to a few minutes (!), there is no response to any keyboard input including C-g, and CPU consumption is ~80-90%. After that, scrolling is resumed, but it's still _very_ slow. This looks like a C++ issue only since xdisp.c of nearly the same size may be scrolled much faster. Dmitry [-- Attachment #2: test.cc.xz --] [-- Type: application/x-xz, Size: 144492 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode 2012-12-18 9:56 Scrolling huge buffers and cc-mode Dmitry Antipov @ 2012-12-18 17:30 ` Eli Zaretskii 2012-12-18 19:58 ` Stefan Monnier 2012-12-18 19:40 ` Stefan Monnier 1 sibling, 1 reply; 12+ messages in thread From: Eli Zaretskii @ 2012-12-18 17:30 UTC (permalink / raw) To: Dmitry Antipov, Alan Mackenzie; +Cc: emacs-devel > Date: Tue, 18 Dec 2012 13:56:13 +0400 > From: Dmitry Antipov <dmantipov@yandex.ru> > > Attached is a C++ source file to reproduce the painfully slow scrolling > under cc-mode. Test is simple: just visit it and press <page-down>, or <next>, > or whatever else bind to scroll-up-command, until buffer end is reached. > The file is ~900K; when scrolling reaches 50%, everything is stopped, may > be up to a few minutes (!), there is no response to any keyboard input > including C-g, and CPU consumption is ~80-90%. After that, scrolling is > resumed, but it's still _very_ slow. This looks like a C++ issue only > since xdisp.c of nearly the same size may be scrolled much faster. (Added Alan to the discussion.) It looks like this is due to font-lock (invoked by JIT Lock each time a new screenful is about to be displayed) that is specific to C++. To see this, fontify the whole buffer, or turn on JIT Stealth and wait for it to finish, and then try scrolling -- you will see a normal scrolling speed. While JIT Stealth was working, I noticed that one of my cores was busy 100%. So I turned on the profiler, and the result of running it for about 1 minute is below. If I understand correctly, it blames scan-sexps for most of the CPU load. - apply 25518 84% - jit-lock-stealth-fontify 25517 84% - jit-lock-fontify-now 25517 84% - byte-code 25517 84% - run-hook-with-args 25517 84% - font-lock-fontify-region 25517 84% - c-font-lock-fontify-region 25517 84% - font-lock-default-fontify-region 25495 84% - font-lock-fontify-keywords-regio 25495 84% - c-font-lock-enclosing-decls 25389 83% - c-beginning-of-decl-1 25388 83% - byte-code 25388 83% - c-beginning-of-statement 25388 83% - byte-code 25387 83% - byte-code 25373 83% scan-sexps 25372 83% scan-lists 1 0% - c-backward-sws 5 0% looking-at 4 0% previous-single-pr 1 0% - c-looking-at-inexpr- 5 0% looking-at 3 0% c-backward-sws 2 0% - c-beginning-of-macro 2 0% beginning-of-line 2 0% - c-crosses-statement- 2 0% append 2 0% - c-at-macro-vsemi-p 1 0% - c-backward-sws 1 0% looking-at 1 0% + c-parse-state 1 0% + c-font-lock-declarations 97 0% + #<compiled 0xea75d7> 5 0% + cwarn-font-lock-match-referenc 2 0% + c-font-lock-<>-arglists 2 0% + mapc 22 0% + auto-revert-buffers 1 0% + timer-event-handler 2496 8% + byte-code 1211 3% + redisplay_internal (C function) 392 1% Automatic GC 294 0% + call-interactively 226 0% + jit-lock-stealth-fontify 95 0% + jit-lock-fontify-now 40 0% + run-hook-with-args 8 0% + font-lock-fontify-region 3 0% + profiler-calltree-walk 1 0% tooltip-hide 1 0% + list 1 0% + input-pending-p 1 0% ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode 2012-12-18 17:30 ` Eli Zaretskii @ 2012-12-18 19:58 ` Stefan Monnier 2012-12-19 6:04 ` Dmitry Antipov 0 siblings, 1 reply; 12+ messages in thread From: Stefan Monnier @ 2012-12-18 19:58 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Alan Mackenzie, Dmitry Antipov, emacs-devel > about 1 minute is below. If I understand correctly, it blames > scan-sexps for most of the CPU load. Indeed, but running scan-sexps on the whole buffer doesn't take that long, so the problem is not that scan-sexps is slow, but that it's called too many times. Stefan ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode 2012-12-18 19:58 ` Stefan Monnier @ 2012-12-19 6:04 ` Dmitry Antipov 2012-12-19 7:20 ` Dmitry Antipov 0 siblings, 1 reply; 12+ messages in thread From: Dmitry Antipov @ 2012-12-19 6:04 UTC (permalink / raw) To: emacs-devel; +Cc: Alan Mackenzie, Eli Zaretskii, Stefan Monnier On 12/18/2012 11:58 PM, Stefan Monnier wrote: >> about 1 minute is below. If I understand correctly, it blames >> scan-sexps for most of the CPU load. > > Indeed, but running scan-sexps on the whole buffer doesn't take that > long, so the problem is not that scan-sexps is slow, but that it's > called too many times. I didn't try to profile elisp, but here is what I'm seeing at the lower level. Since the percentage of exec_byte_code is just 1.21%, I suspect that elisp is not a bottleneck here. Looking at enormous percentage of lookup_char_property and next_interval, I would rather suspect an issue with interval trees. Dmitry # ======== # captured on: Tue Dec 18 19:30:31 2012 # hostname : localhost.localdomain # os release : 3.6.10-2.fc17.x86_64 # perf version : 3.6.10-2.fc17.x86_64 # arch : x86_64 # nrcpus online : 4 # nrcpus avail : 4 # cpudesc : Intel(R) Core(TM) i5-2410M CPU @ 2.30GHz # cpuid : GenuineIntel,6,42,7 # total memory : 3930292 kB # cmdline : /usr/bin/perf record -e stalled-cycles-frontend -e stalled-cycles-backend -F 10000 ./src/emacs -Q ../../misc/huge.cc # event : name = stalled-cycles-frontend, type = 0, config = 0x7, config1 = 0x0, config2 = 0x0, excl_usr = 0, excl_kern = 0, excl_host # event : name = stalled-cycles-backend, type = 0, config = 0x8, config1 = 0x0, config2 = 0x0, excl_usr = 0, excl_kern = 0, excl_host # HEADER_CPU_TOPOLOGY info available, use -I to display # HEADER_NUMA_TOPOLOGY info available, use -I to display # ======== # # Samples: 3M of event 'stalled-cycles-frontend' # Event count (approx.): 284964331659 # # Overhead Command Shared Object Symbol # ........ ........... ............................. ...................................................... # 22.65% emacs emacs [.] lookup_char_property 21.21% emacs emacs [.] scan_sexps_forward 17.65% emacs emacs [.] next_interval 8.29% emacs emacs [.] Fcdr 4.66% emacs emacs [.] update_syntax_table 2.15% emacs emacs [.] scan_lists 1.95% emacs emacs [.] mark_object 1.92% emacs emacs [.] update_interval 1.74% emacs emacs [.] Fassq 1.41% emacs emacs [.] back_comment 1.38% emacs emacs [.] forw_comment 1.21% emacs emacs [.] exec_byte_code 1.08% emacs emacs [.] char_quoted 1.05% emacs emacs [.] re_match_2_internal 0.70% emacs emacs [.] re_compile_pattern 0.67% emacs emacs [.] Fgarbage_collect 0.58% emacs emacs [.] find_interval ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode 2012-12-19 6:04 ` Dmitry Antipov @ 2012-12-19 7:20 ` Dmitry Antipov 2012-12-19 22:39 ` Alan Mackenzie 2012-12-20 22:29 ` Alan Mackenzie 0 siblings, 2 replies; 12+ messages in thread From: Dmitry Antipov @ 2012-12-19 7:20 UTC (permalink / raw) To: emacs-devel; +Cc: Alan Mackenzie, Eli Zaretskii, Stefan Monnier It looks like the reason of slow scrolling is how the C++ namespaces are handled. Basically my test file is: namespace X { namespace Y { /* 13K lines of code */ } } /* middle-point */ namespace X { namespace Y { /* 13K lines of code */ } } When scrolling reaches middle-point, everything hangs, most probably because we need to scan huge regions (first and second namespace blocks). When I remove all namespace definitions (with matched '}'), scrolling works much faster. Corresponding profile is: 13.04% emacs emacs [.] mark_object 7.70% emacs emacs [.] exec_byte_code 6.79% emacs emacs [.] re_match_2_internal 6.48% emacs emacs [.] lookup_char_property 4.70% emacs emacs [.] scan_sexps_forward 4.67% emacs emacs [.] Fgarbage_collect 4.62% emacs emacs [.] re_compile_pattern 4.57% emacs emacs [.] next_interval 3.29% emacs emacs [.] find_interval 3.16% emacs emacs [.] mark_interval 2.15% emacs emacs [.] Fcdr 2.06% emacs emacs [.] update_syntax_table 1.83% emacs emacs [.] Flocal_variable_p 1.33% emacs emacs [.] Fassq 1.19% emacs emacs [.] balance_intervals_internal 1.13% emacs emacs [.] Ffuncall 1.00% emacs emacs [.] compile_pattern This is very similar to what I'm seeing while scrolling over xdisp.c: 13.92% emacs emacs [.] mark_object 8.20% emacs emacs [.] exec_byte_code 4.91% emacs emacs [.] re_match_2_internal 4.78% emacs emacs [.] Fgarbage_collect 4.65% emacs emacs [.] lookup_char_property 3.60% emacs emacs [.] find_interval 3.55% emacs emacs [.] scan_sexps_forward 3.45% emacs emacs [.] re_compile_pattern 2.77% emacs emacs [.] Flocal_variable_p 2.71% emacs emacs [.] mark_interval 2.46% emacs emacs [.] next_interval 1.62% emacs emacs [.] update_syntax_table 1.52% emacs emacs [.] Fassq 1.45% emacs emacs [.] Fcdr 1.35% emacs emacs [.] Ffuncall 1.11% emacs emacs [.] compile_pattern Dmitry ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode 2012-12-19 7:20 ` Dmitry Antipov @ 2012-12-19 22:39 ` Alan Mackenzie 2012-12-20 0:52 ` Michael Welsh Duggan 2012-12-20 22:29 ` Alan Mackenzie 1 sibling, 1 reply; 12+ messages in thread From: Alan Mackenzie @ 2012-12-19 22:39 UTC (permalink / raw) To: Dmitry Antipov; +Cc: Eli Zaretskii, Stefan Monnier, emacs-devel On Wed, Dec 19, 2012 at 11:20:55AM +0400, Dmitry Antipov wrote: > It looks like the reason of slow scrolling is how the C++ namespaces are handled. > Basically my test file is: > namespace X { > namespace Y { > /* 13K lines of code */ > } } > /* middle-point */ > namespace X { > namespace Y { > /* 13K lines of code */ > } } > When scrolling reaches middle-point, everything hangs, most probably because > we need to scan huge regions (first and second namespace blocks). When I remove > all namespace definitions (with matched '}'), scrolling works much faster. OK, I'll have a look at it in the next few days. I have an idea for a solution, which would involve effectively masking out enclosing namespaces' braces, but this might be quite complicated to implement. Am I right in thinking that a namespace declaration _cannot_ be enclosed within any other type of declaration, such as a class? > Dmitry -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode 2012-12-19 22:39 ` Alan Mackenzie @ 2012-12-20 0:52 ` Michael Welsh Duggan 0 siblings, 0 replies; 12+ messages in thread From: Michael Welsh Duggan @ 2012-12-20 0:52 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Eli Zaretskii, Dmitry Antipov, Stefan Monnier, emacs-devel Alan Mackenzie <acm@muc.de> writes: > Am I right in thinking that a namespace declaration _cannot_ be enclosed > within any other type of declaration, such as a class? It took a while to locate, but section 7.3.1 [namespace.def] states: 4 Every namespace-definition shall appear in the global scope or in a namespace scope (3.3.5). So, in essence, you are correct. -- Michael Welsh Duggan (md5i@md5i.com) ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode 2012-12-19 7:20 ` Dmitry Antipov 2012-12-19 22:39 ` Alan Mackenzie @ 2012-12-20 22:29 ` Alan Mackenzie 2012-12-21 5:15 ` Dmitry Antipov 1 sibling, 1 reply; 12+ messages in thread From: Alan Mackenzie @ 2012-12-20 22:29 UTC (permalink / raw) To: Dmitry Antipov; +Cc: Eli Zaretskii, Stefan Monnier, emacs-devel Hello, Dmitry, On Wed, Dec 19, 2012 at 11:20:55AM +0400, Dmitry Antipov wrote: > It looks like the reason of slow scrolling is how the C++ namespaces are handled. > Basically my test file is: > namespace X { > namespace Y { > /* 13K lines of code */ > } } > /* middle-point */ > namespace X { <============================================== > namespace Y { > /* 13K lines of code */ > } } > When scrolling reaches middle-point, everything hangs, most probably because > we need to scan huge regions (first and second namespace blocks). When I remove > all namespace definitions (with matched '}'), scrolling works much faster. What is happening is that (c-beginning-of-statement-1 ...) is being called repeatedly on the <===== line. o - This moves to the beginning of "namespace X {", o - then calls (scan-sexps (point) -1), o - which has to scan over the 13k lines of code. This is slow. The solution appears to be to give the backward search a search limit. Could you try out the following patch please, and let me know if it works OK: diff -r ce04d3763229 cc-fonts.el --- a/cc-fonts.el Thu Dec 20 20:11:22 2012 +0000 +++ b/cc-fonts.el Thu Dec 20 21:58:50 2012 +0000 @@ -1558,6 +1558,7 @@ ;; prevent a repeat invocation. See elisp/lispref page "Search-based ;; Fontification". (let* ((paren-state (c-parse-state)) + (decl-search-lim (c-determine-limit 1000)) decl-context in-typedef ps-elt) ;; Are we in any nested struct/union/class/etc. braces? (while paren-state @@ -1566,7 +1567,7 @@ (when (and (atom ps-elt) (eq (char-after ps-elt) ?\{)) (goto-char ps-elt) - (setq decl-context (c-beginning-of-decl-1) + (setq decl-context (c-beginning-of-decl-1 decl-search-lim) in-typedef (looking-at c-typedef-key)) (if in-typedef (c-forward-token-2)) (when (and c-opt-block-decls-with-vars-key > Dmitry -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode 2012-12-20 22:29 ` Alan Mackenzie @ 2012-12-21 5:15 ` Dmitry Antipov 2012-12-21 10:57 ` Alan Mackenzie 0 siblings, 1 reply; 12+ messages in thread From: Dmitry Antipov @ 2012-12-21 5:15 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Eli Zaretskii, Stefan Monnier, emacs-devel On 12/21/2012 02:29 AM, Alan Mackenzie wrote: > The solution appears to be to give the backward search a search limit. > Could you try out the following patch please, and let me know if it works > OK: Yes, that's much better. Thanks. Dmitry ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode 2012-12-21 5:15 ` Dmitry Antipov @ 2012-12-21 10:57 ` Alan Mackenzie 2012-12-22 3:28 ` Chong Yidong 0 siblings, 1 reply; 12+ messages in thread From: Alan Mackenzie @ 2012-12-21 10:57 UTC (permalink / raw) To: Stefan Monnier; +Cc: Eli Zaretskii, Dmitry Antipov, emacs-devel Stefan, should I commit this fix to the Emacs-24 branch? On Fri, Dec 21, 2012 at 09:15:52AM +0400, Dmitry Antipov wrote: > On 12/21/2012 02:29 AM, Alan Mackenzie wrote: > > The solution appears to be to give the backward search a search limit. > > Could you try out the following patch please, and let me know if it works > > OK: > Yes, that's much better. Thanks. > Dmitry -- Alan Mackenzie (Nuremberg, Germany). ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode 2012-12-21 10:57 ` Alan Mackenzie @ 2012-12-22 3:28 ` Chong Yidong 0 siblings, 0 replies; 12+ messages in thread From: Chong Yidong @ 2012-12-22 3:28 UTC (permalink / raw) To: Alan Mackenzie; +Cc: Eli Zaretskii, Dmitry Antipov, Stefan Monnier, emacs-devel Alan Mackenzie <acm@muc.de> writes: > Stefan, > > should I commit this fix to the Emacs-24 branch? Not Stefan, but I think we should take the risk of committing to emacs-24; this bug would be very annoying for people using Emacs as an editor for C++, and the fix seems safe enough. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Scrolling huge buffers and cc-mode 2012-12-18 9:56 Scrolling huge buffers and cc-mode Dmitry Antipov 2012-12-18 17:30 ` Eli Zaretskii @ 2012-12-18 19:40 ` Stefan Monnier 1 sibling, 0 replies; 12+ messages in thread From: Stefan Monnier @ 2012-12-18 19:40 UTC (permalink / raw) To: Dmitry Antipov; +Cc: Emacs development discussions > resumed, but it's still _very_ slow. This looks like a C++ issue only > since xdisp.c of nearly the same size may be scrolled much faster. M-x profiler-start RET would probably give more hints about where the time is spent. Stefan ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2012-12-22 3:28 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-12-18 9:56 Scrolling huge buffers and cc-mode Dmitry Antipov 2012-12-18 17:30 ` Eli Zaretskii 2012-12-18 19:58 ` Stefan Monnier 2012-12-19 6:04 ` Dmitry Antipov 2012-12-19 7:20 ` Dmitry Antipov 2012-12-19 22:39 ` Alan Mackenzie 2012-12-20 0:52 ` Michael Welsh Duggan 2012-12-20 22:29 ` Alan Mackenzie 2012-12-21 5:15 ` Dmitry Antipov 2012-12-21 10:57 ` Alan Mackenzie 2012-12-22 3:28 ` Chong Yidong 2012-12-18 19:40 ` Stefan Monnier
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.