bug#51692: 29.0.50; High CPU in c++ mode. Type finder?

bug#51692: 29.0.50; High CPU in c++ mode. Type finder?

[David Koppelman]

[-- Attachment #1: Type: text/plain, Size: 355 bytes --]


Load the attached file into a buffer:

./src/emacs --no-init d.cc

Emacs will use 100% CPU on a core while the buffer is visible. CPU usage
goes to normal when switching to another buffer. The attached file is a
reduced version of a much larger file. (The larger file experiences the
high CPU usage only while a certain portion of the code is visible.)


[-- Attachment #2: Testcase. --]
[-- Type: text/plain, Size: 402 bytes --]



{

  if ( indices_segments != opt_segments )
    {
      for ( int i=1; i<chain_length; i++ )
        for ( int t=0; t<opt_segments; t++ )
          for ( int inner=0; inner<2; inner++ )
            bset_spiral2 << ivec4(i,t,inner,0);
    }

  buf_uni_common->chain_length = chain_length;
}


{ for(int i=0;i<chain_length;i++)balls[i].push(amt);}
{ for(int i=0;i<chain_length;i++)balls[i].stop();}



[-- Attachment #3: Type: text/plain, Size: 4199 bytes --]




In GNU Emacs 29.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.22.30, cairo version 1.15.12)
 of 2021-11-08 built on cyc.ece.lsu.edu
Repository revision: 5861b8d027382ecbd4c0d3dffc283b8ac95b5692
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12010000
System Description: Red Hat Enterprise Linux 8.4 (Ootpa)

Configured using:
 'configure
 PATH=/opt/gnu/gcc11/bin:/bin:/usr/bin:/opt/local/bin:/usr/local/bin:/home/faculty/koppel/scripts:/home/faculty/koppel/r/ga/simtools/bin:/home/faculty/koppel/lbin:/home/apps/bin:/usr/kerberos/bin:/usr/X11R6/bin:/usr/local/cuda/bin:/usr/sbin:/apps/linux/cadence/GENUS211/bin:/apps/linux/cadence/SPECTRE211/tools/bin:/apps/linux/cadence/SPECTRE211/bin:/apps/linux/cadence/SPECTRE191/tools/bin:/apps/linux/cadence/SPECTRE191/bin:/apps/linux/cadence/XCELIUM2103/tools/bin/64bit:/apps/linux/cadence/XCELIUM2103/tools/bin:/apps/linux/cadence/XCELIUM2103/bin:/apps/linux/cadence/IC618/tools/bin:/apps/linux/cadence/IC618/bin:/opt/torque/bin:/extra/localpri/build/depot_tools:/usr/lib64/openmpi/bin:/apps/linux/Mathematica/Executables:/opt/gnu/gcc11/bin:/opt/pgi/linux86-64/17.10/bin
 LD_LIBRARY_PATH=/opt/gnu/gcc11/lib:/opt/gnu/gcc11/lib64:
 CC=/opt/gnu/gcc11/bin/gcc 'CFLAGS=-march=native -O2' --without-pop
 --with-native-compilation'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
WEBP X11 XDBE XIM XPM GTK3 ZLIB

Important settings:
  value of $LC_COLLATE: C
  value of $LC_TIME: C
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: C++//l

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug comp comp-cstr warnings rx cl-extra
message mailcap yank-media rmc puny dired dired-loaddefs rfc822 mml
mml-sec epa derived epg rfc6068 epg-config gnus-util rmail
rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map text-property-search time-date
seq gv subr-x byte-opt bytecomp byte-compile cconv mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils vc-git
diff-mode easy-mmode vc-dispatcher cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs help-mode
cl-loaddefs cl-lib iso-transl tooltip eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button
loaddefs faces cus-face macroexp files window text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 129683 8659)
 (symbols 48 10727 1)
 (strings 32 32890 2419)
 (string-bytes 1 1161589)
 (vectors 16 20723)
 (vector-slots 8 397379 17246)
 (floats 8 42 38)
 (intervals 56 413 0)
 (buffers 992 14))

bug#51692: 29.0.50; High CPU in c++ mode. Type finder?

[Lars Ingebrigtsen]

David Koppelman <koppel@ece.lsu.edu> writes:

> Load the attached file into a buffer:
>
> ./src/emacs --no-init d.cc
>
> Emacs will use 100% CPU on a core while the buffer is visible. CPU usage
> goes to normal when switching to another buffer. The attached file is a
> reduced version of a much larger file. (The larger file experiences the
> high CPU usage only while a certain portion of the code is visible.)

I can reproduce this problem on the trunk; Alan added to the CCs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#51692: 29.0.50; High CPU in c++ mode. Type finder?

[Alan Mackenzie]

Hello, David, Zhiwei, Yuri, and Lars.

On Mon, Nov 08, 2021 at 13:00:06 -0600, David Koppelman wrote:

> Load the attached file into a buffer:

> ./src/emacs --no-init d.cc

> Emacs will use 100% CPU on a core while the buffer is visible. CPU usage
> goes to normal when switching to another buffer. The attached file is a
> reduced version of a much larger file. (The larger file experiences the
> high CPU usage only while a certain portion of the code is visible.)

Yes, there is a bug here.  CC Mode is expected to use a high amount of
CPU time (but not 100%) for a few seconds after starting C (etc.) Mode,
or for a few minutes after starting Emacs when there's a desktop with a
lot of CC Mode files in it.

However, with C++ files (and likely Java files, too), a problem with
templates caused this 100% usage.

I'm hoping the following patch will fix it.  Could you (plural) please
apply this patch to the Emacs master branch, byte compile the two
amended files, then load the fixed CC Mode into your Emacs.  Then please
try it out with your real files.  (If anybody wants any help with the
patching or byte compiling, feel free to send me personal email.)

Then please report back to the bug list confirming that the bug is
fixed, or telling me what's still not right.

Thank you!



diff -r 05fd00cb5937 cc-engine.el
--- a/cc-engine.el	Sat Oct 30 15:57:26 2021 +0000
+++ b/cc-engine.el	Thu Nov 11 18:47:47 2021 +0000
@@ -6838,6 +6838,13 @@
 (defvar c-found-types nil)
 (make-variable-buffer-local 'c-found-types)
 
+;; Dynamically bound variable that instructs `c-forward-type' to
+;; record the ranges of types that only are found.  Behaves otherwise
+;; like `c-record-type-identifiers'.  Also when this variable is non-nil,
+;; `c-fontify-new-found-type' doesn't get called (yet) for the purported
+;; type.
+(defvar c-record-found-types nil)
+
 (defsubst c-clear-found-types ()
   ;; Clears `c-found-types'.
   (setq c-found-types
@@ -6851,7 +6858,10 @@
   (let ((type (c-syntactic-content from to c-recognize-<>-arglists)))
     (unless (gethash type c-found-types)
       (puthash type t c-found-types)
-      (when (and (eq (string-match c-symbol-key type) 0)
+      (when (and (not c-record-found-types) ; Only call `c-fontify-new-fount-type'
+					; when we haven't "bound" c-found-types
+					; to itself in c-forward-<>-arglist.
+		 (eq (string-match c-symbol-key type) 0)
 		 (eq (match-end 0) (length type)))
 	(c-fontify-new-found-type type)))))
 
@@ -8248,11 +8258,6 @@
 	   (setq c-record-ref-identifiers
 		 (cons range c-record-ref-identifiers))))))
 
-;; Dynamically bound variable that instructs `c-forward-type' to
-;; record the ranges of types that only are found.  Behaves otherwise
-;; like `c-record-type-identifiers'.
-(defvar c-record-found-types nil)
-
 (defmacro c-forward-keyword-prefixed-id (type)
   ;; Used internally in `c-forward-keyword-clause' to move forward
   ;; over a type (if TYPE is 'type) or a name (otherwise) which
@@ -8480,6 +8485,11 @@
 		(c-forward-<>-arglist-recur all-types)))
 	(progn
 	  (when (consp c-record-found-types)
+	    (let ((cur c-record-found-types))
+	      (while (consp (car-safe cur))
+		(c-fontify-new-found-type
+		 (buffer-substring-no-properties (caar cur) (cdar cur)))
+		(setq cur (cdr cur))))
 	    (setq c-record-type-identifiers
 		  ;; `nconc' doesn't mind that the tail of
 		  ;; `c-record-found-types' is t.
@@ -9203,6 +9213,12 @@
 
 		(when (and (eq res t)
 			   (consp c-record-found-types))
+		  ;; Cause the confirmed types to get fontified.
+		  (let ((cur c-record-found-types))
+		    (while (consp (car-safe cur))
+		      (c-fontify-new-found-type
+		       (buffer-substring-no-properties (caar cur) (cdar cur)))
+		      (setq cur (cdr cur))))
 		  ;; Merge in the ranges of any types found by the second
 		  ;; `c-forward-type'.
 		  (setq c-record-type-identifiers
diff -r 05fd00cb5937 cc-fonts.el
--- a/cc-fonts.el	Sat Oct 30 15:57:26 2021 +0000
+++ b/cc-fonts.el	Thu Nov 11 18:47:47 2021 +0000
@@ -105,6 +105,7 @@
 (cc-bytecomp-defun c-font-lock-objc-method)
 (cc-bytecomp-defun c-font-lock-invalid-string)
 (cc-bytecomp-defun c-before-context-fl-expand-region)
+(cc-bytecomp-defun c-font-lock-fontify-region)
 
 \f
 ;; Note that font-lock in XEmacs doesn't expand face names as
@@ -2431,6 +2432,7 @@
 (defun c-force-redisplay (start end)
   ;; Force redisplay immediately.  This assumes `font-lock-support-mode' is
   ;; 'jit-lock-mode.  Set the variable `c-re-redisplay-timer' to nil.
+  (save-excursion (c-font-lock-fontify-region start end))
   (jit-lock-force-redisplay (copy-marker start) (copy-marker end))
   (setq c-re-redisplay-timer nil))
 
@@ -2458,7 +2460,6 @@
 	    (dolist (win-boundary window-boundaries)
 	      (when (and (< (match-beginning 0) (cdr win-boundary))
 			 (> (match-end 0) (car win-boundary))
-			 (c-get-char-property (match-beginning 0) 'fontified)
 			 (not c-re-redisplay-timer))
 		(setq c-re-redisplay-timer
 		      (run-with-timer 0 nil #'c-force-redisplay


-- 
Alan Mackenzie (Nuremberg, Germany).

bug#51692: 29.0.50; High CPU in c++ mode. Type finder?

[David Koppelman]

I applied the patch and it fixes the problem! I applied the patch to the
checkout on which I reported the problem (I didn't try to update). Also,
I tested both on the reduced testcase and the original file, on both
there was no suspiciously high CPU usage.

Thank you!


Alan Mackenzie <acm@muc.de> writes:

> Hello, David, Zhiwei, Yuri, and Lars.
>
> On Mon, Nov 08, 2021 at 13:00:06 -0600, David Koppelman wrote:
>
>> Load the attached file into a buffer:
>
>> ./src/emacs --no-init d.cc
>
>> Emacs will use 100% CPU on a core while the buffer is visible. CPU usage
>> goes to normal when switching to another buffer. The attached file is a
>> reduced version of a much larger file. (The larger file experiences the
>> high CPU usage only while a certain portion of the code is visible.)
>
> Yes, there is a bug here.  CC Mode is expected to use a high amount of
> CPU time (but not 100%) for a few seconds after starting C (etc.) Mode,
> or for a few minutes after starting Emacs when there's a desktop with a
> lot of CC Mode files in it.
>
> However, with C++ files (and likely Java files, too), a problem with
> templates caused this 100% usage.
>
> I'm hoping the following patch will fix it.  Could you (plural) please
> apply this patch to the Emacs master branch, byte compile the two
> amended files, then load the fixed CC Mode into your Emacs.  Then please
> try it out with your real files.  (If anybody wants any help with the
> patching or byte compiling, feel free to send me personal email.)
>
> Then please report back to the bug list confirming that the bug is
> fixed, or telling me what's still not right.
>
> Thank you!
>
>
>
> diff -r 05fd00cb5937 cc-engine.el
> --- a/cc-engine.el	Sat Oct 30 15:57:26 2021 +0000
> +++ b/cc-engine.el	Thu Nov 11 18:47:47 2021 +0000
> @@ -6838,6 +6838,13 @@
>  (defvar c-found-types nil)
>  (make-variable-buffer-local 'c-found-types)
>  
> +;; Dynamically bound variable that instructs `c-forward-type' to
> +;; record the ranges of types that only are found.  Behaves otherwise
> +;; like `c-record-type-identifiers'.  Also when this variable is non-nil,
> +;; `c-fontify-new-found-type' doesn't get called (yet) for the purported
> +;; type.
> +(defvar c-record-found-types nil)
> +
>  (defsubst c-clear-found-types ()
>    ;; Clears `c-found-types'.
>    (setq c-found-types
> @@ -6851,7 +6858,10 @@
>    (let ((type (c-syntactic-content from to c-recognize-<>-arglists)))
>      (unless (gethash type c-found-types)
>        (puthash type t c-found-types)
> -      (when (and (eq (string-match c-symbol-key type) 0)
> +      (when (and (not c-record-found-types) ; Only call `c-fontify-new-fount-type'
> +					; when we haven't "bound" c-found-types
> +					; to itself in c-forward-<>-arglist.
> +		 (eq (string-match c-symbol-key type) 0)
>  		 (eq (match-end 0) (length type)))
>  	(c-fontify-new-found-type type)))))
>  
> @@ -8248,11 +8258,6 @@
>  	   (setq c-record-ref-identifiers
>  		 (cons range c-record-ref-identifiers))))))
>  
> -;; Dynamically bound variable that instructs `c-forward-type' to
> -;; record the ranges of types that only are found.  Behaves otherwise
> -;; like `c-record-type-identifiers'.
> -(defvar c-record-found-types nil)
> -
>  (defmacro c-forward-keyword-prefixed-id (type)
>    ;; Used internally in `c-forward-keyword-clause' to move forward
>    ;; over a type (if TYPE is 'type) or a name (otherwise) which
> @@ -8480,6 +8485,11 @@
>  		(c-forward-<>-arglist-recur all-types)))
>  	(progn
>  	  (when (consp c-record-found-types)
> +	    (let ((cur c-record-found-types))
> +	      (while (consp (car-safe cur))
> +		(c-fontify-new-found-type
> +		 (buffer-substring-no-properties (caar cur) (cdar cur)))
> +		(setq cur (cdr cur))))
>  	    (setq c-record-type-identifiers
>  		  ;; `nconc' doesn't mind that the tail of
>  		  ;; `c-record-found-types' is t.
> @@ -9203,6 +9213,12 @@
>  
>  		(when (and (eq res t)
>  			   (consp c-record-found-types))
> +		  ;; Cause the confirmed types to get fontified.
> +		  (let ((cur c-record-found-types))
> +		    (while (consp (car-safe cur))
> +		      (c-fontify-new-found-type
> +		       (buffer-substring-no-properties (caar cur) (cdar cur)))
> +		      (setq cur (cdr cur))))
>  		  ;; Merge in the ranges of any types found by the second
>  		  ;; `c-forward-type'.
>  		  (setq c-record-type-identifiers
> diff -r 05fd00cb5937 cc-fonts.el
> --- a/cc-fonts.el	Sat Oct 30 15:57:26 2021 +0000
> +++ b/cc-fonts.el	Thu Nov 11 18:47:47 2021 +0000
> @@ -105,6 +105,7 @@
>  (cc-bytecomp-defun c-font-lock-objc-method)
>  (cc-bytecomp-defun c-font-lock-invalid-string)
>  (cc-bytecomp-defun c-before-context-fl-expand-region)
> +(cc-bytecomp-defun c-font-lock-fontify-region)
>  
>  \f
>  ;; Note that font-lock in XEmacs doesn't expand face names as
> @@ -2431,6 +2432,7 @@
>  (defun c-force-redisplay (start end)
>    ;; Force redisplay immediately.  This assumes `font-lock-support-mode' is
>    ;; 'jit-lock-mode.  Set the variable `c-re-redisplay-timer' to nil.
> +  (save-excursion (c-font-lock-fontify-region start end))
>    (jit-lock-force-redisplay (copy-marker start) (copy-marker end))
>    (setq c-re-redisplay-timer nil))
>  
> @@ -2458,7 +2460,6 @@
>  	    (dolist (win-boundary window-boundaries)
>  	      (when (and (< (match-beginning 0) (cdr win-boundary))
>  			 (> (match-end 0) (car win-boundary))
> -			 (c-get-char-property (match-beginning 0) 'fontified)
>  			 (not c-re-redisplay-timer))
>  		(setq c-re-redisplay-timer
>  		      (run-with-timer 0 nil #'c-force-redisplay

bug#51692: 29.0.50; High CPU in c++ mode. Type finder?

[Yuri D'Elia]

On Thu, Nov 11 2021, Alan Mackenzie wrote:
> Yes, there is a bug here.  CC Mode is expected to use a high amount of
> CPU time (but not 100%) for a few seconds after starting C (etc.) Mode,
> or for a few minutes after starting Emacs when there's a desktop with a
> lot of CC Mode files in it.
>
> However, with C++ files (and likely Java files, too), a problem with
> templates caused this 100% usage.

High-burst is not a problem, however I have a related question. Is it
normal that repeated C-g didn't abort?

There's nothing that seemed to work for me except waiting, which means
that with a few scrolling moves queued I just ended up killing emacs.

I didn't test if there was a difference between JIT/no-JIT versions
(should there ever be in terms of quitting behavior?)

> Then please report back to the bug list confirming that the bug is
> fixed, or telling me what's still not right.

Tested in both the small test I posted and the full source file I was
working with. Applied on master. Seems to be working perfectly now.

bug#51692: 29.0.50; High CPU in c++ mode. Type finder?

[Eli Zaretskii]

> From: Yuri D'Elia <wavexx@thregr.org>
> Date: Fri, 12 Nov 2021 10:47:46 +0100
> Cc: 51692@debbugs.gnu.org, David Koppelman <koppel@ece.lsu.edu>,
>  Zhiwei Chen <condy0919@gmail.com>, Lars Ingebrigtsen <larsi@gnus.org>
> 
> On Thu, Nov 11 2021, Alan Mackenzie wrote:
> > Yes, there is a bug here.  CC Mode is expected to use a high amount of
> > CPU time (but not 100%) for a few seconds after starting C (etc.) Mode,
> > or for a few minutes after starting Emacs when there's a desktop with a
> > lot of CC Mode files in it.
> >
> > However, with C++ files (and likely Java files, too), a problem with
> > templates caused this 100% usage.
> 
> High-burst is not a problem, however I have a related question. Is it
> normal that repeated C-g didn't abort?

What do you expect C-g to do when Emacs is in the middle of redisplay?

bug#51692: 29.0.50; High CPU in c++ mode. Type finder?

[Alan Mackenzie]

Hello, Yuri.

On Fri, Nov 12, 2021 at 10:47:46 +0100, Yuri D'Elia wrote:
> On Thu, Nov 11 2021, Alan Mackenzie wrote:
> > Yes, there is a bug here.  CC Mode is expected to use a high amount of
> > CPU time (but not 100%) for a few seconds after starting C (etc.) Mode,
> > or for a few minutes after starting Emacs when there's a desktop with a
> > lot of CC Mode files in it.

> > However, with C++ files (and likely Java files, too), a problem with
> > templates caused this 100% usage.

> High-burst is not a problem, however I have a related question. Is it
> normal that repeated C-g didn't abort?

No.  In this particular scenario, though, the 100% CPU usage was
happening in a timer function, so your C-g would just abort the
foreground function, i.e. nothing.  At least in this situation Emacs
wasn't completely frozen, since foreground processing took priority over
the background fontification.

> There's nothing that seemed to work for me except waiting, which means
> that with a few scrolling moves queued I just ended up killing emacs.

Ah.  Sorry.

> I didn't test if there was a difference between JIT/no-JIT versions
> (should there ever be in terms of quitting behavior?)

I don't know that.  But jit is pretty much universal now, apart from
for testing purposes by developers.

> > Then please report back to the bug list confirming that the bug is
> > fixed, or telling me what's still not right.

> Tested in both the small test I posted and the full source file I was
> working with. Applied on master. Seems to be working perfectly now.

Thanks!  That's good to hear.  I'll just give Zhiwei Chen a little time
to reply, and assuming (s)he doesn't find problems, I'll commit the fix.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#51692: 29.0.50; High CPU in c++ mode. Type finder?

[Zhiwei Chen]

Alan Mackenzie <acm@muc.de> writes:

> Thanks!  That's good to hear.  I'll just give Zhiwei Chen a little time
> to reply, and assuming (s)he doesn't find problems, I'll commit the fix.

It solves the issue after applied the patch on Emacs trunk. Thanks.

-- 
Zhiwei Chen

bug#51692: 29.0.50; High CPU in c++ mode. Type finder?

[Alan Mackenzie]

Hello, Zhiwei, David, Yuri, and Lars.

On Sat, Nov 13, 2021 at 17:24:22 +0800, Zhiwei Chen wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > Thanks!  That's good to hear.  I'll just give Zhiwei Chen a little time
> > to reply, and assuming (s)he doesn't find problems, I'll commit the fix.

> It solves the issue after applied the patch on Emacs trunk. Thanks.

Thanks to everybody for the testing.  I have now committed the fix to
the master branch, and am closing the bug with this post.

> -- 
> Zhiwei Chen

-- 
Alan Mackenzie (Nuremberg, Germany).