bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[John Wiegley]

>>>>> YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:

> With the Time Profiler in Instrument.app, I found that fontification
> of CC Mode in Emacs 24 is much heavier and causes frequent GCs than
> that in Emacs 23.  Please try the following:
> 
>   1. Start Emacs 24 Mac port with -Q (alternatively, pressing the
>      shift key.)
>   2. M-x load-file PREFIX/share/emacs/23.4/lisp/progmodes/cc-fonts.elc
>      RTE.
>   3. Replay scrolling a large C++ file.
> 
> I'm not sure if this slowdown is intended or expected.

Indeed, this makes the speed situation much better on Emacs 24.0.97.

 - When I scroll a large C++ file in Emacs 24.0.97 the first time, the
   performance is very choppy, even on a powerful Mac Pro machine.

   There are moments toward the end of the file when I can actually count out
   10 seconds or so before it moves on to the next page.  The file is 17,983
   lines long, consisting entirely of type declarations, enum, #define's and
   prototypes.

 - If I press M-<, go back to the top of the file, and then scroll to the
   bottom again, there are basically no pauses.

 - If I delete the buffer and re-open the file, scrolling is the same as
   before.

 - If I delete the buffer and load cc-fonts.elc from Emacs 23.4, scrolling
   performance is *much* better.  It is less choppy, and although it still
   shows one long pause toward the end (garbage collection?), that's it.

 - As before, going to the top with M-< and re-scrolling shows perfect speed,
   no lag whatsoever; and killing the buffer and re-scrolling shows the same
   faster performance as before, with less lag (but still a little bit).

The strange thing is, cc-fonts.el.gz is identical between Emacs 23.4 and Emacs
24.0.97!  Only the .elc's differ.  Have we found a byte-compilation issue in
Emacs 24?

John

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[Andreas Schwab]

John Wiegley <jwiegley@gmail.com> writes:

> The strange thing is, cc-fonts.el.gz is identical between Emacs 23.4 and Emacs
> 24.0.97!  Only the .elc's differ.

Do they also differ semantically?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[Glenn Morris]

John Wiegley wrote:

> The strange thing is, cc-fonts.el.gz is identical between Emacs 23.4
> and Emacs 24.0.97!

I see hundreds of lines of differences between the emacs-23 and emacs-24
branch versions of cc-fonts.

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[Glenn Morris]

PS The context for this bug report is missing; but I imagine the first
thing Alan will ask for is an example that shows how to reproduce the
problem.

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[John Wiegley]

>>>>> Glenn Morris <rgm@gnu.org> writes:

> I see hundreds of lines of differences between the emacs-23 and emacs-24
> branch versions of cc-fonts.

Sorry, tool failure here.  Now I'm seeing:

    660 insertions(+), 243 deletions(-)

> PS The context for this bug report is missing; but I imagine the first thing
> Alan will ask for is an example that shows how to reproduce the problem.

Start either Emacs with -Q -nw.  Open a largish C++ file.  Hold down C-v.  On
my laptop the lagginess was quite obvious, on my desktop a little less so.

Thanks, John

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[Eli Zaretskii]

> Date: Wed, 23 May 2012 05:28:24 -0500
> Cc: mituharu+bug-gnu-emacs-mac@math.s.chiba-u.ac.jp, 11545@debbugs.gnu.org
> 
> Start either Emacs with -Q -nw.  Open a largish C++ file.  Hold down C-v.  On
> my laptop the lagginess was quite obvious, on my desktop a little less so.

Does the sluggishness go away if, right after starting Emacs, you type
"M-x global-font-lock-mode RET" to disable font-lock, and _then_ visit
that largish C++ file?

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

>> Start either Emacs with -Q -nw.  Open a largish C++ file.  Hold down C-v.
>> On my laptop the lagginess was quite obvious, on my desktop a little less
>> so.

> Does the sluggishness go away if, right after starting Emacs, you type "M-x
> global-font-lock-mode RET" to disable font-lock, and _then_ visit that
> largish C++ file?

Yes.  With font-lock off, it is buttery smooth.

So it looks like either the new cc-mode, or Emacs 24, is causing lots of
GC'ing when lazy fontifying a large C++ header.  And in fact, this header is
mostly C, so there's nothing complex in it like templates to throw cc-mode off
the mark.

John

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[Alan Mackenzie]

Hello John,

On Wed, May 23, 2012 at 05:28:24AM -0500, John Wiegley wrote:
> >>>>> Glenn Morris <rgm@gnu.org> writes:

> > I see hundreds of lines of differences between the emacs-23 and emacs-24
> > branch versions of cc-fonts.

> Sorry, tool failure here.  Now I'm seeing:

>     660 insertions(+), 243 deletions(-)

> > PS The context for this bug report is missing; but I imagine the first thing
> > Alan will ask for is an example that shows how to reproduce the problem.

> Start either Emacs with -Q -nw.  Open a largish C++ file.  Hold down C-v.  On
> my laptop the lagginess was quite obvious, on my desktop a little less so.

This has been the case for some while, as you have said.

The offending function is probably c-font-lock-enclosing-decls, a
relatively new function.  c-f-l-e-decls solves the former problem of
misfontification when a jit-lock chunk started within (mainly) a
struct/enum/union/class/... and lacked the context to fontify correctly.
An example of this happening was the first enum construct in
.../emacs/src/gnutls.h.

Could you possibly check this is the case in your file.c++ using elp.
Here's a quick recipe in case you haven't used it before:
[ M-x elp-instrument-package <ret> c- <ret>.
  Scroll with C-v, either once or an arbitrary number of times.
  M-x elp-results.]

The cost of this correct fontification is the "slight" sluggishness being
seen here.  It is likely possible to optimise this function somewhat,
though probably it's now too late for Emacs 24.1.

> Thanks, John

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[John Wiegley]

>>>>> Alan Mackenzie <acm@muc.de> writes:

> Could you possibly check this is the case in your file.c++ using elp.
> Here's a quick recipe in case you haven't used it before: [ M-x
> elp-instrument-package <ret> c- <ret>.  Scroll with C-v, either once or an
> arbitrary number of times.  M-x elp-results.]

Results attached below.

> The cost of this correct fontification is the "slight" sluggishness being
> seen here.  It is likely possible to optimise this function somewhat, though
> probably it's now too late for Emacs 24.1.

I'd rather forgo the correctness for the speed, since cc-mode has never been
100% correct, but it's always been correct enough.

John

Function Name                                  Call Count  Elapsed Time  Average Time
c-font-lock-fontify-region                     1529        31.313269999  0.0204795748
c-beginning-of-decl-1                          3657        11.680622999  0.0031940451
c-font-lock-declarations                       1529        11.070964000  0.0072406566
c-find-decl-spots                              1529        11.035531000  0.0072174826
c-context-set-fl-decl-start                    1529        10.549607     0.0068996775
c-set-fl-decl-start                            1529        10.542452999  0.0068949986
c-beginning-of-statement-1                     3910        9.8793279999  0.0025266823
c-backward-sws                                 123582      8.8908430000  7.194...e-05
c-crosses-statement-barrier-p                  15967       7.1756169999  0.0004494029
c-parse-state                                  14667       6.1785950000  0.0004212582
c-parse-state-1                                14667       5.6588790000  0.0003858238
c-font-lock-enclosing-decls                    1529        4.462361      0.0029184833
c-determine-limit                              5498        3.4111050000  0.0006204265
c-append-to-state-cache                        14467       2.5331440000  0.0001750980
c-remove-stale-state-cache                     14425       2.0322649999  0.0001408849
c-forward-decl-or-cast-1                       6632        1.9189249999  0.0002893433
c-beginning-of-macro                           164245      1.8161729999  1.105...e-05
c-forward-type                                 18312       1.5136799999  8.266...e-05
c-in-knr-argdecl                               3367        1.5124650000  0.0004492025
c-syntactic-skip-backward                      3882        1.3911049999  0.0003583475
c-font-lock-enum-tail                          1529        1.1747370000  0.0007683041
c-font-lock-complex-decl-prepare               1529        0.786506      0.0005143924
c-backward-token-2                             21271       0.7062790000  3.320...e-05
c-literal-limits                               25371       0.7047289999  2.777...e-05
c-at-macro-vsemi-p                             30271       0.7038709999  2.325...e-05
c-font-lock-declarators                        3723        0.6608859999  0.0001775143
c-forward-name                                 18056       0.5404579999  2.993...e-05
c-parse-state-get-strategy                     14667       0.4485610000  3.058...e-05
c-cheap-inside-bracelist-p                     5015        0.4087189999  8.149...e-05
c-forward-sws                                  60915       0.3506809999  5.756...e-06
c-syntactic-re-search-forward                  7741        0.2931619999  3.787...e-05
c-syntactic-content                            10849       0.2439560000  2.248...e-05
c-forward-token-2                              5974        0.2161999999  3.619...e-05
c-add-type                                     3181        0.1633079999  5.133...e-05
c-append-lower-brace-pair-to-state-cache       153         0.1180970000  0.0007718758
c-state-semi-safe-place                        7045        0.1104430000  1.567...e-05
c-remove-stale-state-cache-backwards           242         0.0868989999  0.0003590867
c-beginning-of-current-token                   8514        0.0621019999  7.294...e-06
c-looking-at-inexpr-block                      342         0.0573979999  0.0001678304
c-state-get-min-scan-pos                       14865       0.0395270000  2.659...e-06
c-get-fallback-scan-pos                        106         0.039258      0.0003703584
c-forward-label                                274         0.0353939999  0.0001291751
c-get-cache-scan-pos                           14667       0.0347800000  2.371...e-06
c-beginning-of-syntax                          250         0.0315929999  0.0001263719
c-font-lock-doc-comments                       3058        0.0247469999  8.092...e-06
c-safe-position                                4286        0.0241799999  5.641...e-06
c-state-literal-at                             267         0.0240109999  8.992...e-05
c-on-identifier                                694         0.0237040000  3.415...e-05
c-after-conditional                            79          0.0234840000  0.0002972658
c-syntactic-end-of-macro                       684         0.0193390000  2.827...e-05
c-forward-annotation                           6632        0.0187899999  2.833...e-06
c-skip-comments-and-strings                    7979        0.0155589999  1.949...e-06
c-punctuation-in                               636         0.0123149999  1.936...e-05
c-forward-keyword-clause                       831         0.0096369999  1.159...e-05
c-fontify-recorded-types-and-refs              6869        0.0094569999  1.376...e-06
c-state-safe-place                             267         0.0069390000  2.598...e-05
c-end-of-macro                                 2442        0.0062260000  2.549...e-06
c-font-lock-invalid-string                     1031        0.0052879999  5.129...e-06
c-end-of-current-token                         1207        0.0048019999  3.978...e-06
c-state-balance-parens-backwards               202         0.00426       2.108...e-05
c-most-enclosing-brace                         1789        0.0022139999  1.237...e-06
c-renarrow-state-cache                         10          0.0010699999  0.0001069999
c-state-mark-point-min-literal                 10          0.001004      0.0001003999
c-forward-single-comment                       37          0.0004699999  1.270...e-05
c-at-toplevel-p                                2           0.00028       0.00014
c-forward-to-cpp-define-body                   8           8.099...e-05  1.012...e-05
c-search-uplist-for-classkey                   2           1.2e-05       6e-06
c-leave-cc-mode-mode                           3           4.999...e-06  1.666...e-06

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[John Wiegley]

>>>>> John Wiegley <jwiegley@gmail.com> writes:

>>>>>> YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
>> With the Time Profiler in Instrument.app, I found that fontification
>> of CC Mode in Emacs 24 is much heavier and causes frequent GCs than
>> that in Emacs 23.  Please try the following:
>> 
>>   1. Start Emacs 24 Mac port with -Q (alternatively, pressing the
>>      shift key.)
>>   2. M-x load-file PREFIX/share/emacs/23.4/lisp/progmodes/cc-fonts.elc
>>      RTE.
>>   3. Replay scrolling a large C++ file.
>> 
>> I'm not sure if this slowdown is intended or expected.
> 
> Indeed, this makes the speed situation much better on Emacs 24.0.97.

I can now confirm that loading CC-Mode 5.32.3 into Emacs 23.4 causes the
identical speed issues that I was seeing with Emacs 24.0.97, so this is a
performance issue in the latest CC-Mode, not a bug in Emacs 24 or with
Mac-Port Emacs.

To the CC-Mode maintainers: is there a way to disable the slower, "more
correct" mode in the latest CC-Mode, and go back to the entirely sufficient
(for me) mode of previous versions?

Thanks,
  John

Re: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[Alan Mackenzie]

Hello, John.

On Mon, May 28, 2012 at 06:05:15PM -0500, John Wiegley wrote:
> >>>>> John Wiegley <jwiegley@gmail.com> writes:

> >>>>>> YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
> >> With the Time Profiler in Instrument.app, I found that fontification
> >> of CC Mode in Emacs 24 is much heavier and causes frequent GCs than
> >> that in Emacs 23.  Please try the following:

> >>   1. Start Emacs 24 Mac port with -Q (alternatively, pressing the
> >>      shift key.)
> >>   2. M-x load-file PREFIX/share/emacs/23.4/lisp/progmodes/cc-fonts.elc
> >>      RTE.
> >>   3. Replay scrolling a large C++ file.

> >> I'm not sure if this slowdown is intended or expected.

> > Indeed, this makes the speed situation much better on Emacs 24.0.97.

> I can now confirm that loading CC-Mode 5.32.3 into Emacs 23.4 causes
> the identical speed issues that I was seeing with Emacs 24.0.97, so
> this is a performance issue in the latest CC-Mode, not a bug in Emacs
> 24 or with Mac-Port Emacs.

> To the CC-Mode maintainers: is there a way to disable the slower, "more
> correct" mode in the latest CC-Mode, and go back to the entirely
> sufficient (for me) mode of previous versions?

Not as such, no.  The only workaround at the moment is to use a
"pre-correct" version of CC Mode in place of an up to date one.

I've just done a binary chop on CC Mode versions, and it seems the latest
version before (?the first of) these enhancements was the one created by
this (mercurial) changeset (the repository can be downloaded from
<http://cc-mode.sourceforge.net>):

    changeset:   5109:981fa4f0270c
    parent:      5107:bd4013c5633b
    user:        acmacm
    date:        Wed Sep 15 17:47:52 2010 +0000
    files:       cc-engine.el
    description:
    (c-forward-<>-arglist-recur): Fix an infinite recursion.

To undo these changes would be difficult, since several later
enhancements and bug fixes are based on the new code.  I'll see if I can
find some way of optimising the offending code - most of the time it's
doing expensive checks and finding nothing.

Anyhow, I've got the problem flagged as a bug now.  Thanks again for
reporting it.

> Thanks,
>   John

-- 
Alan Mackenzie (Nuremberg, Germany).

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[John Wiegley]

>>>>> Alan Mackenzie <acm@muc.de> writes:

>> To the CC-Mode maintainers: is there a way to disable the slower, "more
>> correct" mode in the latest CC-Mode, and go back to the entirely sufficient
>> (for me) mode of previous versions?

> Not as such, no.  The only workaround at the moment is to use a
> "pre-correct" version of CC Mode in place of an up to date one.

Ok, I've done that (am now using the version of cc-mode from 23.4 in my 24.1
Emacs, and everything is super-snappy again).

> Anyhow, I've got the problem flagged as a bug now.  Thanks again for
> reporting it.

Thanks!  Just FYI: This doesn't just make scrolling laggy.  I have some long C
files which, if I add a statement that breaks up the fontification (for
example, I'm writing code that isn't yet valid), the buffer slows down to the
point that it takes up to 20 seconds for Emacs to register each keypress!  The
old cc-mode doesn't have this problem at all.

Thanks again,
  John

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[Stefan Kangas]

Hi Alan,

Alan Mackenzie <acm@muc.de> writes:

> On Mon, May 28, 2012 at 06:05:15PM -0500, John Wiegley wrote:
>> >>>>> John Wiegley <jwiegley@gmail.com> writes:
>
>> >>>>>> YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
>> >> With the Time Profiler in Instrument.app, I found that fontification
>> >> of CC Mode in Emacs 24 is much heavier and causes frequent GCs than
>> >> that in Emacs 23.  Please try the following:
>
>> >>   1. Start Emacs 24 Mac port with -Q (alternatively, pressing the
>> >>      shift key.)
>> >>   2. M-x load-file PREFIX/share/emacs/23.4/lisp/progmodes/cc-fonts.elc
>> >>      RTE.
>> >>   3. Replay scrolling a large C++ file.
>
>> >> I'm not sure if this slowdown is intended or expected.
>
>> > Indeed, this makes the speed situation much better on Emacs 24.0.97.
>
>> I can now confirm that loading CC-Mode 5.32.3 into Emacs 23.4 causes
>> the identical speed issues that I was seeing with Emacs 24.0.97, so
>> this is a performance issue in the latest CC-Mode, not a bug in Emacs
>> 24 or with Mac-Port Emacs.
>
>> To the CC-Mode maintainers: is there a way to disable the slower, "more
>> correct" mode in the latest CC-Mode, and go back to the entirely
>> sufficient (for me) mode of previous versions?
>
> Not as such, no.  The only workaround at the moment is to use a
> "pre-correct" version of CC Mode in place of an up to date one.
>
> I've just done a binary chop on CC Mode versions, and it seems the latest
> version before (?the first of) these enhancements was the one created by
> this (mercurial) changeset (the repository can be downloaded from
> <http://cc-mode.sourceforge.net>):
>
>     changeset:   5109:981fa4f0270c
>     parent:      5107:bd4013c5633b
>     user:        acmacm
>     date:        Wed Sep 15 17:47:52 2010 +0000
>     files:       cc-engine.el
>     description:
>     (c-forward-<>-arglist-recur): Fix an infinite recursion.
>
> To undo these changes would be difficult, since several later
> enhancements and bug fixes are based on the new code.  I'll see if I can
> find some way of optimising the offending code - most of the time it's
> doing expensive checks and finding nothing.
>
> Anyhow, I've got the problem flagged as a bug now.  Thanks again for
> reporting it.

Just to follow up on this bug, which has seen no update in the last 7
years.  Has it been fixed by now?

Best regards,
Stefan Kangas

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[Alan Mackenzie]

Hello, Stefan.

On Fri, Nov 01, 2019 at 19:40:31 +0100, Stefan Kangas wrote:
> Hi Alan,

> Alan Mackenzie <acm@muc.de> writes:

> > On Mon, May 28, 2012 at 06:05:15PM -0500, John Wiegley wrote:
> >> >>>>> John Wiegley <jwiegley@gmail.com> writes:

> >> >>>>>> YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp> writes:
> >> >> With the Time Profiler in Instrument.app, I found that fontification
> >> >> of CC Mode in Emacs 24 is much heavier and causes frequent GCs than
> >> >> that in Emacs 23.  Please try the following:

> >> >>   1. Start Emacs 24 Mac port with -Q (alternatively, pressing the
> >> >>      shift key.)
> >> >>   2. M-x load-file PREFIX/share/emacs/23.4/lisp/progmodes/cc-fonts.elc
> >> >>      RTE.
> >> >>   3. Replay scrolling a large C++ file.

> >> >> I'm not sure if this slowdown is intended or expected.

> >> > Indeed, this makes the speed situation much better on Emacs 24.0.97.

> >> I can now confirm that loading CC-Mode 5.32.3 into Emacs 23.4 causes
> >> the identical speed issues that I was seeing with Emacs 24.0.97, so
> >> this is a performance issue in the latest CC-Mode, not a bug in Emacs
> >> 24 or with Mac-Port Emacs.

> >> To the CC-Mode maintainers: is there a way to disable the slower, "more
> >> correct" mode in the latest CC-Mode, and go back to the entirely
> >> sufficient (for me) mode of previous versions?

> > Not as such, no.  The only workaround at the moment is to use a
> > "pre-correct" version of CC Mode in place of an up to date one.

> > I've just done a binary chop on CC Mode versions, and it seems the latest
> > version before (?the first of) these enhancements was the one created by
> > this (mercurial) changeset (the repository can be downloaded from
> > <http://cc-mode.sourceforge.net>):

> >     changeset:   5109:981fa4f0270c
> >     parent:      5107:bd4013c5633b
> >     user:        acmacm
> >     date:        Wed Sep 15 17:47:52 2010 +0000
> >     files:       cc-engine.el
> >     description:
> >     (c-forward-<>-arglist-recur): Fix an infinite recursion.

> > To undo these changes would be difficult, since several later
> > enhancements and bug fixes are based on the new code.  I'll see if I can
> > find some way of optimising the offending code - most of the time it's
> > doing expensive checks and finding nothing.

> > Anyhow, I've got the problem flagged as a bug now.  Thanks again for
> > reporting it.

> Just to follow up on this bug, which has seen no update in the last 7
> years.  Has it been fixed by now?

"Fixed" doesn't seem the right term, really.  There have been several
improvemnts in the scrolling speed over the years (the latest less than
an hour ago ;-).  When I scroll through a typical ~700k C++ buffer, I now
experience mild sluggishness, a C-v taking a small fraction of a second,
but longer than instantaneous.  This is perhaps less than ideal on a
slowish machine (which mine is not), but there's a tradeoff between rapid
fontification and correct fontification.

I think this bug should now be closed.

> Best regards,
> Stefan Kangas

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#11545: 24.0.96-mac-2.92; Strange speed problem scrolling in C++ code

[Stefan Kangas]

Hi Alan,

Alan Mackenzie <acm@muc.de> writes:

>> Just to follow up on this bug, which has seen no update in the last 7
>> years.  Has it been fixed by now?
>
> "Fixed" doesn't seem the right term, really.  There have been several
> improvemnts in the scrolling speed over the years (the latest less than
> an hour ago ;-).  When I scroll through a typical ~700k C++ buffer, I now
> experience mild sluggishness, a C-v taking a small fraction of a second,
> but longer than instantaneous.  This is perhaps less than ideal on a
> slowish machine (which mine is not), but there's a tradeoff between rapid
> fontification and correct fontification.
>
> I think this bug should now be closed.

Thanks for that explanation.  I'm consequently closing the bug.

Best regards,
Stefan Kangas