narrow-to-defun and mark-defun now work properly for CC Mode.

narrow-to-defun and mark-defun now work properly for CC Mode.

[Alan Mackenzie]

Hi, Emacs!

I've patched .../lisp/progmodes/cc-{defs,langs,cmds,awk}.el in savannah
(but not yet at SourceForge) so that these two commands work properly -
up till now, they've enclosed the region between the braces; now they
enclose the entire function, including the function header.

The basic idea is to set \(beginning\|end\)-of-defun-function to the
corresponding CC Mode functions.  However, C-M-[ae] remain bound
directly to these CC Mode functions to allow optimisation for large
argument (factor ~10 in speed).

Some macros have been changed, and there are two new ones.  Here is a
recipe for recompiling CC Mode, which gets these macros compiled in the
right order:

(i) M-x byte-compile-file cc-defs.el.  M-x load-file /path/cc-defs.elc
(ii) M-x byte-compile-file cc-langs.el.  M-x load-file /path/cc-langs.elc
(iii) M-x byte-compile-file cc-engine.el.  M-x load-file ../cc-engine.elc
(iv) M-x byte-compile-file cc-mode.el.  M-x load-file ../cc-mode.elc

(v) compile and load cc-{awk,cmds}.el (no dependence on the other four files).

Please complain to me about any problems.

-- 
Alan Mackenzie (Munich, Germany)

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Eric Hanchrow]

>>>>> "Alan" == Alan Mackenzie <acm@muc.de> writes:

    Alan> Hi, Emacs!  I've patched
    Alan> .../lisp/progmodes/cc-{defs,langs,cmds,awk}.el ... they
    Alan> enclose the entire function, including the function header.

Ooh, thanks -- I think that problem has been subtly annoying me for
ages.

-- 
People studying literature rarely say anything that would be of the
slightest use to those producing it.
        -- Paul Graham

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Nick Roberts]

 > Please complain to me about any problems.

Well, with that invitation!

Fontifying C files takes too long now, so my old machine starts to hang if I
step through a buffer or search for a string.  xbacktrace seems to consistently
include:

...
"c-where-wrt-brace-construct" (0x83752e9)
"c-beginning-of-defun" (0x8)
"beginning-of-defun-raw" (0x83752e9)
"beginning-of-defun" (0x10)
"c-get-fallback-start-pos" (0xd72d0)
"c-parse-state" (0xd72d0)
"c-font-lock-complex-decl-prepare" (0xd73f8)
"font-lock-fontify-keywords-region" (0xd72d0)
"font-lock-default-fontify-region" (0xd72d0)
"font-lock-fontify-region" (0xd72d0)
...

-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[martin rudalics]

>  > Please complain to me about any problems.
> 
> Well, with that invitation!
> 
> Fontifying C files takes too long now, so my old machine starts to hang if I
> step through a buffer or search for a string.  xbacktrace seems to consistently
> include:
> 
> ...

... probably due to disabling stealth fontification a couple of weeks ago.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Chong Yidong]

martin rudalics <rudalics@gmx.at> writes:

>>  > Please complain to me about any problems.
>>
>> Well, with that invitation!
>>
>> Fontifying C files takes too long now, so my old machine starts to hang if I
>> step through a buffer or search for a string.  xbacktrace seems to consistently
>> include:
>>
>> ...
>
> ... probably due to disabling stealth fontification a couple of weeks ago.

No, it has nothing to do with stealth fontification.  The problem is
due to the recent change:

2007-04-09  Alan Mackenzie  <acm@muc.de>

	Changes to make `narrow-to-defun' and `mark-defun' work properly
	in CC Mode:

	* progmodes/cc-defs.el (c-beginning-of-defun-1):
	* progmodes/cc-cmds.el (c-beginning-of-defun, c-end-of-defun):
	Bind beginning/end-of-defun-function to nil around calls to
	beginning/end-of-defun.

	* progmodes/cc-langs.el (beginning-of-defun-function)
	(end-of-defun-function): New c-lang-setvar's.

	* progmodes/cc-awk.el (c-awk-beginning-of-defun): Add "(or arg
	(setq arg 1))" to enable non-interactive call.

One way to see this is to open xdisp.c, and do C-s method.  The
incremental search takes several seconds.  Previously, it was
instantaneous.

I think it may be too late for this change.  If the problem it is
meant to fix is not serious, maybe this patch should be reverted.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Chong Yidong]

Chong Yidong <cyd@stupidchicken.com> writes:

> martin rudalics <rudalics@gmx.at> writes:
>
>>>  > Please complain to me about any problems.
>>>
>>> Well, with that invitation!
>>>
>>> Fontifying C files takes too long now, so my old machine starts to hang if I
>>> step through a buffer or search for a string.  xbacktrace seems to consistently
>>> include:
>
> The problem is due to the recent change:
>
> 2007-04-09  Alan Mackenzie  <acm@muc.de>
>
> One way to see this is to open xdisp.c, and do C-s method.  The
> incremental search takes several seconds.  Previously, it was
> instantaneous.

Another data point: 

- Open xdisp.c
- M->

On my Pentium IV, M-> takes 30 seconds to complete.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Chong Yidong]

Alan Mackenzie <acm@muc.de> writes:

> I've patched .../lisp/progmodes/cc-{defs,langs,cmds,awk}.el in savannah
> (but not yet at SourceForge) so that these two commands work properly -
> up till now, they've enclosed the region between the braces; now they
> enclose the entire function, including the function header.
>
> The basic idea is to set \(beginning\|end\)-of-defun-function to the
> corresponding CC Mode functions.  However, C-M-[ae] remain bound
> directly to these CC Mode functions to allow optimisation for large
> argument (factor ~10 in speed).

The trouble is that font-lock uses {beginning/end}-of-defun-function.
Because c-{beginning/end}-of-defun-function is slow (which is
apparently unavoidable due to syntactic complications), it makes
font-lock unacceptably slow (> 20 seconds for M-> in xdisp.c).

Having narrow-to-defun including the function header would be nice,
but if no solution to the slowness problem arises quickly, I think
this change should be postphoned to post-22.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Wed, 11 Apr 2007 13:53:30 -0400
> Cc: martin rudalics <rudalics@gmx.at>, Nick Roberts <nickrob@snap.net.nz>,
> 	emacs-devel@gnu.org
> 
> Chong Yidong <cyd@stupidchicken.com> writes:
> 
> > martin rudalics <rudalics@gmx.at> writes:
> >
> >>>  > Please complain to me about any problems.
> >>>
> >>> Well, with that invitation!
> >>>
> >>> Fontifying C files takes too long now, so my old machine starts to hang if I
> >>> step through a buffer or search for a string.  xbacktrace seems to consistently
> >>> include:
> >
> > The problem is due to the recent change:
> >
> > 2007-04-09  Alan Mackenzie  <acm@muc.de>
> >
> > One way to see this is to open xdisp.c, and do C-s method.  The
> > incremental search takes several seconds.  Previously, it was
> > instantaneous.
> 
> Another data point: 
> 
> - Open xdisp.c
> - M->
> 
> On my Pentium IV, M-> takes 30 seconds to complete.

FWIW, I cannot reproduce any of these two problems with today's CVS.
Maybe my machine is too fast to notice, I don't know.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Richard Matthew Stallman]

    Fontifying C files takes too long now,

Is this the result of a very recent change?

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Alan Mackenzie]

'Evening, Chong!

On Wed, Apr 11, 2007 at 01:53:30PM -0400, Chong Yidong wrote:
> Chong Yidong <cyd@stupidchicken.com> writes:

> > martin rudalics <rudalics@gmx.at> writes:

> >>>  > Please complain to me about any problems.

> >>> Well, with that invitation!

> >>> Fontifying C files takes too long now, so my old machine starts to
> >>> hang if I step through a buffer or search for a string.  xbacktrace
> >>> seems to consistently include:

> > The problem is due to the recent change:

> > 2007-04-09  Alan Mackenzie  <acm@muc.de>

> One way to see this is to open xdisp.c, and do C-s method.  The
> incremental search takes several seconds.  Previously, it was
> instantaneous.

> Another data point: 

> - Open xdisp.c
> - M->

> On my Pentium IV, M-> takes 30 seconds to complete.

I have a 1.2 GHz Athlon, and for me these operations are instantaneous
enough.  I start Emacs with -Q.  With C-s method, I don't see any delay
at all, and M-> takes well under half a second.  I have checked (by doing
C-x n d) that I have my own latest changes in my Emacs.

Chong, please confirm you're starting your Emacs with -Q.  Would you
please check that C-x n d (`narrow-to-defun') does include functions'
headers.  One other possibility is that the recipe I gave for rebuilding
the CC Mode files.elc was erroneous (or you made an error following it),
and, somehow, one or more of the macros didn't get properly compiled.

I last updated my Emacs a week ago.  I'm going to update it again in the
next few minutes, to see if some other change in the last few days is
interacting badly with my changes.

I would like very much (and that is an understatement) to diagnose this
problem.  It would be very nice to achieve this in hours rather than
days, and I'll do all I can to help here.

-- 
Alan.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Kim F. Storm]

Chong Yidong <cyd@stupidchicken.com> writes:

> Another data point: 
>
> - Open xdisp.c
> - M->
>
> On my Pentium IV, M-> takes 30 seconds to complete.

On my Dell Laptop (1.4 MHz centrino) it takes 57 seconds.

And it is very difficult to interrupt it with C-g.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Alan Mackenzie]

Hallo, again!

On Wed, Apr 11, 2007 at 10:09:16PM +0100, Alan Mackenzie wrote:
> 'Evening, Chong!

> On Wed, Apr 11, 2007 at 01:53:30PM -0400, Chong Yidong wrote:
> > Chong Yidong <cyd@stupidchicken.com> writes:

> > One way to see this [ speed problems caused by the latest CC Mode
> > patch] is to open xdisp.c, and do C-s method.  The incremental search
> > takes several seconds.  Previously, it was instantaneous.

> > Another data point: 

> > - Open xdisp.c
> > - M->

> > On my Pentium IV, M-> takes 30 seconds to complete.

> I have a 1.2 GHz Athlon, and for me these operations are instantaneous
> enough.  I start Emacs with -Q.  With C-s method, I don't see any delay
> at all, and M-> takes well under half a second.  I have checked (by
> doing C-x n d) that I have my own latest changes in my Emacs.

[ .... ]

> I last updated my Emacs a week ago.  I'm going to update it again in the
> next few minutes, to see if some other change in the last few days is
> interacting badly with my changes.

I've just updated and rebuild my Emacs 22.  I still don't see any of the
speed problems others have.

> -- 
> Alan.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Chong Yidong]

Alan Mackenzie <acm@muc.de> writes:

>> - Open xdisp.c
>> - M->
>
>> On my Pentium IV, M-> takes 30 seconds to complete.
>
> I have a 1.2 GHz Athlon, and for me these operations are instantaneous
> enough.  I start Emacs with -Q.  With C-s method, I don't see any delay
> at all, and M-> takes well under half a second.  I have checked (by doing
> C-x n d) that I have my own latest changes in my Emacs.
>
> One other possibility is that the recipe I gave for rebuilding the
> CC Mode files.elc was erroneous (or you made an error following it),
> and, somehow, one or more of the macros didn't get properly
> compiled.

Aha, that was the problem!  I expected `make recompile' to DTRT, but
this was a mistaken assumption for this case.

After doing a `make bootstrap', the performance problem has
disappeared entirely.  Thanks very much, and sorry for the noise!

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Alan Mackenzie]

Hi, Chong!

On Wed, Apr 11, 2007 at 04:51:12PM -0400, Chong Yidong wrote:
> Alan Mackenzie <acm@muc.de> writes:

[ .... ]

> > One other possibility is that the recipe I gave for rebuilding the CC
> > Mode files.elc was erroneous (or you made an error following it),
> > and, somehow, one or more of the macros didn't get properly compiled.

> Aha, that was the problem!  I expected `make recompile' to DTRT, but
> this was a mistaken assumption for this case.

Maybe I should look at fixing the problem in ..../lisp/Makefile.  The
dependencies between the CC Mode source files aren't accurately
represented there - For example, that cc-engine.elc depends on
cc-defs.el.  What do you think?

> After doing a `make bootstrap', the performance problem has disappeared
> entirely.  Thanks very much, and sorry for the noise!

Not a problem!  It's a relief to get it sorted out.  Thanks for the rapid
response!

Good night!

-- 
Alan.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Kim F. Storm]

Chong Yidong <cyd@stupidchicken.com> writes:

> After doing a `make bootstrap', the performance problem has
> disappeared entirely.  Thanks very much, and sorry for the noise!

I can confirm that this fixes the problem for me too.

Sorry for the false alarm.  And thank you for the fix!!

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Edward O'Connor]

> Another data point: 
>
> - Open xdisp.c
> - M->
>
> On my Pentium IV, M-> takes 30 seconds to complete.

In Emacs -Q, on my PowerBook G4 (1.5GHz), this is effectively instantaneous.

GNU Emacs 22.0.97.2 (powerpc-apple-darwin8.9.0, Carbon Version 1.6.0) of
2007-04-09 on roisin.local


-- 
Edward O'Connor

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Robert J. Chassell]

Both before and after fontification
with what amounts to  emacs -Q -D  , but more on the command line,
both     C-s method
and      M->
are fast, very fast.

Fontifying the emacs/src/xdisp.c buffer took nearly 5 minutes.

An IBM A21p laptop running a Pentium III (Coppermine) at 697.429 MHz,
using a Debian testing distribution of GNU/Linux, kernel 2.4.25,
updated this morning at around 11:45 UTC, along with

Today's GNU Emacs CVS snapshot, Wed, 2007 Apr 11  11:52 UTC
GNU Emacs 22.0.97.8 (i686-pc-linux-gnu, GTK+ Version 2.8.20)
started with

    /usr/local/src/emacs/src/emacs -Q -D \
     --eval '(setq-default mode-line-buffer-identification
                           (quote (#("%14b" 0 4 (face (:weight normal))))))' \
     -fn "-Misc-Fixed-Medium-R-Normal--20-200-75-75-C-100-ISO8859-1" \
     --visit=/usr/local/bin/emacs-test                               \
     --eval "(setq frame-title-format '(\"Emacs test:  %b\"))"       \
     --eval '(set-frame-name "Emacs test")'                          \
     --eval '(switch-to-buffer "*scratch*")'                         \
     --eval '(emacs-version t)'



I found the   emacs/src/xdisp.c  buffer and in it, I did:


    M-:
        (progn (load "elp")
               (elp-instrument-function 'font-lock-fontify-buffer)
               (elp-reset-all))
    RET

    M-x font-lock-fontify-buffer RET

    M-:
        (elp-results)
    RET


*ELP Profiling Results*  buffer 


    Function Name             Call Count  Elapsed Time  Average Time
    ========================  ==========  ============  ============
    font-lock-fontify-buffer  1           297.907887    297.907887


-- 
    Robert J. Chassell                          GnuPG Key ID: 004B4AC8
    bob@rattlesnake.com                         bob@gnu.org
    http://www.rattlesnake.com                  http://www.teak.cc

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Nick Roberts]

 > > Aha, that was the problem!  I expected `make recompile' to DTRT, but
 > > this was a mistaken assumption for this case.
 > 
 > Maybe I should look at fixing the problem in ..../lisp/Makefile.  The
 > dependencies between the CC Mode source files aren't accurately
 > represented there - For example, that cc-engine.elc depends on
 > cc-defs.el.  What do you think?

Is it possible to move all/most macros into one file?

-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Richard Matthew Stallman]

    The trouble is that font-lock uses {beginning/end}-of-defun-function.
    Because c-{beginning/end}-of-defun-function is slow (which is
    apparently unavoidable due to syntactic complications), it makes
    font-lock unacceptably slow (> 20 seconds for M-> in xdisp.c).

I see an easy solution: make font-lock use the old
code which CC mode formerly used, but keep the new code for
interactive use of C-M-h.

    Having narrow-to-defun including the function header would be nice,
    but if no solution to the slowness problem arises quickly, I think
    this change should be postphoned to post-22.

It is a mistake to TALK about giving up before investigating possible
easy solutions such as the one I suggested above.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Wed, 11 Apr 2007 16:51:12 -0400
> Cc: martin rudalics <rudalics@gmx.at>, Nick Roberts <nickrob@snap.net.nz>,
> 	emacs-devel@gnu.org
> 
> Aha, that was the problem!  I expected `make recompile' to DTRT, but
> this was a mistaken assumption for this case.
> 
> After doing a `make bootstrap', the performance problem has
> disappeared entirely.  Thanks very much, and sorry for the noise!

I did not need to bootstrap.  I used "make cvs-update".

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Nick Roberts]

 > > After doing a `make bootstrap', the performance problem has
 > > disappeared entirely.  Thanks very much, and sorry for the noise!
 > 
 > I did not need to bootstrap.  I used "make cvs-update".

    # This is useful after "cvs up".
    cvs-update: recompile autoloads finder-data custom-deps

All this does is recompile + extras.  I don't know how this worked in your case
but, in general, this wouldn't solve a problem involving macro dependencies.

I deleted all the cc-*.elc files in the progmodes directory and did `make
recompile'.  It worked in this case and I think deleting the relevant elc files
does generally solve such problems, although `make bootstrap' is probably
needed in other cases.




-- 
Nick                                           http://www.inet.net.nz/~nickrob

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Richard Matthew Stallman]

    Maybe I should look at fixing the problem in ..../lisp/Makefile.  The
    dependencies between the CC Mode source files aren't accurately
    represented there - For example, that cc-engine.elc depends on
    cc-defs.el.  What do you think?

Please give it a try and see how complex it is.
If it is simple, then let's install it.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Alan Mackenzie]

Hi, Nick

On Thu, Apr 12, 2007 at 10:19:56AM +1200, Nick Roberts wrote:
>  > > Aha, that was the problem!  I expected `make recompile' to DTRT, but
>  > > this was a mistaken assumption for this case.
>  > 
>  > Maybe I should look at fixing the problem in ..../lisp/Makefile.  The
>  > dependencies between the CC Mode source files aren't accurately
>  > represented there - For example, that cc-engine.elc depends on
>  > cc-defs.el.  What do you think?

> Is it possible to move all/most macros into one file?

# grep -c '^[^;]*(defmacro' $CC_RELFILESEL | sort -t: -nk2

gives this as a result:

cc-align.el:0
cc-cmds.el:0
cc-compat.el:0
cc-menus.el:0
cc-styles.el:0
cc-subword.el:0
cc-awk.el:1
cc-vars.el:1
cc-mode.el:2
cc-langs.el:3
cc-fonts.el:4
cc-bytecomp.el:16
cc-engine.el:22
cc-defs.el:43

, so pretty much all the macros are in 2 files.  (cc-bytecomp.el
only contains compile-time macros which won't mess other things up
much).

On or two macros could be moved from cc-mode.el to cc-defs.el, but it
seems scarcely worth the bother.  I think all (or very nearly all) the
others are used locally in the files that define them and don't depend
much on other files, so it would degrade maintainability to move them.
cc-defs.el is the home of CC Mode's "general" macros.

I think the problem here was that I changed a macro in cc-defs.el and
added two new c-lang-setvar's into cc-langs.el.  These made
recompilation of cc-engine.el necessary.

> Nick

-- 
Alan.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Stefan Monnier]

>> Aha, that was the problem!  I expected `make recompile' to DTRT, but
>> this was a mistaken assumption for this case.

> Maybe I should look at fixing the problem in ..../lisp/Makefile.  The
> dependencies between the CC Mode source files aren't accurately
> represented there - For example, that cc-engine.elc depends on
> cc-defs.el.  What do you think?

Won't make any difference since `recompile' doesn't use the Makefile's
dependency rules (it runs Emacs once and tells it to recompile everything
that needs it in the directory).


        Stefan

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Eli Zaretskii]

> From: Nick Roberts <nickrob@snap.net.nz>
> Date: Thu, 12 Apr 2007 15:51:31 +1200
> Cc: emacs-devel@gnu.org
> 
>  > > After doing a `make bootstrap', the performance problem has
>  > > disappeared entirely.  Thanks very much, and sorry for the noise!
>  > 
>  > I did not need to bootstrap.  I used "make cvs-update".
> 
>     # This is useful after "cvs up".
>     cvs-update: recompile autoloads finder-data custom-deps
> 
> All this does is recompile + extras.  I don't know how this worked in your case
> but, in general, this wouldn't solve a problem involving macro dependencies.

All I can say is that it did, as a matter of fact.

Re: narrow-to-defun and mark-defun now work properly for CC Mode.

[Nick Roberts]

 > # grep -c '^[^;]*(defmacro' $CC_RELFILESEL | sort -t: -nk2
 > 
 > gives this as a result:
 > 
 > cc-align.el:0
 > cc-cmds.el:0
 > cc-compat.el:0
 > cc-menus.el:0
 > cc-styles.el:0
 > cc-subword.el:0
 > cc-awk.el:1
 > cc-vars.el:1
 > cc-mode.el:2
 > cc-langs.el:3
 > cc-fonts.el:4
 > cc-bytecomp.el:16
 > cc-engine.el:22
 > cc-defs.el:43
 > 
 > , so pretty much all the macros are in 2 files.  (cc-bytecomp.el
 > only contains compile-time macros which won't mess other things up
 > much).

Well cc-defs.el is much smaller than cc-engine.el, so they could be merged.
Thinking about it, though, I'm not sure that it would help, as maybe it depends
on which files use the macros and whether Emacs think they need to be
recompiled.

-- 
Nick                                           http://www.inet.net.nz/~nickrob