end-of-defun acts weirdly in c-mode; also, mark-defun in c-mode

end-of-defun acts weirdly in c-mode; also, mark-defun in c-mode

[Miles Bader]

Instead of moving to the end of the current function, it actually seems
to move one line further if there's whitespace there.  This is
inconsistent with the behavior of end-of-defun elsewhere in emacs.

Also, since "mark-defun" basically does the right thing in c-mode
(except for slightly funny handling of whitespace due to c-mode's wacky
end-of-defun behavior), the current c-mode specific binding of C-M-h
(c-mark-function) should probably be removed.

Thanks,

-Miles

-- 
Freebooter, n. A conqueror in a small way of business, whose annexations lack
of the sanctifying merit of magnitude.

Re: end-of-defun acts weirdly in c-mode; also, mark-defun in c-mode

[Alan Mackenzie]

Hi, Miles!

On Mon, Feb 02, 2009 at 05:13:39PM +0900, Miles Bader wrote:
> Instead of moving to the end of the current function, it actually seems
> to move one line further if there's whitespace there.  This is
> inconsistent with the behavior of end-of-defun elsewhere in emacs.

In

1
2
3 (defun foo ()
4   (bar))
5
6

, C-M-e moves point to BOL 5.

In

1
2
3 int foo ()
4 {
5     return bar () ;
6 }
7
8   

, C-M-e moves pointo to BOL 7.  Honest, I've just tried it, with CVS from
a few days ago, with -Q.  That looks consistent to me.  I must have
missed something, somewhere.

Could you put some elisp and C source up with a precise description of
how they're different, please.

> Also, since "mark-defun" basically does the right thing in c-mode
> (except for slightly funny handling of whitespace due to c-mode's wacky
> end-of-defun behavior), the current c-mode specific binding of C-M-h
> (c-mark-function) should probably be removed.

`c-mark-function' will mark a construct which lacks a brace block, such
as

   int foo (int bar) ;

, whereas C-M-a and C-M-e search for outermost brace blocks ("soon" to be
enhanced to find objects inside C++ namespaces and the like).

I have no idea how useful this marking of a braceless construct is.
_Somebody_ probably uses it.  (Hi, Barry Warsaw, are you listening?)

> Thanks,

> -Miles

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun acts weirdly in c-mode; also, mark-defun in c-mode

[Stefan Monnier]

> `c-mark-function' will mark a construct which lacks a brace block, such
> as

>    int foo (int bar) ;

> , whereas C-M-a and C-M-e search for outermost brace blocks ("soon" to be
> enhanced to find objects inside C++ namespaces and the like).

> I have no idea how useful this marking of a braceless construct is.
> _Somebody_ probably uses it.  (Hi, Barry Warsaw, are you listening?)

Hopefully, end-of-defun-function can be used so that make-defun DTRT.


        Stefan

Re: end-of-defun acts weirdly in c-mode; also, mark-defun in c-mode

[Miles Bader]

Alan Mackenzie <acm@muc.de> writes:
> 3 (defun foo ()
> 4   (bar))
> 5
> 6
>
> , C-M-e moves point to BOL 5.

Yes

> 3 int foo ()
> 4 {
> 5     return bar () ;
> 6 }
> 7
> 8   
>
> , C-M-e moves pointo to BOL 7.  Honest, I've just tried it, with CVS from
> a few days ago, with -Q.  That looks consistent to me.  I must have
> missed something, somewhere.

For me, with emacs -Q, and from-CVS-as-of-this-morning, point always
moves to BOL 8.  I've tried it on a bunch of different C files, and it
always does the same thing.  If there is no blank line at 7, then it
will stop at BOL 7 though.

Could you it try again after updating from CVS?  Maybe it's a recently
introduce bug...

Thanks,

-Miles

-- 
Egotist, n. A person of low taste, more interested in himself than in me.

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com

Re: end-of-defun acts weirdly in c-mode; also, mark-defun in c-mode

[Alan Mackenzie]

Hi, Miles!

On Tue, Feb 03, 2009 at 06:17:06PM +0900, Miles Bader wrote:

> > 3 int foo ()
> > 4 {
> > 5     return bar () ;
> > 6 }
> > 7
> > 8   

> > , C-M-e moves pointo to BOL 7.  Honest, I've just tried it, with CVS
> > from a few days ago, with -Q.  That looks consistent to me.  I must
> > have missed something, somewhere.

> For me, with emacs -Q, and from-CVS-as-of-this-morning, point always
> moves to BOL 8.  I've tried it on a bunch of different C files, and it
> always does the same thing.  If there is no blank line at 7, then it
> will stop at BOL 7 though.

> Could you it try again after updating from CVS?  Maybe it's a recently
> introduce bug...

I've just updated and built CVSmacs.  M-x emacs-versions gives:

    GNU Emacs 23.0.90.1 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of 2009-02-03 on acm

I start it with emacs -Q.  Regardless of whether there's a blank line at
L7 or not, I cannot get C-M-e to go anywhere but BOL 7.

Would you please send me your foo.c as an attachment, tell me where to
put point so that C-M-e will fail.  Thanks!

> -Miles

-- 
Alan Mackenzie (Nuremberg, Germany).

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com

Re: end-of-defun acts weirdly in c-mode; also, mark-defun in c-mode

[Miles Bader]

[-- Attachment #1: Type: text/plain, Size: 610 bytes --]

Alan Mackenzie <acm@muc.de> writes:
> I start it with emacs -Q.  Regardless of whether there's a blank line at
> L7 or not, I cannot get C-M-e to go anywhere but BOL 7.
>
> Would you please send me your foo.c as an attachment, tell me where to
> put point so that C-M-e will fail.  Thanks!

I've attached a file "bug.c".

     (1) Start "emacs -Q"

     (2) Visit "bug.c"

     (3) Put point at the beginning of the first line marked with a comment

     (4) Hit C-M-e

     (5) Point jumps to the beginning of the line which starts defining
         "fun3" (also marked with a comment)
     
Thanks,

-Miles


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: bug.c --]
[-- Type: text/x-csrc, Size: 224 bytes --]

void fun1 ()
{
  x = 5;
  x = 5;
  x = 5;
}

void fun2 ()
{
  x = 5;
  x = 5;	/* put point at beginning of this line */
  x = 5;
}

void fun3 ()	/* C-M-e jumps to the beginning of this line */
{
  x = 5;
  x = 5;
  x = 5;
}

[-- Attachment #3: Type: text/plain, Size: 51 bytes --]



-- 
Insurrection, n. An unsuccessful revolution.

[-- Attachment #4: Type: text/plain, Size: 467 bytes --]

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com

Re: end-of-defun acts weirdly in c-mode; also, mark-defun in c-mode

[Miles Bader]

Incidentally, any idea why the copy of my message I'm seeing via gmane
ends with an advertisement for "Adobe(R)AIR(TM) software"?  I certainly
didn't add it!  Your most recent message also ends with such an
advertisement, but the copy on lists.gnu.org doesn't contain it.

The gmane message appears to be the copy received via the bug-cc-mode
mailing list, so either that mailing list is adding it or gmane is (and
the latter seems a bit unlikely as no other messages I see via gmane
seem to have such adverts)...

Thanks,

-Miles

-- 
Suburbia: where they tear out the trees and then name streets after them.

Re: end-of-defun acts weirdly in c-mode; also, mark-defun in c-mode

[Alan Mackenzie]

Hi, Miles!

It's occurred to me that my "CVS" version wasn't actually clean.  Sorry
for not checking this first.  I've downloaded a fresh copy of the CVS,
and I'm building it at the moment.

I'll let you know soon (assuming you'll not be asleep ;-).

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun acts weirdly in c-mode; also, mark-defun in c-mode

[Alan Mackenzie]

Hi, Miles!

On Tue, Feb 03, 2009 at 12:29:06PM +0000, Alan Mackenzie wrote:
> It's occurred to me that my "CVS" version wasn't actually clean.  Sorry
> for not checking this first.  I've downloaded a fresh copy of the CVS,
> and I'm building it at the moment.

Built.  Yes, I get the same thing as you, now.  Sorry for messing you
around.  I'll look into it.

-- 
Alan Mackenzie (Nuremberg, Germany).

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com

Re: end-of-defun is fubsr. [Was: end-of-defun acts weirdly in c-mode]

[Juanma Barranquero]

On Tue, Feb 3, 2009 at 17:09, Alan Mackenzie <acm@muc.de> wrote:

> end-of-defun (in .../lisp/emacs-lisp/lisp.el) is buggy, at least when an
> end-of-defun-function has been defined:

Is this related to bug#2106? (The title talks about Python, but
according to Chong it was a bug in end-of-defun.)

    Juanma

end-of-defun is fubsr. [Was: end-of-defun acts weirdly in c-mode]

[Alan Mackenzie]

[bug-cc-mode@gnu.org removed from the Cc:.]

Hi, Miles and Stefan!

On Tue, Feb 03, 2009 at 01:00:28PM +0000, Alan Mackenzie wrote:
> On Tue, Feb 03, 2009 at 12:29:06PM +0000, Alan Mackenzie wrote:
> > It's occurred to me that my "CVS" version wasn't actually clean.  Sorry
> > for not checking this first.  I've downloaded a fresh copy of the CVS,
> > and I'm building it at the moment.

> Built.  Yes, I get the same thing as you, now.  Sorry for messing you
> around.  I'll look into it.

end-of-defun (in .../lisp/emacs-lisp/lisp.el) is buggy, at least when an
end-of-defun-function has been defined:

(i) After calling end-of-defun-function, e-o-d takes it upon itself to
advance an arbitrary amount of whitespace/comments.  This is what you
(Miles) were complaining about.

(ii) When point is BETWEEN two C functions (more precisely JUST AFTER
the end of the previous function), C-M-e doesn't move over the next
function.  This is because it gets its knickers in a twist, first
calling BOD-raw, then EOD-function, trying to check if its succeeded
yet, etc. .........   This is crazy!

This mechanism is entirely unsuited to CC Mode.
c-\(beginning\|end\)-of-defun have a very high setup (determining
whether point is within a function's block, or header, etc.) and tear
down (locating the start of a function's header) time, but is lightening
fast zipping through brace blocks in between.  This high setup/teardown
time has been the cause of several "it's too slow" bugs (e.g. for C-x 4
a) in the last few years.

The current implementation of end-of-defun is essentially calling
c-end-of-defun AND c-beginning-of-defun in a loop, sometimes calling
them twice in each iteration.  This is slow for large ARG.  It's crazy!
To see this, go into buffer.c, and do

    C-u 106 C-M-e

.  On my box, this takes 20s.  By contrast, C-u 106 C-M-a takes about
0.5s.

Also, the semantics of end-of-defun-function have been completely
changed (specifically, in lisp.el v1.82, 2007-11-26) so that it now has
only a coincidental connection with what its name suggests.

It is IMAO a Bad Thing that \(beginning\|end\)-of-defun-function don't
have the same meaning modulo direction of travel.

I propose the following changes:
1/- end-of-defun-function should be restored to its prior semantics, and
additionally be passed the ARG argument in the same way as BOD-function.

2/- \(beginning\|end\)-of-defun-function should be trusted to do their
jobs.  These jobs should be defined to include moving to the next blank
line, to BOL, or over comments, and suchlike, as appropriate to the
major mode.  \(beginning\|end\)-of-defun should not tamper with the
position at all.

3/- end-of-defun should be restructured along the lines of
beginning-of-defun.

Alternatively, as a quick kludge, C-M-[ae] should be bound again to
c-\(beginning\|end\)-of-defun so as to make them work properly again.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Chong Yidong]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Tue, Feb 3, 2009 at 17:09, Alan Mackenzie <acm@muc.de> wrote:
>
>> end-of-defun (in .../lisp/emacs-lisp/lisp.el) is buggy, at least when an
>> end-of-defun-function has been defined:
>
> Is this related to bug#2106? (The title talks about Python, but
> according to Chong it was a bug in end-of-defun.)

A while back, Stefan changed end-of-defun, making it do some additional
movement beyond what end-of-defun-function does.

2007-11-26  Stefan Monnier  <monnier@iro.umontreal.ca>

            * emacs-lisp/lisp.el (end-of-defun): Restructure so that
            end-of-defun-function is called consistently, even for
            negative arguments.

This led to an infloop in python under the circumstances of bug#2106,
and I made some additional changes to try to prevent such infloops from
occurring.  But possibly the original, simpler version of end-of-defun
is better, I don't know.

Re: end-of-defun is fubsr. [Was: end-of-defun acts weirdly in c-mode]

[Alan Mackenzie]

Hi, Juanma,

On Tue, Feb 03, 2009 at 04:56:05PM +0100, Juanma Barranquero wrote:
> On Tue, Feb 3, 2009 at 17:09, Alan Mackenzie <acm@muc.de> wrote:

> > end-of-defun (in .../lisp/emacs-lisp/lisp.el) is buggy, at least when an
> > end-of-defun-function has been defined:

> Is this related to bug#2106? (The title talks about Python, but
> according to Chong it was a bug in end-of-defun.)

It's possible.  end-of-defun has a loop jumping backwards and forwards
until point is somewhere sensible.

>     Juanma

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Stefan Monnier]

>> > It's occurred to me that my "CVS" version wasn't actually clean.  Sorry
>> > for not checking this first.  I've downloaded a fresh copy of the CVS,
>> > and I'm building it at the moment.

>> Built.  Yes, I get the same thing as you, now.  Sorry for messing you
>> around.  I'll look into it.

> end-of-defun (in .../lisp/emacs-lisp/lisp.el) is buggy, at least when an
> end-of-defun-function has been defined:

> (i) After calling end-of-defun-function, e-o-d takes it upon itself to
> advance an arbitrary amount of whitespace/comments.  This is what you
> (Miles) were complaining about.

It should only move from "right after the closing }" to BOL 7.
Not "an arbitrary amount of whitespace".  Of course, there might be
a bug, but my guess is that your end-of-defun-function jumpts to BOL
7 rather than right after the brace.  So the problem is a disagreement
between the two.

> (ii) When point is BETWEEN two C functions (more precisely JUST AFTER
> the end of the previous function), C-M-e doesn't move over the next
> function.  This is because it gets its knickers in a twist, first
> calling BOD-raw, then EOD-function, trying to check if its succeeded
> yet, etc. .........   This is crazy!

This might be linked to the above problem.  For Elisp it seems to
work correctly.

> This mechanism is entirely unsuited to CC Mode.
> c-\(beginning\|end\)-of-defun have a very high setup (determining
> whether point is within a function's block, or header, etc.) and tear
> down (locating the start of a function's header) time, but is lightening
> fast zipping through brace blocks in between.  This high setup/teardown
> time has been the cause of several "it's too slow" bugs (e.g. for C-x 4
> a) in the last few years.

> The current implementation of end-of-defun is essentially calling
> c-end-of-defun AND c-beginning-of-defun in a loop, sometimes calling
> them twice in each iteration.  This is slow for large ARG.  It's crazy!
> To see this, go into buffer.c, and do

>     C-u 106 C-M-e

> .  On my box, this takes 20s.  By contrast, C-u 106 C-M-a takes about
> 0.5s.

I don't consider "C-u 106 C-M-e" as a common operation.

> Also, the semantics of end-of-defun-function have been completely
> changed (specifically, in lisp.el v1.82, 2007-11-26) so that it now has
> only a coincidental connection with what its name suggests.

Huh?  It hasn't completely changed.  Some details have changed to make
it easier to implement a simple end-of-defun-function, while making sure
that end-of-defun behaves consistently.
It was mostly a matter of fixing end-of-defun which was completely broken when
passed negative arguments.

> 1/- end-of-defun-function should be restored to its prior semantics, and
> additionally be passed the ARG argument in the same way as BOD-function.

Not sure about restoring the previous semantics.  But I could agree to
the additional ARG argument, which could even let it "take over" (so
beginning-of-defun-raw is not called in that case).

> 3/- end-of-defun should be restructured along the lines of
> beginning-of-defun.

I don't think that's a good idea.  The main reason is to deal with
languages that allow nested functions.


        Stefan

Re: end-of-defun is fubsr.

[Andreas Roehler]

Chong Yidong wrote:
> Juanma Barranquero <lekktu@gmail.com> writes:
> 
>> On Tue, Feb 3, 2009 at 17:09, Alan Mackenzie <acm@muc.de> wrote:
>>
>>> end-of-defun (in .../lisp/emacs-lisp/lisp.el) is buggy, at least when an
>>> end-of-defun-function has been defined:
>> Is this related to bug#2106? (The title talks about Python, but
>> according to Chong it was a bug in end-of-defun.)
> 
> A while back, Stefan changed end-of-defun, making it do some additional
> movement beyond what end-of-defun-function does.
> 
> 2007-11-26  Stefan Monnier  <monnier@iro.umontreal.ca>
> 
>             * emacs-lisp/lisp.el (end-of-defun): Restructure so that
>             end-of-defun-function is called consistently, even for
>             negative arguments.
> 
> This led to an infloop in python under the circumstances of bug#2106,
> and I made some additional changes to try to prevent such infloops from
> occurring.  But possibly the original, simpler version of end-of-defun
> is better, I don't know.
> 
> 
> 


I have remarked buggy/risky implementation already here

http://www.archivum.info/emacs-devel@gnu.org/2007-11/msg01940.html


Andreas Röhler

Re: end-of-defun is fubsr.

[Alan Mackenzie]

Hi, Stefan!

On Tue, Feb 03, 2009 at 12:13:19PM -0500, Stefan Monnier wrote:
> > end-of-defun (in .../lisp/emacs-lisp/lisp.el) is buggy, at least when an
> > end-of-defun-function has been defined:

> > (i) After calling end-of-defun-function, e-o-d takes it upon itself
> > to advance an arbitrary amount of whitespace/comments.  This is what
> > you (Miles) were complaining about.

> It should only move from "right after the closing }" to BOL 7.
> Not "an arbitrary amount of whitespace".  Of course, there might be

Sorry, yes, I was wrong.  It moves at most one line forward.

> a bug, but my guess is that your end-of-defun-function jumpts to BOL
> 7 rather than right after the brace.  So the problem is a disagreement
> between the two.

Exactly.  There is no documentation which says whether a [BE]OD-function
should adjust for WS/Comments or not.  I think it should, because the
variable name doesn't end in "-raw-function", and the major mode knows
better than the Emacs core where the best finishing point is.

How about:
(defvar beginning-of-defun-function ....
  "If non-nil, function for `beginning-of-defun-raw' to call.
  This is used to move to the beginning of the defun instead of using the
  normal recipe (see `beginning-of-defun').  Major modes can define this
  if defining `defun-prompt-regexp' is not sufficient to handle the
  mode's needs.

  The function takes one optional arg, the number of defuns to go back.
  It should leave point at a sensible place just before the defun starts,
  e.g. at the start of the line, or before a block comment preceding it.
  It should return non-nil on success, or nil on failure.

> > (ii) When point is BETWEEN two C functions (more precisely JUST AFTER
> > the end of the previous function), C-M-e doesn't move over the next
> > function.  This is because it gets its knickers in a twist, first
> > calling BOD-raw, then EOD-function, trying to check if its succeeded
> > yet, etc. .........   This is crazy!

> This might be linked to the above problem.  For Elisp it seems to
> work correctly.

The problem is that end-of-defun calls beginning-of-defun-raw at each
iteration (over ARG).  It thus discards the information as to whether
point began in a defun or between 2 defuns.

> > This mechanism is entirely unsuited to CC Mode.
> > c-\(beginning\|end\)-of-defun have a very high setup (determining
> > whether point is within a function's block, or header, etc.) and tear
> > down (locating the start of a function's header) time, but is lightening
> > fast zipping through brace blocks in between.  This high setup/teardown
> > time has been the cause of several "it's too slow" bugs (e.g. for C-x 4
> > a) in the last few years.

> > The current implementation of end-of-defun is essentially calling
> > c-end-of-defun AND c-beginning-of-defun in a loop, sometimes calling
> > them twice in each iteration.  This is slow for large ARG.  It's crazy!
> > To see this, go into buffer.c, and do

> >     C-u 106 C-M-e

> > .  On my box, this takes 20s.  By contrast, C-u 106 C-M-a takes about
> > 0.5s.

> I don't consider "C-u 106 C-M-e" as a common operation.

No, but when a simple operation takes 20s on a faster machine than a
12 MHz 386SX, something's less than optimal.  And when there are enough
top-level declarations without braces, this time will extend to several
minutes.  There are enough CC Mode users that quite a few will be calling
C-M-e with repeat counts.

What's bugging the Hades out of me is that I've put a LOT of effort into
optimising c-\(beginning\|end\)-of-defun, and that's being rendered
completely useless, at least for C-M-e, by an inept way of calling these
functions.  Several bug reports which made this work necessary came
directly from Emacs Developers (for example, C-x 4 a taking a minute to
run, or hassle with potential K&R regions taking just as long).

I foresee that immediately after the release of Emacs 23 there will be at
least one complaint about the sluggishness of C-M-e, and my reply will be
to agree, to apologise for it, and to tell the complainer just to bind
C-M-[ae] in c-mode-base-map.  Chance are, there will be several.  Hello,
lots of unrewarding repetitive and uncreative drudge work.

Having come this far, why can't we just do the job right?  Or, failing
that, at least bind C-M-[ae] in c-mode-base-map in Emacs-23?

Surely there's nobody here who isn't sick and fed up with this defun
movement business?  Surely to goodness, after 25 years, we should be able
to do major-mode specific defun movement as a matter of course?

> > Also, the semantics of end-of-defun-function have been completely
> > changed (specifically, in lisp.el v1.82, 2007-11-26) so that it now
> > has only a coincidental connection with what its name suggests.

> Huh?  It hasn't completely changed.  Some details have changed to make
> it easier to implement a simple end-of-defun-function, while making
> sure that end-of-defun behaves consistently.

It's changed from "move to next end of function" to "move to the end of
the function at whose beginning we now are", and its default value is
`forward-sexp'.  `c-end-of-defun' was a good fit for the variable as it
formerly was, but is now severely suboptimal.

> It was mostly a matter of fixing end-of-defun which was completely
> broken when passed negative arguments.

Surely we have two cases: if the major mode supplies a EOD-function (old
semantics), e-o-d should simply invoke it.  Otherwise (e.g. Elisp), why
not just do (scan-lists -1 1) (on the last iteration) to locate the
pertinent ")"?

> > 1/- end-of-defun-function should be restored to its prior semantics, and
> > additionally be passed the ARG argument in the same way as BOD-function.

> Not sure about restoring the previous semantics.  But I could agree to
> the additional ARG argument, which could even let it "take over" (so
> beginning-of-defun-raw is not called in that case).

:-)  Let's do it!

> > 3/- end-of-defun should be restructured along the lines of
> > beginning-of-defun.

> I don't think that's a good idea.  The main reason is to deal with
> languages that allow nested functions.

Don't follow - In the upcoming CC Mode 5.32 code (in the CVS repository
at SourceForge), C-M-[ae] works just fine for C++ defuns nested inside
classes/namespaces and so on.  The mechanism is entirely within CC Mode.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Stefan Monnier]

>> It should only move from "right after the closing }" to BOL 7.
>> Not "an arbitrary amount of whitespace".  Of course, there might be

> Sorry, yes, I was wrong.  It moves at most one line forward.

Yes, the code should be fixed to only move one line forward if it's
necessary (i.e. if we're not yet at a line beginning).

>> This might be linked to the above problem.  For Elisp it seems to
>> work correctly.

I don't see this problem (actually in my tests, it seems to work fine
in C mode as well).  Can you provide a test case.

> The problem is that end-of-defun calls beginning-of-defun-raw at each
> iteration (over ARG).  It thus discards the information as to whether
> point began in a defun or between 2 defuns.

I don't think so.  Quite the opposite: it uses BOD and EOD to figure out
whether we started inside a defun or between two defuns.

Unless by "the problem" you're talking about the performance problem, in
which case I understand that each time we call BOD (except for the first
call), we know that we're "outside" of a defun (unless there's nesting,
of course) but we don't tell that to BOD which may have to redo the work
of figuring it out.

> What's bugging the Hades out of me is that I've put a LOT of effort into
> optimising c-\(beginning\|end\)-of-defun, and that's being rendered
> completely useless, at least for C-M-e, by an inept way of calling these
> functions.  Several bug reports which made this work necessary came
> directly from Emacs Developers (for example, C-x 4 a taking a minute to
> run, or hassle with potential K&R regions taking just as long).

None of the above invloved EOD as far as I can tell.  These all do
a single call to BOD.

> Surely there's nobody here who isn't sick and fed up with this defun
> movement business?  Surely to goodness, after 25 years, we should be able
> to do major-mode specific defun movement as a matter of course?

Yes, it worked fine and fast in Emacs-21.  I wasn't the one who insisted
that we should scan the whole buffer just in order to make sure that
we're not bumping into the super-rare case of K&R declaration.

I'd *much* rather that C-mode's BOD gets it wrong every blue moon,
rather than the current "let's parse the whole damn thing".
On my 800MHz machine, C-mode is often borderline unusable nowadays.
And it's not the fault of end-of-defun.

> It's changed from "move to next end of function" to "move to the end of
> the function at whose beginning we now are",

Right.  As you may notice, the second is a subset of the first (with
a few caveats for nested functions, of course, but that shouldn't matter
for C-mode), so if your implementation works for the first, it should
work for the second as well.  It's called backward compatibility.

> and its default value is `forward-sexp'.  `c-end-of-defun' was a good
> fit for the variable as it formerly was, but is now
> severely suboptimal.

At most by a factor of 2.  I.e. if it's slow now, it sure wasn't
zippy before.

>> Not sure about restoring the previous semantics.  But I could agree to
>> the additional ARG argument, which could even let it "take over" (so
>> beginning-of-defun-raw is not called in that case).

> :-)  Let's do it!

I knew you'd like it.

>> > 3/- end-of-defun should be restructured along the lines of
>> > beginning-of-defun.
>> I don't think that's a good idea.  The main reason is to deal with
>> languages that allow nested functions.
> Don't follow - In the upcoming CC Mode 5.32 code (in the CVS repository
> at SourceForge), C-M-[ae] works just fine for C++ defuns nested inside
> classes/namespaces and so on.  The mechanism is entirely within CC Mode.

Yes, but I maintain Emacs, not CC-mode, so I care to move the
functionality to the generic code so that all modes (not just CC-modes)
benefit.


        Stefan

Re: end-of-defun is fubsr.

[Alan Mackenzie]

Hi, Stefan!

On Tue, Feb 03, 2009 at 03:50:36PM -0500, Stefan Monnier wrote:

> Yes, the code should be fixed to only move one line forward if it's
> necessary (i.e. if we're not yet at a line beginning).

Agreed.

> >> This might be linked to the above problem.  For Elisp it seems to
> >> work correctly.

> > (ii) When point is BETWEEN two C functions (more precisely JUST AFTER
> > the end of the previous function), C-M-e doesn't move over the next
> > function.  This is because it gets its knickers in a twist, first
> > calling BOD-raw, then EOD-function, trying to check if its succeeded
> > yet, etc. .........   This is crazy!

> I don't see this problem (actually in my tests, it seems to work fine
> in C mode as well).  Can you provide a test case.

Visit src/buffer.c, goto L313 which contains the closing brace of DEFUN
Fget_file_buffer, put point AFTER the "}".  Or put point on the next line
(L314).  In either case C-M-e leaves point at BOL315.  

> > The problem is that end-of-defun calls beginning-of-defun-raw at each
> > iteration (over ARG).  It thus discards the information as to whether
> > point began in a defun or between 2 defuns.

> I don't think so.  Quite the opposite: it uses BOD and EOD to figure out
> whether we started inside a defun or between two defuns.

Doing BOD-raw then (funcall EOD-function) can leave point after where it
started, so the test which should trigger another (funcall EOD-function)

      (unless (or (> (point) pos)
                  (eq (point) retry-point))

, fails.  I'm not sure it's possible to determine in the general case,
using only BOD and EOD, whether a given position is inside a defun or
not.  THIS IS CRITICALLY AND ESSENTIALLY CRUCIAL.

If one were to define a variable end-of-defun-RAW-function, which is
defined to leave point just after the last bit of the function, perhaps
that would work, but it would break existing 3rd party code.

> Unless by "the problem" you're talking about the performance problem, in
> which case I understand that each time we call BOD (except for the first
> call), we know that we're "outside" of a defun (unless there's nesting,
> of course) but we don't tell that to BOD which may have to redo the work
> of figuring it out.

WILL have to do the work.  There's a CC Mode function
`c-where-wrt-brace-construct' which figures it out.  It is
computationally expensive, and is called exactly once per call of
c-[be]-of-defun.

> > What's bugging the Hades out of me is that I've put a LOT of effort into
> > optimising c-\(beginning\|end\)-of-defun, and that's being rendered
> > completely useless, at least for C-M-e, by an inept way of calling these
> > functions.  Several bug reports which made this work necessary came
> > directly from Emacs Developers (for example, C-x 4 a taking a minute to
> > run, or hassle with potential K&R regions taking just as long).

> None of the above invloved EOD as far as I can tell.  These all do
> a single call to BOD.

C-x 4 a used to do, I think, 5 calls in total to c-[be]-of-defun; one or
two of these were certainly to c-end-of-defun.  Instead, it now calls
`c-defun-name' exactly once, which is all it wanted anyway.

> > Surely there's nobody here who isn't sick and fed up with this defun
> > movement business?  Surely to goodness, after 25 years, we should be able
> > to do major-mode specific defun movement as a matter of course?

> Yes, it worked fine and fast in Emacs-21.  I wasn't the one who insisted
> that we should scan the whole buffer just in order to make sure that
> we're not bumping into the super-rare case of K&R declaration.

Now, I wonder, who could that have been?  ;-)  Oh, how I'd love just to
get rid of these stupid K&R declarations, which NOBODY uses at all
nowadays, except on >20 year old code bases.  ;-)  Why don't we flush
them out of Emacs C source?  (No, you don't have to answer.)

Actually, in Emacs-21, ARG was ignored, presumed nil, when
[be]-of-defun-function were non-nil, which was even worse.

> I'd *much* rather that C-mode's BOD gets it wrong every blue moon,
> rather than the current "let's parse the whole damn thing".
> On my 800MHz machine, C-mode is often borderline unusable nowadays.
> And it's not the fault of end-of-defun.

There are one or two "it's too slow" bugs I'm desperately trying to fix.
A dearth of braces makes CC Mode very slow, right now.  I'm aware that
some things are horribly slow.  I'm doing my best to fix them, but
there's only one of me at the moment.

> > It's changed from "move to next end of function" to "move to the end of
> > the function at whose beginning we now are",

> Right.  As you may notice, the second is a subset of the first (with
> a few caveats for nested functions, of course, but that shouldn't matter
> for C-mode), so if your implementation works for the first, it should
> work for the second as well.  It's called backward compatibility.

c-end-of-defun's functionality "needs" to stay the same to support other
Emacsen.  For Emacs-23's EOD-function, It would barely need to be more
than
     (search-forward-regexp "{")
     (backward-char)
     (foward-sexp)

, except "of course", for having to move over any dangling bits at the
end like the "bar ;" of "struct BAR { .... } bar ;".  You've just got to
love C, haven't you?  ;-(

> > and its default value is `forward-sexp'.  `c-end-of-defun' was a good
> > fit for the variable as it formerly was, but is now
> > severely suboptimal.

> At most by a factor of 2.  I.e. if it's slow now, it sure wasn't
> zippy before.

By a factor a great deal more than 2, highly dependent on ARG.

> >> Not sure about restoring the previous semantics.  But I could agree to
> >> the additional ARG argument, which could even let it "take over" (so
> >> beginning-of-defun-raw is not called in that case).

> > :-)  Let's do it!

> I knew you'd like it.

:-)

> >> > 3/- end-of-defun should be restructured along the lines of
> >> > beginning-of-defun.
> >> I don't think that's a good idea.  The main reason is to deal with
> >> languages that allow nested functions.

> > Don't follow - In the upcoming CC Mode 5.32 code (in the CVS
> > repository at SourceForge), C-M-[ae] works just fine for C++ defuns
> > nested inside classes/namespaces and so on.  The mechanism is
> > entirely within CC Mode.

> Yes, but I maintain Emacs, not CC-mode, so I care to move the
> functionality to the generic code so that all modes (not just CC-modes)
> benefit.

OK.  I think I can see what you're saying, now.  Were you thinking of any
(non CC Mode) languages in particular?  Did this ever get discussed on
emacs-devel?  I might well have been asleep at the time.

I've a suspicion that changing the meaning of EOD-function, even if it's
only "restricting" how it's being called, is essentially a loss of
backward compatibility, because any such function needs to be optimised
differently.

Good night for now!

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Stefan Monnier]

> Visit src/buffer.c, goto L313 which contains the closing brace of DEFUN
> Fget_file_buffer, put point AFTER the "}".  Or put point on the next line
> (L314).  In either case C-M-e leaves point at BOL315.  

Oh, so we just disagree about what is the correct behavior.  As far as
I'm concerned, if C-M-e gets you to BOL315, that means that the end of
the function is BOL315, which implies that EOL313 is inside the
function, so it's correct for C-M-e to just move to BOL315.

> If one were to define a variable end-of-defun-RAW-function, which is
> defined to leave point just after the last bit of the function,

That's what end-of-defun-function should do, indeed.

> perhaps that would work, but it would break existing 3rd party code.

Like which code?

>> Unless by "the problem" you're talking about the performance problem, in
>> which case I understand that each time we call BOD (except for the first
>> call), we know that we're "outside" of a defun (unless there's nesting,
>> of course) but we don't tell that to BOD which may have to redo the work
>> of figuring it out.

> WILL have to do the work.

No: in most cases, beginning-of-defun-raw just uses a simple regexp and
doesn't try to determine its position in the syntax tree before doing
its job.  This is pretty much how C-mode's beginning-of-defun worked in
Emacs-21 and incidentally it worked (for the cases I care about) much
better than the current code.

> Now, I wonder, who could that have been?  ;-)  Oh, how I'd love just to
> get rid of these stupid K&R declarations, which NOBODY uses at all
> nowadays, except on >20 year old code bases.  ;-)  Why don't we flush
> them out of Emacs C source?  (No, you don't have to answer.)

Feel free to remove support for those things in C-mode.  If that can
solve the performance issues, I'm all for it.  After all, Emacs-21
didn't get it right and I haven't heard significant complains about it.
It's not like Emacs is always expected to understand every little bit of
a language's syntax anyway.

>> > It's changed from "move to next end of function" to "move to the end of
>> > the function at whose beginning we now are",

>> Right.  As you may notice, the second is a subset of the first (with
>> a few caveats for nested functions, of course, but that shouldn't matter
>> for C-mode), so if your implementation works for the first, it should
>> work for the second as well.  It's called backward compatibility.

> c-end-of-defun's functionality "needs" to stay the same to support other
> Emacsen.  For Emacs-23's EOD-function, It would barely need to be more
> than
>      (search-forward-regexp "{")
>      (backward-char)
>      (foward-sexp)

Yes, simplicity of implementation of EOD-function was indeed an
important motivation in my code changes.

>> > and its default value is `forward-sexp'.  `c-end-of-defun' was a good
>> > fit for the variable as it formerly was, but is now
>> > severely suboptimal.

>> At most by a factor of 2.  I.e. if it's slow now, it sure wasn't
>> zippy before.

> By a factor a great deal more than 2, highly dependent on ARG.

The only case I care about for now is when ARG=1.  As long as this case
is slow, there's no point complaining that end-of-defun makes it too
slow for ARG=106.

>> >> Not sure about restoring the previous semantics.  But I could agree to
>> >> the additional ARG argument, which could even let it "take over" (so
>> >> beginning-of-defun-raw is not called in that case).
>> > :-)  Let's do it!
>> I knew you'd like it.
> :-)

I'm still wondering whether we couldn't just turn end-of-defun into
a simple variant of

   (BOD-raw N)
   (funcall EOD-function)

> OK.  I think I can see what you're saying, now.  Were you thinking of any
> (non CC Mode) languages in particular?  Did this ever get discussed on
> emacs-devel?  I might well have been asleep at the time.

No, I don't remember discussing it much.  It did come out of
a discussion where it became apparent that end-of-defun was broken
(especially when called with negative args).


        Stefan

Re: end-of-defun is fubsr.

[Alan Mackenzie]

'ten Tag, Andreas!

On Tue, Feb 03, 2009 at 06:18:14PM +0100, Andreas Roehler wrote:

> I have remarked buggy/risky implementation already here

> http://www.archivum.info/emacs-devel@gnu.org/2007-11/msg01940.html

Thanks, I hadn't seen that thread.  Quoting from it, Stefan said in
Message-ID: <jwvprxwajza.fsf-monnier+emacs@gnu.org>, Date: Mon, 26 Nov
2007 14:26:59 -0500:

    Rather than argue about the change itself, just give us some use
    case where the new behavior is problematic.

We now have at least two problematic behaviours: it enforces
inefficiencies on some major modes, e.g. CC Mode.  It is difficult to
code bug-free (bug #2106, looping in Python mode).

I agree with Andreas here, this change should be reworked.

> Andreas Röhler

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Alan Mackenzie]

Hi, Stefan!

On Tue, Feb 03, 2009 at 09:21:02PM -0500, Stefan Monnier wrote:
> > Visit src/buffer.c, goto L313 which contains the closing brace of
> > DEFUN Fget_file_buffer, put point AFTER the "}".  Or put point on the
> > next line (L314).  In either case C-M-e leaves point at BOL315.  

> Oh, so we just disagree about what is the correct behavior.  As far as
> I'm concerned, if C-M-e gets you to BOL315, that means that the end of
> the function is BOL315, which implies that EOL313 is inside the
> function, so it's correct for C-M-e to just move to BOL315.

WADR, this is sophistry.  Correct behaviour is to go to the end of the
defun.  In C, this is the closing }.  It CAN'T be anywhere else.  Take a
straw poll of 100 C hackers and ask them.  It's the place where a
compiler would give an error message if you chop off the buffer before
it.

The doc string for `end-of-defun' is crystal clear here: "Move forward to
next end of defun".

> > If one were to define a variable end-of-defun-RAW-function, which is
> > defined to leave point just after the last bit of the function,

> That's what end-of-defun-function should do, indeed.

I would agree.  I detest sloppy definitions, but EOD-function has been
sloppily defined for years.  I think we should accept that EOD-functions
in existing modes DON'T all go to this ideal point, but themselves
correct for WS.

> > perhaps that would work, but it would break existing 3rd party code.

> Like which code?

Well, in my first page of 50 in a google search for
"end-of-defun-function", I came up with these, all of which set
end-of-defun-function:
cfengine.el
espresso.el ; a Javascript mode.
lua-mode.el
js2-sl-mode.el ; One of Lennart Borgman's files
php-mode.el
Slime mode ; probably slime-mode.el

There are certainly several more.  There are likely many more.  Some of
them will do WS jiggling at the end.  I don't think we should make these
end-of-defun-functions broken.

> >> Unless by "the problem" you're talking about the performance problem, in
> >> which case I understand that each time we call BOD (except for the first
> >> call), we know that we're "outside" of a defun (unless there's nesting,
> >> of course) but we don't tell that to BOD which may have to redo the work
> >> of figuring it out.

> > WILL have to do the work.

> No: in most cases, beginning-of-defun-raw just uses a simple regexp and
> doesn't try to determine its position in the syntax tree before doing
> its job.  This is pretty much how C-mode's beginning-of-defun worked in
> Emacs-21 and incidentally it worked (for the cases I care about) much
> better than the current code.

I don't know enough major modes to talk about most, for whatever value of
"most" you have in mind.  I hold it to be wrong to assume that any
BOD-function or EOD-function is implemented "simply", whether or not with
a regexp.

C Mode's beginning-of-defun in Emacs 21 was broken, but that's really a
different matter.  I don't claim that C Mode's C-M-e is optimal.  It is
however reasonably good and it's fast enough for nearly all C files in
Emacs 22.  Any place where it's not fast enough will find its C-M-e's
speed halved or quartered in Emacs 23, and C-u n C-M-e will be slowed
down to loss-of-temper speed.

> > Now, I wonder, who could that have been?  ;-)  Oh, how I'd love just
> > to get rid of these stupid K&R declarations, which NOBODY uses at all
> > nowadays, except on >20 year old code bases.  ;-)  Why don't we flush
> > them out of Emacs C source?  (No, you don't have to answer.)

> Feel free to remove support for those things in C-mode.

A user option c-recognize-k&r would be the way to go, I suppose.  It
would be trivially easy to implement and would improve the performance of
C Mode.  I was thinking more of purging the K&R declarations from the
Emacs source code.  Perhaps for Emacs 24?  This surely would be a forward
step now, even if it wouldn't have been in 1990.

> If that can solve the performance issues, I'm all for it.  After all,
> Emacs-21 didn't get it right and I haven't heard significant complains
> about it.  It's not like Emacs is always expected to understand every
> little bit of a language's syntax anyway.

Well, we could get philosophical about this.  I think that every little
glitch which "isn't quite right" detracts from quality, and if there are
enough such glitches, the user's impression will be of general low
quality.

> >> > It's changed from "move to next end of function" to "move to the
> >> > end of the function at whose beginning we now are",

> >> Right.  As you may notice, the second is a subset of the first (with
> >> a few caveats for nested functions, of course, but that shouldn't matter
> >> for C-mode), so if your implementation works for the first, it should
> >> work for the second as well.  It's called backward compatibility.

This change has caused grief and confusion already, and will cause more
if it gets out to users.  I don't think there is backward compatibility.
In the near future, some EOD-functions will only work from a BOD, some
will work from anywhere, and this will break any 3rd party code which
calls end-of-defun-function (assuming old semantics) for other purposes.

Let me repeat that:  THIS CHANGE WILL CAUSE EXISTING CODE TO BREAK WITH
FUTURE IMPLEMENTATIONS OF end-of-defun-function.

> > c-end-of-defun's functionality "needs" to stay the same to support
> > other Emacsen.  For Emacs-23's EOD-function, It would barely need to
> > be more than
> >      (search-forward-regexp "{")
> >      (backward-char)
> >      (foward-sexp)

> Yes, simplicity of implementation of EOD-function was indeed an
> important motivation in my code changes.

Is there any win with this?  Any hacker writing an EOD-function can get
this simplicity himself, if it's appropriate.

> >> > and its default value is `forward-sexp'.  `c-end-of-defun' was a
> >> > good fit for the variable as it formerly was, but is now severely
> >> > suboptimal.

> >> At most by a factor of 2.  I.e. if it's slow now, it sure wasn't
> >> zippy before.

> > By a factor a great deal more than 2, highly dependent on ARG.

> The only case I care about for now is when ARG=1.  As long as this case
> is slow, there's no point complaining that end-of-defun makes it too
> slow for ARG=106.

It isn't slow for ARG=1, except for some ill-conditioned files (perhaps
src/lisp.h).  In these cases, even a factor of 2 is worth the trouble.
I'm currently working on one of the big slowness problems.  Please
identify any problematic files you know of whose slowness isn't caused by
a paucity of braces.

Somebody other than me will care about ARG > 1.  She'll care a lot when
Emacs 23 hangs for 5, 10, 20, 50 seconds.  People who will use C-M-e to
move large distances over files, for example.  In Emacs-22, this is fast
- the speed is essentially independent of ARG.  Why should we slap them
in the face when we release Emacs-23?

> >> >> Not sure about restoring the previous semantics.  But I could
> >> >> agree to the additional ARG argument, which could even let it
> >> >> "take over" (so beginning-of-defun-raw is not called in that
> >> >> case).
> >> > :-)  Let's do it!
> >> I knew you'd like it.
> > :-)

> I'm still wondering whether we couldn't just turn end-of-defun into
> a simple variant of

>    (BOD-raw N)
>    (funcall EOD-function)

It won't work when point is outside a defun.  And a combination of (BOD)
and (EOD) isn't capable of reliably determining if point is outside a
defun.  This was the main point of my last post, which you didn't
address.  To use this approach, a new hook would be needed, say
`inside-defun-function'.  Then end-of-defun would become:

    (when (funcall inside-defun-function)
          (setq N (1+ N)))
    (BOD-raw (- N))
    (funcall EOD-function)

Well, something like that.  It would be surely be worse than simply
(funcall end-of-defun-function N).

This approach would probably be making hidden assumptions about nested
defuns, or other wierdly formulated defuns where BODs and EODs don't
necessarily alternate.  Also any 3rd party code which uses EOD-function
will be assuming that EOD-function works with point anywhere.  If we're
going to go this road, the (simplified current) EOD-function should get a
different name; perhaps `forward-defun-function'?  This would underline
its unlikeness to beginning-of-defun-function.

I honestly don't see the advantage to this.  Why don't we just assume
that if begininng-of-defun-function and/or end-of-defun-function have
been defined (with the previous semantics + &optional arg) they will do
their job properly?

> > OK.  I think I can see what you're saying, now.  Were you thinking of any
> > (non CC Mode) languages in particular?  Did this ever get discussed on
> > emacs-devel?  I might well have been asleep at the time.

> No, I don't remember discussing it much.  It did come out of
> a discussion where it became apparent that end-of-defun was broken
> (especially when called with negative args).

OK.  Maybe the elisp manual should emphasise the possibility of
negativeness.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Stefan Monnier]

>> > Visit src/buffer.c, goto L313 which contains the closing brace of
>> > DEFUN Fget_file_buffer, put point AFTER the "}".  Or put point on the
>> > next line (L314).  In either case C-M-e leaves point at BOL315.  

>> Oh, so we just disagree about what is the correct behavior.  As far as
>> I'm concerned, if C-M-e gets you to BOL315, that means that the end of
>> the function is BOL315, which implies that EOL313 is inside the
>> function, so it's correct for C-M-e to just move to BOL315.

> WADR, this is sophistry.  Correct behaviour is to go to the end of the
> defun.  In C, this is the closing }.  It CAN'T be anywhere else.  Take a
> straw poll of 100 C hackers and ask them.  It's the place where a
> compiler would give an error message if you chop off the buffer before
> it.
> The doc string for `end-of-defun' is crystal clear here: "Move forward to
> next end of defun".

There are 2 issues:
- Should M-C-e from within that function jump to BOL315 or to EOL313?
- given that it jumps to BOL315, should C-M-e from EOL313 also jump to
  BOL315 or to the end of the next function?

My reply only concerned the second question, whereas your above argument
seems to apply to the first.

>> > If one were to define a variable end-of-defun-RAW-function, which is
>> > defined to leave point just after the last bit of the function,
>> That's what end-of-defun-function should do, indeed.
> I would agree.  I detest sloppy definitions, but EOD-function has been
> sloppily defined for years.  I think we should accept that EOD-functions
> in existing modes DON'T all go to this ideal point, but themselves
> correct for WS.
>> > perhaps that would work, but it would break existing 3rd party code.
>> Like which code?
> Well, in my first page of 50 in a google search for
> "end-of-defun-function", I came up with these, all of which set
> end-of-defun-function:

I know such functions exist, but which ones of these are (would be) broken?

> A user option c-recognize-k&r would be the way to go, I suppose.  It
> would be trivially easy to implement and would improve the performance of
> C Mode.  I was thinking more of purging the K&R declarations from the
> Emacs source code.  Perhaps for Emacs 24?  This surely would be a forward
> step now, even if it wouldn't have been in 1990.

No need to purge them before removing C-mode support for them.
AFAIK they were not supported by Emacs-21's C-mode either (at least in
the sense that BOD/EOD always did the right thing for them).

>> If that can solve the performance issues, I'm all for it.  After all,
>> Emacs-21 didn't get it right and I haven't heard significant complains
>> about it.  It's not like Emacs is always expected to understand every
>> little bit of a language's syntax anyway.
> Well, we could get philosophical about this.  I think that every little
> glitch which "isn't quite right" detracts from quality, and if there are
> enough such glitches, the user's impression will be of general low
> quality.

If you want to go there, you need to use the corresponding tools:
e.g. a real parser.

> Let me repeat that:  THIS CHANGE WILL CAUSE EXISTING CODE TO BREAK WITH
> FUTURE IMPLEMENTATIONS OF end-of-defun-function.

Potential breakage comes with *any* change.  So I'm not interested
in it.  Show me actual breakage.

> It isn't slow for ARG=1, except for some ill-conditioned files (perhaps
> src/lisp.h).

regex.c is pretty painful as well.

> Somebody other than me will care about ARG > 1.  She'll care a lot when
> Emacs 23 hangs for 5, 10, 20, 50 seconds.  People who will use C-M-e to
> move large distances over files, for example.  In Emacs-22, this is fast
> - the speed is essentially independent of ARG.  Why should we slap them
> in the face when we release Emacs-23?

Check Emacs-22's code again: it also calls EOD-function ARG times.
So we're still talking just a constant factor.  The fact that you don't
know that indicates that it's clearly not a big issue.

Mind you, I do agree that it might be good to solve this "ARG times
slow".  I just don't think it's high on the list of things to fix.

> It won't work when point is outside a defun.  And a combination of (BOD)
> and (EOD) isn't capable of reliably determining if point is outside a
> defun.

Why wouldn't it work to compare the starting position with the position
after BOD+EOD?


        Stefan

Re: end-of-defun is fubsr.

[Andreas Roehler]

Alan Mackenzie wrote:
> 'ten Tag, Andreas!
> 
> On Tue, Feb 03, 2009 at 06:18:14PM +0100, Andreas Roehler wrote:
> 
>> I have remarked buggy/risky implementation already here
> 
>> http://www.archivum.info/emacs-devel@gnu.org/2007-11/msg01940.html
> 
> Thanks, I hadn't seen that thread.  Quoting from it, Stefan said in
> Message-ID: <jwvprxwajza.fsf-monnier+emacs@gnu.org>, Date: Mon, 26 Nov
> 2007 14:26:59 -0500:
> 
>     Rather than argue about the change itself, just give us some use
>     case where the new behavior is problematic.
> 
> We now have at least two problematic behaviours: it enforces
> inefficiencies on some major modes, e.g. CC Mode.  It is difficult to
> code bug-free (bug #2106, looping in Python mode).
> 
> I agree with Andreas here, this change should be reworked.
> 
>> Andreas Röhler
> 

Hi Stefan,

AFAIR point here is not to write new stuff, rather to
weed out useless girlands.

Don't felt my contribution welcome in 2007.  If you
wont me to strip it down to function now, please tell.

(Deleting some outgrown should not raise the
copyright/assignment question either.)


Andreas Röhler

Re: end-of-defun is fubsr.

[Alan Mackenzie]

Hi, Stefan,

On Wed, Feb 04, 2009 at 09:29:20AM -0500, Stefan Monnier wrote:
> >> > Visit src/buffer.c, goto L313 which contains the closing brace of
> >> > DEFUN Fget_file_buffer, put point AFTER the "}".  Or put point on the
> >> > next line (L314).  In either case C-M-e leaves point at BOL315.  

> >> Oh, so we just disagree about what is the correct behavior.  As far
> >> as I'm concerned, if C-M-e gets you to BOL315, that means that the
> >> end of the function is BOL315, which implies that EOL313 is inside
> >> the function, so it's correct for C-M-e to just move to BOL315.

> > WADR, this is sophistry.  Correct behaviour is to go to the end of the
> > defun.  In C, this is the closing }.  It CAN'T be anywhere else.  Take a
> > straw poll of 100 C hackers and ask them.  It's the place where a
> > compiler would give an error message if you chop off the buffer before
> > it.
> > The doc string for `end-of-defun' is crystal clear here: "Move forward to
> > next end of defun".

> There are 2 issues:
> - Should M-C-e from within that function jump to BOL315 or to EOL313?

Either EOL313 or BOL314 (was your "315" a typo?).  It would be nice to be
definite here, but history necessitates vagueness.

> - given that it jumps to BOL315, should C-M-e from EOL313 also jump to
>   BOL315 or to the end of the next function?

EOL313 is AT the end of the defun.  C-M-e move to the NEXT EOD, therefore
it MUST jump to L331/332.  From a pragmatic point of view, what hacker
wouldn't be aggravated by C-M-e taking him from EOL313 to BOL314?

> My reply only concerned the second question, whereas your above argument
> seems to apply to the first.

> >> > If one were to define a variable end-of-defun-RAW-function, which is
> >> > defined to leave point just after the last bit of the function,
> >> That's what end-of-defun-function should do, indeed.
> > I would agree.  I detest sloppy definitions, but EOD-function has been
> > sloppily defined for years.  I think we should accept that EOD-functions
> > in existing modes DON'T all go to this ideal point, but themselves
> > correct for WS.
> >> > perhaps that would work, but it would break existing 3rd party code.
> >> Like which code?
> > Well, in my first page of 50 in a google search for
> > "end-of-defun-function", I came up with these, all of which set
> > end-of-defun-function:

> I know such functions exist, but which ones of these are (would be) broken?

Haven't a clue, and can't be bothered to investigate.  It's a fair
assumption that several or many would be broken.  I think the onus is on
the person advocating the change to show that things are OK, rather than
for opponents to show that things will definitely break.

> > A user option c-recognize-k&r would be the way to go, I suppose.  It
> > would be trivially easy to implement and would improve the
> > performance of C Mode.  I was thinking more of purging the K&R
> > declarations from the Emacs source code.  Perhaps for Emacs 24?  This
> > surely would be a forward step now, even if it wouldn't have been in
> > 1990.

> No need to purge them before removing C-mode support for them.
> AFAIK they were not supported by Emacs-21's C-mode either (at least in
> the sense that BOD/EOD always did the right thing for them).

> >> If that can solve the performance issues, I'm all for it.  After all,
> >> Emacs-21 didn't get it right and I haven't heard significant complains
> >> about it.  It's not like Emacs is always expected to understand every
> >> little bit of a language's syntax anyway.
> > Well, we could get philosophical about this.  I think that every little
> > glitch which "isn't quite right" detracts from quality, and if there are
> > enough such glitches, the user's impression will be of general low
> > quality.

> If you want to go there, you need to use the corresponding tools:
> e.g. a real parser.

I do want to go there, and without a real parser.  Therefore I can only
approach closely.  

> > Let me repeat that:  THIS CHANGE WILL CAUSE EXISTING CODE TO BREAK
> > WITH FUTURE IMPLEMENTATIONS OF end-of-defun-function.

> Potential breakage comes with *any* change.  So I'm not interested
> in it.  Show me actual breakage.

That's sophistry again, Stefan.  It's true that any change risks
breakage.  But with most changes we make, we take care to minimise the
risk of breakage.  This particular change positively provokes it.  And
fixing any such breakage will be boring tedious drudgery, regardless of
whether one of us, or J. Random Hacker has to do it.  

> > It isn't slow for ARG=1, except for some ill-conditioned files (perhaps
> > src/lisp.h).

> regex.c is pretty painful as well.

OK.  I'll look at that.

> > Somebody other than me will care about ARG > 1.  She'll care a lot
> > when Emacs 23 hangs for 5, 10, 20, 50 seconds.  People who will use
> > C-M-e to move large distances over files, for example.  In Emacs-22,
> > this is fast - the speed is essentially independent of ARG.  Why
> > should we slap them in the face when we release Emacs-23?

> Check Emacs-22's code again: it also calls EOD-function ARG times.
> So we're still talking just a constant factor.  The fact that you don't
> know that indicates that it's clearly not a big issue.

Sorry, you're wrong, at least as far as CC Mode is concerned. ;-)  In
cc-mode.el appears this:

  ;; (April 2006): RMS has now approved these commands as defaults.
  (unless (memq 'argumentative-bod-function c-emacs-features)
    (define-key c-mode-base-map "\e\C-a"    'c-beginning-of-defun)
    (define-key c-mode-base-map "\e\C-e"    'c-end-of-defun))

In Emacs-22, C-M-[ae] were bound directly to the CC Mode functions.  In
Emacs-23, the unless clause prevents this.  Would you object if I
restored these CC Mode bindings in Emacs 23?  That's not a rhetorical
question.  Then we'd both get what we want, more or less.  Except that
this matter would stay unresolved, and raise its ugly head yet again some
time in the future.  I'm thoroughly sick of it, and I expect you are too.

> Mind you, I do agree that it might be good to solve this "ARG times
> slow".  I just don't think it's high on the list of things to fix.

I regard it as already fixed.  I fixed it, at the cost of a vast amount
of refactoring and many, many evenings' hacking in 2006.  I protest most
strongly at losing the fruits of that work, even if only for the EOD
case.  Should I just restore the C-M-[ae] bindings in c-mode-base-map?

> > It won't work when point is outside a defun.  And a combination of
> > (BOD) and (EOD) isn't capable of reliably determining if point is
> > outside a defun.

> Why wouldn't it work to compare the starting position with the position
> after BOD+EOD?

Because the sloppiness of the general EOD-function will have left point
at some random position in the WS between defun-0 and defun-1.  If you
start somewhere in that WS, BOD+EOD will also leave you somewhere in that
WS, so comparing those positions gives at best heuristic information.

To get firm information, you'd have to start doing things like
c-backward-syntactic-ws and see if you end up before where EOD put you.
You can't rely on checking for a paren/brace/bracket, because not all
languages use them (shell script, haskell, ?pascal for example).

Even (BOD) (BOD -1) isn't necessarily any good, since
beginning-of-defun-function might well be implemented to go to the
textually nearest defun regardless of nesting level.  Martin Stjernholm
suggested this algorithm for CC Mode a while back - it's a good
suggestion.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Andreas Roehler]

Alan Mackenzie wrote:
> Hi, Stefan,
> 
> On Wed, Feb 04, 2009 at 09:29:20AM -0500, Stefan Monnier wrote:
>>>>> Visit src/buffer.c, goto L313 which contains the closing brace of
>>>>> DEFUN Fget_file_buffer, put point AFTER the "}".  Or put point on the
>>>>> next line (L314).  In either case C-M-e leaves point at BOL315.  
> 
>>>> Oh, so we just disagree about what is the correct behavior.  As far
>>>> as I'm concerned, if C-M-e gets you to BOL315, that means that the
>>>> end of the function is BOL315, which implies that EOL313 is inside
>>>> the function, so it's correct for C-M-e to just move to BOL315.
> 
>>> WADR, this is sophistry.  Correct behaviour is to go to the end of the
>>> defun.  In C, this is the closing }.  It CAN'T be anywhere else.  Take a
>>> straw poll of 100 C hackers and ask them.  It's the place where a
>>> compiler would give an error message if you chop off the buffer before
>>> it.
>>> The doc string for `end-of-defun' is crystal clear here: "Move forward to
>>> next end of defun".
> 
>> There are 2 issues:
>> - Should M-C-e from within that function jump to BOL315 or to EOL313?
> 
> Either EOL313 or BOL314 (was your "315" a typo?).  It would be nice to be
> definite here, but history necessitates vagueness.
> 
>> - given that it jumps to BOL315, should C-M-e from EOL313 also jump to
>>   BOL315 or to the end of the next function?
> 
> EOL313 is AT the end of the defun.  C-M-e move to the NEXT EOD, therefore
> it MUST jump to L331/332.  From a pragmatic point of view, what hacker
> wouldn't be aggravated by C-M-e taking him from EOL313 to BOL314?
> 
>> My reply only concerned the second question, whereas your above argument
>> seems to apply to the first.
> 
>>>>> If one were to define a variable end-of-defun-RAW-function, which is
>>>>> defined to leave point just after the last bit of the function,
>>>> That's what end-of-defun-function should do, indeed.
>>> I would agree.  I detest sloppy definitions, but EOD-function has been
>>> sloppily defined for years.  I think we should accept that EOD-functions
>>> in existing modes DON'T all go to this ideal point, but themselves
>>> correct for WS.
>>>>> perhaps that would work, but it would break existing 3rd party code.
>>>> Like which code?
>>> Well, in my first page of 50 in a google search for
>>> "end-of-defun-function", I came up with these, all of which set
>>> end-of-defun-function:
> 
>> I know such functions exist, but which ones of these are (would be) broken?
> 
> Haven't a clue, and can't be bothered to investigate.  It's a fair
> assumption that several or many would be broken.  I think the onus is on
> the person advocating the change to show that things are OK, rather than
> for opponents to show that things will definitely break.
> 
>>> A user option c-recognize-k&r would be the way to go, I suppose.  It
>>> would be trivially easy to implement and would improve the
>>> performance of C Mode.  I was thinking more of purging the K&R
>>> declarations from the Emacs source code.  Perhaps for Emacs 24?  This
>>> surely would be a forward step now, even if it wouldn't have been in
>>> 1990.
> 
>> No need to purge them before removing C-mode support for them.
>> AFAIK they were not supported by Emacs-21's C-mode either (at least in
>> the sense that BOD/EOD always did the right thing for them).
> 
>>>> If that can solve the performance issues, I'm all for it.  After all,
>>>> Emacs-21 didn't get it right and I haven't heard significant complains
>>>> about it.  It's not like Emacs is always expected to understand every
>>>> little bit of a language's syntax anyway.
>>> Well, we could get philosophical about this.  I think that every little
>>> glitch which "isn't quite right" detracts from quality, and if there are
>>> enough such glitches, the user's impression will be of general low
>>> quality.
> 
>> If you want to go there, you need to use the corresponding tools:
>> e.g. a real parser.
> 
> I do want to go there, and without a real parser.  Therefore I can only
> approach closely.  
> 
>>> Let me repeat that:  THIS CHANGE WILL CAUSE EXISTING CODE TO BREAK
>>> WITH FUTURE IMPLEMENTATIONS OF end-of-defun-function.
> 
>> Potential breakage comes with *any* change.  So I'm not interested
>> in it.  Show me actual breakage.
> 
> That's sophistry again, Stefan.  It's true that any change risks
> breakage.  But with most changes we make, we take care to minimise the
> risk of breakage.  This particular change positively provokes it.  And
> fixing any such breakage will be boring tedious drudgery, regardless of
> whether one of us, or J. Random Hacker has to do it.  
> 
>>> It isn't slow for ARG=1, except for some ill-conditioned files (perhaps
>>> src/lisp.h).
> 
>> regex.c is pretty painful as well.
> 
> OK.  I'll look at that.
> 
>>> Somebody other than me will care about ARG > 1.  She'll care a lot
>>> when Emacs 23 hangs for 5, 10, 20, 50 seconds.  People who will use
>>> C-M-e to move large distances over files, for example.  In Emacs-22,
>>> this is fast - the speed is essentially independent of ARG.  Why
>>> should we slap them in the face when we release Emacs-23?
> 
>> Check Emacs-22's code again: it also calls EOD-function ARG times.
>> So we're still talking just a constant factor.  The fact that you don't
>> know that indicates that it's clearly not a big issue.
> 
> Sorry, you're wrong, at least as far as CC Mode is concerned. ;-)  In
> cc-mode.el appears this:
> 
>   ;; (April 2006): RMS has now approved these commands as defaults.
>   (unless (memq 'argumentative-bod-function c-emacs-features)
>     (define-key c-mode-base-map "\e\C-a"    'c-beginning-of-defun)
>     (define-key c-mode-base-map "\e\C-e"    'c-end-of-defun))
> 
> In Emacs-22, C-M-[ae] were bound directly to the CC Mode functions.  In
> Emacs-23, the unless clause prevents this.  Would you object if I
> restored these CC Mode bindings in Emacs 23?  That's not a rhetorical
> question.  Then we'd both get what we want, more or less.  Except that
> this matter would stay unresolved, and raise its ugly head yet again some
> time in the future.  I'm thoroughly sick of it, and I expect you are too.
> 
>> Mind you, I do agree that it might be good to solve this "ARG times
>> slow".  I just don't think it's high on the list of things to fix.
> 
> I regard it as already fixed.  I fixed it, at the cost of a vast amount
> of refactoring and many, many evenings' hacking in 2006.  I protest most
> strongly at losing the fruits of that work, even if only for the EOD
> case.  Should I just restore the C-M-[ae] bindings in c-mode-base-map?
> 
>>> It won't work when point is outside a defun.  And a combination of
>>> (BOD) and (EOD) isn't capable of reliably determining if point is
>>> outside a defun.
> 
>> Why wouldn't it work to compare the starting position with the position
>> after BOD+EOD?
> 
> Because the sloppiness of the general EOD-function will have left point
> at some random position in the WS between defun-0 and defun-1.  If you
> start somewhere in that WS, BOD+EOD will also leave you somewhere in that
> WS, so comparing those positions gives at best heuristic information.
> 
> To get firm information, you'd have to start doing things like
> c-backward-syntactic-ws and see if you end up before where EOD put you.
> You can't rely on checking for a paren/brace/bracket, because not all
> languages use them (shell script, haskell, ?pascal for example).
> 
> Even (BOD) (BOD -1) isn't necessarily any good, since
> beginning-of-defun-function might well be implemented to go to the
> textually nearest defun regardless of nesting level.  Martin Stjernholm
> suggested this algorithm for CC Mode a while back - it's a good
> suggestion.
> 
>>         Stefan
> 

Hi Alan,

seeing you struggle with well known and long-time
disputed issues, lets take a IMO related source of bugs
and confusion into consideration:

`forward-word' and a couple of similar functions don't
stop at the last char of THING, but one char behind.

Surely there are reasons for it. However, from the
perspective of editing, that seems not comfortable.

After a `forward' you are outside already. So what
should `word-at-point' for example return then?
Logically nil.

Now the fixing may start saying: well, let's consider
just one char behind as inside. Although, with cursor
at single whitespace between two words, where remains
justice, if the word before is taken, but the word
after not?

People must remember illogical behaviour at these
occasions, workarounds with bug-risks are the result.

That's the point I started out to write
`thing-at-point-utils.el'. Fixing this opened
the path to a whole range of useful functions.

For editing C-modes all the delimiting functions as
`braced-atpt' should be of interest.

You get it at

https://code.launchpad.net/s-x-emacs-werkstatt/


Thanks all

Andreas Röhler

Problems C Mode has with src/regex.c [was: end-of-defun is fubsr.]

[Alan Mackenzie]

Hi, Stefan!

On Wed, Feb 04, 2009 at 09:29:20AM -0500, Stefan Monnier wrote:

> regex.c is pretty painful as well.

I've had a wee look at this file.

C-M-a doesn't work at all from BOD re_comp.  There's some fairly
unit-testy twisted #if thingies around there.  I've submitted a bug
report for this.

Is there anything else troublesome in this file, unconnected with this
bug?

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun acts weirdly in c-mode

[Alan Mackenzie]

Hi, Miles and Stefan!

On Tue, Feb 03, 2009 at 04:09:42PM +0000, Alan Mackenzie wrote:

[ .... ]

> Alternatively, as a quick kludge, C-M-[ae] should be bound again to
> c-\(beginning\|end\)-of-defun so as to make them work properly again.

I've put more rigour into a test in cc-defs.el, which has the above
effect.  This should at least fix the symptoms of the various bugs.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun acts weirdly in c-mode

[Glenn Morris]

Alan Mackenzie wrote:

> I've put more rigour into a test in cc-defs.el, which has the above
> effect.  This should at least fix the symptoms of the various bugs.

cc-defs.el:1453:22:Warning: reference to free variable `bod-param'
cc-defs.el:1454:23:Warning: assignment to free variable `bod-param'
cc-defs.el:1459:23:Warning: reference to free variable `eod-param'
cc-defs.el:1460:25:Warning: assignment to free variable `eod-param'

Re: end-of-defun acts weirdly in c-mode

[Alan Mackenzie]

Hi, Glenn!

On Thu, Feb 05, 2009 at 08:03:23PM -0500, Glenn Morris wrote:

> cc-defs.el:1453:22:Warning: reference to free variable `bod-param'
> cc-defs.el:1454:23:Warning: assignment to free variable `bod-param'
> cc-defs.el:1459:23:Warning: reference to free variable `eod-param'
> cc-defs.el:1460:25:Warning: assignment to free variable `eod-param'

Sorry.  I've fixed it now.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Stefan Monnier]

BTW, as you may have noticed I rewrote end-of-defun.  It should fix the
asymptotic complexity issue (it still calls BOD-function typically
twice, tho).


        Stefan

Re: end-of-defun is fubsr.

[Alan Mackenzie]

Hi, Stefan,

On Thu, Feb 12, 2009 at 04:35:04PM -0500, Stefan Monnier wrote:
> BTW, as you may have noticed I rewrote end-of-defun.  It should fix the
> asymptotic complexity issue (it still calls BOD-function typically
> twice, tho).

It still suffers an off-by-1 error.  For example, load a C file and
execute the following to make sure that the keys are appropriately
bound:

    (define-key c-mode-base-map "\C-\M-a" 'beginning-of-defun)
    (define-key c-mode-base-map "\C-\M-e" 'end-of-defun)

Now go to just after the end of a defun:

void (foo)
{
   return ;
}
 ^
 |

Now do C-M-e.  Point dribbles a bit of WS forward.

#########################################################################

The problem here is that the generic end-of-defun-function moves to some
arbitrary point in the WS between defuns.  c-end-of-defun-function is
typical of EOD-functions in this respect.

If point also begins somewhere in the inter-defun WS, after calling
BOD-function and EOD-function, it will land back at an arbitrary place
in the same WS, which may be before, at, or after the starting point.

We cannot, in the general case, deduce whether we started within or
outwith a defun by this mechanism.
 
>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Stefan Monnier]

> If point also begins somewhere in the inter-defun WS, after calling
> BOD-function and EOD-function, it will land back at an arbitrary place
> in the same WS, which may be before, at, or after the starting point.

It's not arbitrary: it's carefully chosen by the implementor of
[EB]OD-function.  If you make your choices arbitrarily, you get what you
ask for, of course ;-)

I.e. "notabug" (or at least "wontfix") for what I can see.

Did you notice any other problem?


        Stefan

Re: end-of-defun is fubsr.

[Alan Mackenzie]

[N.B.  There is no Andreas@muc.de]

Hi, Stefan,

On Fri, Feb 13, 2009 at 09:31:19AM -0500, Stefan Monnier wrote:
> > If point also begins somewhere in the inter-defun WS, after calling
> > BOD-function and EOD-function, it will land back at an arbitrary place
> > in the same WS, which may be before, at, or after the starting point.

> It's not arbitrary: it's carefully chosen by the implementor of
> [EB]OD-function.

An unknown number of implementors of EOD-functions will have chosen
carefully; well, maybe some of them weren't so careful, who knows?  The
point is, they probably haven't all chosen the "same" place in that WS,
for whatever value of "same".  Most of them will expect their
EOD-functions to work the same in Emacs 23 as Emacs n, n < 23.

> If you make your choices arbitrarily, you get what you ask for, of
> course ;-)

No.  The whole point of Miles's bug report was that he DIDN'T get what
he asked for, neither did he get what c-end-of-defun asked for.

> I.e. "notabug" (or at least "wontfix") for what I can see.

> Did you notice any other problem?

Well, the ones I pointed out before:

(i) The meaning of end-of-defun-function has been radically changed, so
that uses and definitions of them pre- and post- Emacs-23 will no longer
be compatible with eachother.

The Emacs 23 handling of this variable isn't compatible with its
docstring in Emacs 22, which states that the EOD-function is used when
the "normal method" is not appropriate.

(ii) The symmetry between beginning/end-of-defun-function has been
broken, which will lead to confusion.

How about renaming `end-of-defun-function' to, say,
`forward-defun-function', thus precluding any possible confusion?

(iii) Infinite recursion will occur if an existing BOD-function with
-ve argument calls EOD-function.  I don't know if this happens.  If so,
I think it would have happend in Emacs 22 too.

(iv) The way things are done introduces the unnecessary restriction that
BODs and EODs alternate strictly in a source file.  Although this is
virtually universally true, it might not be in a language which defined
EOD as a match for "^)", for example, as Elisp could well have done.  It
might also constrain a major mode's handling of nested defuns, though I
haven't got the energy to think this through at the moment.

(v) Most of all, using BOD/EOD-function is so difficult to understand
that it's more work than just binding C-M-[aeh] in a major-mode map,
even if starting from scratch.

Additionally, not all the doc strings are optimal, but that doesn't seem
particularly urgent at the moment; for example, that for
end-of-defun-function says why a function is called, but not what it
should do, nor what its result means, nor where it should leave point.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Stefan Monnier]

>> Did you notice any other problem?
> Well, the ones I pointed out before:

Great, thanks,


        Stefan

Re: end-of-defun is fubsr.

[Alan Mackenzie]

Hi, Stefan,

On Fri, Feb 13, 2009 at 12:06:26PM -0500, Stefan Monnier wrote:
> >> Did you notice any other problem?
> > Well, the ones I pointed out before:

> Great, thanks,

Sorry, here's something a bit more constructive:

#########################################################################
*** lisp.el~	2009-02-12 12:10:53.000000000 +0000
--- lisp.el	2009-02-13 18:49:26.780916880 +0000
***************
*** 314,322 ****
  
  (defvar end-of-defun-function #'forward-sexp
    "Function for `end-of-defun' to call.
! This is used to find the end of the defun.
! It is called with no argument, right after calling `beginning-of-defun-raw'.
! So the function can assume that point is at the beginning of the defun body.")
  
  (defun buffer-end (arg)
    "Return the \"far end\" position of the buffer, in direction ARG.
--- 314,326 ----
  
  (defvar end-of-defun-function #'forward-sexp
    "Function for `end-of-defun' to call.
! This should move point to the next end of defun and return t on success,
! nil on failure.  It should not move further forward into any whitespace or
! onto the next line.
! 
! It is called with no argument.
! The function can assume that point is at the beginning of a defun
! body or before the first defun in the buffer.")
  
  (defun buffer-end (arg)
    "Return the \"far end\" position of the buffer, in direction ARG.
#########################################################################

Another thing: in an elisp buffer, if (point-min) is in the middle of a
defun, C-M-e takes point to the next but one EOD.  CC Mode is just as
bad.  ;-)

Should end-of-defun perhaps widen the buffer around its machinations?

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Stefan Monnier]

> ! This should move point to the next end of defun

I don't think it's quite true.  It should really be "the end of the
defun at point".  There might be a nearer EOD in a nested function.

> ! and return t on success, nil on failure.

I don't think the return result is ever used as of now.  So I'd rather
not document such a thing.

> ! It should not move further forward into any whitespace or
> ! onto the next line.

I used slightly different wording.

> ! It is called with no argument.
> ! The function can assume that point is at the beginning of a defun
> ! body or before the first defun in the buffer.")
  
I prefer to keep the reference to BOD-raw, which makes this starting
position much more precise.

> Another thing: in an elisp buffer, if (point-min) is in the middle of a
> defun, C-M-e takes point to the next but one EOD.  CC Mode is just as
> bad.  ;-)
> Should end-of-defun perhaps widen the buffer around its machinations?

Widening is not a good option, no.  Rather we may want to consider what
should happen if a file starts in the middle of a defun.
I am not going to lose too much sleep over this.


        Stefan

Re: end-of-defun is fubsr.

[Alan Mackenzie]

On Fri, Feb 13, 2009 at 11:22:48PM -0500, Stefan Monnier wrote:
> > ! This should move point to the next end of defun

> I don't think it's quite true.  It should really be "the end of the
> defun at point".  There might be a nearer EOD in a nested function.

Yes, there might be, and this might be the appropriate EOD.  (See
below.)

OK:  Maybe it should be

    This should move point to the \"next\" end of defun

, where "next" is to be interpreted appropriately for the nesting
strategy in use by the major mode.  For example, consider this fragment
of C++:

1   namespace baz
2   {
3       int bar = 666;
4                          // <==== Do C-M-e here.
5       int foo ()
6       {
7           return bar ;
8       }
9
......
42  }
43

Suppose in C++ Mode, the strategy for C-M-[ae] with nested defuns is "go
to the lexically next one, regardless of nesting depth" (which was
Martin Stjernholm's suggestion a few years back, and it's a good one).
Then starting at L4, C-M-e should go to L9.  However, (BOD-raw) will go
to L1, and "end of the defun at point" will be L43.  Whoops!

"Next"'s meaning should be at the discretion of the major mode author.

> > ! and return t on success, nil on failure.

> I don't think the return result is ever used as of now.  So I'd rather
> not document such a thing.

Emacs 22 and 21 end-of-defun return end-of-defun's result as
EOD-function's own return value, even if it wasn't documented.  We
shouldn't change this, surely?

> > ! It should not move further forward into any whitespace or
> > ! onto the next line.

> I used slightly different wording.

I don't see the previous wording specifying this at all.

> > ! It is called with no argument.
> > ! The function can assume that point is at the beginning of a defun
> > ! body or before the first defun in the buffer.")

> I prefer to keep the reference to BOD-raw, which makes this starting
> position much more precise.

Ah, OK.  But it shouldn't give the impression that the function may be
called ONLY after calling BOD-raw, and a bit less brusque than saying
"RTFS!!".

> > Another thing: in an elisp buffer, if (point-min) is in the middle of a
> > defun, C-M-e takes point to the next but one EOD.  CC Mode is just as
> > bad.  ;-)
> > Should end-of-defun perhaps widen the buffer around its machinations?

> Widening is not a good option, no.  Rather we may want to consider what
> should happen if a file starts in the middle of a defun.
> I am not going to lose too much sleep over this.

Nah, maybe not!  Anybody doing C-M-e can just get annoyed, he set the
narrrowing himself, and anybody using EOD in a program ought to widen
the buffer himself before manipulating it.

How about this, then:?

#########################################################################
(defvar end-of-defun-function #'forward-sexp
  "Function for `end-of-defun' to call.
This should move point to the \"next\" end of defun.  It should
not move further forward into any whitespace or onto the next
line.

It is called with no argument.  Its return value will become the
return value of `end-of-defun' when called from it.

The function can assume that point is at a position where
`beginning-of-defun-raw' could have left it, typically at the
beginning of a defun body or before the first defun in the
buffer.")
#########################################################################

>         Stefan

-- 
Alan.

Re: end-of-defun is fubsr.

[Stefan Monnier]

>> I don't think the return result is ever used as of now.  So I'd rather
>> not document such a thing.
> Emacs 22 and 21 end-of-defun return end-of-defun's result as
> EOD-function's own return value, even if it wasn't documented.

It wasn't documented and should stay that way.

> We shouldn't change this, surely?

We have changed this.

> How about this, then:?

I prefer the current text.


        Stefan

Re: end-of-defun is fubsr.

[Alan Mackenzie]

Hi, Stefan,

On Sat, Feb 14, 2009 at 04:16:53PM -0500, Stefan Monnier wrote:
> >> I don't think the return result is ever used as of now.  So I'd rather
> >> not document such a thing.
> > Emacs 22 and 21 end-of-defun return end-of-defun's result as
> > EOD-function's own return value, even if it wasn't documented.

It was documented, sort of, in that the `beginning-of-defun' documents
its own return value, and `end-of-defun' says "see function
`beginning-of-defun'".

The doc-string of `end-of-defun' (very nearly) explicitly states that it
is only for use in languages which have close parentheses at EODs.

> It wasn't documented and should stay that way.

> > We shouldn't change this, surely?

> We have changed this.

Should I put an entry into NEWS for it?

> > How about this, then:?

> I prefer the current text.

Was my patch really that bad?

Look, I'm as sick of all this as you must be.  This whole area of
beginning/end-of-defuns is riven with vagueness of documentation and
inconsistencies between fine manuals and doc strings, between doc strings
and code, between beginning- and end-.

Defun movement commands are _important_ in major modes for programming
languages.

I have solid experience with writing and maintaining
beginning/end-of-defun functions, and the frustration over several years
of trying to link them smoothly into Emacs, something which should be
utterly trivial, has been intense in the extreme.  Basically, I've pretty
much given up trying to hook in the CC Mode defun stuff.  Not good.

I won't say it's unusable to a major mode maintainer, because it clearly
isn't.  I will say that it's a wretched experience - you can't write
B/EOD-functions without reading the fine source code in its finest
detail, which for something which should be so easy, is lacking fun in
the extreme.

How about me collecting together the existing work, rendering it
consistent, logical, backward compatible (in so far as that is possible),
as usable as possible, sprucing up all the pertinent doc strings and
manual pages, and submitting a patch for it to emacs-devel?

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: end-of-defun is fubsr.

[Miles Bader]

Alan Mackenzie <acm@muc.de> writes:
> The doc-string of `end-of-defun' (very nearly) explicitly states that it
> is only for use in languages which have close parentheses at EODs.

Hmm, this doesn't right though, does it?  Surely it's very useful if a
language with definitions ending in "end" handles those commands too
(tho it's more work, so understandable when not implemented).

[I guess it'd be desirable even for extreme languages like python, but
even more difficult]

-Miles

-- 
XML is like violence.  If it doesn't solve your problem, you're not
using enough of it.

Re: end-of-defun is fubsr.

[Stefan Monnier]

>> I prefer the current text.
> Was my patch really that bad?

I'm not saying it's bad, I'm saying that I find it less good than the
current text.  Most of the differences are nitpicks on which
we disagree for reasons explained over and over in this thread.

> Look, I'm as sick of all this as you must be.

AFAIK the only thing still missing is the addition of ARG to
EOD-function, which is something I prefer to postpone because it's not
a bug fix and we're in pretest.


        Stefan

Re: end-of-defun is fubsr.

[Alan Mackenzie]

On Sun, Feb 15, 2009 at 02:26:57PM -0500, Stefan Monnier wrote:
> >> I prefer the current text.
> > Was my patch really that bad?

> I'm not saying it's bad, I'm saying that I find it less good than the
> current text.  Most of the differences are nitpicks on which
> we disagree for reasons explained over and over in this thread.

The current text doesn't specify an EOD-F sufficiently to enable a major
mode writer to construct one.  He is forced to read the source in
lisp.el and work it out for himself.  This is a Bad Thing.

> > Look, I'm as sick of all this as you must be.

> AFAIK the only thing still missing is the addition of ARG to
> EOD-function, which is something I prefer to postpone because it's not
> a bug fix and we're in pretest.

There's a lot more "missing" than that.

With all due respect, a change this big should have been discussed on
emacs-devel long before we got to the pretest.  The only discussion I
think there's been so far was in the thread "Useless change in lisp.el?"
started by Andreas Röhler in November 2007.  Your verdict at the time
was, and I quote: "Rather than argue abuot the change itself, just give
us some use case where the new behavior is problematic."

We have now seen such a use case, if not two or three.

Why did this change not get proper discussion on emacs-devel?  Or did I
miss it?

I'd disagree with you about the bug fixness of the thing, but the last
thing we need to do is squabble about what words mean.  The effort
needed to fix it now is of a lesser order of magnitude than what will be
needed later if the current implementation is released in Emacs 23.  In
fact, the effort to fix it now is probably less than what the two of us
have just spent composing emails on the topic.

A suggestion: Let's just revert to the Emacs 22 code and documentation.
That was at least stable and consistent, and reverting won't introduce
new problems.  The way things are at the moment will introduce new
problems for people outside the core Emacs team.  It has already done so
for people inside the team.

If we release with lisp.el the way it currently is in the CVS, the
problems it will cause will be unfixable in the future, or at least
fixable only with an inordinate degree of uncreative effort.  I suspect
a lot of major mode maintainers just won't bother.

Then we can discuss things properly and fix them properly for Emacs 24.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).