Don't move to eol in end-of-defun?

Don't move to eol in end-of-defun?

[Filipp Gunbin]

In Java, it's a common style to have hanging parentheses:

class C <1>{<2>
  void foo() {
  }<3>
}<4>

If we're at position <1> and press C-M-e, then it's logical to move to
<4>, and that's what I implement in javaimp package, by defining my own
beginning-of-defun-function / end-of-defun-function.  But end-of-defun
first calls (end-of-line 1), before doing everything else, and so my
function finds itself at <2>!  And moves to <3>.

OTOH, end-of-defun is documented to move to "next end of defun", and one
could say that moving from <1> to <3> is just that.  I can agree, but
then I get strange behavior of narrow-to-defun, mark-defun and others,
because they rely on beginning / end of defun.

I can suggest this patch, but I don't very much like it.

Thoughts?

TIA
Filipp


diff --git a/lisp/emacs-lisp/lisp.el b/lisp/emacs-lisp/lisp.el
index 4b85414943..cc8185e453 100644
--- a/lisp/emacs-lisp/lisp.el
+++ b/lisp/emacs-lisp/lisp.el
@@ -507,6 +507,13 @@ end-of-defun-function
 So the function can assume that point is at the beginning of the defun body.
 It should move point to the first position after the defun.")
 
+(defvar end-of-defun-moves-to-eol t
+  "Defines whether `end-of-defun' moves to eol before doing
+everything else.
+
+Set this to nil in major mode if this movement affects mode's
+decisions about context in an unwanted way.")
+
 (defun buffer-end (arg)
   "Return the \"far end\" position of the buffer, in direction ARG.
 If ARG is positive, that's the end of the buffer.
@@ -538,7 +545,9 @@ end-of-defun
         (push-mark))
     (if (or (null arg) (= arg 0)) (setq arg 1))
     (let ((pos (point))
-          (beg (progn (end-of-line 1) (beginning-of-defun-raw 1) (point)))
+          (beg (progn (when end-of-defun-moves-to-eol
+                        (end-of-line 1))
+                      (beginning-of-defun-raw 1) (point)))
 	  (skip (lambda ()
 		  ;; When comparing point against pos, we want to consider that
 		  ;; if point was right after the end of the function, it's

Re: Don't move to eol in end-of-defun?

[Daniel Martín]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> In Java, it's a common style to have hanging parentheses:
>
> class C <1>{<2>
>   void foo() {
>   }<3>
> }<4>
>
> If we're at position <1> and press C-M-e, then it's logical to move to
> <4>, and that's what I implement in javaimp package, by defining my own
> beginning-of-defun-function / end-of-defun-function.

If I yank your sample program in a buffer in Java mode, with point at
<1>, C-M-e goes to <4>, as expected.  So I don´t know why you need to
implement your own end-of-defun-function in your package.

>
> OTOH, end-of-defun is documented to move to "next end of defun", and one
> could say that moving from <1> to <3> is just that.

The docstring is perhaps a bit confusing, but the Emacs manual and
glossary defines what end-of-defun does in each major mode more
precisely: It moves point to the end of the current defun.  When the
point is in between two defuns, the current defun is defined as the next
defun that follows the point.  So the expected navigation is <1> -> <4>,
<2> -> <3>, which is how the included java-mode works.

I'm not sure about the problem your solution is trying to fix.  Perhaps
I'm missing something.

Re: Don't move to eol in end-of-defun?

[Filipp Gunbin]

On 31/07/2022 01:40 +0200, Daniel Martín wrote:

> Filipp Gunbin <fgunbin@fastmail.fm> writes:
>
>> In Java, it's a common style to have hanging parentheses:
>>
>> class C <1>{<2>
>>   void foo() {
>>   }<3>
>> }<4>
>>
>> If we're at position <1> and press C-M-e, then it's logical to move to
>> <4>, and that's what I implement in javaimp package, by defining my own
>> beginning-of-defun-function / end-of-defun-function.
>
> If I yank your sample program in a buffer in Java mode, with point at
> <1>, C-M-e goes to <4>, as expected.  So I don´t know why you need to
> implement your own end-of-defun-function in your package.

I'd like to not rely on java-mode (part of cc-mode).

Thanks.

Re: Don't move to eol in end-of-defun?

[Alan Mackenzie]

Hello, Filipp.

On Mon, Aug 01, 2022 at 01:31:28 +0300, Filipp Gunbin wrote:
> On 31/07/2022 01:40 +0200, Daniel Martín wrote:

> > Filipp Gunbin <fgunbin@fastmail.fm> writes:

> >> In Java, it's a common style to have hanging parentheses:

> >> class C <1>{<2>
> >>   void foo() {
> >>   }<3>
> >> }<4>

> >> If we're at position <1> and press C-M-e, then it's logical to move to
> >> <4>, and that's what I implement in javaimp package, by defining my own
> >> beginning-of-defun-function / end-of-defun-function.

Be aware that end-of-defun-function is broken.  Before calling e-o-d-f,
end-of-defun first moves point somewhere else and then calls it.  That
somewhere else could easily be in a different function.  Say if you have

class C <1>{ <2>void foo () {
             }<3>
}<4>

and start at <1>, that somewhere else is likely to be <2> from where
your e-o-d-f will probably go to <3>.

> > If I yank your sample program in a buffer in Java mode, with point at
> > <1>, C-M-e goes to <4>, as expected.  So I don´t know why you need to
> > implement your own end-of-defun-function in your package.

> I'd like to not rely on java-mode (part of cc-mode).

Be further aware that in languages like Java, you can't do a fully
accurate job without scanning braces back to BOB.  CC Mode has its own
c-end-of-defun function, which uses a cache of brace blocks, for the
reasons in this post (amongst others).  You're welcome to take stuff
from it, if that might help.

> Thanks.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Don't move to eol in end-of-defun?

[Filipp Gunbin]

Hello Alan,

On 01/08/2022 17:50 +0000, Alan Mackenzie wrote:

> Hello, Filipp.
>
> On Mon, Aug 01, 2022 at 01:31:28 +0300, Filipp Gunbin wrote:
>> On 31/07/2022 01:40 +0200, Daniel Martín wrote:
>
>> > Filipp Gunbin <fgunbin@fastmail.fm> writes:
>
>> >> In Java, it's a common style to have hanging parentheses:
>
>> >> class C <1>{<2>
>> >>   void foo() {
>> >>   }<3>
>> >> }<4>
>
>> >> If we're at position <1> and press C-M-e, then it's logical to move to
>> >> <4>, and that's what I implement in javaimp package, by defining my own
>> >> beginning-of-defun-function / end-of-defun-function.
>
> Be aware that end-of-defun-function is broken.  Before calling e-o-d-f,
> end-of-defun first moves point somewhere else and then calls it.  That
> somewhere else could easily be in a different function.  Say if you have
>
> class C <1>{ <2>void foo () {
>              }<3>
> }<4>
>
> and start at <1>, that somewhere else is likely to be <2> from where
> your e-o-d-f will probably go to <3>.

.. and that is exactly what I'm trying to prevent.

>> > If I yank your sample program in a buffer in Java mode, with point at
>> > <1>, C-M-e goes to <4>, as expected.  So I don´t know why you need to
>> > implement your own end-of-defun-function in your package.
>
>> I'd like to not rely on java-mode (part of cc-mode).
>
> Be further aware that in languages like Java, you can't do a fully
> accurate job without scanning braces back to BOB.  CC Mode has its own
> c-end-of-defun function, which uses a cache of brace blocks, for the
> reasons in this post (amongst others).  You're welcome to take stuff
> from it, if that might help.

Of course, I already do context parsing (via syntax-ppss), it's in
<elpa>/packages/javaimp/javaimp-parse.el.  That is, my code knows enough
about context.  I just need to prevent end-of-defun from messing with
position.

Thanks.

Re: Don't move to eol in end-of-defun?

[Filipp Gunbin]

Here's a bit more on the current state of javaimp:
https://lists.gnu.org/archive/html/help-gnu-emacs/2022-06/msg00324.html

I've released the mentioned "next version" last week, it's 0.9.

Re: Don't move to eol in end-of-defun?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > In Java, it's a common style to have hanging parentheses:

  > class C <1>{<2>
  >   void foo() {
  >   }<3>
  > }<4>

  > If we're at position <1> and press C-M-e, then it's logical to move to
  > <4>,

I'm not convinced.  I made end-of-defun move to the newline between defuns,
usually, because there are situations where it is useful to end up there.
One of them is in C-M-h (mark-defun), but it's not that alone.

I think these reasons apply to all languages, especially when there's
a blank line between the defuns.

If you want to end up at the closing delimiter, you can use C-M-a C-M-f.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Don't move to eol in end-of-defun?

[Filipp Gunbin]

On 02/08/2022 23:46 -0400, Richard Stallman wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > In Java, it's a common style to have hanging parentheses:
>
>   > class C <1>{<2>
>   >   void foo() {
>   >   }<3>
>   > }<4>
>
>   > If we're at position <1> and press C-M-e, then it's logical to move to
>   > <4>,
>
> I'm not convinced.  I made end-of-defun move to the newline between defuns,
> usually, because there are situations where it is useful to end up there.
> One of them is in C-M-h (mark-defun), but it's not that alone.
>
> I think these reasons apply to all languages, especially when there's
> a blank line between the defuns.

Yes, that will be the behavior.  This post and example were about moving
over the braces vs. moving into them, and <4> will really be on blank
line between defuns.

Thanks.

Re: Don't move to eol in end-of-defun?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Yes, that will be the behavior.  This post and example were about moving
  > over the braces vs. moving into them, and <4> will really be on blank
  > line between defuns.

We may be partly miscommunicating.  With other languages, such as
Lisp, C-M-e moves to the blank line after the defun from _anywhere_
inside the defun.  (Except in weird cases, such as this,

    (defun foo (a b)
      x y z))

where the extra closeparen after the defun means the only valid place
to stop is before the extra closeparen.)

You're planning to break that rule, right?

I think that this rule behavior will be useful in other languages too;
maybe breaking it is not a good idea.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Don't move to eol in end-of-defun?

[Filipp Gunbin]

On 04/08/2022 00:04 -0400, Richard Stallman wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Yes, that will be the behavior.  This post and example were about moving
>   > over the braces vs. moving into them, and <4> will really be on blank
>   > line between defuns.
>
> We may be partly miscommunicating.  With other languages, such as
> Lisp, C-M-e moves to the blank line after the defun from _anywhere_
> inside the defun.  (Except in weird cases, such as this,
>
>     (defun foo (a b)
>       x y z))
>
> where the extra closeparen after the defun means the only valid place
> to stop is before the extra closeparen.)
>
> You're planning to break that rule, right?
>
> I think that this rule behavior will be useful in other languages too;
> maybe breaking it is not a good idea.

No-no, I don't intend to break it.

Returning to my example:

class C <1>{<2>
  void foo() {
  }<3>
}<4>
<5>
class D ...

We're at point <1>.  I'd like C-M-e to go to <5> (in my first email I
said <4>, but that was my inaccuracy; it will be <5> except in weird
cases like you showed).

We have nested defuns here.  When we're at <1>, we're "in class C's
declaration", and C-M-e should move to after its end.  But currently
end-of-defun will first unconditionally move to eol (<2>), where the
context is different: now we're "before foo()'s declaration", and C-M-e
should move to <3>, which it does.  The knowledge about nested contexts
and their boundaries is provided by my mode, in
beginning/end-of-defun-function.

I just want to make movement to eol conditional, with default value
meaning "like before", to not break anything.

Filipp

Re: Don't move to eol in end-of-defun?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > We have nested defuns here.

A "defun" in Emacs is not the same thing as a function definition
(or class definition).

    class C {
      void foo() {
      }
    }

has two nested definitions, but only the outermost one counts as a
defun in Emacs parlance.

A defun is a construct which is top-level, or appears locally to be.
In Lisp that usually means an open-paren in column 0.  In some other
languages, there are other ways to find defuns.

  > I just want to make movement to eol conditional, with default value
  > meaning "like before", to not break anything.

Doing it that way might be ok.  At any rate, no disaster.  But it
leaves the question, should we really try to support nested defuns?
It is a can of worms.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Don't move to eol in end-of-defun?

[Alan Mackenzie]

Hello, Richard.

On Fri, Aug 05, 2022 at 23:41:39 -0400, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

>   > We have nested defuns here.

> A "defun" in Emacs is not the same thing as a function definition
> (or class definition).

>     class C {
>       void foo() {
>       }
>     }

> has two nested definitions, but only the outermost one counts as a
> defun in Emacs parlance.

> A defun is a construct which is top-level, or appears locally to be.
> In Lisp that usually means an open-paren in column 0.  In some other
> languages, there are other ways to find defuns.

>   > I just want to make movement to eol conditional, with default value
>   > meaning "like before", to not break anything.

To clarify, Filipp's problem is the current implementation of
end-of-defun in lisp/emacs-lisp/lisp.el.  There are two alternatives in
this function.  Either it moves up the parenthesis structure, like for
Emacs Lisp Mode, or it calls the mode specific function
end-of-defun-function instead.

The problem is that e-o-d moves point somewhere else _before_ it calls
end-of-defun-function, and that somewhere else can easily be in a
different (nested) defun.

I think Filipp is asking for the coding of end-of-defun to be revisited.

> Doing it that way might be ok.  At any rate, no disaster.  But it
> leaves the question, should we really try to support nested defuns?
> It is a can of worms.

CC Mode has supported these nested defuns for many years, now.

For example, in C++, it is common for a source file to begin with

    namespace foo {

, and the rest of the functions/methods/classes in the file to be
enclosed within that namespace.

In these circumstances, for C-M-a to go to the outermost "defun"
wouldn't be useful.  CC Mode has C-M-a moving to the previous start of
defun at the current level of namespace/class/struct nesting, or the
next level outwards when we bump up against the defining
namespace/class/struct start.  C-M-e works likewise.

This works well, and there have been remarkably few bug reports about it
(I can't actually remember any).

> -- 
> Dr Richard Stallman (https://stallman.org)
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Don't move to eol in end-of-defun?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The problem is that e-o-d moves point somewhere else _before_ it calls
  > end-of-defun-function, and that somewhere else can easily be in a
  > different (nested) defun.

Is there a case which is incorrect _now_?  If so, what case is it,
and what happens now in that case?

  > I think Filipp is asking for the coding of end-of-defun to be revisited.

Code can be changed if it is broken, but the question behind that
needs to be, "What is the right thing in this case?"

  > In these circumstances, for C-M-a to go to the outermost "defun"
  > wouldn't be useful.

Yes, we do need to be able to have outer groupings which we designate
as "does not count as a defun", so that things inside it which look like
defuns do count as  defuns.

This requires a way for the programmer to mark them so that C-M-a
mostly ignores them.

However, any old nested function definition shouldn't be treated as a
defun.

  >   CC Mode has C-M-a moving to the previous start of
  > defun at the current level of namespace/class/struct nesting, or the
  > next level outwards when we bump up against the defining
  > namespace/class/struct start.  C-M-e works likewise.

Can that method be used in this language too?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Don't move to eol in end-of-defun?

[Filipp Gunbin]

On 07/08/2022 00:24 -0400, Richard Stallman wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > The problem is that e-o-d moves point somewhere else _before_ it calls
>   > end-of-defun-function, and that somewhere else can easily be in a
>   > different (nested) defun.
>
> Is there a case which is incorrect _now_?  If so, what case is it,
> and what happens now in that case?

Yes, my example from previous messages.

>   > I think Filipp is asking for the coding of end-of-defun to be revisited.
>
> Code can be changed if it is broken, but the question behind that
> needs to be, "What is the right thing in this case?"

It's not broken, but rather clearly wasn't intended for nested defuns.
AFAICS, cc-mode overrides beginning/end-of-defun completely.

In my mode, I implemented just beginning/end-of-defun-function, while
using standard beginning/end-of-defun functions, and it works well
except for this case (and, yes, one more rare bug during narrowing, I'm
yet to investigate it).

Currently cc-mode and my implementation for Java work differently: as
Alan said, cc-mode goes outside the enclosing block, while I stay
inside.  I find it convenient to think about nested things in this way:
C-M-a and C-M-e should behave as if we were narrowed to the current
enclosing block, like if its contained methods/whatever were top-level.

>   > In these circumstances, for C-M-a to go to the outermost "defun"
>   > wouldn't be useful.
>
> Yes, we do need to be able to have outer groupings which we designate
> as "does not count as a defun", so that things inside it which look like
> defuns do count as  defuns.
>
> This requires a way for the programmer to mark them so that C-M-a
> mostly ignores them.
>
> However, any old nested function definition shouldn't be treated as a
> defun.

In Java, you can have "local" classes, these are classes defined and
used inside a method:

class C {

  void foo() {
    class D {
      void bar() {
      }
    }

    D d = new D();
    ...
  }
}

So I chose to "honestly" parse & handle context right from the start.

Filipp