Something is rotten with end-of-line and move-end-of-line

Something is rotten with end-of-line and move-end-of-line

[David Kastrup]

Here are the online docs:

move-end-of-line is an interactive compiled Lisp function in `simple.el'.
It is bound to C-e, <end>.
(move-end-of-line ARG)

Move point to end of current line.
With argument ARG not nil or 1, move forward ARG - 1 lines first.
If point reaches the beginning or end of buffer, it stops there.
To ignore intangibility, bind `inhibit-point-motion-hooks' to t.

This command does not move point across a field boundary unless doing so
would move beyond there to a different line; if ARG is nil or 1, and
point starts at a field boundary, point does not move.  To ignore field
boundaries bind `inhibit-field-text-motion' to t.

[back]

end-of-line is an interactive built-in function in `C source code'.
(end-of-line &optional N)

Move point to end of current line.
With argument N not nil or 1, move forward N - 1 lines first.
If point reaches the beginning or end of buffer, it stops there.
To ignore intangibility, bind `inhibit-point-motion-hooks' to t.

This command does not move point across a field boundary unless doing so
would move beyond there to a different line; if N is nil or 1, and
point starts at a field boundary, point does not move.  To ignore field
boundaries bind `inhibit-field-text-motion' to t.


Apart from the argument names, both functions have identical
descriptions.  However, move-end-of-line is quite complicated and
implemented in Lisp.  The DOC strings don't give the slightest clue
about what might be different.  It appears like move-end-of-line
(which is bound to C-e) might have been intended for interactive use
and behaves special with regard to field boundaries.  However, since
end-of-line-position and other things seem to behave special, too,
according to their docs, it would appear that both tend to do the same
thing, but differently.

Personally, I'd prefer to have functions typically used in programs
(like end-of-line-position and end-of-line) rather not heed field
boundaries in order to save programmers from unexpected surprises.

However this is supposed to be resolved: it does not appear to make
much sense to have both move-end-of-line as well as end-of-line do
exactly the same thing, one in Lisp, one in C.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Something is rotten with end-of-line and move-end-of-line

[Richard M. Stallman]

    However this is supposed to be resolved: it does not appear to make
    much sense to have both move-end-of-line as well as end-of-line do
    exactly the same thing, one in Lisp, one in C.

They are not the same; I wrote move-end-of-line because it needs to be
different.  It was to fix a bug.

Unfortunately, I don't remember what the difference is.

Re: Something is rotten with end-of-line and move-end-of-line

[David Kastrup]

"Richard M. Stallman" <rms@gnu.org> writes:

>     However this is supposed to be resolved: it does not appear to
>     make much sense to have both move-end-of-line as well as
>     end-of-line do exactly the same thing, one in Lisp, one in C.
>
> They are not the same; I wrote move-end-of-line because it needs to
> be different.  It was to fix a bug.
>
> Unfortunately, I don't remember what the difference is.

Neither do the doc strings remember the difference.  And that means
that whatever bug you fixed by introducing the difference is bound to
be repeated by others, since there is no way to guess which of the two
functions is supposed to be used for what purpose.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Something is rotten with end-of-line and move-end-of-line

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Date: Mon, 28 Nov 2005 12:35:40 +0100
> Cc: emacs-devel@gnu.org
> 
> "Richard M. Stallman" <rms@gnu.org> writes:
> 
> >     However this is supposed to be resolved: it does not appear to
> >     make much sense to have both move-end-of-line as well as
> >     end-of-line do exactly the same thing, one in Lisp, one in C.
> >
> > They are not the same; I wrote move-end-of-line because it needs to
> > be different.  It was to fix a bug.
> >
> > Unfortunately, I don't remember what the difference is.
> 
> Neither do the doc strings remember the difference.  And that means
> that whatever bug you fixed by introducing the difference is bound to
> be repeated by others, since there is no way to guess which of the two
> functions is supposed to be used for what purpose.

There _is_ a way, albeit not an easy one: one could look at
emacs-devel archives around the time this change was committed, and
try to find the discussion which led to the change.

Re: Something is rotten with end-of-line and move-end-of-line

[Richard M. Stallman]

    Neither do the doc strings remember the difference.  And that means
    that whatever bug you fixed by introducing the difference is bound to
    be repeated by others, since there is no way to guess which of the two
    functions is supposed to be used for what purpose.

Yes, I should have written a comment to explain.

Can you find any message I posted about it?  If so, maybe
we can see the bug report I was dealing with, and thus figure out
the point.

Re: Something is rotten with end-of-line and move-end-of-line

[Lőrentey Károly]

Richard Stallman <rms@gnu.org> writes:
>     However this is supposed to be resolved: it does not appear to make
>     much sense to have both move-end-of-line as well as end-of-line do
>     exactly the same thing, one in Lisp, one in C.
>
> They are not the same; I wrote move-end-of-line because it needs to be
> different.  It was to fix a bug.
>
> Unfortunately, I don't remember what the difference is.

The difference (a difference?) is inside fields:
`{beginning,end}-of-line' moves to the beginning/end of the field,
while `move-{beginning,end}-of-line' stops at the field boundaries:

(let ((s "Test Field Test"))
  (put-text-property 5 10 'field 'foo s)
  (insert s))

Try pressing C-e and End inside "Field" to see the difference.

-- 
Károly

Re: Something is rotten with end-of-line and move-end-of-line

[Lőrentey Károly]

Lőrentey Károly <lorentey@elte.hu> writes:
> The difference (a difference?) is inside fields:
> `{beginning,end}-of-line' moves to the beginning/end of the field,
> while `move-{beginning,end}-of-line' stops at the field boundaries:

Sorry, I meant `move-{beginning,end}-of-line' goes to the actual line ends.

-- 
Károly

Re: Something is rotten with end-of-line and move-end-of-line

[David Kastrup]

lorentey@elte.hu (Lőrentey Károly) writes:

> Lőrentey Károly <lorentey@elte.hu> writes:
>> The difference (a difference?) is inside fields:
>> `{beginning,end}-of-line' moves to the beginning/end of the field,
>> while `move-{beginning,end}-of-line' stops at the field boundaries:
>
> Sorry, I meant `move-{beginning,end}-of-line' goes to the actual line ends.

Both doc strings claim to merely go to the field boundaries:

beginning-of-line is an interactive built-in function in `C source code'.
(beginning-of-line &optional N)

Move point to beginning of current line.
With argument N not nil or 1, move forward N - 1 lines first.
If point reaches the beginning or end of buffer, it stops there.

This command does not move point across a field boundary unless doing so
would move beyond there to a different line; If N is nil or 1, and point
starts at a field boundary, point does not move.  To ignore field
boundaries, either bind `inhibit-field-text-motion' to t, or use the
`forward-line' function instead.  For instance, `(forward-line 0)' does
the same thing as `(beginning-of-line)', except that it ignores field
boundaries.

[back]


move-beginning-of-line is an interactive compiled Lisp function in `simple.el'.
It is bound to <home>.
(move-beginning-of-line ARG)

Move point to beginning of current display line.
With argument ARG not nil or 1, move forward ARG - 1 lines first.
If point reaches the beginning or end of buffer, it stops there.
To ignore intangibility, bind `inhibit-point-motion-hooks' to t.

This command does not move point across a field boundary unless doing so
would move beyond there to a different line; if ARG is nil or 1, and
point starts at a field boundary, point does not move.  To ignore field
boundaries bind `inhibit-field-text-motion' to t.

[back]


-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Something is rotten with end-of-line and move-end-of-line

[Lőrentey Károly]

David Kastrup <dak@gnu.org> writes:
> lorentey@elte.hu (Lőrentey Károly) writes:
>> Lőrentey Károly <lorentey@elte.hu> writes:
>>> The difference (a difference?) is inside fields:
>>> `{beginning,end}-of-line' moves to the beginning/end of the field,
>>> while `move-{beginning,end}-of-line' stops at the field boundaries:
>>
>> Sorry, I meant `move-{beginning,end}-of-line' goes to the actual line ends.
>
> Both doc strings claim to merely go to the field boundaries:

Indeed, there are some bugs.

`end-of-line' does stop at the current field's boundary, although
repeated executions escape from the field.  However,
`move-end-of-line' always jumps over the end of the field, which is
clearly wrong.

`move-beginning-of-line' is similarly broken, but only if the field
property isn't front-sticky.  Otherwise both `beginning-of-line' and
`move-beginning-of-line' stop at the field boundary, and they stay
there if repeated.

Try this test in a fundamental buffer:

	(let ((s "Test Normal Front-sticky Test"))
	  (add-text-properties 5 11 '(face widget-field field foo) s)
	  (add-text-properties 12 24 '(face widget-field field bar front-sticky t) s)
          (insert s))

Thus, the current behaviour is clearly broken.  It is not clear,
however, what C-a/C-e is supposed to do when point is on a field
boundary.

* * *

Meanwhile, I found an archived discussion thread which explains the
intended difference between the two flavours:

	http://thread.gmane.org/gmane.emacs.devel/34012

        From: Kim F. Storm
        Subject: move-beginning-of-line
        Date: 2005-03-01 23:01:16 GMT
	Message-ID: <m3fyzfm0n7.fsf@kfs-l.imdomain.dk>

There is a test case given in the discussion that demonstrates the
original problem:

	(progn
	 (insert "\nab")
	 (insert-image-file "../etc/splash.xpm")
	 (move-end-of-line 1) 
	 (insert "def\n"))

move-b/e-of-line move across the image to the line ending that
corresponds to what is displayed on the screen, while the builtin
b/e-of-line variants stop at the hidden newlines embedded in the image
file.

Funnily enough, on closer inspection the doc string do indicate this
difference:

,----[ C-h f move-beginning-of-line RET
| Move point to beginning of current display line.
|                                    ^^^^^^^
`----

,----[ C-h f beginning-of-line RET
| Move point to beginning of current line.
`----

Obviously this needs to be elaborated.

-- 
Károly

Re: Something is rotten with end-of-line and move-end-of-line

[David Kastrup]

lorentey@elte.hu (Lőrentey Károly) writes:

> Thus, the current behaviour is clearly broken.  It is not clear,
> however, what C-a/C-e is supposed to do when point is on a field
> boundary.
>
> * * *
>
> Meanwhile, I found an archived discussion thread which explains the
> intended difference between the two flavours:
>
> 	http://thread.gmane.org/gmane.emacs.devel/34012
>
>         From: Kim F. Storm
>         Subject: move-beginning-of-line
>         Date: 2005-03-01 23:01:16 GMT
> 	Message-ID: <m3fyzfm0n7.fsf@kfs-l.imdomain.dk>
>
> There is a test case given in the discussion that demonstrates the
> original problem:
>
> 	(progn
> 	 (insert "\nab")
> 	 (insert-image-file "../etc/splash.xpm")
> 	 (move-end-of-line 1) 
> 	 (insert "def\n"))
>
> move-b/e-of-line move across the image to the line ending that
> corresponds to what is displayed on the screen, while the builtin
> b/e-of-line variants stop at the hidden newlines embedded in the image
> file.

Correct me if I am wrong, but we move point away from invisible areas
anyway in the keyboard input loop, so there is no necessity to let
move-end-of-line do the deed explicitly in order to get
user-comprehensible behavior, right?

> Funnily enough, on closer inspection the doc string do indicate this
> difference:
>
> ,----[ C-h f move-beginning-of-line RET
> | Move point to beginning of current display line.
> |                                    ^^^^^^^
> `----
>
> ,----[ C-h f beginning-of-line RET
> | Move point to beginning of current line.
> `----
>
> Obviously this needs to be elaborated.

Oh wow.  You can say that again.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Something is rotten with end-of-line and move-end-of-line

[Lőrentey Károly]

David Kastrup <dak@gnu.org> writes:
>> There is a test case given in the discussion that demonstrates the
>> original problem:
>>
>> 	(progn
>> 	 (insert "\nab")
>> 	 (insert-image-file "../etc/splash.xpm")
>> 	 (move-end-of-line 1) 
>> 	 (insert "def\n"))
>>
>> move-b/e-of-line move across the image to the line ending that
>> corresponds to what is displayed on the screen, while the builtin
>> b/e-of-line variants stop at the hidden newlines embedded in the image
>> file.
>
> Correct me if I am wrong, but we move point away from invisible areas
> anyway in the keyboard input loop, so there is no necessity to let
> move-end-of-line do the deed explicitly in order to get
> user-comprehensible behavior, right?

Consider the above example, when point is after the inserted image:

	ab[IMAGE]def<!>

Executing `beginning-of-line' stops right after the image, which is
not what the user expects.

	ab[IMAGE]<!>def

Similarly, from the other way around, `end-of-line' stops just before
the image.  (Note that the text that is hidden behind the image has
display and intangible properties only, it is not invisible.  However,
`end-of-line' and `beginning-of-line' behave the same (well, similar)
way with invisible newlines.)

-- 
Károly

Re: Something is rotten with end-of-line and move-end-of-line

[David Kastrup]

lorentey@elte.hu (Lőrentey Károly) writes:

> David Kastrup <dak@gnu.org> writes:
>>> There is a test case given in the discussion that demonstrates the
>>> original problem:
>>>
>>> 	(progn
>>> 	 (insert "\nab")
>>> 	 (insert-image-file "../etc/splash.xpm")
>>> 	 (move-end-of-line 1) 
>>> 	 (insert "def\n"))
>>>
>>> move-b/e-of-line move across the image to the line ending that
>>> corresponds to what is displayed on the screen, while the builtin
>>> b/e-of-line variants stop at the hidden newlines embedded in the image
>>> file.
>>
>> Correct me if I am wrong, but we move point away from invisible areas
>> anyway in the keyboard input loop, so there is no necessity to let
>> move-end-of-line do the deed explicitly in order to get
>> user-comprehensible behavior, right?
>
> Consider the above example, when point is after the inserted image:
>
> 	ab[IMAGE]def<!>
>
> Executing `beginning-of-line' stops right after the image, which is
> not what the user expects.
>
> 	ab[IMAGE]<!>def

Why would it do that?  beginning-of-line moved backwards, so I'd expect

ab<!>[IMAGE]def

> Similarly, from the other way around, `end-of-line' stops just
> before the image.  (Note that the text that is hidden behind the
> image has display and intangible properties only, it is not
> invisible.

Pretty much the same with regard to the keyboard loop.

> However, `end-of-line' and `beginning-of-line' behave the same
> (well, similar) way with invisible newlines.)

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Something is rotten with end-of-line and move-end-of-line

[Lőrentey Károly]

[-- Attachment #1: Type: text/plain, Size: 249 bytes --]

Lőrentey Károly <lorentey@elte.hu> writes:
> Indeed, there are some bugs.
[...]
> Obviously this needs to be elaborated.

Here is a patch that updates the doc strings and fixes
move-end-of-line behaviour with respect to field boundaries.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 3681 bytes --]

diff -u -3 -p -r1.774 simple.el
*** simple.el	28 Nov 2005 19:59:34 -0000	1.774
--- simple.el	28 Nov 2005 22:10:28 -0000
***************
*** 3641,3658 ****
  	    (goto-char (previous-char-property-change (point) line-beg))))))))
  
  (defun move-end-of-line (arg)
!   "Move point to end of current line.
  With argument ARG not nil or 1, move forward ARG - 1 lines first.
  If point reaches the beginning or end of buffer, it stops there.
  To ignore intangibility, bind `inhibit-point-motion-hooks' to t.
  
! This command does not move point across a field boundary unless doing so
! would move beyond there to a different line; if ARG is nil or 1, and
! point starts at a field boundary, point does not move.  To ignore field
! boundaries bind `inhibit-field-text-motion' to t."
    (interactive "p")
    (or arg (setq arg 1))
!   (let (done)
      (while (not done)
        (let ((newpos
  	     (save-excursion
--- 3641,3664 ----
  	    (goto-char (previous-char-property-change (point) line-beg))))))))
  
  (defun move-end-of-line (arg)
!   "Move point to end of current display line.
  With argument ARG not nil or 1, move forward ARG - 1 lines first.
  If point reaches the beginning or end of buffer, it stops there.
  To ignore intangibility, bind `inhibit-point-motion-hooks' to t.
  
! Unlike `end-of-line' (which see), this command ignores invisible
! newline characters while it searches for the end of the current
! line.  It moves the point to the end of the displayed screen
! line, which is not necessarily the same as the end of the current
! line in the buffer.
! 
! This command does not move point across the boundary of the
! current field.  To ignore field boundaries bind
! `inhibit-field-text-motion' to t."
    (interactive "p")
    (or arg (setq arg 1))
!   (let ((orig (point))
! 	done)
      (while (not done)
        (let ((newpos
  	     (save-excursion
***************
*** 3664,3670 ****
  			  (goto-char (previous-char-property-change (point))))
  			(backward-char 1)))
  		 (point)))))
! 	(goto-char newpos)
  	(if (and (> (point) newpos)
  		 (eq (preceding-char) ?\n))
  	    (backward-char 1)
--- 3670,3676 ----
  			  (goto-char (previous-char-property-change (point))))
  			(backward-char 1)))
  		 (point)))))
! 	(goto-char (constrain-to-field newpos orig (/= arg 1) t nil))
  	(if (and (> (point) newpos)
  		 (eq (preceding-char) ?\n))
  	    (backward-char 1)
***************
*** 3682,3691 ****
  If point reaches the beginning or end of buffer, it stops there.
  To ignore intangibility, bind `inhibit-point-motion-hooks' to t.
  
! This command does not move point across a field boundary unless doing so
! would move beyond there to a different line; if ARG is nil or 1, and
! point starts at a field boundary, point does not move.  To ignore field
! boundaries bind `inhibit-field-text-motion' to t."
    (interactive "p")
    (or arg (setq arg 1))
    (if (/= arg 1)
--- 3688,3703 ----
  If point reaches the beginning or end of buffer, it stops there.
  To ignore intangibility, bind `inhibit-point-motion-hooks' to t.
  
! Unlike `beginning-of-line' (which see), this command ignores
! invisible newline characters while it searches for the beginning
! of the current line.  It moves the point to the beginning of the
! displayed screen line, which is not necessarily the same as the
! end of the current line in the buffer.
! 
! This command does not move point across the boundary of the
! current field.  If ARG is nil or 1, and point starts at a field
! boundary, point does not move.  To ignore field boundaries bind
! `inhibit-field-text-motion' to t."
    (interactive "p")
    (or arg (setq arg 1))
    (if (/= arg 1)

[-- Attachment #3: Type: text/plain, Size: 892 bytes --]


I have a feeling that C-e should behave like C-a on a field boundary,
and leave the point unchanged when it is at the field end.  This would
make these operations idempotent.

For example, let's say the buffer contents are

	other stuff FIELD other stuff

Suppose that the characters "FIELD" have the same (front-sticky) field
property, while the rest are non-field.  Currently,
move-beginning-of-line (C-a) is a no-op when point is before F:

        other stuff <!>FIELD other stuff

I argue that either C-a should be changed to go the line beginning, or
move-end-of-line should behave symmetrically, i.e. it should have the
following fixed point:

	other stuff FIELD<!> other stuff

This could be achieved by changing `constrain-to-field'.

The various behaviours of C-a/C-e near field-field boundaries are also
somewhat surprising at the moment.

-- 
Károly

[-- Attachment #4: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Something is rotten with end-of-line and move-end-of-line

[Richard M. Stallman]

    `end-of-line' does stop at the current field's boundary, although
    repeated executions escape from the field.  However,
    `move-end-of-line' always jumps over the end of the field, which is
    clearly wrong.

It isn't wrong, it's intentional.

    `move-beginning-of-line' is similarly broken, but only if the field
    property isn't front-sticky.  Otherwise both `beginning-of-line' and
    `move-beginning-of-line' stop at the field boundary, and they stay
    there if repeated.

If move-beginning-of-line stops at a field beginning, I think that is
a bug.  Would you like to investigate it?

Re: Something is rotten with end-of-line and move-end-of-line

[Lőrentey Károly]

Richard Stallman <rms@gnu.org> writes:
>     `move-beginning-of-line' is similarly broken, but only if the field
>     property isn't front-sticky.  Otherwise both `beginning-of-line' and
>     `move-beginning-of-line' stop at the field boundary, and they stay
>     there if repeated.
>
> If move-beginning-of-line stops at a field beginning, I think that is
> a bug.  Would you like to investigate it?

Sure, we just need to remove the call to constrain-to-field in its
definition.  However, I would argue it's not a bug, but an important
feature: when I use an editable field, I have come to expect C-a to
jump back only to the beginning of the field, not all the way back to
the first column.  This is especially useful in a customize buffer.

Note that the original `beginning-of-line' also stops at field
boundaries.

(By symmetry, I also expect C-e to stop at the end of the field, but
despite the docstring of `move-end-of-line' that doesn't happen.)

-- 
Károly

Re: Something is rotten with end-of-line and move-end-of-line

[Richard M. Stallman]

    The difference (a difference?) is inside fields:
    `{beginning,end}-of-line' moves to the beginning/end of the field,
    while `move-{beginning,end}-of-line' stops at the field boundaries:

I am not sure of the difference between "beginning/end of the field"
and "stops at the field boundaries".  Could you explain the
difference in different words?

    Sorry, I meant `move-{beginning,end}-of-line' goes to the actual line ends.

Oh, that makes sense.

So I think this should explain the difference.


*** simple.el	28 Nov 2005 16:42:47 -0500	1.774
--- simple.el	28 Nov 2005 17:04:06 -0500	
***************
*** 3644,3655 ****
    "Move point to end of current line.
  With argument ARG not nil or 1, move forward ARG - 1 lines first.
  If point reaches the beginning or end of buffer, it stops there.
! To ignore intangibility, bind `inhibit-point-motion-hooks' to t.
! 
! This command does not move point across a field boundary unless doing so
! would move beyond there to a different line; if ARG is nil or 1, and
! point starts at a field boundary, point does not move.  To ignore field
! boundaries bind `inhibit-field-text-motion' to t."
    (interactive "p")
    (or arg (setq arg 1))
    (let (done)
--- 3644,3650 ----
    "Move point to end of current line.
  With argument ARG not nil or 1, move forward ARG - 1 lines first.
  If point reaches the beginning or end of buffer, it stops there.
! To ignore intangibility, bind `inhibit-point-motion-hooks' to t."
    (interactive "p")
    (or arg (setq arg 1))
    (let (done)
***************
*** 3680,3691 ****
    "Move point to beginning of current display line.
  With argument ARG not nil or 1, move forward ARG - 1 lines first.
  If point reaches the beginning or end of buffer, it stops there.
! To ignore intangibility, bind `inhibit-point-motion-hooks' to t.
! 
! This command does not move point across a field boundary unless doing so
! would move beyond there to a different line; if ARG is nil or 1, and
! point starts at a field boundary, point does not move.  To ignore field
! boundaries bind `inhibit-field-text-motion' to t."
    (interactive "p")
    (or arg (setq arg 1))
    (if (/= arg 1)
--- 3675,3681 ----
    "Move point to beginning of current display line.
  With argument ARG not nil or 1, move forward ARG - 1 lines first.
  If point reaches the beginning or end of buffer, it stops there.
! To ignore intangibility, bind `inhibit-point-motion-hooks' to t."
    (interactive "p")
    (or arg (setq arg 1))
    (if (/= arg 1)