moving point and invisible text

moving point and invisible text

[martin rudalics]

The Elisp manual describes the behavior of commands moving point with
respect to invisible text as follows:

"However, if a command ends with point inside or immediately after
invisible text, the main editing loop moves point further forward or
further backward (in the same direction that the command already moved
it) until that condition is no longer true.  ...  If the command moved
point forward into an invisible range, Emacs moves point forward past
the first visible character that follows the invisible text."

This description seems archaic.  With Emacs -q evaluate

(with-current-buffer (get-buffer-create "*test*")
   (insert "foo\n")
   (let ((at (point)))
     (insert "bar\nbar\nbar\n")
     (overlay-put (make-overlay at (point-max)) 'invisible t))
   (insert "baz\n"))

move point to position 18 in *test* (between the "b" and "a" of "baz")
and do C-b.  On my system this moves point to position 17 which is the
position immediately after the invisible text, thus contradicting the
first sentence of the description.  According to that sentence point
should move to position 5 here.

Now move point to position 4 (the end of the first line) and do C-2 C-f.
Another time point moves to position 17.  According to the manual point
should move to position 18 here.  I'm glad it doesn't.

Finally, move point to position 4 and execute C-f.  Once more point
moves to position 17.  This behavior is not mentioned in the manual.
Moreover, the manual fails to say what happens when the invisible text
resides at the end of a buffer.

Also, I was not able to detect a consistent behavior with different
values of `line-move-ignore-invisible' and with respect to whether an
invisibility overlay or an invisibility text-property was used.


Would it be difficult to implement this as:

"However, if a command moved point backward into or to the end of
invisible text, Emacs moves point backward to the beginning of that
text.  If a command moved point forward into or to the beginning of
invisible text, Emacs moves point forward to the end of that text."

Otherwise I'd change the text to something like:

"However, if a command ends with point inside invisible text, the main
editing loop moves point to some position before or after the invisible
text."

Re: moving point and invisible text

[Stefan Monnier]

> Would it be difficult to implement this as:

> "However, if a command moved point backward into or to the end of
> invisible text, Emacs moves point backward to the beginning of that
> text.  If a command moved point forward into or to the beginning of
> invisible text, Emacs moves point forward to the end of that text."

The behavior depends on many things: whether the invisible text is replaced
by ellipsis, whether the invisible property is front-sticky and/or
rear-sticky, whether the command made changes to the buffer, ...

My impression is that the text you quoted describes the behavior for
invisible+intangible text rather than just invisible.

> Otherwise I'd change the text to something like:

> "However, if a command ends with point inside invisible text, the main
> editing loop moves point to some position before or after the invisible
> text."

Sounds much more reasonable.  If you want to be more precise, the only thing
that might be worth mentioning is the driving principle behind the code: try
to ensure that a subsequent self-insert-command will insert a visible char.


        Stefan

Re: moving point and invisible text

[Richard M. Stallman]

    (with-current-buffer (get-buffer-create "*test*")
       (insert "foo\n")
       (let ((at (point)))
	 (insert "bar\nbar\nbar\n")
	 (overlay-put (make-overlay at (point-max)) 'invisible t))
       (insert "baz\n"))

    move point to position 18 in *test* (between the "b" and "a" of "baz")
    and do C-b.  On my system this moves point to position 17

    Finally, move point to position 4 and execute C-f.  Once more point
    moves to position 17.  This behavior is not mentioned in the manual.

The manual follows a previous version of the code, in which the position
after the invisible text was an impossible position.  The motive for this
was to avoid having two different point positions which display the cursor
in the same place.

The present code seems to treat the position after the invisible text
as possible, and treat the position just before it as impossible.
It's different, but it still achieves the goal of not having two
point positions that display the cursor in the same place.

I don't see any particular reason to prefer the old behavior, so I will
update the manual to describe the new behavior.

    Would it be difficult to implement this as:

    "However, if a command moved point backward into or to the end of
    invisible text, Emacs moves point backward to the beginning of that
    text.  If a command moved point forward into or to the beginning of
    invisible text, Emacs moves point forward to the end of that text."

It would be easy enough to implement, but then there would be two
different point positions that appear to be the same place.

Re: moving point and invisible text

[martin rudalics]

 > The manual follows a previous version of the code, in which the position
 > after the invisible text was an impossible position.  The motive for this
 > was to avoid having two different point positions which display the cursor
 > in the same place.
 >
 > The present code seems to treat the position after the invisible text
 > as possible, and treat the position just before it as impossible.
 > It's different, but it still achieves the goal of not having two
 > point positions that display the cursor in the same place.
 >
 > I don't see any particular reason to prefer the old behavior, so I will
 > update the manual to describe the new behavior.

I don't follow you.

(with-current-buffer (get-buffer-create "*test*")
   (insert "foo\n")
   (let ((at (point)))
     (insert "bar\nbar\nbar\n")
     (overlay-put (make-overlay at (point-max)) 'invisible t))
   (insert "baz\n"))

and point at position 4 in *test* C-f moves to position 17.

(with-current-buffer (get-buffer-create "*test*")
   (insert "foo\n")
   (let ((at (point)))
     (insert "bar\nbar\nbar\n")
     (put-text-property at (point-max) 'invisible t))
   (insert "baz\n"))

and point at position 4 in *test* C-f moves to position 5.  Hence when
using a text-property the position before invisible text is possible.

Re: moving point and invisible text

[Richard M. Stallman]

    (with-current-buffer (get-buffer-create "*test*")
       (insert "foo\n")
       (let ((at (point)))
	 (insert "bar\nbar\nbar\n")
	 (overlay-put (make-overlay at (point-max)) 'invisible t))
       (insert "baz\n"))

    and point at position 4 in *test* C-f moves to position 17.

That's what I'm talking about.  It skips position 5, the position
before the invisible text.

    (with-current-buffer (get-buffer-create "*test*")
       (insert "foo\n")
       (let ((at (point)))
	 (insert "bar\nbar\nbar\n")
	 (put-text-property at (point-max) 'invisible t))
       (insert "baz\n"))

    and point at position 4 in *test* C-f moves to position 5.

You did not mention that case before, and I did not know about it.

Further experimentation shows that this case works the way the
manual previously said: it allows position 5 but does not allow
position 17.

That inconsistency is a bug; overlay invisibility and text property
invisibility ought to work the same.

Could you debug why they are different?

Re: moving point and invisible text

[martin rudalics]

 >     (with-current-buffer (get-buffer-create "*test*")
 >        (insert "foo\n")
 >        (let ((at (point)))
 > 	 (insert "bar\nbar\nbar\n")
 > 	 (put-text-property at (point-max) 'invisible t))
 >        (insert "baz\n"))
 >
 >     and point at position 4 in *test* C-f moves to position 5.
 >
 > You did not mention that case before, and I did not know about it.

In the first mail in this thread I wrote

   Also, I was not able to detect a consistent behavior with different
   values of `line-move-ignore-invisible' and with respect to whether an
   invisibility overlay or an invisibility text-property was used.

Stefan then told me that

   The behavior depends on many things: whether the invisible text is replaced
   by ellipsis, whether the invisible property is front-sticky and/or
   rear-sticky, whether the command made changes to the buffer, ...

 > Further experimentation shows that this case works the way the
 > manual previously said: it allows position 5 but does not allow
 > position 17.

Not really: It rarely ever moves "point forward past the first visible
character that follows the invisible text", that is, position 18.  (It
does so only with `line-move-ignore-invisible' nil.)

 > That inconsistency is a bug; overlay invisibility and text property
 > invisibility ought to work the same.
 >
 > Could you debug why they are different?

I tried that already but got completely lost in the labyrinths of
redisplay.

Re: moving point and invisible text

[Richard M. Stallman]

It looks like the your code to move point out of invisible text is not
behaving consistently.  The previous invisible/intangible code, which
your code replaced, worked consistently.

Here is what martin rudalics <rudalics@gmx.at> said:

    Stefan then told me that

       The behavior depends on many things: whether the invisible text is replaced
       by ellipsis, whether the invisible property is front-sticky and/or
       rear-sticky, whether the command made changes to the buffer, ...

     > Further experimentation shows that this case works the way the
     > manual previously said: it allows position 5 but does not allow
     > position 17.

    Not really: It rarely ever moves "point forward past the first visible
    character that follows the invisible text", that is, position 18.  (It
    does so only with `line-move-ignore-invisible' nil.)

It needs to consistently allow one of the two possible point positions
next to invisible text--either just before it, or just after it--and
not allow the other.

Would you please work on fixing this?

Re: moving point and invisible text

[Stefan Monnier]

> It looks like the your code to move point out of invisible text is not
> behaving consistently.  The previous invisible/intangible code, which
> your code replaced, worked consistently.

AFAIK my code did not replace it.  Both codes are still with us.

> Here is what martin rudalics <rudalics@gmx.at> said:

>     Stefan then told me that

>        The behavior depends on many things: whether the invisible text is
>        replaced by ellipsis, whether the invisible property is
>        front-sticky and/or rear-sticky, whether the command made changes
>        to the buffer, ...

>> Further experimentation shows that this case works the way the
>> manual previously said: it allows position 5 but does not allow
>> position 17.

>     Not really: It rarely ever moves "point forward past the first visible
>     character that follows the invisible text", that is, position 18.  (It
>     does so only with `line-move-ignore-invisible' nil.)

> It needs to consistently allow one of the two possible point positions
> next to invisible text--either just before it, or just after it--and
> not allow the other.

> Would you please work on fixing this?

This keep coming up and the answer is still the same: no, I won't work on
it, because I'm deeply convinced it cannot be done without introducing
worse problems.  Such consistency is actually pretty low in the list
of priorities.  Much more important is e.g. to ensure that repeated movement
commands do not get stuck.

I have neither time nor inclination to get into more details again.
Check past discussions about it,


        Stefan

Re: moving point and invisible text

[Richard M. Stallman]

    > It looks like the your code to move point out of invisible text is not
    > behaving consistently.  The previous invisible/intangible code, which
    > your code replaced, worked consistently.

    AFAIK my code did not replace it.  Both codes are still with us.

I have a memory of deleting my invisible-intangible code, so that your
code would be the only one.  But I checked now, and that appears to be
a false memory; the code is still there.  Perhaps I just thought of
deleting it and did not actually do so.

Anyway, that code does not apply to the test cases we have been
looking at, since they are just invisible, not intangible.  We need to
clean up what is done with moving point outside invisible text at the
command loop level.

    This keep coming up and the answer is still the same: no, I won't work on
    it, because I'm deeply convinced it cannot be done without introducing
    worse problems.  Such consistency is actually pretty low in the list
    of priorities.  Much more important is e.g. to ensure that repeated movement
    commands do not get stuck.

We need to do both.  There is no conflict between them.  "Not getting
stuck" is a weaker criterion; "not appearing momentarily to be stuck"
is a stronger one which includes the other.  Satisfying the latter
will not interfere with the former.

This is a new feature and we should make it work properly now.  I will
make time to work on this myself, but I don't know how soon.  I hope
someone else can do it sooner.

I want to be able to get back to checking the Emacs Manual.  That also
needs to be done, and in two weeks around the beginning of this month
I checked 2/3 of it, but I've been bogged down by other work for the
past two weeks.  I hope to get back to it soon and finish it.