bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Ihor Radchenko]

Hi,

This is a followup of https://orgmode.org/list/87ilag1pth.fsf@localhost

We recently ran into some subtleties of point adjustment near invisible
regions with sticky 'invisible property.

The actual code in `set_point_both' makes sure to move point across
invisible sticky region if point happens to end up right before front
sticky invisible or right after rear sticky invisible region.

However, 22.6 Adjusting Point After Commands does not mention this
scenario.

Further, `set_point_both' contains the following comments:

	  /* If the preceding character is both intangible and invisible,
	     and the invisible property is `rear-sticky', perturb it so
	     that the search starts one character earlier -- this ensures
	     that point can never move to the end of an invisible/
	     intangible/rear-sticky region.  */
	  charpos = adjust_for_invis_intang (charpos, -1, -1, 1);
          <...>
      	  /* If the following character is both intangible and invisible,
	     and the invisible property is `front-sticky', perturb it so
	     that the search starts one character later -- this ensures
	     that point can never move to the beginning of an
	     invisible/intangible/front-sticky region.  */
	  charpos = adjust_for_invis_intang (charpos, 0, 1, 1);

However, 'intanglible text property is, in fact, not necessary to
trigger the adjustment.

May you (1) document this behaviour in the manual; (2) clarify what's up
with intanglible.

In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.38, cairo version 1.17.8) of 2023-07-18 built on localhost
Repository revision: aba3d13ea8d712bca1b3f23ac7db7e38c2165b3c
Repository branch: feature/named-lambdas
Windowing system distributor 'The X.Org Foundation', version 11.0.12101008
System Description: Gentoo Linux

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Date: Wed, 19 Jul 2023 07:02:21 +0000
> 
> This is a followup of https://orgmode.org/list/87ilag1pth.fsf@localhost
> 
> We recently ran into some subtleties of point adjustment near invisible
> regions with sticky 'invisible property.
> 
> The actual code in `set_point_both' makes sure to move point across
> invisible sticky region if point happens to end up right before front
> sticky invisible or right after rear sticky invisible region.
> 
> However, 22.6 Adjusting Point After Commands does not mention this
> scenario.
> 
> Further, `set_point_both' contains the following comments:
> 
> 	  /* If the preceding character is both intangible and invisible,
> 	     and the invisible property is `rear-sticky', perturb it so
> 	     that the search starts one character earlier -- this ensures
> 	     that point can never move to the end of an invisible/
> 	     intangible/rear-sticky region.  */
> 	  charpos = adjust_for_invis_intang (charpos, -1, -1, 1);
>           <...>
>       	  /* If the following character is both intangible and invisible,
> 	     and the invisible property is `front-sticky', perturb it so
> 	     that the search starts one character later -- this ensures
> 	     that point can never move to the beginning of an
> 	     invisible/intangible/front-sticky region.  */
> 	  charpos = adjust_for_invis_intang (charpos, 0, 1, 1);
> 
> However, 'intanglible text property is, in fact, not necessary to
> trigger the adjustment.
> 
> May you (1) document this behaviour in the manual; (2) clarify what's up
> with intanglible.

I think you confuse two different point-adjustment behaviors.  What
the ELisp manual describes is the one implemented in
adjust_point_for_property and the code which calls it from
command_loop_1.  set_point_both does a different job for different
situations, AFAIU.

If you ignore set_point_both, what exactly is wrong about that short
section in the manual?

Adding Stefan, in case he has comments.

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> I think you confuse two different point-adjustment behaviors.  What
> the ELisp manual describes is the one implemented in
> adjust_point_for_property and the code which calls it from
> command_loop_1.  set_point_both does a different job for different
> situations, AFAIU.

Indeed, the `set_point_both` adjustment is an obsolete (mis-)feature
that has been deprecated since Emacs-25, when
`inhibit-point-motion-hooks`s default has been changed to t (and
replaced with `cursor-sensor-mode` and
`cursor-intangible-mode`).


        Stefan

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Indeed, the `set_point_both` adjustment is an obsolete (mis-)feature
> that has been deprecated since Emacs-25, when
> `inhibit-point-motion-hooks`s default has been changed to t (and
> replaced with `cursor-sensor-mode` and
> `cursor-intangible-mode`).

Will it ever be removed though?
It appears to affect outline mode a lot.

We just recently received a somewhat related report with yet another
point adjustment subtlety:

* Short heading...
* Very very very very long heading<point>...

C-p will move after hidden text above:

* Short heading...<point>
* Very very very very long heading...

And I am pretty sure that there are many edge cases like this, which
rely on the implementation details with point adjustment.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Ihor Radchenko [2023-07-20 09:57:02] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> Indeed, the `set_point_both` adjustment is an obsolete (mis-)feature
>> that has been deprecated since Emacs-25, when
>> `inhibit-point-motion-hooks`s default has been changed to t (and
>> replaced with `cursor-sensor-mode` and
>> `cursor-intangible-mode`).
>
> Will it ever be removed though?

That's the plan, yes.  In Emacs-29, I removed all the places where we
defensively let-bound it to non-nil, in preparation for the
actual removal.

> It appears to affect outline mode a lot.
>
> We just recently received a somewhat related report with yet another
> point adjustment subtlety:
>
> * Short heading...
> * Very very very very long heading<point>...
>
> C-p will move after hidden text above:
>
> * Short heading...<point>
> * Very very very very long heading...

I suspect this has nothing to do with `inhibit-point-motion-hooks` nor
`set-point-both` but with the post-command point-adjustment, instead.

> And I am pretty sure that there are many edge cases like this, which
> rely on the implementation details with point adjustment.

point-adjustment cannot get all cases right, because (like in the above
example), the "right" thing to do depends on the specific semantics of
the last command (e.g. was the last command supposed to move by one
logical line or one screen line?).

IIRC, `C-p` has ad-hoc code to try and handle cases like the above, but
apparently it doesn't handle this specific case.


        Stefan

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> May you (1) document this behaviour in the manual; (2) clarify what's up
>> with intanglible.
>
> I think you confuse two different point-adjustment behaviors.  What
> the ELisp manual describes is the one implemented in
> adjust_point_for_property and the code which calls it from
> command_loop_1.  set_point_both does a different job for different
> situations, AFAIU.

> If you ignore set_point_both, what exactly is wrong about that short
> section in the manual?

Let me describe the full sequence how I arrived to that manual page:

1. We had a situation where `backward-word' moved further than expected:

   <invisible>=</invisible>word<point>
   M-b
   <point>=word
   
   This was despite "=" not considered to be a word constituent in
   syntax table.

2. I noticed that this behavior is related to point movement and
   invisible text and searched manual index for "point", "invisible".
   The only match was "22.6 Adjusting Point After Commands"
   I assumed that it is describing all the cases related to interaction
   between point and invisible text.

3. I've read that section of the manual and did not find anything about
   point ending up not inside, but at the boundary of invisible text.

4. I reached to sources and tried to track down the cause, eventually
   arriving to `set_point_both'.

So, there is no problem with this section of the manual. The problem is
that manual does not describe the behavior with invisible text
boundaries I was trying to figure out. (Or does not properly mark the
relevant section with index keywords).

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, 64724@debbugs.gnu.org
> Date: Thu, 20 Jul 2023 09:51:52 +0000
> 
> So, there is no problem with this section of the manual. The problem is
> that manual does not describe the behavior with invisible text
> boundaries I was trying to figure out.

Which aspects of the behavior you describe are not mentioned in the
manual.  It does refer to invisible text.

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> So, there is no problem with this section of the manual. The problem is
>> that manual does not describe the behavior with invisible text
>> boundaries I was trying to figure out.
>
> Which aspects of the behavior you describe are not mentioned in the
> manual.  It does refer to invisible text.

It talks about "in the middle of a sequence .. that ... is invisible"

    Emacs cannot display the cursor when point is in the middle of a
    sequence of text that has the ‘display’ or ‘composition’ property, or is
    invisible.  Therefore, after a command finishes and returns to the
    command loop, if point is within such a sequence, the command loop
    normally moves point to the edge of the sequence, making this sequence
    effectively intangible.

The situation I encountered is when point ends up not in the middle, but
already on the edge. So, Emacs moving point to _other_ edge was
surprising.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: monnier@iro.umontreal.ca, 64724@debbugs.gnu.org
> Date: Fri, 21 Jul 2023 07:50:55 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> So, there is no problem with this section of the manual. The problem is
> >> that manual does not describe the behavior with invisible text
> >> boundaries I was trying to figure out.
> >
> > Which aspects of the behavior you describe are not mentioned in the
> > manual.  It does refer to invisible text.
> 
> It talks about "in the middle of a sequence .. that ... is invisible"
> 
>     Emacs cannot display the cursor when point is in the middle of a
>     sequence of text that has the ‘display’ or ‘composition’ property, or is
>     invisible.  Therefore, after a command finishes and returns to the
>     command loop, if point is within such a sequence, the command loop
>     normally moves point to the edge of the sequence, making this sequence
>     effectively intangible.
> 
> The situation I encountered is when point ends up not in the middle, but
> already on the edge. So, Emacs moving point to _other_ edge was
> surprising.

That's because where to move point in these situations depends on
where it was before.  We could describe these details of the
heuristics we use, but I very much doubt it will be useful in
practice.  And the heuristics could change in the future, which makes
this a maintenance problem.

So I tend to think this bug should be closed with no further action.
The main purpose of that node is to explain how to disable this point
adjustment, not describe in detail what it does and how.

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> The situation I encountered is when point ends up not in the middle, but
>> already on the edge. So, Emacs moving point to _other_ edge was
>> surprising.
>
> That's because where to move point in these situations depends on
> where it was before.  We could describe these details of the
> heuristics we use, but I very much doubt it will be useful in
> practice.  And the heuristics could change in the future, which makes
> this a maintenance problem.

I think you misunderstood.
I know that the logic how Emacs chooses which boundary to move to is not
documented.

However, my problem is not with this logic. My problem is that info page
says that point adjustment happens when point is _inside_ invisible
region. However, in my situation the point is already at the boundary
(that is: _not inside_).

> So I tend to think this bug should be closed with no further action.
> The main purpose of that node is to explain how to disable this point
> adjustment, not describe in detail what it does and how.

Please remember that info page was just one of the points I listed in
the initial report.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: monnier@iro.umontreal.ca, 64724@debbugs.gnu.org
> Date: Fri, 21 Jul 2023 12:14:55 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > That's because where to move point in these situations depends on
> > where it was before.  We could describe these details of the
> > heuristics we use, but I very much doubt it will be useful in
> > practice.  And the heuristics could change in the future, which makes
> > this a maintenance problem.
> 
> I think you misunderstood.
> I know that the logic how Emacs chooses which boundary to move to is not
> documented.
> 
> However, my problem is not with this logic. My problem is that info page
> says that point adjustment happens when point is _inside_ invisible
> region. However, in my situation the point is already at the boundary
> (that is: _not inside_).

Stefan explained that aspect, so I didn't see the point in reiterating
it.

> > So I tend to think this bug should be closed with no further action.
> > The main purpose of that node is to explain how to disable this point
> > adjustment, not describe in detail what it does and how.
> 
> Please remember that info page was just one of the points I listed in
> the initial report.

I must have missed them (even though I've skimmed the discussion again
now), so please point them out again.

(And please don't use the incorrect term "Info page".  Unlike "man
pages", the Info documentation doesn't come as a collection of "pages"
it comes in the form of complete manuals.)

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Please remember that info page was just one of the points I listed in
>> the initial report.
>
> I must have missed them (even though I've skimmed the discussion again
> now), so please point them out again.

Further, `set_point_both' contains the following comments:

	  /* If the preceding character is both intangible and invisible,
	     and the invisible property is `rear-sticky', perturb it so
	     that the search starts one character earlier -- this ensures
	     that point can never move to the end of an invisible/
	     intangible/rear-sticky region.  */
	  charpos = adjust_for_invis_intang (charpos, -1, -1, 1);
	  <...>
	  /* If the following character is both intangible and invisible,
	     and the invisible property is `front-sticky', perturb it so
	     that the search starts one character later -- this ensures
	     that point can never move to the beginning of an
	     invisible/intangible/front-sticky region.  */
	  charpos = adjust_for_invis_intang (charpos, 0, 1, 1);

However, 'intanglible text property is, in fact, not necessary to
trigger the adjustment.

> (And please don't use the incorrect term "Info page".  Unlike "man
> pages", the Info documentation doesn't come as a collection of "pages"
> it comes in the form of complete manuals.)

I meant info node.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: monnier@iro.umontreal.ca, 64724@debbugs.gnu.org
> Date: Fri, 21 Jul 2023 12:56:19 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Please remember that info page was just one of the points I listed in
> >> the initial report.
> >
> > I must have missed them (even though I've skimmed the discussion again
> > now), so please point them out again.
> 
> Further, `set_point_both' contains the following comments:
> 
> 	  /* If the preceding character is both intangible and invisible,
> 	     and the invisible property is `rear-sticky', perturb it so
> 	     that the search starts one character earlier -- this ensures
> 	     that point can never move to the end of an invisible/
> 	     intangible/rear-sticky region.  */
> 	  charpos = adjust_for_invis_intang (charpos, -1, -1, 1);
> 	  <...>
> 	  /* If the following character is both intangible and invisible,
> 	     and the invisible property is `front-sticky', perturb it so
> 	     that the search starts one character later -- this ensures
> 	     that point can never move to the beginning of an
> 	     invisible/intangible/front-sticky region.  */
> 	  charpos = adjust_for_invis_intang (charpos, 0, 1, 1);
> 
> However, 'intanglible text property is, in fact, not necessary to
> trigger the adjustment.

Didn't we establish that set_point_both is not relevant to the
behavior you see, and in fact is a deprecated feature on its way out?

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Further, `set_point_both' contains the following comments:
>> ...
>
> Didn't we establish that set_point_both is not relevant to the
> behavior you see, and in fact is a deprecated feature on its way out?

True. The comments are still confusing though.
Of course, if that code is going to be removed soon, there is no point
fixing the comments.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Ihor Radchenko [2023-07-21 07:50:55] wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> So, there is no problem with this section of the manual. The problem is
>>> that manual does not describe the behavior with invisible text
>>> boundaries I was trying to figure out.
>>
>> Which aspects of the behavior you describe are not mentioned in the
>> manual.  It does refer to invisible text.
>
> It talks about "in the middle of a sequence .. that ... is invisible"
>
>     Emacs cannot display the cursor when point is in the middle of a
>     sequence of text that has the ‘display’ or ‘composition’ property, or is
>     invisible.  Therefore, after a command finishes and returns to the
>     command loop, if point is within such a sequence, the command loop
>     normally moves point to the edge of the sequence, making this sequence
>     effectively intangible.
>
> The situation I encountered is when point ends up not in the middle, but
> already on the edge. So, Emacs moving point to _other_ edge was
> surprising.

How 'bout the rephrasing below?


        Stefan


diff --git a/doc/lispref/commands.texi b/doc/lispref/commands.texi
index 037f42124cc..ca009d97b39 100644
--- a/doc/lispref/commands.texi
+++ b/doc/lispref/commands.texi
@@ -1253,12 +1253,13 @@ Adjusting Point
 @cindex @code{display} property, and point display
 @cindex @code{composition} property, and point display
 
-  Emacs cannot display the cursor when point is in the middle of a
-sequence of text that has the @code{display} or @code{composition}
-property, or is invisible.  Therefore, after a command finishes and
-returns to the command loop, if point is within such a sequence, the
-command loop normally moves point to the edge of the sequence, making this
-sequence effectively intangible.
+  When point is in the middle of a sequence of text that has the
+@code{display} or @code{composition} property, or is invisible, Emacs
+there can be several buffer positions that result in the cursor being
+displayed at same place.  Therefore, after a command finishes and
+returns to the command loop, if point is in such a sequence, the
+command loop normally moves point to one of the two edges of the
+sequence, making this sequence effectively intangible.
 
   A command can inhibit this feature by setting the variable
 @code{disable-point-adjustment}:

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  64724@debbugs.gnu.org
> Date: Fri, 21 Jul 2023 10:58:11 -0400
> 
> -  Emacs cannot display the cursor when point is in the middle of a
> -sequence of text that has the @code{display} or @code{composition}
> -property, or is invisible.  Therefore, after a command finishes and
> -returns to the command loop, if point is within such a sequence, the
> -command loop normally moves point to the edge of the sequence, making this
> -sequence effectively intangible.
> +  When point is in the middle of a sequence of text that has the
> +@code{display} or @code{composition} property, or is invisible, Emacs
                                                                   ^^^^^
That "Emacs" is redundant.

> +there can be several buffer positions that result in the cursor being
> +displayed at same place.
             ^^^^^^^^^^^^^
I'd say "at the same place on screen".

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

2nd try!


        Stefan


diff --git a/doc/lispref/commands.texi b/doc/lispref/commands.texi
index 037f42124cc..ae379c664d4 100644
--- a/doc/lispref/commands.texi
+++ b/doc/lispref/commands.texi
@@ -1253,12 +1253,13 @@ Adjusting Point
 @cindex @code{display} property, and point display
 @cindex @code{composition} property, and point display
 
-  Emacs cannot display the cursor when point is in the middle of a
-sequence of text that has the @code{display} or @code{composition}
-property, or is invisible.  Therefore, after a command finishes and
-returns to the command loop, if point is within such a sequence, the
-command loop normally moves point to the edge of the sequence, making this
-sequence effectively intangible.
+  When a sequence of text has the @code{display} or @code{composition}
+property, or is invisible, there can be several buffer positions that
+result in the cursor being displayed at same place on the screen.
+Therefore, after a command finishes and returns to the command loop,
+if point is in such a sequence, the command loop normally moves point
+to one of the two edges of the sequence, making this sequence
+effectively intangible.
 
   A command can inhibit this feature by setting the variable
 @code{disable-point-adjustment}:

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> 2nd try!

Thanks!
This sounds better. Although, I would also specifically mention that
adjustment can happen even when point is already at the edge. To make
sure that it is 100% clear.

And should the detail about stickiness be documented? Or do you want to
leave this as unspecified behaviour?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> +  When point is in the middle of a sequence of text that has the

I still find "in the middle of a sequence" confusing.

> +@code{display} or @code{composition} property, or is invisible, Emacs
> +there can be several buffer positions that result in the cursor being
> +displayed at same place.  Therefore, after a command finishes and
> +returns to the command loop, if point is in such a sequence, the
> +command loop normally moves point to one of the two edges of the
> +sequence, making this sequence effectively intangible.
>  
>    A command can inhibit this feature by setting the variable
>  @code{disable-point-adjustment}:
>

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Let me describe the full sequence how I arrived to that manual page:
>
> 1. We had a situation where `backward-word' moved further than expected:
>
>    <invisible>=</invisible>word<point>
>    M-b
>    <point>=word
>    
>    This was despite "=" not considered to be a word constituent in
>    syntax table.
>
> 2. I noticed that this behavior is related to point movement and
>    invisible text and searched manual index for "point", "invisible".
>    The only match was "22.6 Adjusting Point After Commands"
>    I assumed that it is describing all the cases related to interaction
>    between point and invisible text.
>
> 3. I've read that section of the manual and did not find anything about
>    point ending up not inside, but at the boundary of invisible text.

The boundary is usually considered as "inside" for this specific case.
More specifically, in your above example, at the end of the command
point ends up being *displayed* at a position that could correspond to
various buffer positions, and the 22.6 point-adjustment will thus try
and choose which of those buffer positions to use (based on the
stickiness, as well as the direction of movement).

> 4. I reached to sources and tried to track down the cause, eventually
>    arriving to `set_point_both'.

Hmm... I'd be very surprised if `set_point_both` is the cause:
`inhibit-point-motion-hooks` should be non-nil, so that code will simply
be skipped.

If `inhibit-point-motion-hooks` is nil, please send me the recipe so
I can try and change the caller that set this var.


        Stefan

bug#64724: 30.0.50; Inconsistency between manual, comments in the code, and implementation of point adjustment

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> 3. I've read that section of the manual and did not find anything about
>>    point ending up not inside, but at the boundary of invisible text.
>
> The boundary is usually considered as "inside" for this specific case.

I did not expect this. It should be clarified, if true. Because the same
paragraph says that point will be moved to the boundary. Combined with
motion to _another_ boundary, I see not how I can understand what is
happening from the info node.

>> 4. I reached to sources and tried to track down the cause, eventually
>>    arriving to `set_point_both'.
>
> Hmm... I'd be very surprised if `set_point_both` is the cause:
> `inhibit-point-motion-hooks` should be non-nil, so that code will simply
> be skipped.
>
> If `inhibit-point-motion-hooks` is nil, please send me the recipe so
> I can try and change the caller that set this var.

You are right.
I now tried

   =word=<point>

M-: (progn (backward-word) (message "%s" (point)))

The message displays point at "w". Yet, when I follow this up with
M-: (point), the output is at "=".

The actual cause if probably `adjust_point_for_property'. Although I do
not fully understand how it works.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>