Re: invisible

Re: invisible

[Stephen Berman]

On Sat, 28 Feb 2004 20:24:44 -0600 (CST) Luc Teirlinck
<teirllm@dms.auburn.edu> wrote: 

> ===File ~/invlines==========================================
> line1
> line2
> line3
> line4
> line5
> line6
> line7
> ============================================================
>
> No trailing whitespace but with final newline.
>
> Visit using emacs -q
>
> (put-text-property 6 7 `invisible t)
> (put-text-property 12 13 `invisible t)
> (put-text-property 18 19 `invisible t)
> (put-text-property 24 25 `invisible t)
> (put-text-property 30 31 `invisible t)
>
> Result:
>
> line1line2line3line4line5line6
> line7
>
> line-move-ignore-invisible is now nil.
>
> With point at beginning of buffer keep doing C-n.  Everything goes at
> expected.  We arrive at the end of the buffer, the newline after line7.
>
> C-p beginning of line7. OK.
> C-p beginning of line6.  OK.
> C-p beginning of line4.  Why did we skip line5?
> C-p beginning of line2.  Skipped line3.

This behavior still exists in GNU Emacs 23.0.50.4 (i686-pc-linux-gnu,
GTK+ Version 2.12.0) of 2007-11-09 on escher.[1]  In addition, starting
with the cursor at (point-min), the line number indicator in the mode
line displays L1, and it continues to display L1 as you advance either
by C-n or by C-f until it reaches the `l' of `line7', then it changes to
L7.  From there if you type C-b moving point to just after `6', the mode
line displays L6.  Now continuing backwards either by C-p or C-b the
mode line usually continues to display L6, even at (point-min).  Typing
C-b again here then changes the display to L1.

Steve Berman

Footnotes: 
[1] But the behavior seems to be unstable: sometimes line2 is skipped
advancing by C-n, sometimes line6 is skipped receding by C-p (then the
mode line displays L5 and doesn't change as you continue to
(point-min)).

Re: invisible

[Stephen Berman]

RMS requested that I post a reminder about the following.  I append to
it a status report of a related bug report.

On Fri, 09 Nov 2007 22:59:08 +0100 Stephen Berman <Stephen.Berman@gmx.net> wrote:

> On Sat, 28 Feb 2004 20:24:44 -0600 (CST) Luc Teirlinck
> <teirllm@dms.auburn.edu> wrote: 
>
>> ===File ~/invlines==========================================
>> line1
>> line2
>> line3
>> line4
>> line5
>> line6
>> line7
>> ============================================================
>>
>> No trailing whitespace but with final newline.
>>
>> Visit using emacs -q
>>
>> (put-text-property 6 7 `invisible t)
>> (put-text-property 12 13 `invisible t)
>> (put-text-property 18 19 `invisible t)
>> (put-text-property 24 25 `invisible t)
>> (put-text-property 30 31 `invisible t)
>>
>> Result:
>>
>> line1line2line3line4line5line6
>> line7
>>
>> line-move-ignore-invisible is now nil.
>>
>> With point at beginning of buffer keep doing C-n.  Everything goes at
>> expected.  We arrive at the end of the buffer, the newline after line7.
>>
>> C-p beginning of line7. OK.
>> C-p beginning of line6.  OK.
>> C-p beginning of line4.  Why did we skip line5?
>> C-p beginning of line2.  Skipped line3.
>
> This behavior still exists in GNU Emacs 23.0.50.4 (i686-pc-linux-gnu,
> GTK+ Version 2.12.0) of 2007-11-09 on escher.[1]  In addition, starting
> with the cursor at (point-min), the line number indicator in the mode
> line displays L1, and it continues to display L1 as you advance either
> by C-n or by C-f until it reaches the `l' of `line7', then it changes to
> L7.  From there if you type C-b moving point to just after `6', the mode
> line displays L6.  Now continuing backwards either by C-p or C-b the
> mode line usually continues to display L6, even at (point-min).  Typing
> C-b again here then changes the display to L1.
>
> Steve Berman
>
> Footnotes: 
> [1] But the behavior seems to be unstable: sometimes line2 is skipped
> advancing by C-n, sometimes line6 is skipped receding by C-p (then the
> mode line displays L5 and doesn't change as you continue to
> (point-min)).

=======================================================================
Here's a status report of a related bug report.  I tested the following
with -Q in GNU Emacs 23.0.50.2 (i686-pc-linux-gnu, GTK+ Version 2.12.0)
of 2007-11-13 on escher.

On Fri, 29 Oct 2004 12:07:10 +0900 (JST) Kazu Yamamoto (山本和彦)
<kazu@iijlab.net> wrote: 

> Hello,
>
> This is the third trial to ask to fix bugs related to the "invisible"
> property. I'm really suffering from these bugs.
>
> Consider the following content (without invisible text) of a buffer:
>
> ---
> line1
> line2
> line3
> line4
> ---
>
> At the beginning of line N, C-n goes to the beginning of line N+1.
> At the beginning of line N, C-p goes to the beginning of line N-1.
> At the end of line N, C-n goes to the end of line N+1.
> At the end of line N, C-p goes to the end of line N-1.
>
> This is the right behavior.
>
> Then put the invisible property onto line 2 including "\n".
>
> ---
> line1
> line3
> line4
> ---
>
> (1) If line-move-ignore-invisible is nil:
>
> At the beginning of line 3, C-n goes to "i" in line 3.
> 	(should go to the beginning of line 4)

The undesired behavior still obtains.

> At the end of line 3, C-p goes to the beginning of line 3.
> 	(should go to the end of line 2)

In current development Emacs, the cursor stays put, i.e., C-p is a no-op
here; the same goes for every position in line 3, except the beginning
of the line: here C-p goes to the beginning of line 1 (the parenthetical
comment doesn't make sense, unless it is a typo for line 1, or maybe he
means the mode line should display L2 instead of L3, even though the
cursor stays put).

> At the beginning of line 3, C-e goes to "i" in line 3.
> 	(should go to the end of line 3)

In current development Emacs, the cursor goes to the end of line 1
(anywhere else on line 3, C-e goes to the end of line 3).  On the other
hand, C-a at the beginning of line 3 goes to "i" in line 3 (elsewhere it
goes to the beginning of the line, so that repeatedly typing C-a
starting from the beginning of line 3 makes the cursor oscillate between
the first and second columns of the line).

> (2) If line-move-ignore-invisible is t:
>
> At the end of line 1, C-n goes to the beginning of line 3.
> 	(should go to the end of line 3)
> At the end of line 4, C-p goes to the beginning of line 3.
> 	(should go to the end of line 3)
> At the beginning of the line 3, C-e goes to "i" in line 3.
> 	(should go to the end of line 3)

In current development Emacs, the desired behavior (expressed in the
parentheses) obtains.

> I guess the best solution is removing the line-move-ignore-invisible
> variable and behaving as if the invisible text does not exist.
>
> If this is not possible, Emacs should behave as if the invisible text
> does not exist when line-move-ignore-invisible is non-nil.
>
> --Kazu Yamamoto

Steve Berman

Re: invisible

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 3212 bytes --]

 >>>(put-text-property 6 7 `invisible t)
 >>>(put-text-property 12 13 `invisible t)
 >>>(put-text-property 18 19 `invisible t)
 >>>(put-text-property 24 25 `invisible t)
 >>>(put-text-property 30 31 `invisible t)

These will cause problems with `forward-line'.

I'd propose to use

(add-text-properties 6 7 '(invisible t rear-nonsticky t))
(add-text-properties 12 13 '(invisible t rear-nonsticky t))
(add-text-properties 18 19 '(invisible t rear-nonsticky t))
(add-text-properties 24 25 '(invisible t rear-nonsticky t))
(add-text-properties 30 31 '(invisible t rear-nonsticky t))

instead.

 >>>C-p beginning of line4.  Why did we skip line5?
 >>>C-p beginning of line2.  Skipped line3.

See above.  Verify by running (forward-line -1) on these.  Your results
may differ according to the value of `track-eol'.

 >>This behavior still exists in GNU Emacs 23.0.50.4 (i686-pc-linux-gnu,
 >>GTK+ Version 2.12.0) of 2007-11-09 on escher.[1]  In addition, starting
 >>with the cursor at (point-min), the line number indicator in the mode
 >>line displays L1, and it continues to display L1 as you advance either
 >>by C-n or by C-f until it reaches the `l' of `line7', then it changes to
 >>L7.  From there if you type C-b moving point to just after `6', the mode
 >>line displays L6.  Now continuing backwards either by C-p or C-b the
 >>mode line usually continues to display L6, even at (point-min).  Typing
 >>C-b again here then changes the display to L1.

We have been discussing this at the beginning of this year but I don't
recall the thread.  I seem to recall that Emacs tries to optimize line
number calculations as long as the buffer is not modified.

 > In current development Emacs, the cursor stays put, i.e., C-p is a no-op
 > here; the same goes for every position in line 3, except the beginning
 > of the line: here C-p goes to the beginning of line 1 (the parenthetical
 > comment doesn't make sense, unless it is a typo for line 1, or maybe he
 > means the mode line should display L2 instead of L3, even though the
 > cursor stays put).

This seems like a bug in `line-move-finish', please try the attached
patch (untested).

 >>At the beginning of line 3, C-e goes to "i" in line 3.
 >>	(should go to the end of line 3)
 >
 >
 > In current development Emacs, the cursor goes to the end of line 1
 > (anywhere else on line 3, C-e goes to the end of line 3).  On the other
 > hand, C-a at the beginning of line 3 goes to "i" in line 3 (elsewhere it
 > goes to the beginning of the line, so that repeatedly typing C-a
 > starting from the beginning of line 3 makes the cursor oscillate between
 > the first and second columns of the line).
 >
 >
 >>(2) If line-move-ignore-invisible is t:
 >>
 >>At the end of line 1, C-n goes to the beginning of line 3.
 >>	(should go to the end of line 3)
 >>At the end of line 4, C-p goes to the beginning of line 3.
 >>	(should go to the end of line 3)
 >>At the beginning of the line 3, C-e goes to "i" in line 3.
 >>	(should go to the end of line 3)

There's likely also a bug in `line-move-to-column': `move-to-column'
apparently skips invisible text (I didn't find any documentation for
that).  I tried to fix it in a really awkward way in the same patch.

[-- Attachment #2: simple.patch --]
[-- Type: text/plain, Size: 1681 bytes --]

*** simple.el.~1.888.~	Sat Nov 10 09:23:20 2007
--- simple.el	Sun Nov 18 16:02:42 2007
***************
*** 3862,3868 ****
  	     (save-excursion
  	       ;; Like end-of-line but ignores fields.
  	       (skip-chars-forward "^\n")
! 	       (while (and (not (eobp)) (invisible-p (point)))
  		 (goto-char (next-char-property-change (point)))
  		 (skip-chars-forward "^\n"))
  	       (point))))
--- 3862,3869 ----
  	     (save-excursion
  	       ;; Like end-of-line but ignores fields.
  	       (skip-chars-forward "^\n")
! 	       (while (and line-move-ignore-invisible
! 			   (not (eobp)) (invisible-p (point)))
  		 (goto-char (next-char-property-change (point)))
  		 (skip-chars-forward "^\n"))
  	       (point))))
***************
*** 3940,3948 ****
  This function works only in certain cases,
  because what we really need is for `move-to-column'
  and `current-column' to be able to ignore invisible text."
!   (if (zerop col)
!       (beginning-of-line)
      (move-to-column col))

    (when (and line-move-ignore-invisible
  	     (not (bolp)) (invisible-p (1- (point))))
--- 3941,3956 ----
  This function works only in certain cases,
  because what we really need is for `move-to-column'
  and `current-column' to be able to ignore invisible text."
!   (cond
!    ((zerop col)
!     (beginning-of-line))
!    (line-move-ignore-invisible
      (move-to-column col))
+    (t
+     ;; Tedious.
+     (save-restriction
+       (narrow-to-region (line-beginning-position) (line-end-position))
+       (move-to-column col))))

    (when (and line-move-ignore-invisible
  	     (not (bolp)) (invisible-p (1- (point))))

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: invisible

[Stephen Berman]

Sorry for the late response.

On Sun, 18 Nov 2007 16:27:09 +0100 martin rudalics <rudalics@gmx.at> wrote:

>>>>(put-text-property 6 7 `invisible t)
>>>>(put-text-property 12 13 `invisible t)
>>>>(put-text-property 18 19 `invisible t)
>>>>(put-text-property 24 25 `invisible t)
>>>>(put-text-property 30 31 `invisible t)
>
> These will cause problems with `forward-line'.
>
> I'd propose to use
>
> (add-text-properties 6 7 '(invisible t rear-nonsticky t))
> (add-text-properties 12 13 '(invisible t rear-nonsticky t))
> (add-text-properties 18 19 '(invisible t rear-nonsticky t))
> (add-text-properties 24 25 '(invisible t rear-nonsticky t))
> (add-text-properties 30 31 '(invisible t rear-nonsticky t))
>
> instead.
>
>>>>C-p beginning of line4.  Why did we skip line5?
>>>>C-p beginning of line2.  Skipped line3.
>
> See above.  Verify by running (forward-line -1) on these.  Your results
> may differ according to the value of `track-eol'.

With the rear-nonsticky property set to t C-p does not skip lines 5 and
3, i.e. it DTRT.  With (forward-line -1) there is also no line skipping,
but this is also the case without setting the rear-nonsticky property to
t, i.e., I see no difference, in contrast to C-p.  The value of
track-eol is nil (the default).  So I am not sure what problems you
mean.

>>>This behavior still exists in GNU Emacs 23.0.50.4 (i686-pc-linux-gnu,
>>>GTK+ Version 2.12.0) of 2007-11-09 on escher.[1]  In addition, starting
>>>with the cursor at (point-min), the line number indicator in the mode
>>>line displays L1, and it continues to display L1 as you advance either
>>>by C-n or by C-f until it reaches the `l' of `line7', then it changes to
>>>L7.  From there if you type C-b moving point to just after `6', the mode
>>>line displays L6.  Now continuing backwards either by C-p or C-b the
>>>mode line usually continues to display L6, even at (point-min).  Typing
>>>C-b again here then changes the display to L1.
>
> We have been discussing this at the beginning of this year but I don't
> recall the thread.  I seem to recall that Emacs tries to optimize line
> number calculations as long as the buffer is not modified.

I don't recall this discussion, but in any case Emacs normally does
update the line number display when the cursor moves to another line,
which does not involve buffer modification, so I don't see why
invisibility should make a difference here.

>> In current development Emacs, the cursor stays put, i.e., C-p is a no-op
>> here; the same goes for every position in line 3, except the beginning
>> of the line: here C-p goes to the beginning of line 1 (the parenthetical
>> comment doesn't make sense, unless it is a typo for line 1, or maybe he
>> means the mode line should display L2 instead of L3, even though the
>> cursor stays put).
>
> This seems like a bug in `line-move-finish', please try the attached
> patch (untested).

With your patch typing C-p, with the cursor at any position in line 3
but the beginning of the line, moves the cursor to the beginning of line
3, i.e., does the same thing as C-a.  I think I would have expected it
to put the cursor on line 1.  (At the beginning of line 3 the behavior
is as before, C-p goes to the beginning of line 1.)

>>>At the beginning of line 3, C-e goes to "i" in line 3.
>>>	(should go to the end of line 3)
>>
>>
>> In current development Emacs, the cursor goes to the end of line 1
>> (anywhere else on line 3, C-e goes to the end of line 3).  On the other
>> hand, C-a at the beginning of line 3 goes to "i" in line 3 (elsewhere it
>> goes to the beginning of the line, so that repeatedly typing C-a
>> starting from the beginning of line 3 makes the cursor oscillate between
>> the first and second columns of the line).
>>
>>
>>>(2) If line-move-ignore-invisible is t:
>>>
>>>At the end of line 1, C-n goes to the beginning of line 3.
>>>	(should go to the end of line 3)
>>>At the end of line 4, C-p goes to the beginning of line 3.
>>>	(should go to the end of line 3)
>>>At the beginning of the line 3, C-e goes to "i" in line 3.
>>>	(should go to the end of line 3)
>
> There's likely also a bug in `line-move-to-column': `move-to-column'
> apparently skips invisible text (I didn't find any documentation for
> that).  I tried to fix it in a really awkward way in the same patch.

Is this patch intended to fix the behavior I described above (2)?  For
that case, if I do as you suggested above and set both invisible and
rear-nonsticky to t, then no patch is needed: with these properties both
C-e and C-a DTRT at all positions.  But leaving rear-nonsticky nil, the
misbehavior remains even with your patch.

Summarizing, it appears that rear-nonsticky should be set to t when a
position is given the invisible property.  In particular,
facemenu-set-invisible should be changed accordingly, otherwise the menu
choice Edit->Text Properties->Special Properties->Invisible admits the
observed motion misbehavior.  (But I don't know if there are other cases
where facemenu-set-invisible should leave the invisible position
rear-sticky.)  This still does not fix the problem with C-p at positions
other than the beginning of the line after an invisible line, but your
patch for that also results in unexpected behavior.

Steve Berman

Re: invisible

[martin rudalics]

 > With (forward-line -1) there is also no line skipping,
 > but this is also the case without setting the rear-nonsticky property to
 > t, i.e., I see no difference, in contrast to C-p.  The value of
 > track-eol is nil (the default).  So I am not sure what problems you
 > mean.

Assigning the rear-sticky property affects where `point' is set by
scan_newline which does the line-end serching in `forward-line'.  Not
setting the rear-nonsticky property skips the line ends with the
invisible property, at least on my system.  Since `previous-line' and
`next-line' both rely on `forward-line' to skip lines, I can't imagine
how you do not see the problem with the latter.  Please try again.

 > I don't recall this discussion, but in any case Emacs normally does
 > update the line number display when the cursor moves to another line,
 > which does not involve buffer modification, so I don't see why
 > invisibility should make a difference here.

My memory was bad, have a look at the thread starting here:

http://lists.gnu.org/archive/html/emacs-devel/2006-02/msg00338.html

 >>In current development Emacs, the cursor stays put, i.e., C-p is a no-op
 >>>here; the same goes for every position in line 3, except the beginning
 >>>of the line: here C-p goes to the beginning of line 1 (the parenthetical
 >>>comment doesn't make sense, unless it is a typo for line 1, or maybe he
 >>>means the mode line should display L2 instead of L3, even though the
 >>>cursor stays put).
 >>
 >>This seems like a bug in `line-move-finish', please try the attached
 >>patch (untested).
 >
 >
 > With your patch typing C-p, with the cursor at any position in line 3
 > but the beginning of the line, moves the cursor to the beginning of line
 > 3, i.e., does the same thing as C-a.  I think I would have expected it
 > to put the cursor on line 1.  (At the beginning of line 3 the behavior
 > is as before, C-p goes to the beginning of line 1.)

Did you assign the rear-nonsticky property?  It won't work without that
property due to the behavior of the point setting mechanism.

 > Summarizing, it appears that rear-nonsticky should be set to t when a
 > position is given the invisible property.  In particular,
 > facemenu-set-invisible should be changed accordingly, otherwise the menu
 > choice Edit->Text Properties->Special Properties->Invisible admits the
 > observed motion misbehavior.  (But I don't know if there are other cases
 > where facemenu-set-invisible should leave the invisible position
 > rear-sticky.)  This still does not fix the problem with C-p at positions
 > other than the beginning of the line after an invisible line, but your
 > patch for that also results in unexpected behavior.

Setting rear-nonsticky is _not_ sufficient, you have to do something as
in my patch in order to handle `line-move-ignore-invisible' correctly.
Please try again (1) with the property set, (2) with and without my
patch applied, and (3) with both values for `line-move-ignore-invisible'
and `track-eol'.

Re: invisible

[Stephen Berman]

On Fri, 23 Nov 2007 15:25:19 +0100 martin rudalics <rudalics@gmx.at> wrote:

>> With (forward-line -1) there is also no line skipping,
>> but this is also the case without setting the rear-nonsticky property to
>> t, i.e., I see no difference, in contrast to C-p.  The value of
>> track-eol is nil (the default).  So I am not sure what problems you
>> mean.
>
> Assigning the rear-sticky property affects where `point' is set by
> scan_newline which does the line-end serching in `forward-line'.  Not
> setting the rear-nonsticky property skips the line ends with the
> invisible property, at least on my system.  Since `previous-line' and
> `next-line' both rely on `forward-line' to skip lines, I can't imagine
> how you do not see the problem with the latter.  Please try again.

I cannot see a difference with forward-line.  I did this:

1. I inserted the following lines into each of the buffers a and b:
line1
line2
line3
line4
line5
line6
line7

2. In buffer a I did 
M-: (progn (put-text-property 6 7 `invisible t)
           (put-text-property 12 13 `invisible t)
           (put-text-property 18 19 `invisible t)
           (put-text-property 24 25 `invisible t)
           (put-text-property 30 31 `invisible t))


3. In buffer b I did
M-: (progn (add-text-properties 6 7 '(invisible t rear-nonsticky t))
           (add-text-properties 12 13 '(invisible t rear-nonsticky t))
           (add-text-properties 18 19 '(invisible t rear-nonsticky t))
           (add-text-properties 24 25 '(invisible t rear-nonsticky t))
           (add-text-properties 30 31 '(invisible t rear-nonsticky t)))

4. Via Customize I toggled line-move-ignore-invisible off and set it for
the current session.  I left track-eol at its default value nil.

5. In buffer a with point at (point-max), repeatedly typing C-p goes
like this, with `^' marking successive positions of the cursor:
line1line2line3line4line5line6
^    ^         ^         ^
line7
^

6. In buffer b with point at (point-max), repeatedly typing C-p goes
like this:
line1line2line3line4line5line6
^    ^   ^     ^    ^    ^
line7
^

7. In both buffer a and buffer b with point at (point-max), repeatedly
typing 'M-: (forward-line -1)' goes like in 6:
line1line2line3line4line5line6
^    ^         ^         ^
line7
^

Set track-eol to t does not make any difference for 5, 6, 7.  Setting
line-move-ignore-invisible to t makes C-p move point from the beginning
of line 7 to the beginning of line 1 in both buffers a and b, while 'M-:
(forward-line -1)' continues to behave as in 7.

>>>In current development Emacs, the cursor stays put, i.e., C-p is a no-op
>>>>here; the same goes for every position in line 3, except the beginning
>>>>of the line: here C-p goes to the beginning of line 1 (the parenthetical
>>>>comment doesn't make sense, unless it is a typo for line 1, or maybe he
>>>>means the mode line should display L2 instead of L3, even though the
>>>>cursor stays put).
>>>
>>>This seems like a bug in `line-move-finish', please try the attached
>>>patch (untested).
>>
>>
>> With your patch typing C-p, with the cursor at any position in line 3
>> but the beginning of the line, moves the cursor to the beginning of line
>> 3, i.e., does the same thing as C-a.  I think I would have expected it
>> to put the cursor on line 1.  (At the beginning of line 3 the behavior
>> is as before, C-p goes to the beginning of line 1.)
>
> Did you assign the rear-nonsticky property?  It won't work without that
> property due to the behavior of the point setting mechanism.

I inserted the following lines into buffer c:

line1
line2
line3
line4

and typed `M-: (add-text-properties 7 13 '(invisible t rear-nonsticky
t))'.  Then I evalled line-move-finish with your patch applied.  With
line-move-ignore-invisible set to nil I get the behavior I described
(C-p on line 3 behaving like C-a except at the beginning of the line).
(The value of track-eol makes no difference here.)

>> Summarizing, it appears that rear-nonsticky should be set to t when a
>> position is given the invisible property.  In particular,
>> facemenu-set-invisible should be changed accordingly, otherwise the menu
>> choice Edit->Text Properties->Special Properties->Invisible admits the
>> observed motion misbehavior.  (But I don't know if there are other cases
>> where facemenu-set-invisible should leave the invisible position
>> rear-sticky.)  This still does not fix the problem with C-p at positions
>> other than the beginning of the line after an invisible line, but your
>> patch for that also results in unexpected behavior.
>
> Setting rear-nonsticky is _not_ sufficient, you have to do something as
> in my patch in order to handle `line-move-ignore-invisible' correctly.
> Please try again (1) with the property set, (2) with and without my
> patch applied, and (3) with both values for `line-move-ignore-invisible'
> and `track-eol'.

I hope my descriptions above are clear enough.

On Fri, 23 Nov 2007 15:37:28 +0100 martin rudalics <rudalics@gmx.at> wrote:

>> Setting rear-nonsticky is _not_ sufficient, you have to do something as
>> in my patch in order to handle `line-move-ignore-invisible' correctly.
>> Please try again (1) with the property set, (2) with and without my
>> patch applied, and (3) with both values for `line-move-ignore-invisible'
>> and `track-eol'.
>
> Forgot to say (4) starting C-p / C-n at line-beginnings and -ends in
> order to test the "track-eol" behavior.

The only difference I see is in buffer a when the cursor is at the end
of line 7 (actually, anywhere but at the beginning), then typing C-p
repeatedly does not skip any lines.  This is regardless of the setting
of track-eol.

Steve Berman

Re: invisible

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 1386 bytes --]

 > I cannot see a difference with forward-line.  I did this:
...
 > 5. In buffer a with point at (point-max), repeatedly typing C-p goes
 > like this, with `^' marking successive positions of the cursor:
 > line1line2line3line4line5line6
 > ^    ^         ^         ^
 > line7
 > ^
 >
 > 6. In buffer b with point at (point-max), repeatedly typing C-p goes
 > like this:
 > line1line2line3line4line5line6
 > ^    ^   ^     ^    ^    ^
 > line7
 > ^
 >
 > 7. In both buffer a and buffer b with point at (point-max), repeatedly
 > typing 'M-: (forward-line -1)' goes like in 6:
 > line1line2line3line4line5line6
 > ^    ^         ^         ^
 > line7
 > ^

I don't understand: You say it "goes like in 6" but according to your
"^" indicators you skip line5 and line3 in "7.".  Anyway, my Emacs has
for (forward-line -1) the behavior you sketched for cases 5 and 6: line
5 and line3 are skipped when the rear-nonsticky property is not set.

 > The only difference I see is in buffer a when the cursor is at the end
 > of line 7 (actually, anywhere but at the beginning), then typing C-p
 > repeatedly does not skip any lines.  This is regardless of the setting
 > of track-eol.

Please experiment with the attached text file.  You probably need a
visible line before and/or after the lines with the invisible endings.
And it's easier to move the cursor to the end of a line before moving.

[-- Attachment #2: lmii.txt --]
[-- Type: text/plain, Size: 568 bytes --]

line1
line2
line3
line4
line5
line6
line7

(progn
  (add-text-properties 12 13 '(invisible t))
  (add-text-properties 18 19 '(invisible t))
  (add-text-properties 24 25 '(invisible t))
  (add-text-properties 30 31 '(invisible t))
  (setq line-move-ignore-invisible nil))

(progn
  (add-text-properties 12 13 '(invisible t rear-nonsticky t))
  (add-text-properties 18 19 '(invisible t rear-nonsticky t))
  (add-text-properties 24 25 '(invisible t rear-nonsticky t))
  (add-text-properties 30 31 '(invisible t rear-nonsticky t))
  (setq line-move-ignore-invisible nil))

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: invisible

[Stephen Berman]

On Fri, 23 Nov 2007 20:59:41 +0100 martin rudalics <rudalics@gmx.at> wrote:

>> I cannot see a difference with forward-line.  I did this:
> ...
>> 5. In buffer a with point at (point-max), repeatedly typing C-p goes
>> like this, with `^' marking successive positions of the cursor:
>> line1line2line3line4line5line6
>> ^    ^         ^         ^
>> line7
>> ^
>>
>> 6. In buffer b with point at (point-max), repeatedly typing C-p goes
>> like this:
>> line1line2line3line4line5line6
>> ^    ^   ^     ^    ^    ^
>> line7
>> ^
>>
>> 7. In both buffer a and buffer b with point at (point-max), repeatedly
>> typing 'M-: (forward-line -1)' goes like in 6:
>> line1line2line3line4line5line6
>> ^    ^         ^         ^
>> line7
>> ^
>
> I don't understand: You say it "goes like in 6" but according to your
> "^" indicators you skip line5 and line3 in "7.".  Anyway, my Emacs has
> for (forward-line -1) the behavior you sketched for cases 5 and 6: line
> 5 and line3 are skipped when the rear-nonsticky property is not set.

Oh, sorry, that was a copy and paste error: the cursor indicators in 7
should be the same as in 6.

>> The only difference I see is in buffer a when the cursor is at the end
>> of line 7 (actually, anywhere but at the beginning), then typing C-p
>> repeatedly does not skip any lines.  This is regardless of the setting
>> of track-eol.
>
> Please experiment with the attached text file.  You probably need a
> visible line before and/or after the lines with the invisible endings.
> And it's easier to move the cursor to the end of a line before moving.

It's the same as before: C-p skips lines 5 and 3 with rear-nonsticky
nil, does not skip with rear-nonsticky t, and (forward-line -1) does not
skip regardless of rear-nonstickiness.  I tested on GNU Emacs 23.0.50.1
(i686-pc-linux-gnu, GTK+ Version 2.12.0) of 2007-11-21.  Could the
differences between what you and I see be related to the EOL difference
between MS Windows and Unix?  Can other users of these systems who are
following this thread try these tests and report their observations?

Steve Berman

Re: invisible

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 791 bytes --]

 > It's the same as before: C-p skips lines 5 and 3 with rear-nonsticky
 > nil, does not skip with rear-nonsticky t, and (forward-line -1) does not
 > skip regardless of rear-nonstickiness.  I tested on GNU Emacs 23.0.50.1
 > (i686-pc-linux-gnu, GTK+ Version 2.12.0) of 2007-11-21.  Could the
 > differences between what you and I see be related to the EOL difference
 > between MS Windows and Unix?  Can other users of these systems who are
 > following this thread try these tests and report their observations?

Hmmm... I think I see why you don't see it.  With emacs -Q visit the
attached file, evaluate fl-1 and the first progn, move point before
line7 and do M-x fl-1 twice.  Undo the changes, evaluate the second
progn and do the same movement again.  Can you see the difference now?

[-- Attachment #2: lmii.txt --]
[-- Type: text/plain, Size: 621 bytes --]

line1
line2
line3
line4
line5
line6
line7

(defun fl-1 ()
  (interactive)
  (forward-line -1))

(progn
  (add-text-properties 12 13 '(invisible t))
  (add-text-properties 18 19 '(invisible t))
  (add-text-properties 24 25 '(invisible t))
  (add-text-properties 30 31 '(invisible t))
  (setq line-move-ignore-invisible nil))

(progn
  (add-text-properties 12 13 '(invisible t rear-nonsticky t))
  (add-text-properties 18 19 '(invisible t rear-nonsticky t))
  (add-text-properties 24 25 '(invisible t rear-nonsticky t))
  (add-text-properties 30 31 '(invisible t rear-nonsticky t))
  (setq line-move-ignore-invisible nil))

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: invisible

[Stephen Berman]

On Fri, 23 Nov 2007 22:52:12 +0100 martin rudalics <rudalics@gmx.at> wrote:

>> It's the same as before: C-p skips lines 5 and 3 with rear-nonsticky
>> nil, does not skip with rear-nonsticky t, and (forward-line -1) does not
>> skip regardless of rear-nonstickiness.  I tested on GNU Emacs 23.0.50.1
>> (i686-pc-linux-gnu, GTK+ Version 2.12.0) of 2007-11-21.  Could the
>> differences between what you and I see be related to the EOL difference
>> between MS Windows and Unix?  Can other users of these systems who are
>> following this thread try these tests and report their observations?
>
> Hmmm... I think I see why you don't see it.  With emacs -Q visit the
> attached file, evaluate fl-1 and the first progn, move point before
> line7 and do M-x fl-1 twice.  Undo the changes, evaluate the second
> progn and do the same movement again.  Can you see the difference now?

Yes.  So why does (interactive) have this effect?

Steve Berman

Re: invisible

[martin rudalics]

 > So why does (interactive) have this effect?

I don't know.  I'd have to gdb set_point_both to find out but doing so
is awkward here.  Maybe you could give it a try.  I could imagine that
invisibility and intangibility properties get mixed up.  On the other
hand it might be simply TRT to do for redisplay after an interactive
command.  Miles and Stefan used to work on this but I'm afraid they
won't tell us ...

Meanwhile could you please check my patch for simple.el too?

(1) With emacs -Q visit my txt file, evaluate the second progn, move the
cursor after "line7" and try to do C-p repeatedly.  On my system point
gets stuck after "line5".  Setting `track-eol' to t has point get stuck
after "line6".

(2) With emacs -Q visit my txt file, evaluate the second progn, set
`track-eol' to t, move point after "line1", and hit C-n.  `point' moves
after "line6" instead of after "line2".

Now please apply my patch and check whether it DTRT in these cases.

Re: invisible

[Johan Bockgård]

martin rudalics <rudalics@gmx.at> writes:

>> So why does (interactive) have this effect?
>
> I don't know.  I'd have to gdb set_point_both to find out but doing so
> is awkward here.  Maybe you could give it a try.  I could imagine that
> invisibility and intangibility properties get mixed up.  On the other
> hand it might be simply TRT to do for redisplay after an interactive
> command.  Miles and Stefan used to work on this but I'm afraid they
> won't tell us ...

I haven't been following the thread, but is this what you are talking
about?

    Ordinarily, functions that operate on text or move point do not care
    whether the text is invisible.  The user-level line motion commands
    explicitly ignore invisible newlines if `line-move-ignore-invisible'
    is non-`nil' (the default), but only because they are explicitly
    programmed to do so.

    However, if a command ends with point inside or immediately before
    invisible text, the main editing loop moves point further forward or
    further backward (in the same direction that the command already
    moved it) until that condition is no longer true.

(info "(elisp) Invisible Text")

-- 
Johan Bockgård

Re: invisible

[martin rudalics]

> I haven't been following the thread, but is this what you are talking
> about?
> 
>     Ordinarily, functions that operate on text or move point do not care
>     whether the text is invisible.  The user-level line motion commands
>     explicitly ignore invisible newlines if `line-move-ignore-invisible'
>     is non-`nil' (the default), but only because they are explicitly
>     programmed to do so.
> 
>     However, if a command ends with point inside or immediately before
>     invisible text, the main editing loop moves point further forward or
>     further backward (in the same direction that the command already
>     moved it) until that condition is no longer true.

Yes.  But where in the code does set_point_both base its decision
on whether a "command ends"?

Re: invisible

[martin rudalics]

> Yes.  But where in the code does set_point_both base its decision
> on whether a "command ends"?

Ah, I see.  It's in adjust_point_for_property.

Re: invisible

[Stephen Berman]

On Sat, 24 Nov 2007 10:33:28 +0100 martin rudalics <rudalics@gmx.at> wrote:

>> So why does (interactive) have this effect?
>
> I don't know.  I'd have to gdb set_point_both to find out but doing so
> is awkward here.  Maybe you could give it a try.  I could imagine that
> invisibility and intangibility properties get mixed up.  On the other
> hand it might be simply TRT to do for redisplay after an interactive
> command.  Miles and Stefan used to work on this but I'm afraid they
> won't tell us ...

It seems you have already answered this to your satisfaction; I confess
I still don't get it, but when I have time I'll try gdb and see if that
helps me understand.

> Meanwhile could you please check my patch for simple.el too?
>
> (1) With emacs -Q visit my txt file, evaluate the second progn, move the
> cursor after "line7" and try to do C-p repeatedly.  On my system point
> gets stuck after "line5".  Setting `track-eol' to t has point get stuck
> after "line6".

I reproduced these results.  (At first I understood "after line7" to mean
on the next line and did not get your results.  Then I put the cursor on
line7 immediately to right of "7" and got your results.)

> (2) With emacs -Q visit my txt file, evaluate the second progn, set
> `track-eol' to t, move point after "line1", and hit C-n.  `point' moves
> after "line6" instead of after "line2".

I reproduced this as well.

> Now please apply my patch and check whether it DTRT in these cases.

It DTRT: in case (1) point does not stop moving, in case (2) point moves
to the position after line2.

Do you think facemenu-set-invisible should set rear-nonsticky to t?

Steve Berman

Re: invisible

[martin rudalics]

 > It seems you have already answered this to your satisfaction; I confess
 > I still don't get it, but when I have time I'll try gdb and see if that
 > helps me understand.

I was confused.  What happens is that when Emacs executes an arbitrary
command it adjusts point to assure that it doesn't end up in invisible
or intangible text.  It does this _in addition_ to point adjustment
whenever scanning for a newline.

If you really want to debug something you could try to find out why line
numbers are not updated correctly.  But there's probably an easy
explanation for that, maybe some optimization for scrolling quickly.

 > Do you think facemenu-set-invisible should set rear-nonsticky to t?

Probably.  There's no use typing text you don't see.  The same holds for
intangible and read-only text.  FWIW, these properties should have an
entry in `text-property-default-nonsticky' which would solve the problem
in a more general and intuitive fashion.

Re: invisible

[Stefan Monnier]

>> Do you think facemenu-set-invisible should set rear-nonsticky to t?
> Probably.

I don't think it would be significiantly better.  Since the text is
invisible, the user has generally no way to know whether she's inserting
text before or after the invisible text anyway.


        Stefan

Re: invisible

[martin rudalics]

 >>>Do you think facemenu-set-invisible should set rear-nonsticky to t?
 >>
 >>Probably.
 >
 >
 > I don't think it would be significiantly better.  Since the text is
 > invisible, the user has generally no way to know whether she's inserting
 > text before or after the invisible text anyway.

I think Stephen was talking about invisible text followed by visible
text where the user (1) wants to insert text precisely at the beginning
of the visible text and (2) probably wants to see new text while typing.

Re: invisible

[Stefan Monnier]

>>>> Do you think facemenu-set-invisible should set rear-nonsticky to t?
>>> Probably.
>> I don't think it would be significiantly better.  Since the text is
>> invisible, the user has generally no way to know whether she's inserting
>> text before or after the invisible text anyway.

> I think Stephen was talking about invisible text followed by visible
> text where the user (1) wants to insert text precisely at the beginning
> of the visible text and (2) probably wants to see new text while typing.

The point-adjustment already tries to move point to the non-sticky side
of an invisible text so that text inserted is indeed visible.
The rear-nonsticky change proposed above will just make both ends
non-sticky so the point-adjustment will not have a preference and will
sometimes choose one sometimes the other.

In general facemenu-set-invisible can't know whether the user will want
to insert text "precisely at the beginning of the visible text" or
"precisely at the end of the visible text".

This said, I think you're right: the rear-nonsticky property would be
beneficial for the case where the invisible text is shown as an
ellipsis, in which case the user can indeed choose where she inserts
the text.


        Stefan

Re: invisible

[martin rudalics]

 > The point-adjustment already tries to move point to the non-sticky side
 > of an invisible text so that text inserted is indeed visible.

When I do `facemenu-set-invisible' on a region, subsequently inserted
text is invisible.  I have hardly an opinion on that because I didn't
even know about this function before.  But I think inserted text should
go before the invisible text if `point' was at the beginning and after
invisible text if `point' was at the end of the region.

 > The rear-nonsticky change proposed above will just make both ends
 > non-sticky so the point-adjustment will not have a preference and will
 > sometimes choose one sometimes the other.

I agree that would be bad.  Is that the way it behaves now when I set
the rear-nonsticky property?

 > In general facemenu-set-invisible can't know whether the user will want
 > to insert text "precisely at the beginning of the visible text" or
 > "precisely at the end of the visible text".

In general you're right.  I was referring to the effect immediately
after executing that command.

 > This said, I think you're right: the rear-nonsticky property would be
 > beneficial for the case where the invisible text is shown as an
 > ellipsis, in which case the user can indeed choose where she inserts
 > the text.

That would be fine.

Re: invisible

[Stefan Monnier]

>> The rear-nonsticky change proposed above will just make both ends
>> non-sticky so the point-adjustment will not have a preference and will
>> sometimes choose one sometimes the other.

> I agree that would be bad.  Is that the way it behaves now when I set
> the rear-nonsticky property?

Yes.

>> In general facemenu-set-invisible can't know whether the user will want
>> to insert text "precisely at the beginning of the visible text" or
>> "precisely at the end of the visible text".

> In general you're right.  I was referring to the effect immediately
> after executing that command.

I see.

>> This said, I think you're right: the rear-nonsticky property would be
>> beneficial for the case where the invisible text is shown as an
>> ellipsis, in which case the user can indeed choose where she inserts
>> the text.

> That would be fine.

Maybe you want to post this suggestion in a new thread.


        Stefan

Re: invisible

[Stefan Monnier]

> I don't know.  I'd have to gdb set_point_both to find out but doing so
> is awkward here.  Maybe you could give it a try.  I could imagine that
> invisibility and intangibility properties get mixed up.  On the other
> hand it might be simply TRT to do for redisplay after an interactive
> command.  Miles and Stefan used to work on this but I'm afraid they
> won't tell us ...

Indeed.  I don't care about the behavior being right in all conceivable
cases because I consider it simply impossible.  So I have no intention
spending time trying to analyze each and every problematic case
presented in this thread.  No time for that.

If you have a conrete situation where the current behavior is
problematic, firsyt you need to understand how those things work:
there are 2 places where invisibility impacts movement:
- when combined with `intangible', it impacts every single movement of
  `point', including within elisp functions.  This may "do the right
  thing" in some sense, but it tends to break a lot of code, and
  it can be terribly difficult to write code that works in the face of
  intangible properties.  So I highly recommend against the use of
  `intangible' except for those rare cases where it's *really*
  absolutely needed.
- when not combined with `intangible', invisible properties have the
  effect (just like composition and display properties) that after each
  *command* (i..e more or less just after running post-command-hook)
  point is moved outside of the invisible text.  Now since it's done at
  the end of a command (which may have moved point many times in
  arbitrarily complex ways), you can't do it right 100% of the time.
  OTOH this interacts fine with pretty much any elisp code.

Many of the problems you point out have to do with interactions with C-n
and C-p which are surprisingly complex functions.  The interaction
between all three is even worse.

So if you can try and reproduce the problem with only C-n/C-p (i.e. no
intangible text and with non-nil disable-point-adjustement) or with
only intangible text or with only invisible text, that makes it more
likely we can try and fix it.

Also the interaction with those things is sufficiently bad, that there
are many problematic cases.  So if the problem only appears when you mix
those, to have a better chance of seeing your bug fixed, try to make
sure the symptom is really serious: e.g. C-n does move at all (or moves
backward).  And focus on *1* problem at a time.

> Meanwhile could you please check my patch for simple.el too?

Please post it again,


        Stefan

Re: invisible

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 2812 bytes --]

 > If you have a conrete situation where the current behavior is
 > problematic, firsyt you need to understand how those things work:
 > there are 2 places where invisibility impacts movement:
 > - when combined with `intangible', it impacts every single movement of
 >   `point', including within elisp functions.  This may "do the right
 >   thing" in some sense, but it tends to break a lot of code, and
 >   it can be terribly difficult to write code that works in the face of
 >   intangible properties.  So I highly recommend against the use of
 >   `intangible' except for those rare cases where it's *really*
 >   absolutely needed.

Intangible text was never mentioned in this thread.

 > - when not combined with `intangible', invisible properties have the
 >   effect (just like composition and display properties) that after each
 >   *command* (i..e more or less just after running post-command-hook)
 >   point is moved outside of the invisible text.  Now since it's done at
 >   the end of a command (which may have moved point many times in
 >   arbitrarily complex ways), you can't do it right 100% of the time.
 >   OTOH this interacts fine with pretty much any elisp code.
 >
 > Many of the problems you point out have to do with interactions with C-n
 > and C-p which are surprisingly complex functions.  The interaction
 > between all three is even worse.
 >
 > So if you can try and reproduce the problem with only C-n/C-p (i.e. no
 > intangible text and with non-nil disable-point-adjustement) or with
 > only intangible text or with only invisible text, that makes it more
 > likely we can try and fix it.

This thread is about `line-move-ignore-invisible' - an option defined in
simple.el with the following doc-string:

	  *Non-nil means C-n and C-p ignore invisible lines.
	  Outline mode sets this.

Either this option makes sense - then we have to talk about C-n/C-p
_and_ invisible text - or it doesn't.  In the latter case let's remove
the option and the problem is resolved.

 > Also the interaction with those things is sufficiently bad, that there
 > are many problematic cases.  So if the problem only appears when you mix
 > those, to have a better chance of seeing your bug fixed, try to make
 > sure the symptom is really serious: e.g. C-n does move at all (or moves
 > backward).

I never use "those things" hence I don't care about whether these
qualify as "serious symptoms".  I just reacted to a couple of problems
reported by others.  One of these was

	 "the cursor stays put, i.e., C-p is a no-op here"

reported in 2004.  Apparently, it didn't have much chance getting fixed
then.

 > And focus on *1* problem at a time.

I focused on *1* solution at a time.

 >>Meanwhile could you please check my patch for simple.el too?
 >
 >
 > Please post it again,

Attached.

[-- Attachment #2: simple.patch --]
[-- Type: text/plain, Size: 1633 bytes --]

*** simple.el.~1.888.~	Sat Nov 10 09:23:20 2007
--- simple.el	Sun Nov 18 16:02:42 2007
***************
*** 3862,3868 ****
  	     (save-excursion
  	       ;; Like end-of-line but ignores fields.
  	       (skip-chars-forward "^\n")
! 	       (while (and (not (eobp)) (invisible-p (point)))
  		 (goto-char (next-char-property-change (point)))
  		 (skip-chars-forward "^\n"))
  	       (point))))
--- 3862,3869 ----
  	     (save-excursion
  	       ;; Like end-of-line but ignores fields.
  	       (skip-chars-forward "^\n")
! 	       (while (and line-move-ignore-invisible
! 			   (not (eobp)) (invisible-p (point)))
  		 (goto-char (next-char-property-change (point)))
  		 (skip-chars-forward "^\n"))
  	       (point))))
***************
*** 3940,3948 ****
  This function works only in certain cases,
  because what we really need is for `move-to-column'
  and `current-column' to be able to ignore invisible text."
!   (if (zerop col)
!       (beginning-of-line)
      (move-to-column col))

    (when (and line-move-ignore-invisible
  	     (not (bolp)) (invisible-p (1- (point))))
--- 3941,3956 ----
  This function works only in certain cases,
  because what we really need is for `move-to-column'
  and `current-column' to be able to ignore invisible text."
!   (cond
!    ((zerop col)
!     (beginning-of-line))
!    (line-move-ignore-invisible
      (move-to-column col))
+    (t
+     ;; Tedious.
+     (save-restriction
+       (narrow-to-region (line-beginning-position) (line-end-position))
+       (move-to-column col))))

    (when (and line-move-ignore-invisible
  	     (not (bolp)) (invisible-p (1- (point))))

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: invisible

[Stefan Monnier]

>> If you have a conrete situation where the current behavior is
>> problematic, firsyt you need to understand how those things work:
>> there are 2 places where invisibility impacts movement:
>> - when combined with `intangible', it impacts every single movement of
>> `point', including within elisp functions.  This may "do the right
>> thing" in some sense, but it tends to break a lot of code, and
>> it can be terribly difficult to write code that works in the face of
>> intangible properties.  So I highly recommend against the use of
>> `intangible' except for those rare cases where it's *really*
>> absolutely needed.

> Intangible text was never mentioned in this thread.

Great.  (I had no idea because the thread was just too big for
me to follow it)

> This thread is about `line-move-ignore-invisible' - an option defined in
> simple.el with the following doc-string:

> 	  *Non-nil means C-n and C-p ignore invisible lines.
> 	  Outline mode sets this.

> Either this option makes sense - then we have to talk about C-n/C-p
> _and_ invisible text - or it doesn't.  In the latter case let's remove
> the option and the problem is resolved.

It makes sense and we need to keep it.  But it interacts with point
adjustment.  So try and reproduce the bug first with
disable-point-adjustment so as to eliminate a variable.

[ We could also dispense with the variable and use
disable-point-adjustment for it, but then one wouldn't be able to
disable one without disabling the other which is sometimes inconvenient
while debugging.  ]

> I never use "those things" hence I don't care about whether these
> qualify as "serious symptoms".

You use point-adjustment all the time since it's enabled by default.

> I just reacted to a couple of problems
> reported by others.  One of these was

I wasn't ranting against you.  Sorry.  It was against this thread with
its umpteen recipes.

> 	 "the cursor stays put, i.e., C-p is a no-op here"

Great: a clear bug.  Thanks for helping.

> *** simple.el.~1.888.~	Sat Nov 10 09:23:20 2007
> --- simple.el	Sun Nov 18 16:02:42 2007
> ***************
> *** 3862,3868 ****
>   	     (save-excursion
>   	       ;; Like end-of-line but ignores fields.
>   	       (skip-chars-forward "^\n")
> ! 	       (while (and (not (eobp)) (invisible-p (point)))
>   		 (goto-char (next-char-property-change (point)))
>   		 (skip-chars-forward "^\n"))
>   	       (point))))
> --- 3862,3869 ----
>   	     (save-excursion
>   	       ;; Like end-of-line but ignores fields.
>   	       (skip-chars-forward "^\n")
> ! 	       (while (and line-move-ignore-invisible
> ! 			   (not (eobp)) (invisible-p (point)))
>   		 (goto-char (next-char-property-change (point)))
>   		 (skip-chars-forward "^\n"))
>   	       (point))))

Looks good.

> ***************
> *** 3940,3948 ****
>   This function works only in certain cases,
>   because what we really need is for `move-to-column'
>   and `current-column' to be able to ignore invisible text."

This text seems odd: AFAIK they *do* ignore invisible text.  Does it
want to say that they sould ignore the `invisible' text property?

>   This function works only in certain cases,
>   because what we really need is for `move-to-column'
>   and `current-column' to be able to ignore invisible text."
> !   (cond
> !    ((zerop col)
> !     (beginning-of-line))
> !    (line-move-ignore-invisible
>       (move-to-column col))
> +    (t
> +     ;; Tedious.
> +     (save-restriction
> +       (narrow-to-region (line-beginning-position) (line-end-position))
> +       (move-to-column col))))
>   
>     (when (and line-move-ignore-invisible
>   	     (not (bolp)) (invisible-p (1- (point))))

The docstring of line-move-to-column is too vague: it doesn't say what
it intends to do, really: "considering invisibility" is not much help.
If someone knows what it intends to do, really, maybe we can figure out
how to fix it (probably the best fix will be to change the C code).


        Stefan

Re: invisible

[martin rudalics]

 >>Either this option makes sense - then we have to talk about C-n/C-p
 >>_and_ invisible text - or it doesn't.  In the latter case let's remove
 >>the option and the problem is resolved.
 >
 >
 > It makes sense and we need to keep it.  But it interacts with point
 > adjustment.  So try and reproduce the bug first with
 > disable-point-adjustment so as to eliminate a variable.

Both bugs of my test file persist with `global-disable-point-adjustment'
non-nil, maybe Stephen can verify that.

 > [ We could also dispense with the variable and use
 > disable-point-adjustment for it, but then one wouldn't be able to
 > disable one without disabling the other which is sometimes inconvenient
 > while debugging.  ]

I lost you here.  `line-move-ignore-invisible' nil means do not skip
invisible newlines and explicitly should affect only C-n and C-p.
`disable-point-adjustment' is about a lot of other things as well.

 >>I never use "those things" hence I don't care about whether these
 >>qualify as "serious symptoms".
 >
 >
 > You use point-adjustment all the time since it's enabled by default.

Yes.  But the bug is with `line-move-ignore-invisible' which is by
default t and never changed by me.

 >>***************
 >>*** 3940,3948 ****
 >>  This function works only in certain cases,
 >>  because what we really need is for `move-to-column'
 >>  and `current-column' to be able to ignore invisible text."
 >
 >
 > This text seems odd: AFAIK they *do* ignore invisible text.  Does it
 > want to say that they sould ignore the `invisible' text property?

Personally I think they should ignore all properties.  Moving into
invisible text should be handled by point-adjustment.

 > The docstring of line-move-to-column is too vague: it doesn't say what
 > it intends to do, really: "considering invisibility" is not much help.
 > If someone knows what it intends to do, really, maybe we can figure out
 > how to fix it (probably the best fix will be to change the C code).

`line-move-ignore-invisible' per se is vague.  What is an invisible
line?

Re: invisible

[Stefan Monnier]

> I lost you here.  `line-move-ignore-invisible' nil means do not skip
> invisible newlines and explicitly should affect only C-n and C-p.
> `disable-point-adjustment' is about a lot of other things as well.

AFAIK in all circumstances where you want line-move-ignore-invisible to
be non-nil, you'll also want disable-point-adjustment to be nil.
And vice-versa.

>>> This function works only in certain cases,
>>> because what we really need is for `move-to-column'
>>> and `current-column' to be able to ignore invisible text."
>> This text seems odd: AFAIK they *do* ignore invisible text.  Does it
>> want to say that they sould ignore the `invisible' text property?
> Personally I think they should ignore all properties.

You're saying that line-move-to-column would like current-column and
friends should to treat invisible text as if it were visible?

For reference: the pre-21 behavior was to treat all invisible text as if
it weren't there.  In Emacs-21, I changed it so that text that's
replaced by an ellipsis is counted as if it were visible: I needed this
to be able to re-indent blocks of code while hidden by outline-minor-mode.
I didn't change it for non-ellipsis invisible text to reduce the
backward compatibility problems.

It turns out that it's also a good behavior in the sense that you can
get the other two behaviors (ignore all invisible text properties and
obey all invisible text properties) in the following way:

   (let ((buffer-invisibility-spec nil)) (current-column))
and
   (let ((buffer-invisibility-spec t)) (current-column))

>> The docstring of line-move-to-column is too vague: it doesn't say what
>> it intends to do, really: "considering invisibility" is not much help.
>> If someone knows what it intends to do, really, maybe we can figure out
>> how to fix it (probably the best fix will be to change the C code).

> `line-move-ignore-invisible' per se is vague.
> What is an invisible line?

Indeed.  We also need to improve that docstring.


        Stefan

Re: invisible

[martin rudalics]

 > AFAIK in all circumstances where you want line-move-ignore-invisible to
 > be non-nil, you'll also want disable-point-adjustment to be nil.
 > And vice-versa.

I never set them hence I conclude you're right.

 > You're saying that line-move-to-column would like current-column and
 > friends should to treat invisible text as if it were visible?

For `line-move-ignore-invisible' nil at least, yes.

 > In Emacs-21, I changed it so that text that's
 > replaced by an ellipsis is counted as if it were visible: I needed this
 > to be able to re-indent blocks of code while hidden by outline-minor-mode.
 > I didn't change it for non-ellipsis invisible text to reduce the
 > backward compatibility problems.
 >
 > It turns out that it's also a good behavior in the sense that you can
 > get the other two behaviors (ignore all invisible text properties and
 > obey all invisible text properties) in the following way:
 >
 >    (let ((buffer-invisibility-spec nil)) (current-column))
 > and
 >    (let ((buffer-invisibility-spec t)) (current-column))

Neat.  Indeed, writing

   (cond
    ((zerop col)
     (beginning-of-line))
    (line-move-ignore-invisible
     (move-to-column col))
    (t
     (let ((buffer-invisibility-spec nil))
       ;; Don't ignore invisible text when moving to the end of an
       ;; invisible line.
       (move-to-column col))))

seems to work perfectly.

Re: invisible

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 477 bytes --]

> It turns out that it's also a good behavior in the sense that you can
> get the other two behaviors (ignore all invisible text properties and
> obey all invisible text properties) in the following way:
> 
>    (let ((buffer-invisibility-spec nil)) (current-column))

It's not just neat, it's splendid.  We can dispense with that
rear-nonstickyness and DTRT with plain rear-sticky, invisible
text.  Stephen, please try with the attached patch, it should
work out of the box.


[-- Attachment #2: simple.patch --]
[-- Type: text/plain, Size: 3943 bytes --]

*** simple.el.~1.888.~	Sat Nov 10 09:23:20 2007
--- simple.el	Tue Nov 27 19:39:14 2007
***************
*** 3862,3868 ****
  	     (save-excursion
  	       ;; Like end-of-line but ignores fields.
  	       (skip-chars-forward "^\n")
! 	       (while (and (not (eobp)) (invisible-p (point)))
  		 (goto-char (next-char-property-change (point)))
  		 (skip-chars-forward "^\n"))
  	       (point))))
--- 3862,3869 ----
  	     (save-excursion
  	       ;; Like end-of-line but ignores fields.
  	       (skip-chars-forward "^\n")
! 	       (while (and line-move-ignore-invisible
! 			   (not (eobp)) (invisible-p (point)))
  		 (goto-char (next-char-property-change (point)))
  		 (skip-chars-forward "^\n"))
  	       (point))))
***************
*** 3936,3971 ****
  	  (setq repeat t))))))

  (defun line-move-to-column (col)
!   "Try to find column COL, considering invisibility.
! This function works only in certain cases,
! because what we really need is for `move-to-column'
! and `current-column' to be able to ignore invisible text."
!   (if (zerop col)
!       (beginning-of-line)
!     (move-to-column col))
! 
!   (when (and line-move-ignore-invisible
! 	     (not (bolp)) (invisible-p (1- (point))))
!     (let ((normal-location (point))
! 	  (normal-column (current-column)))
!       ;; If the following character is currently invisible,
!       ;; skip all characters with that same `invisible' property value.
!       (while (and (not (eobp))
! 		  (invisible-p (point)))
! 	(goto-char (next-char-property-change (point))))
!       ;; Have we advanced to a larger column position?
!       (if (> (current-column) normal-column)
! 	  ;; We have made some progress towards the desired column.
! 	  ;; See if we can make any further progress.
! 	  (line-move-to-column (+ (current-column) (- col normal-column)))
! 	;; Otherwise, go to the place we originally found
! 	;; and move back over invisible text.
! 	;; that will get us to the same place on the screen
! 	;; but with a more reasonable buffer position.
! 	(goto-char normal-location)
! 	(let ((line-beg (save-excursion (beginning-of-line) (point))))
! 	  (while (and (not (bolp)) (invisible-p (1- (point))))
! 	    (goto-char (previous-char-property-change (point) line-beg))))))))

  (defun move-end-of-line (arg)
    "Move point to end of current line as displayed.
--- 3937,3975 ----
  	  (setq repeat t))))))

  (defun line-move-to-column (col)
!   "Try to find column COL, considering invisibility."
!   (cond
!    ((zerop col)
!     (beginning-of-line))
!    (line-move-ignore-invisible
!     (move-to-column col)
!     (when (and (not (bolp)) (invisible-p (1- (point))))
!       (let ((normal-location (point))
! 	    (normal-column (current-column)))
! 	;; If the following character is currently invisible, skip all
! 	;; characters with that same `invisible' property value.
! 	(while (and (not (eobp)) (invisible-p (point)))
! 	  (goto-char (next-char-property-change (point))))
! 	;; Have we advanced to a larger column position?
! 	(if (> (current-column) normal-column)
! 	    ;; We have made some progress towards the desired column.
! 	    ;; See if we can make any further progress.
! 	    (line-move-to-column (+ (current-column) (- col normal-column)))
! 	  ;; Otherwise, go to the place we originally found and move
! 	  ;; back over invisible text.  That will get us to the same
! 	  ;; place on the screen but with a more reasonable buffer
! 	  ;; position.
! 	  (goto-char normal-location)
! 	  (let ((line-beg (line-beginning-position)))
! 	    (while (and (not (bolp)) (invisible-p (1- (point))))
! 	      (goto-char
! 	       (previous-char-property-change (point) line-beg))))))))
!    (t
!     (let ((buffer-invisibility-spec nil))
!       ;; Don't ignore invisible text when `line-move-ignore-invisible'
!       ;; is nil.
!       (move-to-column col)))))
! 

  (defun move-end-of-line (arg)
    "Move point to end of current line as displayed.

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: invisible

[Stefan Monnier]

>   (defun line-move-to-column (col)
> !   "Try to find column COL, considering invisibility."

Could you try to write a more detailed docstring that explains what is
meant by "considering invisibility"?  Or else try and explain to me what
the function is supposed to do (and I'll then try and write a docstring
for it)?


        Stefan

Re: invisible

[martin rudalics]

> Could you try to write a more detailed docstring that explains what is
> meant by "considering invisibility"?  Or else try and explain to me what
> the function is supposed to do (and I'll then try and write a docstring
> for it)?

I guess:

   "Move to a visible position at a column as close as possible to COL.
If no such position can be found by moving forwards, move back to last
visible position on current line or to the beginning of the line if
there's none."

If you think of replacing this by `move-to-column', you'd have to find
out what Richard's 2001-12-28 change intended to resolve.

Re: invisible

[Stefan Monnier]

>> Could you try to write a more detailed docstring that explains what is
>> meant by "considering invisibility"?  Or else try and explain to me what
>> the function is supposed to do (and I'll then try and write a docstring
>> for it)?

> I guess:

>   "Move to a visible position at a column as close as possible to COL.
> If no such position can be found by moving forwards, move back to last
> visible position on current line or to the beginning of the line if
> there's none."

Thanks.
But how are lines and columns counted?  Does an invisible newline cause
the intended column numbersto fall back to 0?  Does an invisible "a"
cause the intended column numnber to stay put or to be incremented by 1?


        Stefan

Re: invisible

[martin rudalics]

 > But how are lines and columns counted?  Does an invisible newline cause
 > the intended column numbersto fall back to 0?  Does an invisible "a"
 > cause the intended column numnber to stay put or to be incremented by 1?

We are talking about the default `line-move-ignore-invisible' t case, I
presume.  `col' is the column to move to (presumably the `goal-column'
but I didn't check this).  This code starts with

     (move-to-column col)

and the first question is whether we can find a case where this fails to
DTRT.  Apparently it might fail in the case

     (when (and (not (bolp)) (invisible-p (1- (point))))

which raises the question whether `move-to-column' would move to a
position following invisible text.  As I know now it does so iff that
text has a rear-nonsticky property (but I don't know about overlays).

Next our code goes into a loop

	(while (and (not (eobp)) (invisible-p (point)))
	  (goto-char (next-char-property-change (point))))

which just skips invisible text and we can conclude that lines are _not_
counted.  Moreover, it's strange that `move-to-column' didn't skip that
invisible text as well, but maybe it's a newline.  The following piece
of code is contrived and I leave it as an exercise to anyone interested
to find out what it's intended to do:

	(if (> (current-column) normal-column)
	    ;; We have made some progress towards the desired column.
	    ;; See if we can make any further progress.
	    (line-move-to-column (+ (current-column) (- col normal-column)))

This must have been the motivation to splice out `line-move-to-column'
back in 2001 - make it callable recursively.  I don't understand the ">"
here (`normal-column' is the column `move-to-column' moved to before).
And I completely fail to understand the argument calculation: Suppose
the desired `col' is 17 and `move-to-column' moved to column 14 on that
line because everything after 14 was invisible.  Also suppose the
invisible text ends at the end of some line hence `current-column' would
report 0 - or can you imagine `current-column' to report anything but
zero in this case?  Now should we move to column 3 here - or what am I
missing?

The else part goes as

	  (goto-char normal-location)
	  (let ((line-beg (line-beginning-position)))
	    (while (and (not (bolp)) (invisible-p (1- (point))))
	      (goto-char
	       (previous-char-property-change (point) line-beg))))))))

which simply moves to the first visible position on the line where
`move-to-column' stopped.  IIUC `move-to-column' never stops at a
position following invisible text unless it's rear-nonsticky, hence this
loop would be needed only to handle that special case.

All this got you is more questions asked than answered :-(

Re: invisible

[Stephen Berman]

On Tue, 27 Nov 2007 19:46:23 +0100 martin rudalics <rudalics@gmx.at> wrote:

>> It turns out that it's also a good behavior in the sense that you can
>> get the other two behaviors (ignore all invisible text properties and
>> obey all invisible text properties) in the following way:
>>
>>    (let ((buffer-invisibility-spec nil)) (current-column))
>
> It's not just neat, it's splendid.  We can dispense with that
> rear-nonstickyness and DTRT with plain rear-sticky, invisible
> text.  Stephen, please try with the attached patch, it should
> work out of the box.

I'm afraid it did not, which puzzles me in view of the confidence you
expressed.  I wonder if we did something different again.  I did emacs
-Q, evalled line-move-finish and line-move-to-column with your patch
applied and evalled the sexps in your test file lmii.txt.  Here are the
results I got:

1. 
  i. with '(invisible t):
    fl-1
     Starting with point on or after the `l' of `line7', cursor moves
     to the `l' of line6, then the `l' of line4, then the `l' of line2,
     then the `l' of line1.
    C-p
    a. Starting with point on or after the `i' of `line7', cursor moves
       successively to the corresponding column of the preceding line.
    b. Starting with point on the `l' of `line7', cursor moves to the `l'
       of line6, then the `l' of line4, then the `l' of line2, then the
       `l' of line1.

  ii. with '(invisible t rear-nonsticky t):
    fl-1
     Starting with point on or after the `l' of `line7', cursor moves
     successively to the `l' of the preceding line.
    C-p
     Starting with point on the `l' of `line7', cursor moves
     successively to the corresponding column of the preceding line.

I also tested making line4 invisible (add-text-properties 19 25 ...),
which one of the original bug reports was about:

2. 
  i. with '(invisible t):
    fl-1
     Starting with point with point anywhere on line7, cursor moves
     successively to the `l' of the preceding line.
    C-p
     Starting with point anywhere on line7, cursor moves to the
     corresponding column of line6, then to the corresponding 
     column of line5, then to the `l' of line5, then back to the
     goal column in line3 and likewise with line2 and line1.
    C-n
     Starting with point anywhere on line1 or line2, cursor moves to the
     corresponding column of the next line.  Continuing from line3, C-n
     move cursor to the `l' of line5, and with the next C-n to the
     previous goal column in line5.  Subsequent C-n's move the cursor to
     the next line also at the same column.

  ii. with '(invisible t rear-nonsticky t):
    fl-1
     Starting with point with point anywhere on line7, cursor moves
     to the `l' of line6, then to the `l' of line5, then to the end of
     line3, then to the `l' of line2 and likewise line1.
    C-p
     Starting with point anywhere on line7, cursor moves to the
     corresponding column of line6, then to the corresponding column of
     line5, then to the `l' of line5, then to the end of line3, then
     back to the goal column in line2 and likewise in line1.
    C-n
     a. Starting with point on the `l' of line1, line2, or line3, cursor
        moves to the `l' of the next line.  Continuing with C-n at
        line3, cursor moves to the `i' of line5, then to the `l' of
        line6 and likewise line7.  However, if, after reaching the `i'
        line5 with C-n, I then type C-p, cursor moves first to the `l'
        of line5, then to the `l' of line3 (not, as above, to the end of
        line2), and to the `l' of line2 and line1.
     b. Starting with point on any position but the `l' of line1 or
        line2, cursor moves to the corresponding column of the next
        line.  Continuing from line3, cursor moves to the `l' of line5,
        then to the original goal column in line5 and likewise in line6
        and line7.

Steve Berman

Re: invisible

[martin rudalics]

 > I'm afraid it did not, which puzzles me in view of the confidence you
 > expressed.  I wonder if we did something different again.

Yes and no.  I've been experimenting with a different approach to make
parts of lines invisible.  Everything you did was correct.

I'm afraid the problem cannot be solved satisfactorily without using the
rear-nonsticky property.  In fact the "bug" happens _before_ we can see
it.  Consider the following configuration where the "\n" are visible
newlines, the "(\n)" invisible rear-sticky ones, dots are plain visible
non-newline text, and "^" indicates the position of `point':

.....\n
.....(\n).....(\n).....(\n).....(\n).....\n
^.....\n

C-p now should move to

.....\n
.....(\n).....(\n).....(\n).....(\n)^.....\n
.....\n

but the point-setting in scan_newline really moves to

.....\n
.....(\n).....(\n).....(\n).....^(\n).....\n
.....\n

which _appears_ correct because the newline doesn't appear on screen.
Recall that the point-setting mechanism doesn't permit to position
`point' after the invisible newline because the text property is
rear-sticky.

Hitting C-p again first invokes (forward-line -1) to move `point' to the
beginning of the previous line as

.....\n
.....(\n).....(\n)^.....(\n).....(\n).....\n
.....\n

but the point-setting mechanism in scan_newline again corrects this to

.....\n
.....(\n).....^(\n).....(\n).....(\n).....\n
.....\n

The next C-p gets us to

.....\n
^.....(\n).....(\n).....(\n).....(\n).....\n
.....\n

because we now are after a visible newline and our `goal-column' is
still zero.

I think we first have to specify what we really want and then fix this
accordingly.  I can think of the following alternatives:

(1) Provide for disabling the point adjustment mechanism in
scan_newline.  The disadvantage of this approach is that `point' may
appear after rear-sticky invisible text and doing a self-insert there
won't show the character.

(2) Make self-insertion after rear-sticky invisible (intangible, ...)
text by default _not_ inherit the invisibility property.  Intuitively,
this should do the right thing but the outcome would be different for
not self-inserting commands that try to insert text at `point'.

(3) Use the rear-nonsticky property and live with the fact that
insertions may occur on either end of the invisible text depending on
whether you arrived at the corresponding screen position either by
moving forwards or backwards.  As Stefan said we could ameliorate this
when using ellipses.  However, what should `forward-char' do with
invisible text replaced by an ellipis and `point' immediately before it:

text^...text

Should it go here

text...^text

or here

text...t^ext

Re: invisible

[Stefan Monnier]

> (1) Provide for disabling the point adjustment mechanism in
> scan_newline.  The disadvantage of this approach is that `point' may
> appear after rear-sticky invisible text and doing a self-insert there
> won't show the character.

This approach sounds also simply impossible to implement (at least not
without major surgery and/or big ugly hacks).

> (3) Use the rear-nonsticky property and live with the fact that
> insertions may occur on either end of the invisible text depending on
> whether you arrived at the corresponding screen position either by
> moving forwards or backwards.  As Stefan said we could ameliorate this
> when using ellipses.  However, what should `forward-char' do with
> invisible text replaced by an ellipis and `point' immediately before it:

> text^...text

> Should it go here

> text...^text

> or here

> text...t^ext

The first.  And it's already what it does.


        Stefan

Re: invisible

[Richard Stallman]

I think it would be good to start accumulating a list of test cases
for which line-move and line-move-to-column have failed.
Several times I've seen that fixing one case breaks another.
Can you do that?

Re: invisible

[martin rudalics]

 > I think it would be good to start accumulating a list of test cases
 > for which line-move and line-move-to-column have failed.
 > Several times I've seen that fixing one case breaks another.
 > Can you do that?

Currently, I don't have a test case where this fails with the
default value of `line-move-ignore-invisible'.  All problems stem
from `line-move-ignore-invisible' nil.

Re: invisible

[Stefan Monnier]

>> I think it would be good to start accumulating a list of test cases
>> for which line-move and line-move-to-column have failed.
>> Several times I've seen that fixing one case breaks another.
>> Can you do that?

> Currently, I don't have a test case where this fails with the
> default value of `line-move-ignore-invisible'.  All problems stem
> from `line-move-ignore-invisible' nil.

Hmm... I didn't notice that part.  But then do they also appear with
a line-move-ignore-invisible set to nil and a disable-point-adjustment
set to t?  I would guess not (this case being trivial).

If so, maybe the solution is to do as I suggested: get rid of
line-move-ignore-invisible and use (not disable-point-adjustment) in
its place.


        Stefan

Re: invisible

[martin rudalics]

 >>Currently, I don't have a test case where this fails with the
 >>default value of `line-move-ignore-invisible'.  All problems stem
 >>from `line-move-ignore-invisible' nil.
 >
 >
 > Hmm... I didn't notice that part.  But then do they also appear with
 > a line-move-ignore-invisible set to nil and a disable-point-adjustment
 > set to t?  I would guess not (this case being trivial).
 >
 > If so, maybe the solution is to do as I suggested: get rid of
 > line-move-ignore-invisible and use (not disable-point-adjustment) in
 > its place.

No because the problem is with point adjustment during line movement.
It's got nothing to do with the command loop.

I'm afraid my earlier discussion with Stephen about whether calling
`forward-line' interactively or via `eval-expression' have the same
effect is still muddying this discussion.

To recapitulate: C-n/C-p call `forward-line' to move to the
next/previous line.  `forward-line' first invokes scan_newline to find
the next/previous newline and position point after/before it.
Thereafter it calls

SET_PT_BOTH (pos, pos_byte);

The latter refuses to position `point' immediately after rear-sticky
invisible text.

Re: invisible

[Stefan Monnier]

> SET_PT_BOTH (pos, pos_byte);
> The latter refuses to position `point' immediately after rear-sticky
> invisible text.

SET_PT_BOTH should only pay attention to `invisible' in the case where
there is also `intangible' properties involved.  IIRC you said this
thread was not using intangibility but only invisibility, if so, then
it's a bug in SET_PT_BOTH.

If you do use intangibility, then indeed all bets are off, and
odd/undesirable results are to be expected (yes, I'm biased).


        Stefan

Re: invisible

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

 > SET_PT_BOTH should only pay attention to `invisible' in the case where
 > there is also `intangible' properties involved.

The emphasis is on _should_ I presume.

 > IIRC you said this
 > thread was not using intangibility but only invisibility, if so, then
 > it's a bug in SET_PT_BOTH.

Please have a look.  I attached a short version of my canonical file.
With Emacs -Q put this in scratch, eval the progn, move to the beginning
of the third visible line where it says "line7", and type C-p twice.
The (forward-line -1) of the second C-p makes the "bug" apparent
(although it occurs already in the first C-p).

 > If you do use intangibility, then indeed all bets are off, and
 > odd/undesirable results are to be expected (yes, I'm biased).

I hope my attachment makes it clear that I don't use intangibility.

martin, who fully shares your biases in this regard.


[-- Attachment #2: lmiis.txt --]
[-- Type: text/plain, Size: 271 bytes --]

line1
line2
line3
line4
line5
line6
line7

(progn
  (add-text-properties 12 13 '(invisible t))
  (add-text-properties 18 19 '(invisible t))
  (add-text-properties 24 25 '(invisible t))
  (add-text-properties 30 31 '(invisible t))
  (setq line-move-ignore-invisible nil))

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: invisible

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 577 bytes --]

 > Hmm... I didn't notice that part.  But then do they also appear with
 > a line-move-ignore-invisible set to nil and a disable-point-adjustment
 > set to t?  I would guess not (this case being trivial).
 >
 > If so, maybe the solution is to do as I suggested: get rid of
 > line-move-ignore-invisible and use (not disable-point-adjustment) in
 > its place.

FWIW, the following seems to work: Disable point-adjustment in
interactive use of `next-line' and `previous-line' and apply one of my
earlier proposed changes.  Stephen, please try again with default
rear-stickiness.

[-- Attachment #2: simple.patch --]
[-- Type: text/plain, Size: 2162 bytes --]

*** simple.el.~1.888.~	Sat Nov 10 09:23:20 2007
--- simple.el	Fri Nov 30 18:13:12 2007
***************
*** 3595,3601 ****
  	(line-move arg nil nil try-vscroll))
      (if (interactive-p)
  	(condition-case nil
! 	    (line-move arg nil nil try-vscroll)
  	  ((beginning-of-buffer end-of-buffer) (ding)))
        (line-move arg nil nil try-vscroll)))
    nil)
--- 3595,3606 ----
  	(line-move arg nil nil try-vscroll))
      (if (interactive-p)
  	(condition-case nil
! 	    (progn
! 	      (unless line-move-ignore-invisible
! 		;; Interactively, disable point-adjustment when
! 		;; `line-move-ignore-invisible' is nil.
! 		(setq disable-point-adjustment t))
! 	      (line-move arg nil nil try-vscroll))
  	  ((beginning-of-buffer end-of-buffer) (ding)))
        (line-move arg nil nil try-vscroll)))
    nil)
***************
*** 3621,3627 ****
    (or arg (setq arg 1))
    (if (interactive-p)
        (condition-case nil
! 	  (line-move (- arg) nil nil try-vscroll)
  	((beginning-of-buffer end-of-buffer) (ding)))
      (line-move (- arg) nil nil try-vscroll))
    nil)
--- 3626,3637 ----
    (or arg (setq arg 1))
    (if (interactive-p)
        (condition-case nil
! 	    (progn
! 	      (unless line-move-ignore-invisible
! 		;; Interactively, disable point-adjustment when
! 		;; `line-move-ignore-invisible' is nil.
! 		(setq disable-point-adjustment t))
! 	      (line-move (- arg) nil nil try-vscroll))
  	((beginning-of-buffer end-of-buffer) (ding)))
      (line-move (- arg) nil nil try-vscroll))
    nil)
***************
*** 3862,3868 ****
  	     (save-excursion
  	       ;; Like end-of-line but ignores fields.
  	       (skip-chars-forward "^\n")
! 	       (while (and (not (eobp)) (invisible-p (point)))
  		 (goto-char (next-char-property-change (point)))
  		 (skip-chars-forward "^\n"))
  	       (point))))
--- 3872,3879 ----
  	     (save-excursion
  	       ;; Like end-of-line but ignores fields.
  	       (skip-chars-forward "^\n")
! 	       (while (and line-move-ignore-invisible
! 			   (not (eobp)) (invisible-p (point)))
  		 (goto-char (next-char-property-change (point)))
  		 (skip-chars-forward "^\n"))
  	       (point))))

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: invisible

[Stefan Monnier]

>> Hmm... I didn't notice that part.  But then do they also appear with
>> a line-move-ignore-invisible set to nil and a disable-point-adjustment
>> set to t?  I would guess not (this case being trivial).
>> 
>> If so, maybe the solution is to do as I suggested: get rid of
>> line-move-ignore-invisible and use (not disable-point-adjustment) in
>> its place.

> FWIW, the following seems to work: Disable point-adjustment in
> interactive use of `next-line' and `previous-line' and apply one of my
> earlier proposed changes.

So IIUC replacing line-move-ignore-invisible with
(not global-disable-point-adjustment) would solve the problem as well, right?

> ! 	      (unless line-move-ignore-invisible
> ! 		;; Interactively, disable point-adjustment when
> ! 		;; `line-move-ignore-invisible' is nil.
> ! 		(setq disable-point-adjustment t))

Why not put this directly inside `line-move' so it's not duplicated?


        Stefan

Re: invisible

[martin rudalics]

 >>FWIW, the following seems to work: Disable point-adjustment in
 >>interactive use of `next-line' and `previous-line' and apply one of my
 >>earlier proposed changes.
 >
 >
 > So IIUC replacing line-move-ignore-invisible with
 > (not global-disable-point-adjustment) would solve the problem as well, right?

I don't know.  `line-move-ignore-invisible' is a user option (although I
fail to see how it's useful).  `(global-)disable-point-adjustment' is
not.  IIUC a user might want to set `line-move-ignore-invisible' to nil
in order to have C-n/C-p stop at or near invisible newlines.  In order
to make this possible I set `disable-point-adjustment' to t.  I do this
because for this particular goal the adjustment step is too clever.  But
I don't see how replacing the one by the negation of the other would
solve the problem.

 >>! 	      (unless line-move-ignore-invisible
 >>! 		;; Interactively, disable point-adjustment when
 >>! 		;; `line-move-ignore-invisible' is nil.
 >>! 		(setq disable-point-adjustment t))
 >
 >
 > Why not put this directly inside `line-move' so it's not duplicated?

Because I wanted to emphasize that this is for interactive use only and
`interactive-p' is tested in next-/previous-line.  If for whatever
reason people want to use next-/previous-line in a function, they should
be allowed to disable point-adjustment as they like.  But I do not have
a strong opinion about this, let's see whether my patch DTRT at all.  As
Richard mentioned earlier adjusting one thing here breaks another ...

Re: invisible

[Stefan Monnier]

> I don't know.  `line-move-ignore-invisible' is a user option (although I
> fail to see how it's useful).  `(global-)disable-point-adjustment' is not.

global-disable-point-adjustment *is* a user option.

> IIUC a user might want to set `line-move-ignore-invisible' to nil
> in order to have C-n/C-p stop at or near invisible newlines.  In order
> to make this possible I set `disable-point-adjustment' to t.  I do this
> because for this particular goal the adjustment step is too clever.  But
> I don't see how replacing the one by the negation of the other would
> solve the problem.

I must be missing something: the relationship is pretty obvious to me
since your code sets disable-point-adjustment to t (i.e. forces Emacs to
behave as if global-disable-point-adjustment were t for this one
command) if and only if line-move-ignore-invisible is nil.

>>> ! 	      (unless line-move-ignore-invisible
>>> ! 		;; Interactively, disable point-adjustment when
>>> ! 		;; `line-move-ignore-invisible' is nil.
>>> ! 		(setq disable-point-adjustment t))
>> 
>> Why not put this directly inside `line-move' so it's not duplicated?

> Because I wanted to emphasize that this is for interactive use only
> and `interactive-p' is tested in next-/previous-line.  If for whatever
> reason people want to use next-/previous-line in a function, they
> should be allowed to disable point-adjustment as they like.  But I do
> not have a strong opinion about this, let's see whether my patch DTRT
> at all.  As Richard mentioned earlier adjusting one thing here breaks
> another ...

Since those functions are discouraged in Elisp code anyway I don't think
it matters much.


        Stefan

Re: invisible

[martin rudalics]

 >>I don't know.  `line-move-ignore-invisible' is a user option (although I
 >>fail to see how it's useful).  `(global-)disable-point-adjustment' is not.
 >
 >
 > global-disable-point-adjustment *is* a user option.

OK.  But according to the Elisp manual it's a "plain" variable and it's
not customizable.  Also `line-move-ignore-invisible' is an option that
applies to C-n/C-p exclusively.  `global-disable-point-adjustment', on
the other hand, applies to all commands.

 >>IIUC a user might want to set `line-move-ignore-invisible' to nil
 >>in order to have C-n/C-p stop at or near invisible newlines.  In order
 >>to make this possible I set `disable-point-adjustment' to t.  I do this
 >>because for this particular goal the adjustment step is too clever.  But
 >>I don't see how replacing the one by the negation of the other would
 >>solve the problem.
 >
 >
 > I must be missing something: the relationship is pretty obvious to me
 > since your code sets disable-point-adjustment to t (i.e. forces Emacs to
 > behave as if global-disable-point-adjustment were t for this one
 > command) if and only if line-move-ignore-invisible is nil.

Because I'm not allowed to disable point-adjustment for the normal
`line-move-ignore-invisible' t case.  What shall I do if someone wants
to disable point-adjustment for the `line-move-ignore-invisible' t case
too?  We have

line-move-ignore-invisible => disable-point-adjustment

but not

disable-point-adjustment => line-move-ignore-invisible

or what am I missing?

 >>Because I wanted to emphasize that this is for interactive use only
 >>and `interactive-p' is tested in next-/previous-line.  If for whatever
 >>reason people want to use next-/previous-line in a function, they
 >>should be allowed to disable point-adjustment as they like.  But I do
 >>not have a strong opinion about this, let's see whether my patch DTRT
 >>at all.  As Richard mentioned earlier adjusting one thing here breaks
 >>another ...
 >
 >
 > Since those functions are discouraged in Elisp code anyway I don't think
 > it matters much.

Neither do I.  But why do these have an `interactive-p' check in the
first place?  BTW, there are a few instances where packages do rely on
next-/previous-line (I've just checked in a corresponding fix for
blackbox.el).

Re: invisible

[Stefan Monnier]

>>> I don't know.  `line-move-ignore-invisible' is a user option (although I
>>> fail to see how it's useful).  `(global-)disable-point-adjustment' is not.
>> global-disable-point-adjustment *is* a user option.
> OK.  But according to the Elisp manual it's a "plain" variable and it's
> not customizable.

Sure.

> Also `line-move-ignore-invisible' is an option that applies to C-n/C-p
> exclusively.  `global-disable-point-adjustment', on the other hand,
> applies to all commands.

Obviously conflating the two variables into one is a change visible to
the user.  I'm just saying that it would fix your bug as well.
And I think it's a choice that might make a lot of sense.

> Because I'm not allowed to disable point-adjustment for the normal
> `line-move-ignore-invisible' t case.

AFAICT the reason is not that you're not allowed but that you haven't
needed to do it.  If you needed to do it, I don't think it'd be a big
problem since line-move already tries to avoid invisible text (so
point-adjustment shouldn't make much difference if any).

> What shall I do if someone wants to disable point-adjustment for the
> `line-move-ignore-invisible' t case too?

We'll cross this bridge only if we ever get there.

> Neither do I.  But why do these have an `interactive-p' check in the
> first place?

No idea.  It was introduced by:

  revision 1.183
  date: 1994-12-23 12:25:50 -0500;  author: rms;  state: Exp;  lines: +13 -5;
  (next-line, previous-line): If interactive and not in
  kbd macro, catch the error and beep instead.
  ----------------------------

Note that the difference is very small: in the case where it beeps,
signalling an error would beep as well and would just complain a bit
more loudly.  I.e. it only changes the way in which the error is
signalled to the user.  Not the actual behavior of the command.

So I now think that setting disable-point-adjustment only in the
interactive case is actually incorrect: it should also be set in the
non-interactive case.


        Stefan

Re: invisible

[Johan Bockgård]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Neither do I.  But why do these have an `interactive-p' check in the
>> first place?
>
> No idea.  It was introduced by:
>
>   revision 1.183
>   date: 1994-12-23 12:25:50 -0500;  author: rms;  state: Exp;  lines: +13 -5;
>   (next-line, previous-line): If interactive and not in
>   kbd macro, catch the error and beep instead.
>   ----------------------------

This predates `debug-ignored-errors' by a year...

-- 
Johan Bockgård

Re: invisible

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 1857 bytes --]

 > Obviously conflating the two variables into one is a change visible to
 > the user.  I'm just saying that it would fix your bug as well.

It's not my bug.  Which means, I don't have a user perspective here and
everything I say is purely speculative in this regard.

 > And I think it's a choice that might make a lot of sense.

Maybe. Just that people have to be aware that any character they insert
after rear-sticky invisible text will become invisible too.

 >>Because I'm not allowed to disable point-adjustment for the normal
 >>`line-move-ignore-invisible' t case.
 >
 >
 > AFAICT the reason is not that you're not allowed but that you haven't
 > needed to do it.  If you needed to do it, I don't think it'd be a big
 > problem since line-move already tries to avoid invisible text (so
 > point-adjustment shouldn't make much difference if any).

`line-move-ignore-invisible' is about _invisibility_ only.  Adjusting
point is about invisibility, composition, intangibility, fields ...
Now, if I set `line-move-ignore-invisible' to nil I might want to stop
at the next invisible newline but still respect the remaining
properties.  I understand that's what "conflating" is all about and I
agree that there's no peculiar reason why we should just consider
invisibility and not the rest.  Moreover `line-move-ignore-invisible'
seems broken anyway and hard to get right.

 > So I now think that setting disable-point-adjustment only in the
 > interactive case is actually incorrect: it should also be set in the
 > non-interactive case.

Currently point-adjustment affects point-setting only _after_ a command
is executed.  `line-move-ignore-invisible' affects point-setting for
every single line move, regardless of whether this happens in a command
or how often this happens in a command.

Anyway.  Is the attached patch about what you have in mind?

[-- Attachment #2: simple.patch --]
[-- Type: text/plain, Size: 3422 bytes --]

*** simple.el.~1.888.~	Sat Nov 10 09:23:20 2007
--- simple.el	Sun Dec  2 10:40:22 2007
***************
*** 3593,3603 ****
  	    (end-of-line)
  	    (insert (if use-hard-newlines hard-newline "\n")))
  	(line-move arg nil nil try-vscroll))
!     (if (interactive-p)
! 	(condition-case nil
! 	    (line-move arg nil nil try-vscroll)
! 	  ((beginning-of-buffer end-of-buffer) (ding)))
!       (line-move arg nil nil try-vscroll)))
    nil)

  (defun previous-line (&optional arg try-vscroll)
--- 3593,3602 ----
  	    (end-of-line)
  	    (insert (if use-hard-newlines hard-newline "\n")))
  	(line-move arg nil nil try-vscroll))
!     ;; Disable point-adjustment with `line-move-ignore-invisible' nil.
!     (unless line-move-ignore-invisible
!       (setq disable-point-adjustment t))
!     (line-move arg nil nil try-vscroll))
    nil)

  (defun previous-line (&optional arg try-vscroll)
***************
*** 3619,3629 ****
  to use and more reliable (no dependence on goal column, etc.)."
    (interactive "p\np")
    (or arg (setq arg 1))
!   (if (interactive-p)
!       (condition-case nil
! 	  (line-move (- arg) nil nil try-vscroll)
! 	((beginning-of-buffer end-of-buffer) (ding)))
!     (line-move (- arg) nil nil try-vscroll))
    nil)

  (defcustom track-eol nil
--- 3618,3627 ----
  to use and more reliable (no dependence on goal column, etc.)."
    (interactive "p\np")
    (or arg (setq arg 1))
!   ;; Disable point-adjustment with `line-move-ignore-invisible' nil.
!   (unless line-move-ignore-invisible
!     (setq disable-point-adjustment t))
!   (line-move (- arg) nil nil try-vscroll)
    nil)

  (defcustom track-eol nil
***************
*** 3746,3753 ****
  			most-positive-fixnum
  		      (current-column))))

! 	  (if (not (or (integerp selective-display)
!                        line-move-ignore-invisible))
  	      ;; Use just newline characters.
  	      ;; Set ARG to 0 if we move as many lines as requested.
  	      (or (if (> arg 0)
--- 3744,3751 ----
  			most-positive-fixnum
  		      (current-column))))

! 	  (if (and (not (integerp selective-display))
!                    disable-point-adjustment)
  	      ;; Use just newline characters.
  	      ;; Set ARG to 0 if we move as many lines as requested.
  	      (or (if (> arg 0)
***************
*** 3862,3868 ****
  	     (save-excursion
  	       ;; Like end-of-line but ignores fields.
  	       (skip-chars-forward "^\n")
! 	       (while (and (not (eobp)) (invisible-p (point)))
  		 (goto-char (next-char-property-change (point)))
  		 (skip-chars-forward "^\n"))
  	       (point))))
--- 3860,3867 ----
  	     (save-excursion
  	       ;; Like end-of-line but ignores fields.
  	       (skip-chars-forward "^\n")
! 	       (while (and (not disable-point-adjustment)
! 			   (not (eobp)) (invisible-p (point)))
  		 (goto-char (next-char-property-change (point)))
  		 (skip-chars-forward "^\n"))
  	       (point))))
***************
*** 3944,3950 ****
        (beginning-of-line)
      (move-to-column col))

!   (when (and line-move-ignore-invisible
  	     (not (bolp)) (invisible-p (1- (point))))
      (let ((normal-location (point))
  	  (normal-column (current-column)))
--- 3943,3949 ----
        (beginning-of-line)
      (move-to-column col))

!   (when (and (not disable-point-adjustment)
  	     (not (bolp)) (invisible-p (1- (point))))
      (let ((normal-location (point))
  	  (normal-column (current-column)))

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: invisible

[martin rudalics]

 > Setting rear-nonsticky is _not_ sufficient, you have to do something as
 > in my patch in order to handle `line-move-ignore-invisible' correctly.
 > Please try again (1) with the property set, (2) with and without my
 > patch applied, and (3) with both values for `line-move-ignore-invisible'
 > and `track-eol'.

Forgot to say (4) starting C-p / C-n at line-beginnings and -ends in
order to test the "track-eol" behavior.