bug#17511: 24.4.50; `line-move-ignore-invisible': doc and purpose not clear

bug#17511: 24.4.50; `line-move-ignore-invisible': doc and purpose not clear

[Drew Adams]

1. The doc string says only that `next-line' and `previous-line' ignore
invisible lines.  What does it mean for these commands to "ignore
invisible lines" - what does "ignore" mean here?  What's the
user-visible BEHAVIOR difference between a nil and a non-nil value?
Why/when might a user change the value to nil?

2. The doc string speaks of invisible lines.  But (elisp) `Invisible
Text' speaks of "invisible newlines" (not lines), which is presumably
something different (newline chars vs lines of any chars except newline,
possibly including the separating newlines).  Are both true?  Which?

3 The manual speaks of "the user-level line motion commands", not just
`next-line' and `previous-line'.  Is there a difference?  What other
commands are involved here, besides those two?

4. This Elisp manual node is not very clear in general.  Again, what
does it mean for these line commands to ignore invisible newlines?  Or,
for that matter, for them to "not care whether the text is invisible"?
What user-visible BEHAVIOR difference is there?

5. After saying that we provide option `line-move-ignore-invisible'
specifically to let you prevent line motion commands from ignoring
invisible newlines (whatever that might mean!), this node says that if
ANY command ends with point in a certain position relative to invisible
text, then the cursor is automatically moved past that stretch of
invisible text (one direction or the other).  How is this related to the
previous text about line motion commands and
`line-move-ignore-invisible'?

If a user ends up understanding something coherent and useful from this
text, s?he is lucky indeed.



In GNU Emacs 24.4.50.1 (i686-pc-mingw32)
 of 2014-04-29 on ODIEONE
Bzr revision: 117031 monnier@iro.umontreal.ca-20140429151607-qnkgbymwfaj5ut08
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --prefix=/c/Devel/emacs/snapshot/trunk
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -g3'
 LDFLAGS=-Lc:/Devel/emacs/lib 'CPPFLAGS=-DGC_MCHECK=1
 -Ic:/Devel/emacs/include''

bug#17511: 24.4.50; `line-move-ignore-invisible': doc and purpose not clear

[Eli Zaretskii]

> Date: Fri, 16 May 2014 13:52:58 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> 
> 1. The doc string says only that `next-line' and `previous-line' ignore
> invisible lines.  What does it mean for these commands to "ignore
> invisible lines" - what does "ignore" mean here?  What's the
> user-visible BEHAVIOR difference between a nil and a non-nil value?
> Why/when might a user change the value to nil?

I changed the doc string to this:

  "Non-nil means commands that move by lines ignore invisible newlines.

When this option is non-nil, \\[next-line], \\[previous-line], \\[move-end-of-line], and \\[move-beginning-of-line] behave
as if newlines that are invisible didn't exist, and count
only visible newlines.  Thus, moving across across 2 newlines
one of which is invisible will be counted as a one-line move.
Also, a non-nil value causes invisible text to be ignored when
counting columns for the purposes of keeping point in the same
column by \\[next-line] and \\[previous-line].

Outline mode sets this."

I hope this answers all of your questions in #1.

> 2. The doc string speaks of invisible lines.  But (elisp) `Invisible
> Text' speaks of "invisible newlines" (not lines), which is presumably
> something different (newline chars vs lines of any chars except newline,
> possibly including the separating newlines).  Are both true?  Which?

I think the doc string now clarifies this as well.

> 3 The manual speaks of "the user-level line motion commands", not just
> `next-line' and `previous-line'.  Is there a difference?  What other
> commands are involved here, besides those two?

Again, I believe the doc string now clarifies that.

> 4. This Elisp manual node is not very clear in general.  Again, what
> does it mean for these line commands to ignore invisible newlines?  Or,
> for that matter, for them to "not care whether the text is invisible"?
> What user-visible BEHAVIOR difference is there?

I clarified in that paragraph that "ignoring" invisible text means
behaving as if it didn't exist in the buffer.  This implies that an
invisible newline does not count as a newline, so when invisible text
is ignored, such a line is considered to be part of the next line.
IOW, commands that count lines will not count a line whose newline is
invisible, when this option is non-nil, so, e.g., "C-u 10 C-n" will
not decrement its counter when it moves across an invisible newline.

> 5. After saying that we provide option `line-move-ignore-invisible'
> specifically to let you prevent line motion commands from ignoring
> invisible newlines (whatever that might mean!), this node says that if
> ANY command ends with point in a certain position relative to invisible
> text, then the cursor is automatically moved past that stretch of
> invisible text (one direction or the other).  How is this related to the
> previous text about line motion commands and
> `line-move-ignore-invisible'?

It isn't related to line-move-ignore-invisible.  It is related to the
broader issue described by that node, which is invisible text in
general.

> If a user ends up understanding something coherent and useful from this
> text, s?he is lucky indeed.

Unhelpful comment, I wish you'd stop making them.

bug#17511: 24.4.50; `line-move-ignore-invisible': doc and purpose not clear

[Drew Adams]

Thanks for improving this.

> > 1. The doc string says only that `next-line' and `previous-line' ignore
> > invisible lines.  What does it mean for these commands to "ignore
> > invisible lines" - what does "ignore" mean here?  What's the
> > user-visible BEHAVIOR difference between a nil and a non-nil value?
> > Why/when might a user change the value to nil?
> 
> I changed the doc string to this:
> 
>   "Non-nil means commands that move by lines ignore invisible newlines.
>
> When this option is non-nil, \\[next-line], \\[previous-line], \\[move-end-
> of-line], and \\[move-beginning-of-line] behave
> as if newlines that are invisible didn't exist, and count
> only visible newlines.  Thus, moving across across 2 newlines
> one of which is invisible will be counted as a one-line move.
> Also, a non-nil value causes invisible text to be ignored when
> counting columns for the purposes of keeping point in the same
> column by \\[next-line] and \\[previous-line].
> 
> Outline mode sets this."
> 
> I hope this answers all of your questions in #1.

Very good; thanks.  (I don't think there should be a blank line after the
first line, but maybe that is just a mail artifact.)

> > 2. The doc string speaks of invisible lines.  But (elisp) `Invisible
> > Text' speaks of "invisible newlines" (not lines), which is presumably
> > something different (newline chars vs lines of any chars except newline,
> > possibly including the separating newlines).  Are both true?  Which?
> 
> I think the doc string now clarifies this as well.

Yes, thanks.  But the manual speaks only of invisible newlines, and to
me this part is not clear.  And whenever we speak of newlines (especially
where we are also talking about doing something wrt lines in general),
we should say "newline characters" or "newline chars".  A "newline" as
such doesn't really exist in our vocabulary (or at least it shouldn't),
and some people might read it as meaning a "new line".

> > 3 The manual speaks of "the user-level line motion commands", not just
> > `next-line' and `previous-line'.  Is there a difference?  What other
> > commands are involved here, besides those two?
> 
> Again, I believe the doc string now clarifies that.

OK.

> > 4. This Elisp manual node is not very clear in general.  Again, what
> > does it mean for these line commands to ignore invisible newlines?  Or,
> > for that matter, for them to "not care whether the text is invisible"?
> > What user-visible BEHAVIOR difference is there?
> 
> I clarified in that paragraph that "ignoring" invisible text means
> behaving as if it didn't exist in the buffer.  This implies that an
> invisible newline does not count as a newline, so when invisible text
> is ignored, such a line is considered to be part of the next line.
> IOW, commands that count lines will not count a line whose newline is
> invisible, when this option is non-nil, so, e.g., "C-u 10 C-n" will
> not decrement its counter when it moves across an invisible newline.

Very good (modulo newline -> newline char).

> > 5. After saying that we provide option `line-move-ignore-invisible'
> > specifically to let you prevent line motion commands from ignoring
> > invisible newlines (whatever that might mean!), this node says that if
> > ANY command ends with point in a certain position relative to invisible
> > text, then the cursor is automatically moved past that stretch of
> > invisible text (one direction or the other).  How is this related to the
> > previous text about line motion commands and
> > `line-move-ignore-invisible'?
> 
> It isn't related to line-move-ignore-invisible.  It is related to the
> broader issue described by that node, which is invisible text in
> general.

Yes, I sensed that.  I found (find) the juxtaposition confusing.
Maybe separate the two discussions better, and perhaps give an example
of interaction (or lack thereof) between the two.

bug#17511: 24.4.50; `line-move-ignore-invisible': doc and purpose not clear

[Eli Zaretskii]

> Date: Sat, 17 May 2014 07:45:44 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 17511@debbugs.gnu.org
> 
> Thanks for improving this.

Can this bug be closed, then?

> > > 1. The doc string says only that `next-line' and `previous-line' ignore
> > > invisible lines.  What does it mean for these commands to "ignore
> > > invisible lines" - what does "ignore" mean here?  What's the
> > > user-visible BEHAVIOR difference between a nil and a non-nil value?
> > > Why/when might a user change the value to nil?
> > 
> > I changed the doc string to this:
> > 
> >   "Non-nil means commands that move by lines ignore invisible newlines.
> >
> > When this option is non-nil, \\[next-line], \\[previous-line], \\[move-end-
> > of-line], and \\[move-beginning-of-line] behave
> > as if newlines that are invisible didn't exist, and count
> > only visible newlines.  Thus, moving across across 2 newlines
> > one of which is invisible will be counted as a one-line move.
> > Also, a non-nil value causes invisible text to be ignored when
> > counting columns for the purposes of keeping point in the same
> > column by \\[next-line] and \\[previous-line].
> > 
> > Outline mode sets this."
> > 
> > I hope this answers all of your questions in #1.
> 
> Very good; thanks.


> (I don't think there should be a blank line after the first line,
> but maybe that is just a mail artifact.)

It's not; it's a standard formatting of a doc string, AFAIK.

> > > 2. The doc string speaks of invisible lines.  But (elisp) `Invisible
> > > Text' speaks of "invisible newlines" (not lines), which is presumably
> > > something different (newline chars vs lines of any chars except newline,
> > > possibly including the separating newlines).  Are both true?  Which?
> > 
> > I think the doc string now clarifies this as well.
> 
> Yes, thanks.  But the manual speaks only of invisible newlines, and to
> me this part is not clear.

The doc string now speaks about that as well.  What's not clear about
that?  A newline is just a character, and as such can be invisible.

> And whenever we speak of newlines (especially
> where we are also talking about doing something wrt lines in general),
> we should say "newline characters" or "newline chars".  A "newline" as
> such doesn't really exist in our vocabulary (or at least it shouldn't),
> and some people might read it as meaning a "new line".

I'd never suspect this could be a source of confusion in Emacs.  The
Glossary says:

  Newline
       Control-J characters in the buffer terminate lines of text and are
       therefore also called newlines.

> > > 5. After saying that we provide option `line-move-ignore-invisible'
> > > specifically to let you prevent line motion commands from ignoring
> > > invisible newlines (whatever that might mean!), this node says that if
> > > ANY command ends with point in a certain position relative to invisible
> > > text, then the cursor is automatically moved past that stretch of
> > > invisible text (one direction or the other).  How is this related to the
> > > previous text about line motion commands and
> > > `line-move-ignore-invisible'?
> > 
> > It isn't related to line-move-ignore-invisible.  It is related to the
> > broader issue described by that node, which is invisible text in
> > general.
> 
> Yes, I sensed that.  I found (find) the juxtaposition confusing.
> Maybe separate the two discussions better, and perhaps give an example
> of interaction (or lack thereof) between the two.

It's a separate paragraph already, and I removed the leading
"However", which might hint on some too tight relation.

bug#17511: 24.4.50; `line-move-ignore-invisible': doc and purpose not clear

[Drew Adams]

> > Thanks for improving this.
> 
> Can this bug be closed, then?

It's up to you.  I have some comments below, but you are of course
free to ignore them or disagree with them.

> > (I don't think there should be a blank line after the first line,
> > but maybe that is just a mail artifact.)
> 
> It's not; it's a standard formatting of a doc string, AFAIK.

I don't think so.  Perhaps I've been operating under a misconception
all these years.  For Lisp code I've seen such a blank line only
occasionally (rarely), and nearly always in 3rd-party code and typically
from newbies.

Take a look at the Emacs Lisp sources.  You will see such a blank line
only rarely.

Look at simple.el, for example. There are only a few doc strings that
have such a blank line.  I count only these 17 out of the *hundreds*
of doc strings in simple.el:

`next-error-buffer-p', `next-error-find-buffer', `next-error',
`previous-error', `shell-command-history', `async-shell-command',
`process-file-side-effects', `start-file-process', `mark',
`region-active-p', `shift-select-mode', `default-line-height',
`window-screen-lines', `set-variable-value-history',
`create-indirect-buffer', `normal-erase-is-backspace',
`define-alternatives'.

I thought that not adding a blank line after the first line was a
doc-string convention/guideline/standard (the opposite of what you
say), in addition to reflecting the overwhelming practice.

But I do not see it mentioned at (elisp) `Documentation Tips', so
I guess I was wrong about that (as are you?).  (I do see mention
of the first paragraph being used for disabled command help, but
is not so important here.)

So take my input on this as just one opinion.  I don't see that
adding a blank line after the first helps users - in *Help* output,
for example.  But clearly it is not proscribed, and there is not
even a published guideline against it.

> > > > 2. The doc string speaks of invisible lines.  But (elisp) `Invisible
> > > > Text' speaks of "invisible newlines" (not lines), which is presumably
> > > > something different (newline chars vs lines of any chars except
> > > > newline, possibly including the separating newlines).  Are both true?
> > > > Which?
> > >
> > > I think the doc string now clarifies this as well.
> >
> > Yes, thanks.  But the manual speaks only of invisible newlines, and to
> > me this part is not clear.
> 
> The doc string now speaks about that as well.  What's not clear about
> that?  A newline is just a character, and as such can be invisible.

I told you it was not clear to me, as one reader.  Previously, the doc
string spoke only of invisible lines, and the manual spoke only of
invisible newlines.  The doc string now mentions invisible newline
chars too - good.  Does the manual mention invisible lines of text?

If the fact that I found this confusing helps you improve it, fine.
If not, fine.

> > And whenever we speak of newlines (especially
> > where we are also talking about doing something wrt lines in general),
> > we should say "newline characters" or "newline chars".  A "newline" as
> > such doesn't really exist in our vocabulary (or at least it shouldn't),
> > and some people might read it as meaning a "new line".
> 
> I'd never suspect this could be a source of confusion in Emacs.  The
> Glossary says:
> 
>   Newline
>        Control-J characters in the buffer terminate lines of text and are
>        therefore also called newlines.

OK.  I disagree that that is the right approach, but it is the approach
taken, so fine.  If it were I, I would always say "newline char", to be
clear.

> > Yes, I sensed that.  I found (find) the juxtaposition confusing.
> > Maybe separate the two discussions better, and perhaps give an example
> > of interaction (or lack thereof) between the two.
> 
> It's a separate paragraph already, and I removed the leading
> "However", which might hint on some too tight relation.

I'm sure it's better.  If you find it clear enough in this respect now,
that's good enough for me.

Feel free to close.

bug#17511: 24.4.50; `line-move-ignore-invisible': doc and purpose not clear

[Eli Zaretskii]

> Date: Sat, 17 May 2014 09:03:06 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: 17511@debbugs.gnu.org
> 
> > > (I don't think there should be a blank line after the first line,
> > > but maybe that is just a mail artifact.)
> > 
> > It's not; it's a standard formatting of a doc string, AFAIK.
> 
> I don't think so.  Perhaps I've been operating under a misconception
> all these years.  For Lisp code I've seen such a blank line only
> occasionally (rarely), and nearly always in 3rd-party code and typically
> from newbies.

OK, I removed the empty line.

> > > > > 2. The doc string speaks of invisible lines.  But (elisp) `Invisible
> > > > > Text' speaks of "invisible newlines" (not lines), which is presumably
> > > > > something different (newline chars vs lines of any chars except
> > > > > newline, possibly including the separating newlines).  Are both true?
> > > > > Which?
> > > >
> > > > I think the doc string now clarifies this as well.
> > >
> > > Yes, thanks.  But the manual speaks only of invisible newlines, and to
> > > me this part is not clear.
> > 
> > The doc string now speaks about that as well.  What's not clear about
> > that?  A newline is just a character, and as such can be invisible.
> 
> I told you it was not clear to me, as one reader.  Previously, the doc
> string spoke only of invisible lines, and the manual spoke only of
> invisible newlines.  The doc string now mentions invisible newline
> chars too - good.  Does the manual mention invisible lines of text?

I see no reason to mention invisible lines, because that might be
confusing: what matters are not the lines, but the newlines.
Therefore, the doc string now only talks about newlines, and the
manual now says:

    Ordinarily, functions that operate on text or move point do not care
  whether the text is invisible, they process invisible characters and
  visible characters alike.  The user-level line motion commands,
  such as @code{next-line}, @code{previous-line}, ignore invisible
  newlines if @code{line-move-ignore-invisible} is non-@code{nil} (the
  default), i.e., behave like these invisible newlines didn't exist in
  the buffer, but only because they are explicitly programmed to do so.

> > > Yes, I sensed that.  I found (find) the juxtaposition confusing.
> > > Maybe separate the two discussions better, and perhaps give an example
> > > of interaction (or lack thereof) between the two.
> > 
> > It's a separate paragraph already, and I removed the leading
> > "However", which might hint on some too tight relation.
> 
> I'm sure it's better.  If you find it clear enough in this respect now,
> that's good enough for me.

Feel free to file another bug report if you find the new text in the
manual still confusing about the relation between
line-move-ignore-invisible and point adjustments.

bug#17511: 24.4.50; `line-move-ignore-invisible': doc and purpose not clear

[Drew Adams]

> OK, I removed the empty line.

Thx.  Maybe it would be good to decide on the convention to use and
document it in the guidelines.  Just a suggestion.

> I see no reason to mention invisible lines, because that might be
> confusing: what matters are not the lines, but the newlines.
> Therefore, the doc string now only talks about newlines, and the
> manual now says:
> 
>     Ordinarily, functions that operate on text or move point do not care
>   whether the text is invisible, they process invisible characters and
>   visible characters alike.  The user-level line motion commands,
>   such as @code{next-line}, @code{previous-line}, ignore invisible
>   newlines if @code{line-move-ignore-invisible} is non-@code{nil} (the
>   default), i.e., behave like these invisible newlines didn't exist in
>   the buffer, but only because they are explicitly programmed to do so.

Looks good.  I will close the bug.