Native display of line numbers: visual line-counting

Native display of line numbers: visual line-counting

[Eli Zaretskii]

So a few people requested this feature, and I started thinking about
its implementation.  Immediately I bumped into several issues that I'm
not sure how to resolve, since I don't really understand how this will
be used in Emacs.

First, this makes sense only for relative line numbers, right?  If not
I don't really understand what would be the definition of "visually
counted first line shown in a window".  If everyone agrees, it would
mean that the current line will have the number of zero, right?  or do
people still want to see the "physical line number" for the current
line in this case?

Next, the only use case which I fully understand is the one with
some of the lines folded, as in Org or Outline modes.  More
accurately, the case where text is not displayed because it was made
invisible by the 'invisible' property of some kind -- these lines are
not to be counted under this mode.

But that is not the only case where what is displayed does not
correspond to buffer text 1:1.  Some text on the screen might come
from display properties or from before- and after-strings.  These can
include embedded newlines, and this contribute to "visual lines" Emacs
displays.  Are these lines to be counted or not?  Should Emacs display
something for their "line numbers" or not?

We need to resolve all these and possibly other subtleties before the
feature can be implemented.  Ideas and opinions are welcome.

TIA

Re: Native display of line numbers: visual line-counting

[Joseph Garvin]

[-- Attachment #1: Type: text/plain, Size: 2296 bytes --]

For my own use of relative line numbers (speech recognition controlled
programming) what's important is that the line numbering matches the number
of times I'd press the up/down arrow keys or C-p/C-n to move my cursor to
the labeled line. The whole reason for me to use the feature is to make it
easier by eyeballing to predict where the cursor will land after I say "up
4", "down 7", etc. So I don't care at all about the numbering matching
physical lines in the file.

This isn't the case with any of the relative line number packages I've
tried so far with visual line mode so it'd be awesome to have a reliable
mode available. I actually filed a bug against one of the packages awhile
back:

https://github.com/Fanael/relative-line-numbers/issues/6



On Jun 24, 2017 1:15 PM, "Eli Zaretskii" <eliz@gnu.org> wrote:

> So a few people requested this feature, and I started thinking about
> its implementation.  Immediately I bumped into several issues that I'm
> not sure how to resolve, since I don't really understand how this will
> be used in Emacs.
>
> First, this makes sense only for relative line numbers, right?  If not
> I don't really understand what would be the definition of "visually
> counted first line shown in a window".  If everyone agrees, it would
> mean that the current line will have the number of zero, right?  or do
> people still want to see the "physical line number" for the current
> line in this case?
>
> Next, the only use case which I fully understand is the one with
> some of the lines folded, as in Org or Outline modes.  More
> accurately, the case where text is not displayed because it was made
> invisible by the 'invisible' property of some kind -- these lines are
> not to be counted under this mode.
>
> But that is not the only case where what is displayed does not
> correspond to buffer text 1:1.  Some text on the screen might come
> from display properties or from before- and after-strings.  These can
> include embedded newlines, and this contribute to "visual lines" Emacs
> displays.  Are these lines to be counted or not?  Should Emacs display
> something for their "line numbers" or not?
>
> We need to resolve all these and possibly other subtleties before the
> feature can be implemented.  Ideas and opinions are welcome.
>
> TIA
>
>

[-- Attachment #2: Type: text/html, Size: 2999 bytes --]

Re: Native display of line numbers: visual line-counting

[Eli Zaretskii]

> From: Joseph Garvin <k04jg02@gmail.com>
> Date: Sat, 24 Jun 2017 14:03:27 -0500
> Cc: emacs-devel@gnu.org
> 
> For my own use of relative line numbers (speech recognition controlled programming) what's important is
> that the line numbering matches the number of times I'd press the up/down arrow keys or C-p/C-n to move
> my cursor to the labeled line. The whole reason for me to use the feature is to make it easier by eyeballing to
> predict where the cursor will land after I say "up 4", "down 7", etc. So I don't care at all about the numbering
> matching physical lines in the file.

What about lines where cursor cannot enter?  Are you saying you don't
want to count them?

Also, you seem to be saying that lines that span multiple screen lines
should be counted that many times.  If so, the numbers are not really
"line numbers" anymore, but something else.  Moreover, these lines
cannot be communicated to anyone else, as their window geometry could
be different.

Re: Native display of line numbers: visual line-counting

[Joseph Garvin]

[-- Attachment #1: Type: text/plain, Size: 2298 bytes --]

On Jun 24, 2017 2:22 PM, "Eli Zaretskii" <eliz@gnu.org> wrote:

> From: Joseph Garvin <k04jg02@gmail.com>
> Date: Sat, 24 Jun 2017 14:03:27 -0500
> Cc: emacs-devel@gnu.org
>
> For my own use of relative line numbers (speech recognition controlled
programming) what's important is
> that the line numbering matches the number of times I'd press the up/down
arrow keys or C-p/C-n to move
> my cursor to the labeled line. The whole reason for me to use the feature
is to make it easier by eyeballing to
> predict where the cursor will land after I say "up 4", "down 7", etc. So
I don't care at all about the numbering
> matching physical lines in the file.

What about lines where cursor cannot enter?  Are you saying you don't
want to count them?


I was just trying to illustrate my use case, I didn't mean to imply
anything about lines you can't enter. I typically am editing source files
where all lines can be entered. But now that you mention it, if for example
there was an overlay making it appear as if more lines were in the buffer
than there actually were, then yes ideally those would not count. That's
not something I run into very often though. A solution that didn't handle
this edge case but otherwise had relative visual line numbers would still
be a big improvement.


Also, you seem to be saying that lines that span multiple screen lines
should be counted that many times.  If so, the numbers are not really
"line numbers" anymore, but something else.


From a text editor programmer's perspective line number may be defined by
something like, "number of preceding newline characters" but I think most
users think in terms of the lines of text as they are rendered on the
display. Most non-emacs editors and IDEs nowadays use wrapping that acts
like visual line mode by default. I would call them "visual line numbers."

Moreover, these lines
cannot be communicated to anyone else, as their window geometry could
be different.


Relative line numbers already can't be communicated anyway, since the other
person would have to have their cursor on the same line. But I don't think
users of relative lines are using it for communicating with other people.
They're using it so they can more easily plan a series of keystrokes or
voice commands that they're going to issue themselves.

[-- Attachment #2: Type: text/html, Size: 3492 bytes --]

Re: Native display of line numbers: visual line-counting

[James Nguyen]

>> First, this makes sense only for relative line numbers, right?  If not
>> I don't really understand what would be the definition of "visually
>> counted first line shown in a window".  If everyone agrees, it would
>> mean that the current line will have the number of zero, right?  or do
>> people still want to see the "physical line number" for the current
>> line in this case?

I am thinking about it in terms of code folding.
Each fold becomes the mark of the new line number.

1. static int internal_self_insert (int, EMACS_INT); // Absolute line number of 1
1. DEFUN ("forward-point", Fforward_point, Sforward_point, 1, 1, 0,
2.        doc: /* Return buffer position N characters after (before if N negative) point.  */)
3.   (Lisp_Object n)
4. {
5.   CHECK_NUMBER (n);
6. 
7.   return make_number (PT + XINT (n));
8. }
9. 
10. 
11. static Lisp_Object
12. move_point (Lisp_Object n, bool forward)
13. {
14.   SET_PT (new_point);
15.   return Qnil;
16. }

Becomes

1. static int internal_self_insert (int, EMACS_INT); {} // FOLD // Absolute Line number of 1
1. 
2. 
3. static Lisp_Object
4. move_point (Lisp_Object n, bool forward) {} // FOLD

I'd like the current line to still represent the current absolute line number
of the file.


> On Jun 24, 2017, at 11:15 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> So a few people requested this feature, and I started thinking about
> its implementation.  Immediately I bumped into several issues that I'm
> not sure how to resolve, since I don't really understand how this will
> be used in Emacs.
> 
> First, this makes sense only for relative line numbers, right?  If not
> I don't really understand what would be the definition of "visually
> counted first line shown in a window".  If everyone agrees, it would
> mean that the current line will have the number of zero, right?  or do
> people still want to see the "physical line number" for the current
> line in this case?
> 
> Next, the only use case which I fully understand is the one with
> some of the lines folded, as in Org or Outline modes.  More
> accurately, the case where text is not displayed because it was made
> invisible by the 'invisible' property of some kind -- these lines are
> not to be counted under this mode.
> 
> But that is not the only case where what is displayed does not
> correspond to buffer text 1:1.  Some text on the screen might come
> from display properties or from before- and after-strings.  These can
> include embedded newlines, and this contribute to "visual lines" Emacs
> displays.  Are these lines to be counted or not?  Should Emacs display
> something for their "line numbers" or not?
> 
> We need to resolve all these and possibly other subtleties before the
> feature can be implemented.  Ideas and opinions are welcome.
> 
> TIA
> 

Re: Native display of line numbers: visual line-counting

[Stefan Monnier]

> So a few people requested this feature, and I started thinking about
> its implementation.  Immediately I bumped into several issues that I'm
> not sure how to resolve, since I don't really understand how this will
> be used in Emacs.

Indeed, it's non-trivial.  E.g. if it's used for things like "go up
N lines", then it's important to make sure that the notion of "visual
line" is exactly the same for the "go up N lines" command as it is for
the line-number display code.


        Stefan

Re: Native display of line numbers: visual line-counting

[Joseph Garvin]

[-- Attachment #1: Type: text/plain, Size: 1047 bytes --]

On Sun, Jun 25, 2017 at 2:16 PM, Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > So a few people requested this feature, and I started thinking about
> > its implementation.  Immediately I bumped into several issues that I'm
> > not sure how to resolve, since I don't really understand how this will
> > be used in Emacs.
>
> Indeed, it's non-trivial.  E.g. if it's used for things like "go up
> N lines", then it's important to make sure that the notion of "visual
> line" is exactly the same for the "go up N lines" command as it is for
> the line-number display code.


I take it for granted when using emacs that there is some configuration
work involved. As long as there is a function I can bind for up/down that
matches the the relative line numbering, it's not super important to me
that they automatically are synced somehow. FWIW, visual line mode
automagically rebinds the navigation keys already, so if the relative line
numbering was just labeling the lines as visual line mode renders them,
rendering and input should line up.

[-- Attachment #2: Type: text/html, Size: 1487 bytes --]

Re: Native display of line numbers: visual line-counting

[Stefan Monnier]

> I take it for granted when using emacs that there is some configuration
> work involved. As long as there is a function I can bind for up/down that
> matches the the relative line numbering, it's not super important to me
> that they automatically are synced somehow. FWIW, visual line mode
> automagically rebinds the navigation keys already, so if the relative line
> numbering was just labeling the lines as visual line mode renders them,
> rendering and input should line up.

The problem I was alluding to is that the efficient way for the
redisplay to provide relative visual-line numbers (i.e. working
directly on the matrices) would provide a semantics which I'm not sure
matches faithfully the functions available to Elisp
(e.g. vertical-motion).

So, this argues for the use of the more expensive move_it_* functions to
compute the relative visual-line numbers (IIUC these are also easier to
integrate in the current code, according to Eli).


        Stefan

Re: Native display of line numbers: visual line-counting

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Sun, 25 Jun 2017 16:20:30 -0400
> Cc: emacs-devel@gnu.org
> 
> The problem I was alluding to is that the efficient way for the
> redisplay to provide relative visual-line numbers (i.e. working
> directly on the matrices) would provide a semantics which I'm not sure
> matches faithfully the functions available to Elisp
> (e.g. vertical-motion).

Right.  One other problem with using the glyph matrices is that it's
not easy to glean from them what object with what properties each
glyph came from, especially when the glyphs came from an overlay.

Re: Native display of line numbers: visual line-counting

[Eli Zaretskii]

> From: Joseph Garvin <joseph.h.garvin@gmail.com>
> Date: Sun, 25 Jun 2017 14:31:43 -0500
> Cc: emacs-devel@gnu.org
> 
> I take it for granted when using emacs that there is some configuration work involved. As long as there is a
> function I can bind for up/down that matches the the relative line numbering, it's not super important to me
> that they automatically are synced somehow. FWIW, visual line mode automagically rebinds the navigation
> keys already, so if the relative line numbering was just labeling the lines as visual line mode renders them,
> rendering and input should line up.

I don't think it's a good idea to invent yet another way of vertical
cursor motion.  We have enough trouble with the 2 ways we have now.

Re: Native display of line numbers: visual line-counting

[Eli Zaretskii]

> From: Filipe Silva <filipe.silva@gmail.com>
> Date: Tue, 27 Jun 2017 10:29:28 -0300
> 
> Having the real line number instead of 0 would give us a more useful information instead of just 0, but showing
> 0 is totally fine if it makes the implementation easier.

It doesn't matter from the implementation POV, so real line number it
is.

> in vim, the visual relative line number does not take into account wrapped lines. if a line is big enough that
> spans multiple lines in the buffer, all of that count as 1 and only the first segment of the wrapper line is
> numbered. you can always use 'gj' to navigate wrapper lines.

This contradicts the only set of requirements I saw posted about this
(by several people): show line numbers that are consistent with
vertical movement commands like C-n and C-p.  Which means wrapped
lines should be counted, as Emacs moves visually, at least by default.

> Yes, org-mode and folding are the only cases I know off that would require a visual relative line number
> implementation. I don't know what is display properties, nor before and after strings. Can someone state an
> example ?

If you use magit, you have a lot of them, I think.

Here's a toy example, due to Dmitry.  Evaluate this in *scratch*, then
switch to the popup-test.el buffer.  The multi-line "aaaaa" part is an
overlay after-string.

(with-current-buffer (get-buffer-create "popup-test.el")
   (setq display-line-numbers t)
   (insert "aaaaaaa
aaaaaaa
aaaaaaa
aaaaaaa
aaaaaaa
")

   (let ((ov (make-overlay (point-max) (point-max))))
     (overlay-put ov 'after-string "bbbbbb\nbbbbbb\n")
     (overlay-put ov 'display-line-numbers-disable t)))

Re: Native display of line numbers: visual line-counting

[Filipe Silva]

[-- Attachment #1: Type: text/plain, Size: 886 bytes --]

​I see Eli, you are absolutelly right. I did not know that this was the
default. It makes sense to behave as you described by default.

I'd like to configure the default to what I described though, maybe through
some elisp hackage​. Will that be possible. It's not a unsolveable problem
if this is not possible. I can always turn off the truncation.

About the Overlays, now I see what you mean. It would make sense that the
relative line numbers would give me only possible places to jump to with
C-n and C-p.

On Tue, Jun 27, 2017 at 12:00 PM, Eli Zaretskii <eliz@gnu.org> wrote:

>
> This contradicts the only set of requirements I saw posted about this
> (by several people): show line numbers that are consistent with
> vertical movement commands like C-n and C-p.  Which means wrapped
> lines should be counted, as Emacs moves visually, at least by default.

[-- Attachment #2: Type: text/html, Size: 1643 bytes --]