bug#31716: move_it_in_display_line / horizontal scrolling / tab stretch

[Eli Zaretskii <eliz@gnu.org>]

> Date: Mon, 04 Jun 2018 17:38:17 -0700
> From: Keith David Bershatsky <esq@lawlist.com>
> 
> Step 3.  Paste the following Lisp code into a scratch buffer and evaluate the 
> code.
> 
>     (setq display-line-numbers t)
>     (setq buffer-display-table (make-display-table))
>     (aset buffer-display-table
>           ?\t
>           (vector (make-glyph-code ?\u00BB 'font-lock-warning-face)
>                   (make-glyph-code ?\t 'highlight)))
>     (setq tab-width 8)
> 
> Step 4.  On a new line in the scratch buffer type:  C-q TAB Hello
> 
> Step 5.  Place the cursor somewhere on the word "Hello".
> 
> Step 6.  Evaluate:  (scroll-left 2)
> 
> Step 7.  Evaluate:  (bug-christmas-ghost)
> 
> OBSERVATIONS:
> 
> A.  When w->hscroll is >= 2 and IT is within the tab STRETCH, 
> move_it_in_display_line_to reports an erroneous it.hpos subsequent to the line 
> numbers.  In this example, it.hpos is reported as being either 0 or 5, even 
> though the tab STRETCH visibly begins at an it.hpos of 4 when line numbers are 
> 2 digits wide.
> 
> B.  When w->hscroll is >= 2 and IT is within the tab STRETCH, the latter 
> portion contains one or more it.c ghosts of Christmas future.  I.e., the letter 
> "H" gets returned at several locations along the tab STRETCH.
> 
> EXPECTATIONS / DESIRED BEHAVIOR.
> 
> i.  Loops 7 to 12 should report an it.hpos of 4.
> 
> ii.  Loops 7 to 12 should report an it.c of 9 (aka a tab character).

Summary: Your expectations are incorrect, and the code works as
expected, AFAICT.

Details:

There seems to be a misunderstanding of what MOVE_TO_X means for
move_it_in_display_line_to.  It sounds like you expect it to end up
and the new_x coordinate, or report data about what's at new_x on
display.  But that's not what MOVE_TO_X actually does.  In reality,
move_it_in_display_line_to cannot move with 1-pixel granularity, it
can only move one "display element" at a time.  MOVE_TO_X then tells
it to stop at a display element whose display _includes_ the
coordinate new_x.  A display element that begins at X and has pixel
width of WX includes the coordinates in [X..X+WX), i.e. X+WX itself is
NOT included.  So if, for some display element that starts at X, new_x
satisfies the inequality

        X <= new_x < X+WX

then move_it_in_display_line_to will stop at X and return
MOVE_X_REACHED.

On a GUI frame, a TAB produces a single display element: a stretch
glyph whose width is the required TAB width.  In this case, it's 56
pixels, 14 of which are invisible due to hscroll.

With that knowledge in hand, you need to re-asses the results you
obtain.  Most importantly, it.current_x tells you where did
move_it_in_display_line_to actually stop; it doesn't matter where you
asked it to stop (which is new_x).

> [IT.HPOS IS 4, NOT 0.]
> 7.  TAB STRETCH
>     it.c (9)
>     char (      )
>     it.first_visible_x (14)
>     w->hscroll (2)
>     it.hpos (0)
>     it.first_visible_x (14)
>     it.current_x (7)  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>     relative_x (28)
>     new_x (42)
>     font->space_width (7)
>     it.pixel_width (49)
>     it.lnum_pixel_width (28)
>     rc (MOVE_X_REACHED)

> [IT.HPOS IS 4, NOT 0.]
> 8.  TAB STRETCH
>     it.c (9)
>     char (      )
>     it.first_visible_x (14)
>     w->hscroll (2)
>     it.hpos (0)
>     it.first_visible_x (14)
>     it.current_x (7)  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>     relative_x (35)
>     new_x (49)
>     font->space_width (7)
>     it.pixel_width (49)
>     it.lnum_pixel_width (28)
>     rc (MOVE_X_REACHED)

> [IT.C IS 9, NOT 72; AND, IT.HPOS IS 4, NOT 5.]
> 9.  TEXT
>     it.c (72)
>     char (H)
>     it.first_visible_x (14)
>     w->hscroll (2)
>     it.hpos (5)
>     it.first_visible_x (14)
>     it.current_x (84)  <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>     relative_x (42)
>     new_x (56)

As you see, the first two calls ended up at the same X coordinate of
7, while the 3rd call stopped at X = 84, and first_visible_x is 14.
Since 7 < 14, the first two calls stopped on the TAB, and HPOS is zero
because the TAB is hscrolled (HPOS is only advanced for X coordinates
beyond first_visible_x).  The 3rd call stopped at X = 84, which is the
coordinate of 'H', because 84 = 56 + 28, the width needed for 56
pixels of the TAB and 28 pixels of the line number.  So the 3rd call
reports on 'H', and its HPOS is 5, because 4 columns are taken by the
line-number digits and one more is taken by the (partially visible)
stretch glyph that displays the TAB.

Then you see 4 more calls, all of which stop at X = 84, so they all
report on 'H':

> [IT.C IS 9, NOT 72; AND, IT.HPOS IS 4, NOT 5.]
> 10.  TEXT
>     it.c (72)
>     char (H)
>     it.first_visible_x (14)
>     w->hscroll (2)
>     it.hpos (5)
>     it.first_visible_x (14)
>     it.current_x (84)
>     relative_x (49)
>     new_x (63)
>     font->space_width (7)
>     it.pixel_width (7)
>     it.lnum_pixel_width (28)
>     rc (MOVE_X_REACHED)

> [IT.C IS 9, NOT 72; AND, IT.HPOS IS 4, NOT 5.]
> 11.  TEXT
>     it.c (72)
>     char (H)
>     it.first_visible_x (14)
>     w->hscroll (2)
>     it.hpos (5)
>     it.first_visible_x (14)
>     it.current_x (84)
>     relative_x (56)
>     new_x (70)
>     font->space_width (7)
>     it.pixel_width (7)
>     it.lnum_pixel_width (28)
>     rc (MOVE_X_REACHED)

> [IT.C IS 9, NOT 72; AND, IT.HPOS IS 4, NOT 5.]
> 12.  TEXT
>     it.c (72)
>     char (H)
>     it.first_visible_x (14)
>     w->hscroll (2)
>     it.hpos (5)
>     it.first_visible_x (14)
>     it.current_x (84)
>     relative_x (63)
>     new_x (77)
>     font->space_width (7)
>     it.pixel_width (7)
>     it.lnum_pixel_width (28)
>     rc (MOVE_X_REACHED)

> [THIS IS THE REAL LETTER "H"]
> 13.  TEXT
>     it.c (72)
>     char (H)
>     it.first_visible_x (14)
>     w->hscroll (2)
>     it.hpos (5)
>     it.first_visible_x (14)
>     it.current_x (84)
>     relative_x (70)
>     new_x (84)
>     font->space_width (7)
>     it.pixel_width (7)
>     it.lnum_pixel_width (28)
>     rc (MOVE_X_REACHED)

IOW, these 'H' are not "ghosts", they are the actual character
displayed at the same location X = 84.  You just get 4 reports on the
same character because move_it_in_display_line_to is at the same
coordinate.  So I see nothing wrong here.

Note that on TTY frames, a TAB is displayed as several separate SPC
characters, so in that case you _can_ move by individual increments
inside the TAB display.