bug#51590: Tab-line breaks windows of follow-mode

bug#51590: Tab-line breaks windows of follow-mode

[Juri Linkov]

In emacs -Q visit a long buffer, split the window horizontally,
and enable 'M-x follow-mode'.  A partially-visible bottom line
of the left window will help to observe that the same line
is completely visible at the top of the right window.
This is a nice feature that allows follow-mode to work correctly.

Now enable 'M-x global-tab-line-mode' and observe that
the same line is still visible at the top of the right window,
whereas the bottom line of the left window shows another
partially-visible line.  This is a bug.

The most low-level function of follow-mode is follow-calc-win-end.
When global-tab-line-mode is enabled, follow-calc-win-end
returns the same values as when global-tab-line-mode is disabled.

I don't understand what more low-level function doesn't take
into account the height of the tab-line.  Maybe the problem is
in pos-visible-in-window-p?  Or maybe different values returned
by window-inside-pixel-edges and window-end?

bug#51590: Tab-line breaks windows of follow-mode

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Date: Wed, 03 Nov 2021 20:24:18 +0200
> 
> The most low-level function of follow-mode is follow-calc-win-end.
> When global-tab-line-mode is enabled, follow-calc-win-end
> returns the same values as when global-tab-line-mode is disabled.

Can you step through that function and see which part is misbehaving
there?

> I don't understand what more low-level function doesn't take
> into account the height of the tab-line.  Maybe the problem is
> in pos-visible-in-window-p?  Or maybe different values returned
> by window-inside-pixel-edges and window-end?

I don't know the answer, but I see that header-line-format is
mentioned in one place in follow.el, and that is not a good sign...

bug#51590: follow-mode is broken with header-line and tab-line

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 971 bytes --]

retitle 51590 follow-mode is broken with header-line and tab-line
quit

>> The most low-level function of follow-mode is follow-calc-win-end.
>> When global-tab-line-mode is enabled, follow-calc-win-end
>> returns the same values as when global-tab-line-mode is disabled.
>
> Can you step through that function and see which part is misbehaving
> there?
>
>> I don't understand what more low-level function doesn't take
>> into account the height of the tab-line.  Maybe the problem is
>> in pos-visible-in-window-p?  Or maybe different values returned
>> by window-inside-pixel-edges and window-end?
>
> I don't know the answer, but I see that header-line-format is
> mentioned in one place in follow.el, and that is not a good sign...

Indeed, I tested with the header-line, and it has the same problem,
so retitled this bug report.

After trial and error, I arrived to the following patch that completely
fixes these problems.  However, I can't explain how it works.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: follow-line-height.patch --]
[-- Type: text/x-diff, Size: 963 bytes --]

diff --git a/lisp/follow.el b/lisp/follow.el
index b64f4cb734..327c66d5ff 100644
--- a/lisp/follow.el
+++ b/lisp/follow.el
@@ -679,6 +679,7 @@ follow-scroll-down
 	     (goto-char start)
 	     (vertical-motion (- (- (window-height win)
 				    (if header-line-format 2 1)
+				    (if tab-line-format 2 1)
 				    next-screen-context-lines)))
 	     (set-window-start win (point))
 	     (goto-char start)
@@ -946,8 +947,10 @@ follow-calc-win-end
 used."
   (let* ((win (or win (selected-window)))
 	 (edges (window-inside-pixel-edges win))
-	 (ht (- (nth 3 edges) (nth 1 edges)))
-	 (last-line-pos (posn-point (posn-at-x-y 0 (1- ht) win))))
+	 (ht (+ (- (nth 3 edges) (nth 1 edges))
+		(window-header-line-height win)
+		(window-tab-line-height win)))
+	 (last-line-pos (posn-point (posn-at-x-y 0 (- ht 1) win))))
     (if (pos-visible-in-window-p last-line-pos win)
 	(let ((end (window-end win t)))
 	  (list end (pos-visible-in-window-p (point-max) win)))

bug#51590: follow-mode is broken with header-line and tab-line

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: 51590@debbugs.gnu.org
> Date: Thu, 04 Nov 2021 19:29:28 +0200
> 
> After trial and error, I arrived to the following patch that completely
> fixes these problems.  However, I can't explain how it works.

Alan, any comments?

bug#51590: follow-mode is broken with header-line and tab-line

[martin rudalics]

MHO.> After trial and error, I arrived to the following patch that completely
 > fixes these problems.  However, I can't explain how it works.

I didn't try but suppose it's still broken with a horizontal scroll bar,
high mode lines, remapped faces and the like.  This is mostly suited for
TTYs I think.

martin

bug#51590: follow-mode is broken with header-line and tab-line

[Alan Mackenzie]

Hello, Eli and Juri.

On Thu, Nov 04, 2021 at 20:46:45 +0200, Eli Zaretskii wrote:
> > From: Juri Linkov <juri@linkov.net>
> > Cc: 51590@debbugs.gnu.org
> > Date: Thu, 04 Nov 2021 19:29:28 +0200

> > After trial and error, I arrived to the following patch that completely
> > fixes these problems.  However, I can't explain how it works.

> Alan, any comments?

I'll have a look.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#51590: follow-mode is broken with header-line and tab-line

[Stefan Kangas]

Juri Linkov <juri@linkov.net> writes:

> Now enable 'M-x global-tab-line-mode' and observe that
> the same line is still visible at the top of the right window,
> whereas the bottom line of the left window shows another
> partially-visible line.  This is a bug.

Could this be related to Bug#47498?

bug#51590: follow-mode is broken with header-line and tab-line

[Juri Linkov]

>> Now enable 'M-x global-tab-line-mode' and observe that
>> the same line is still visible at the top of the right window,
>> whereas the bottom line of the left window shows another
>> partially-visible line.  This is a bug.
>
> Could this be related to Bug#47498?

Indeed, this is the same problem fixed by the same proposed patch.

bug#51590: follow-mode is broken with header-line and tab-line

[Stefan Kangas]

merge 47498 51590
thanks

Juri Linkov <juri@linkov.net> writes:

>> Could this be related to Bug#47498?
>
> Indeed, this is the same problem fixed by the same proposed patch.

Thanks, I'm merging the bugs.

bug#51590: follow-mode is broken with header-line and tab-line

[Alan Mackenzie]

Hello, Eli and Juri.

On Thu, Nov 04, 2021 at 20:46:45 +0200, Eli Zaretskii wrote:
> > From: Juri Linkov <juri@linkov.net>
> > Cc: 51590@debbugs.gnu.org
> > Date: Thu, 04 Nov 2021 19:29:28 +0200

> > After trial and error, I arrived to the following patch that completely
> > fixes these problems.  However, I can't explain how it works.

> Alan, any comments?

Yes.  There's a bug in posn-at-x-y, in that either the function or the
documentation is wrong.

The doc says, rather unhelpfully, "By default, X and Y are relative to
[the] text area of the selected window.".  What does "[the] text area"
mean?  In other places, it means the same as "the body" of a window,
which is the area of text not including the mode line, header line or
tab-bar line.

In posn-at-x-y, "the text area" _INCLUDES_ THE HEADER LINE.  This can be
verified, in a GUI Emacs with

    M-: (posn-point (posn-at-x-y 0 18))
                                   ^^
This returns the buffer position of the earliest character in the
window.  The "18" means 18 pixels after the top of the header line.
Surely it ought to mean 18 pixels after the top of the first buffer line
in the window, i.e. the second window line.

This is a bug.

It has unfortunate consequences in (the current) follow-calc-win-end:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(defun follow-calc-win-end (&optional win)
  "Calculate the end position for window WIN.
Return (END-POS END-OF-BUFFER).

Actually, the position returned is the start of the line after
the last fully-visible line in WIN.  END-OF-BUFFER is t when EOB
is fully-visible in WIN.  If WIN is nil, the selected window is
used."
  (let* ((win (or win (selected-window)))
         (edges (window-inside-pixel-edges win))
         (ht (- (nth 3 edges) (nth 1 edges)))                 <======= 1
         (last-line-pos (posn-point (posn-at-x-y 0 (1- ht) win)))) <== 2
    (if (pos-visible-in-window-p last-line-pos win)           <======= 3
        (let ((end (window-end win t)))
          (list end (pos-visible-in-window-p (point-max) win)))
      (list last-line-pos nil))))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

At 1, ht "the height" is the height of the "body" of the window, i.e.
all lines not including the header-line or mode-line.

At 2, posn-point intends to find the bottom pixel in the text part of
the window.  If there is a partial line there, posn-at-x-y will be in
the partial line, otherwise it will be in the last line of the window.
last-line-pos is the buffer position of whichever line that was.

At 3, pos-visible-in-window-p will return nil for the partial line or t
for the full line.  In the nil case, the function result is the partial
line, in the t case, it's (window-end ...).

This is all well and good.  But because of the bug in posn-at-x-y, when
there's a partial line there, at 2 last-line-pos is set one line too
far towards point-max.  Thus, at 3, pos-visible-in-window-p spuriously
returns t.  (window-end) now returns the line after the partial line,
which is wrong.

It would appear this behaviour of posn-at-x-y is also in Emacs 27.2.  It
would probably be catastrophic to fix posn-at-x-y, since so many
programs will have compensated for this bug.  So, we should probably fix
the doc of the function.

But we can fix follow-calc-win-end, by replacing the form at 2 with:

    (last-line-pos (posn-point (+ (posn-at-x-y 0 (1- ht) win)
                                  (window-header-line-height win))))

And I think we (probably me ;-) should go through the elisp manual and
doc strings replacing vague phrases like "the text area of the window"
with explicit descriptions involving "the header line", etc.

Juri, I'm sorry I haven't looked at the first hunk of your patch yet,
the one in follow-scroll-down.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#51590: follow-mode is broken with header-line and tab-line

[Eli Zaretskii]

> Date: Fri, 5 Nov 2021 21:45:32 +0000
> Cc: Juri Linkov <juri@linkov.net>, 51590@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> Yes.  There's a bug in posn-at-x-y, in that either the function or the
> documentation is wrong.

It isn't a bug, it is a deliberately implemented behavior.  See also
bug#15783.

> The doc says, rather unhelpfully, "By default, X and Y are relative to
> [the] text area of the selected window.".  What does "[the] text area"
> mean?

In the ELisp manual, type "i text area of a window RET" and read there.

I've now made changes there to make clear that the header-line and the
tab-line _are_ included in the text area.

> In posn-at-x-y, "the text area" _INCLUDES_ THE HEADER LINE.

Yes.  As it should.

> It would appear this behaviour of posn-at-x-y is also in Emacs 27.2.  It
> would probably be catastrophic to fix posn-at-x-y, since so many
> programs will have compensated for this bug.  So, we should probably fix
> the doc of the function.

Done on the emacs-28 branch.

> But we can fix follow-calc-win-end, by replacing the form at 2 with:
> 
>     (last-line-pos (posn-point (+ (posn-at-x-y 0 (1- ht) win)
>                                   (window-header-line-height win))))

What about the tab-line (which AFAIU was the trigger for this bug)?

> And I think we (probably me ;-) should go through the elisp manual and
> doc strings replacing vague phrases like "the text area of the window"
> with explicit descriptions involving "the header line", etc.

That'd be going overboard.  We could place a cross-reference in some
of those places to the "Frame Layout" node, though.  Suggestions for
such places are welcome.

bug#51590: follow-mode is broken with header-line and tab-line

[Alan Mackenzie]

Hello, Eli.

On Sat, Nov 06, 2021 at 09:00:44 +0200, Eli Zaretskii wrote:
> > Date: Fri, 5 Nov 2021 21:45:32 +0000
> > Cc: Juri Linkov <juri@linkov.net>, 51590@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > Yes.  There's a bug in posn-at-x-y, in that either the function or the
> > documentation is wrong.

> It isn't a bug, it is a deliberately implemented behavior.  See also
> bug#15783.

> > The doc says, rather unhelpfully, "By default, X and Y are relative to
> > [the] text area of the selected window.".  What does "[the] text area"
> > mean?

> In the ELisp manual, type "i text area of a window RET" and read there.

> I've now made changes there to make clear that the header-line and the
> tab-line _are_ included in the text area.

I'm just confused at the moment, after having spent several hours trying
to sort things out in my head.

I think "window body" and "window text area" mean the same thing, but
I'm not sure.

The picture in elisp page "Basic Windows" seems to show "window body
height" as NOT including the header line or tab line.  That picture
seems to show the header line as being ABOVE the text area, not part of
it.

The current implementation of posn-at-x-y includes the header line in
its Y coordinate, BUT NOT THE TAB LINE.  I think this is a bug.

window-inside-pixel-edges, aka (window-edges &optional WINDOW BODY
ABSOLUTE PIXELWISE) with BODY and PIXELWISE non-nil, returns (according
to the elisp page) "the edges of WINDOW's body".  This "body" DOES NOT
include the tab line or the header line.  When I run the function on an
info page with tab-bar-mode enabled in my tty I get:

    (0 2 240 65)
       ^

..  This would appear to contradict the above notion of "body" as
including the header and tab lines.

It seems to me that we really need two distinct terms here, one for the
text area of the buffer including the header and tab lines, and another
for the "pure" text area without those lines.  Maybe something like
"extended body" vs. "text area", or something like that.

> > In posn-at-x-y, "the text area" _INCLUDES_ THE HEADER LINE.

> Yes.  As it should.

OK.

> > It would appear this behaviour of posn-at-x-y is also in Emacs 27.2.  It
> > would probably be catastrophic to fix posn-at-x-y, since so many
> > programs will have compensated for this bug.  So, we should probably fix
> > the doc of the function.

> Done on the emacs-28 branch.

Thanks!

> > But we can fix follow-calc-win-end, by replacing the form at 2 with:

> >     (last-line-pos (posn-point (+ (posn-at-x-y 0 (1- ht) win)
> >                                   (window-header-line-height win))))

Actually, that's not right.  It should be

(last-line-pos (posn-point (posn-at-x-y 
                            0 (+ (1- ht) (window-header-line-height
			                  win)))))
, but that's a mere detail.  It appears to solve much of Juri's bug.

> What about the tab-line (which AFAIU was the trigger for this bug)?

Yes, that needs to be incorporated, too.

The other hunk of Juri's patch, in follow-scroll-down, is needed, but I
don't think it's quite right when combined with my suggestion for
follow-calc-win-end above.

> > And I think we (probably me ;-) should go through the elisp manual and
> > doc strings replacing vague phrases like "the text area of the window"
> > with explicit descriptions involving "the header line", etc.

> That'd be going overboard.  We could place a cross-reference in some
> of those places to the "Frame Layout" node, though.  Suggestions for
> such places are welcome.

OK.  But I think there's confusion in the manual and doc strings, and
even the code, and that it's not just me.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#51590: follow-mode is broken with header-line and tab-line

[Eli Zaretskii]

> Date: Sat, 6 Nov 2021 11:50:48 +0000
> Cc: juri@linkov.net, 51590@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > I've now made changes there to make clear that the header-line and the
> > tab-line _are_ included in the text area.
> 
> I'm just confused at the moment, after having spent several hours trying
> to sort things out in my head.
> 
> I think "window body" and "window text area" mean the same thing, but
> I'm not sure.

They aren't.  "Text area" includes the header-line and the tab-line,
the body doesn't.

> The picture in elisp page "Basic Windows" seems to show "window body
> height" as NOT including the header line or tab line.  That picture
> seems to show the header line as being ABOVE the text area, not part of
> it.

The updated picture doesn't have "text area" written on it at all.

> The current implementation of posn-at-x-y includes the header line in
> its Y coordinate, BUT NOT THE TAB LINE.  I think this is a bug.

Yes.  Please fix it.

> > > And I think we (probably me ;-) should go through the elisp manual and
> > > doc strings replacing vague phrases like "the text area of the window"
> > > with explicit descriptions involving "the header line", etc.
> 
> > That'd be going overboard.  We could place a cross-reference in some
> > of those places to the "Frame Layout" node, though.  Suggestions for
> > such places are welcome.
> 
> OK.  But I think there's confusion in the manual and doc strings, and
> even the code, and that it's not just me.

_After_ my today's changes?  If so, please show where is the confusing
text.

bug#51590: follow-mode is broken with header-line and tab-line

[martin rudalics]

 >> I think "window body" and "window text area" mean the same thing, but
 >> I'm not sure.
 >
 > They aren't.  "Text area" includes the header-line and the tab-line,
 > the body doesn't.
 >
 >> The picture in elisp page "Basic Windows" seems to show "window body
 >> height" as NOT including the header line or tab line.  That picture
 >> seems to show the header line as being ABOVE the text area, not part of
 >> it.
 >
 > The updated picture doesn't have "text area" written on it at all.

I'm afraid this is a change for the worse.  The text area does not
contain the header line.  If you look at a version of
'coordinates-in-window-p' from the past century you will see that

If COORDINATES are in the text portion of WINDOW,\n\
    the coordinates relative to the window are returned.\n\
If they are in the mode line of WINDOW, `mode-line' is returned.\n\
If they are in the top mode line of WINDOW, `header-line' is returned.\n\

and this has never changed.  The text area is what window_box_height
tells us.

According to your change we'd now have to rewrite doc-strings and info
of lots of functions like 'window-text-height', 'window-body-height' or
'window-text-pixel-size'.


If 'posn-at-x-y' has a problem, let's fix it.  Just that I don't really
know what the problem is.  When, with emacs -Q, I do

(setq tab-line-format "Würstelstand ...")
(setq header-line-format "... dafür ist unser Land bekannt")

and then evaluate

(posn-point (posn-at-x-y 0 18))

it gets me nil because point is _not_ in the text area but rather
between the tab and the header line.  Digging further

(posn-area (posn-at-x-y 0 18))

gets me header-line here and I can't complain.

(posn-point (posn-at-x-y 0 36))

gets me 1 which is, as I'd expect, 'point-min' of *scratch*.  So where
is the problem?

martin

bug#51590: follow-mode is broken with header-line and tab-line

[Eli Zaretskii]

> Cc: 51590@debbugs.gnu.org, juri@linkov.net
> From: martin rudalics <rudalics@gmx.at>
> Date: Sat, 6 Nov 2021 19:31:11 +0100
> 
>  >> The picture in elisp page "Basic Windows" seems to show "window body
>  >> height" as NOT including the header line or tab line.  That picture
>  >> seems to show the header line as being ABOVE the text area, not part of
>  >> it.
>  >
>  > The updated picture doesn't have "text area" written on it at all.
> 
> I'm afraid this is a change for the worse.

I respectfully disagree.

> The text area does not contain the header line.

It does in my book.

> If you look at a version of 'coordinates-in-window-p' from the past
> century you will see that
> 
> If COORDINATES are in the text portion of WINDOW,\n\
>     the coordinates relative to the window are returned.\n\
> If they are in the mode line of WINDOW, `mode-line' is returned.\n\
> If they are in the top mode line of WINDOW, `header-line' is returned.\n\
> 
> and this has never changed.  The text area is what window_box_height
> tells us.

I don't think I understand how that follows.  And last-century
documentation may need updating anyway.

> According to your change we'd now have to rewrite doc-strings and info
> of lots of functions like 'window-text-height', 'window-body-height' or
> 'window-text-pixel-size'.

If we must, yes.  Why is that a catastrophe?

> If 'posn-at-x-y' has a problem, let's fix it.  Just that I don't really
> know what the problem is.

See bug#51632.  And let's continue the discussion there.

bug#51590: follow-mode is broken with header-line and tab-line

[martin rudalics]

 > So where is the problem?

I think I see it now.  It's the doc-string of 'posn-at-x-y'.  It should
probably say that

"Y is relative to the top edge of the window.  X is relative to the left
edge of the text area of the selected window.  If optional fourth arg
WHOLE is non-nil, X is relative to the left edge of the window."

martin

bug#51590: follow-mode is broken with header-line and tab-line

[Alan Mackenzie]

Hello, Juri and Eli.

On Thu, Nov 04, 2021 at 19:29:28 +0200, Juri Linkov wrote:
> retitle 51590 follow-mode is broken with header-line and tab-line
> quit

> >> The most low-level function of follow-mode is follow-calc-win-end.
> >> When global-tab-line-mode is enabled, follow-calc-win-end
> >> returns the same values as when global-tab-line-mode is disabled.

> > Can you step through that function and see which part is misbehaving
> > there?

> >> I don't understand what more low-level function doesn't take
> >> into account the height of the tab-line.  Maybe the problem is
> >> in pos-visible-in-window-p?  Or maybe different values returned
> >> by window-inside-pixel-edges and window-end?

> > I don't know the answer, but I see that header-line-format is
> > mentioned in one place in follow.el, and that is not a good sign...

> Indeed, I tested with the header-line, and it has the same problem,
> so retitled this bug report.

> After trial and error, I arrived to the following patch that completely
> fixes these problems.  However, I can't explain how it works.

I've spent some time on this patch, and I've transformed it into a
similar patch that I can explain, and I believe is nearly correct[*].
Could you possibly try it out in your system.  Thanks!


diff --git a/lisp/follow.el b/lisp/follow.el
index b64f4cb734..2ca2c1f17b 100644
--- a/lisp/follow.el
+++ b/lisp/follow.el
@@ -667,7 +667,8 @@ follow-scroll-down
 	 (scroll-down-command arg))
 	(arg (follow-scroll-down-arg arg))
         (t
-	 (let* ((windows (follow-all-followers))
+	 (let* ((orig-point (point))
+                (windows (follow-all-followers))
 		(win (car (reverse windows)))
 		(start (window-start (car windows))))
 	   (if (eq start (point-min))
@@ -678,11 +679,14 @@ follow-scroll-down
 	     (select-window win)
 	     (goto-char start)
 	     (vertical-motion (- (- (window-height win)
-				    (if header-line-format 2 1)
-				    next-screen-context-lines)))
+				    (if header-line-format 2 1) ; always mode-line
+				    (if tab-line-format 1 0)
+                                    next-screen-context-lines)))
 	     (set-window-start win (point))
-	     (goto-char start)
-	     (vertical-motion (- next-screen-context-lines 1))
+             (if (< orig-point (window-end win t))
+                 (goto-char orig-point)
+               (goto-char start)
+	       (vertical-motion (- next-screen-context-lines 1)))
 	     (setq follow-internal-force-redisplay t))))))
 (put 'follow-scroll-down 'scroll-command t)
 
@@ -947,7 +951,11 @@ follow-calc-win-end
   (let* ((win (or win (selected-window)))
 	 (edges (window-inside-pixel-edges win))
 	 (ht (- (nth 3 edges) (nth 1 edges)))
-	 (last-line-pos (posn-point (posn-at-x-y 0 (1- ht) win))))
+	 (last-line-pos (posn-point
+                         (posn-at-x-y 0 (+ (window-header-line-height win)
+                                           (window-tab-line-height win)
+                                           (1- ht))
+                                      win))))
     (if (pos-visible-in-window-p last-line-pos win)
 	(let ((end (window-end win t)))
 	  (list end (pos-visible-in-window-p (point-max) win)))


[*] There is a situation which is not new, where if the buffer is too
short to fill all the windows, and it is already scrolled up far,
follow-scroll-down will scroll a correct amount, but leave point in a
random position.


-- 
Alan Mackenzie (Nuremberg, Germany).

bug#51590: follow-mode is broken with header-line and tab-line

[Eli Zaretskii]

> Date: Sun, 7 Nov 2021 12:48:22 +0000
> Cc: Eli Zaretskii <eliz@gnu.org>, 51590@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > Indeed, I tested with the header-line, and it has the same problem,
> > so retitled this bug report.
> 
> > After trial and error, I arrived to the following patch that completely
> > fixes these problems.  However, I can't explain how it works.
> 
> I've spent some time on this patch, and I've transformed it into a
> similar patch that I can explain, and I believe is nearly correct[*].
> Could you possibly try it out in your system.  Thanks!

So what's the verdict about posn-at-x-y? does it have a bug when
tab-line is present, or doesn't it?

bug#51590: follow-mode is broken with header-line and tab-line

[Alan Mackenzie]

Hello, Eli.

On Sun, Nov 07, 2021 at 15:14:13 +0200, Eli Zaretskii wrote:
> > Date: Sun, 7 Nov 2021 12:48:22 +0000
> > Cc: Eli Zaretskii <eliz@gnu.org>, 51590@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > Indeed, I tested with the header-line, and it has the same problem,
> > > so retitled this bug report.

> > > After trial and error, I arrived to the following patch that completely
> > > fixes these problems.  However, I can't explain how it works.

> > I've spent some time on this patch, and I've transformed it into a
> > similar patch that I can explain, and I believe is nearly correct[*].
> > Could you possibly try it out in your system.  Thanks!

> So what's the verdict about posn-at-x-y? does it have a bug when
> tab-line is present, or doesn't it?

There is no bug there.  It was merely me getting confused between tab
lines and tab bars.  The first of these is part of the window, the
second part of the frame.

I closed bug #51632 as notabug late last night.  Sorry I neglected to
put you onto the Cc:.

I think there are still inconsistencies in the documentation with the
terms "window body" and "window text area", but I'll have to look at
that more systematically to find them, if they are there.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#51590: follow-mode is broken with header-line and tab-line

[Juri Linkov]

> I've spent some time on this patch, and I've transformed it into a
> similar patch that I can explain, and I believe is nearly correct[*].
> Could you possibly try it out in your system.  Thanks!

Thanks, I tested it out, and everything works without a hitch:
with tab-line-format, or with header-line-format, or with both.

bug#51590: follow-mode is broken with header-line and tab-line

[Alan Mackenzie]

Hello, Juri.

On Sun, Nov 07, 2021 at 19:46:38 +0200, Juri Linkov wrote:
> > I've spent some time on this patch, and I've transformed it into a
> > similar patch that I can explain, and I believe is nearly correct[*].
> > Could you possibly try it out in your system.  Thanks!

> Thanks, I tested it out, and everything works without a hitch:
> with tab-line-format, or with header-line-format, or with both.

That's great.  If it's OK with you, I will write a commit message,
commit the change to the emacs-28 branch, then merge it to master.

Thanks for spotting the problem with the header line; it's been there
for a long time.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#51590: follow-mode is broken with header-line and tab-line

[Juri Linkov]

>> Thanks, I tested it out, and everything works without a hitch:
>> with tab-line-format, or with header-line-format, or with both.
>
> That's great.  If it's OK with you, I will write a commit message,
> commit the change to the emacs-28 branch, then merge it to master.
>
> Thanks for spotting the problem with the header line; it's been there
> for a long time.

Fixing it on the emacs-28 branch would be great.  I hadn't realized
until now that some glitches in follow-mode were related to the
tab-line/header-line.  Thanks in advance for pushing the fix
to the emacs-28 branch.

bug#51590: follow-mode is broken with header-line and tab-line

[Alan Mackenzie]

Hello, Juri.

On Sun, Nov 07, 2021 at 21:56:50 +0200, Juri Linkov wrote:
> >> Thanks, I tested it out, and everything works without a hitch:
> >> with tab-line-format, or with header-line-format, or with both.

> > That's great.  If it's OK with you, I will write a commit message,
> > commit the change to the emacs-28 branch, then merge it to master.

> > Thanks for spotting the problem with the header line; it's been there
> > for a long time.

> Fixing it on the emacs-28 branch would be great.  I hadn't realized
> until now that some glitches in follow-mode were related to the
> tab-line/header-line.  Thanks in advance for pushing the fix
> to the emacs-28 branch.

Done.  I'm closing the bug with this post.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#51590: follow-mode is broken with header-line and tab-line

[martin rudalics]

 >> The text area does not contain the header line.
 >
 > It does in my book.

How would your book describe the text area?  A T-shaped area comprising
the window's body, its header and tab line?  What would be the width of
that area?  Why would it exclude the mode line?

 >> If you look at a version of 'coordinates-in-window-p' from the past
 >> century you will see that
 >>
 >> If COORDINATES are in the text portion of WINDOW,\n\
 >>      the coordinates relative to the window are returned.\n\
 >> If they are in the mode line of WINDOW, `mode-line' is returned.\n\
 >> If they are in the top mode line of WINDOW, `header-line' is returned.\n\
 >>
 >> and this has never changed.  The text area is what window_box_height
 >> tells us.
 >
 > I don't think I understand how that follows.

Because text area and header line were mutually exclusive ever since.
At least so until last week.

 > And last-century
 > documentation may need updating anyway.

It _was_ updated continuously in a sense that tried to preserve the
original meaning of concepts and allowed them to live together with
concepts that were introduced at a later stage.  Consider the
'window-height' and 'window-width' controversy, for example.

 >> According to your change we'd now have to rewrite doc-strings and info
 >> of lots of functions like 'window-text-height', 'window-body-height' or
 >> 'window-text-pixel-size'.
 >
 > If we must, yes.  Why is that a catastrophe?

Because the person who has to do that would have to check every single
occurrence of the string "text" in all files that deal with "windows".

 >> If 'posn-at-x-y' has a problem, let's fix it.  Just that I don't really
 >> know what the problem is.
 >
 > See bug#51632.  And let's continue the discussion there.

That bug has been closed meanwhile, like the present one.  'posn-at-x-y'
is a function with ill-conceived arguments (why does WHOLE affect the X
coordinate only) and a doc-string I've never been able to understand.

   By default, X and Y are relative to text area of the selected window.
   Note that the text area includes the header-line and the tab-line of
   the window, if any of them are present.

This is at least as wrong as before: For example, with emacs -Q

(posn-area (posn-at-x-y 0 0))

gives me 'nil' here which is correct since the left fringe is not part of
the text area.  But

(posn-area (posn-at-x-y 0 (window-body-height nil t)))

gives me 'mode-line' here which is wrong since, according to your new
definition, the mode line is _not_ part of the text area.  Same holds
for a horizontal scroll bar, if present.  And with a bottom divider

(posn-area (posn-at-x-y 0 (1- (window-pixel-height))))

gets me 'bottom-divider'.

So I'd suggest to revert your changes wrt the text area.  And, since
'posn-at-x-y' deals with coordinates and not with "areas", simply say
that Y is always relative to the top edge of the window while X is
relative to the left body edge of the window if WHOLE is nil and Y is
within the top and bottom body edge of the window and relative to the
left window edge otherwise.  Do we have a deal?  Not yet ...

(posn-area (posn-at-x-y (1- (window-pixel-width)) 0))

currently gives 'nil' regardless of whether it's done with a header or
tab line and

(posn-area (posn-at-x-y
	    (1- (window-pixel-width))
	    (1- (window-pixel-height))))

gives 'nil' on the mode line.  Only when I remove _both_ fringes and the
vertical scroll bar I get the expected results.  This _is_ a bug and we
should fix it.

martin

bug#51590: follow-mode is broken with header-line and tab-line

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

 > (posn-area (posn-at-x-y (1- (window-pixel-width)) 0))
 >
 > currently gives 'nil' regardless of whether it's done with a header or
 > tab line and
 >
 > (posn-area (posn-at-x-y
 >          (1- (window-pixel-width))
 >          (1- (window-pixel-height))))
 >
 > gives 'nil' on the mode line.  Only when I remove _both_ fringes and the
 > vertical scroll bar I get the expected results.  This _is_ a bug and we
 > should fix it.

I think the attached patch should fix that modulo some ">=" and "<"
glitches.

martin

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: posn-at-x-y.diff --]
[-- Type: text/x-patch; name="posn-at-x-y.diff", Size: 1230 bytes --]

diff --git a/src/keyboard.c b/src/keyboard.c
index a99d14cb4c..d24a0ffe07 100644
--- a/src/keyboard.c
+++ b/src/keyboard.c
@@ -11374,7 +11374,9 @@ DEFUN ("posn-at-x-y", Fposn_at_x_y, Sposn_at_x_y, 2, 4, 0,

       XSETINT (x, (XFIXNUM (x)
 		   + WINDOW_LEFT_EDGE_X (w)
-		   + (NILP (whole)
+		   + ((NILP (whole)
+		       && XFIXNUM (y) >= WINDOW_BOX_TOP_PIXEL_EDGE (w)
+		       && XFIXNUM (y) < WINDOW_BOX_HEIGHT_NO_MODE_LINE (w))
 		      ? window_box_left_offset (w, TEXT_AREA)
 		      : 0)));
       XSETINT (y, WINDOW_TO_FRAME_PIXEL_Y (w, XFIXNUM (y)));
diff --git a/src/window.h b/src/window.h
index 2400c422c1..6420706b5f 100644
--- a/src/window.h
+++ b/src/window.h
@@ -800,6 +800,12 @@ #define WINDOW_BOX_RIGHT_PIXEL_EDGE(W)		\
    - WINDOW_RIGHT_DIVIDER_WIDTH (W)		\
    - WINDOW_RIGHT_SCROLL_BAR_AREA_WIDTH (W))

+/* Return the pixel value where the text in window W starts.  */
+#define WINDOW_BOX_TOP_PIXEL_EDGE(W)		\
+  (WINDOW_TOP_PIXEL_EDGE (W)			\
+   + WINDOW_HEADER_LINE_HEIGHT (W)		\
+   + WINDOW_TAB_LINE_HEIGHT (W))
+
 /* Return the frame x-position at which the text (or left fringe) in
    window W starts.  This does not include a left-hand scroll bar if
    any.  */

bug#51590: follow-mode is broken with header-line and tab-line

[Alan Mackenzie]

Hello, Eli and Martin.

On Sat, Nov 06, 2021 at 14:12:50 +0200, Eli Zaretskii wrote:
> > Date: Sat, 6 Nov 2021 11:50:48 +0000
> > Cc: juri@linkov.net, 51590@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

[ .... ]

> > OK.  But I think there's confusion in the manual and doc strings, and
> > even the code, and that it's not just me.

This confusion being the meanings of "window body" and "window text
area", in particular as to whether or not they include any header line or
tab line.

> _After_ my today's changes?  If so, please show where is the confusing
> text.

I've done a survey of window.c, keyboard.c, window.el for all occurrences
of "text area" and "body" in function names, doc strings, some comments,
and parameter names.  The source I've grepped is an almost up to date
copy of the emacs-28 branch.  The function name is at the left, followed
by annotations for text area and body.  "BIG" means the term includes the
header and tab lines, "small" means it does not, "?" means it is not
immediately clear, blank means it is irrelevant or not mentioned.

Function                       Text Area            Body
--------                       ---------            ----
Fwindow_body_width
Fwindow_body_height                small         (small)
Fwindow_old_body_pixel_width
Fwindow_old_body_pixel_height          ?
Fwindow_lines_pixel_dimensions                  confused.

Fwindow_text_height                small
Fset_window_fringes
window_body_height                                 small
window_body_width
window_change_record_windows                           ?

run_window_change_functions
Vwindow_size_change_functions
Vwindow_state_change_functions
Vwindow_state_change_hook
Vwindow_configuration_change_hook


make_lispy_position		  small
make_lispy_event                                       ?
read_key_sequence                 small
Fposn_at_x_y                        BIG


window-body-size                      ?
window-edges                          ?                ?
window-absolute-body-pixel-edges      ?                ?
window-largest-empty-rectangle        ?
window-preserve-size                                   ?

window-body-edges                                      ?
window-body-pixel-edges                                ?


It seems clear that, at least in the places where the meaning of "text
area" and "body" are clear, that they refer to the area which doesn't
include the header line and tab line.

Fposn_at_x_y stands out as the only function with BIG.  Maybe the picture
would change on examining the elisp manual.  Maybe some of the unclear
annotations would resolve to BIG, but that doesn't seem all that likely.

Given this, it seems it would be better to amend the documentation of
Fposn_at_x_y to refer to the "text area _plus_ any header line or tab
line".

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#51590: follow-mode is broken with header-line and tab-line

[Eli Zaretskii]

> Date: Mon, 8 Nov 2021 17:59:44 +0000
> Cc: juri@linkov.net, martin rudalics <rudalics@gmx.at>, 51590@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> Given this, it seems it would be better to amend the documentation of
> Fposn_at_x_y to refer to the "text area _plus_ any header line or tab
> line".

I would like to stop using "text area" in the doc strings and the user
manual.  It is a confusing term, more so for me, because it has a very
specific meaning in the display code.

So from my POV, the ideal solution is to go through all of those
places and replace "text area" with "window body" (and if the latter
is already there, just delete the former).

Thanks.

bug#51590: follow-mode is broken with header-line and tab-line

[Eli Zaretskii]

> Cc: acm@muc.de, 51590@debbugs.gnu.org, juri@linkov.net
> From: martin rudalics <rudalics@gmx.at>
> Date: Mon, 8 Nov 2021 16:36:36 +0100
> 
>    By default, X and Y are relative to text area of the selected window.
>    Note that the text area includes the header-line and the tab-line of
>    the window, if any of them are present.
> 
> This is at least as wrong as before: For example, with emacs -Q
> 
> (posn-area (posn-at-x-y 0 0))
> 
> gives me 'nil' here which is correct since the left fringe is not part of
> the text area.  But
> 
> (posn-area (posn-at-x-y 0 (window-body-height nil t)))
> 
> gives me 'mode-line' here which is wrong since, according to your new
> definition, the mode line is _not_ part of the text area.

"Wrong" why, because the doc string of window-body-height mentions the
text area?  Or for some other reason?

And, btw, why do you use (window-body-height nil t) instead of
(1- (window-body-height nil t)) ?  The last pixel of the window's body
has Y coordinate (1- (window-body-height nil t)), no?

> And with a bottom divider
> 
> (posn-area (posn-at-x-y 0 (1- (window-pixel-height))))
> 
> gets me 'bottom-divider'.

And that is wrong why?

> So I'd suggest to revert your changes wrt the text area.

I wrote in another message what I'd like to do with "text area" in doc
strings.  Given that they will disappear from the doc strings, what
other problems do you see?

> (posn-area (posn-at-x-y (1- (window-pixel-width)) 0))
> 
> currently gives 'nil' regardless of whether it's done with a header or
> tab line

Isn't that because window-pixel-width includes the fringes?

> and
> 
> (posn-area (posn-at-x-y
> 	    (1- (window-pixel-width))
> 	    (1- (window-pixel-height))))
> 
> gives 'nil' on the mode line.  Only when I remove _both_ fringes and the
> vertical scroll bar I get the expected results.  This _is_ a bug and we
> should fix it.

I don't think it's a bug, because window-pixel-height includes the
fringes, according to its doc string.  Try

  (posn-area (posn-at-x-y
	      (- (window-pixel-width) 9)
	      (1- (window-pixel-height))))

bug#51590: follow-mode is broken with header-line and tab-line

[martin rudalics]

 > I've done a survey of window.c, keyboard.c, window.el for all occurrences
 > of "text area" and "body" in function names, doc strings, some comments,
 > and parameter names.  The source I've grepped is an almost up to date
 > copy of the emacs-28 branch.  The function name is at the left, followed
 > by annotations for text area and body.  "BIG" means the term includes the
 > header and tab lines, "small" means it does not, "?" means it is not
 > immediately clear, blank means it is irrelevant or not mentioned.

Thanks.  Note that searching for "text area" alone is not sufficient.
In the past, the "thing" (the body of a window) we talk about here has
been described in various ways, usually according to the personal taste
of the author of a change.

You missed, for example, 'window-text-width' which uses the term "text
display area" for this.  I've usually tried to retain the nomenclature
originally used by the authors because it's hardly ever immediately
evident what they really had in mind and also in the hope that the use
of functions like 'window-text-width' would eventually fade out.

The greatest problem in this area is that Emacs traditionally uses the
terms 'window-height' to refer to the total height of a window and
'window-width' to refer to its body width.  Obviously, all occurrences
of these two terms should be replaced with 'window-total-height' and
'window-body-width' respectively but you can easily see for yourself the
progress I have made.

Hence, rather than caring about the (IMO) very minor "text area" issue,
fixing these occurrences should be our major aim if we want to remove
inconsistencies.  Sadly, people add new occurrences of these at a faster
pace than I'm able to remove the older ones.

 > Function                       Text Area            Body
 > --------                       ---------            ----
 > Fwindow_body_width

Any function that has "body" in its name is "safe" (and means "small" in
your nomenclature) so we can usually ignore them.  Their doc-strings,
however, may use some variant of "text area" since that was the term
commonly in use at the time the "body" functions were added.

 > Fwindow_body_height                small         (small)
 > Fwindow_old_body_pixel_width
 > Fwindow_old_body_pixel_height          ?
 > Fwindow_lines_pixel_dimensions                  confused.

'window-lines-pixel-dimensions' is like 'window-text-pixel-size' but
works line-wise.  More precisely, it can be used to walk the buffer text
shown in a window and get the number of pixels occupied by each
individual line.  The BODY argument serves to control the impact the
presence of a header or mode line could have on the return value.

 > Fwindow_text_height                small
 > Fset_window_fringes
 > window_body_height                                 small
 > window_body_width
 > window_change_record_windows                           ?

'window_change_record_windows' does not make any reference to the text
area so why the "?".

 > run_window_change_functions
 > Vwindow_size_change_functions
 > Vwindow_state_change_functions
 > Vwindow_state_change_hook
 > Vwindow_configuration_change_hook

I tried to very carefully distinguish between body and total sizes here
because, for example, a  change in the width of a fringe may or may not
change the body width of a window and leave the total width alone.  So
there should not be any problems with these.

 > make_lispy_position		  small
 > make_lispy_event                                       ?
 > read_key_sequence                 small
 > Fposn_at_x_y                        BIG

I never thoroughly checked the keyboard and mouse defined routines.
There anything is possible.

 > window-body-size                      ?
 > window-edges                          ?                ?
 > window-absolute-body-pixel-edges      ?                ?
 > window-largest-empty-rectangle        ?
 > window-preserve-size                                   ?
 >
 > window-body-edges                                      ?
 > window-body-pixel-edges                                ?

All of these are new and should be "small" although their docs may
still contain references to the "text area".

But the far greater problem is to find all occurrences of "text" in the
manuals, for example, where it stands for the width or height of the
buffer text shown in a window.  Often I read and re-read the manual a
couple of times in order to find out what is really meant and whether
what I read is not biased because I've read some different part of the
manual before.

These parts of the manual suffer most from the transition of Emacs from
TTYs to GUIs over time and the accompanying changes of its nomenclature.

 > It seems clear that, at least in the places where the meaning of "text
 > area" and "body" are clear, that they refer to the area which doesn't
 > include the header line and tab line.
 >
 > Fposn_at_x_y stands out as the only function with BIG.  Maybe the picture
 > would change on examining the elisp manual.  Maybe some of the unclear
 > annotations would resolve to BIG, but that doesn't seem all that likely.
 >
 > Given this, it seems it would be better to amend the documentation of
 > Fposn_at_x_y to refer to the "text area _plus_ any header line or tab
 > line".

Or _not use_ the term "text area" here in the first place.

martin

bug#51590: follow-mode is broken with header-line and tab-line

[martin rudalics]

 > I would like to stop using "text area" in the doc strings and the user
 > manual.

OK.  But can we do that on the development branch please?  I recently
spent some time rewriting the window chapter in the Elisp manual and
just do not feel confident enough to install the necessary changes in a
safe manner on the release branch soon.

 > It is a confusing term, more so for me, because it has a very
 > specific meaning in the display code.

While a comment like

/* Return the window-relative coordinate of the right edge of display
    area AREA of window W.  ANY_AREA means return the right edge of the
    whole window, to the left of the right fringe of W.  */

may look entirely clear to you, it doesn't to me.  So maybe we could try
to first say what

- a "display area" of a window is and where its "right edge" is located,

- what a "window-relative coordinate" is,

- what the "right edge of the whole window" is, and

- what "left of the right fringe of W" stands for in the presence of
   fringes in and outside of margins or a scroll bar on the left.

Next we could try fixing things like

     int text_area_x, text_area_y, text_area_width, text_area_height;

     window_box (w, TEXT_AREA, &text_area_x, &text_area_y, &text_area_width,
		&text_area_height);

in frame.c and

   int text_area_x, text_area_y, text_area_width, text_area_height;

   window_box (s->w, TEXT_AREA, &text_area_x, &text_area_y,
               &text_area_width, &text_area_height);

in xwidget.c if we want to say that the box size of a window is not the
same as the size of its text area (deduced from you earlier comment).

 > So from my POV, the ideal solution is to go through all of those
 > places and replace "text area" with "window body" (and if the latter
 > is already there, just delete the former).

Agreed.  But after the release, pretty please.

martin

bug#51590: follow-mode is broken with header-line and tab-line

[martin rudalics]

 >> But
 >>
 >> (posn-area (posn-at-x-y 0 (window-body-height nil t)))
 >>
 >> gives me 'mode-line' here which is wrong since, according to your new
 >> definition, the mode line is _not_ part of the text area.
 >
 > "Wrong" why, because the doc string of window-body-height mentions the
 > text area?  Or for some other reason?

If you had answered my question

   How would your book describe the text area?  A T-shaped area comprising
   the window's body, its header and tab line?  What would be the width of
   that area?  Why would it exclude the mode line?

I would be able to tell.  A definition like

   The @dfn{text area} of the window includes
   the header line and the tab line, if they are present in the window.

does _not_ define what the text area is.

 > And, btw, why do you use (window-body-height nil t) instead of
 > (1- (window-body-height nil t)) ?  The last pixel of the window's body
 > has Y coordinate (1- (window-body-height nil t)), no?

Right.  I wanted to be below the body.

 >> And with a bottom divider
 >>
 >> (posn-area (posn-at-x-y 0 (1- (window-pixel-height))))
 >>
 >> gets me 'bottom-divider'.
 >
 > And that is wrong why?

See above.

 >> So I'd suggest to revert your changes wrt the text area.
 >
 > I wrote in another message what I'd like to do with "text area" in doc
 > strings.  Given that they will disappear from the doc strings, what
 > other problems do you see?

That the manual still defines the "text area" and now even in a form
that lets me only guess what it is.

 >> (posn-area (posn-at-x-y (1- (window-pixel-width)) 0))
 >>
 >> currently gives 'nil' regardless of whether it's done with a header or
 >> tab line
 >
 > Isn't that because window-pixel-width includes the fringes?

The header line doesn't have any fringes.

 >> and
 >>
 >> (posn-area (posn-at-x-y
 >> 	    (1- (window-pixel-width))
 >> 	    (1- (window-pixel-height))))
 >>
 >> gives 'nil' on the mode line.  Only when I remove _both_ fringes and the
 >> vertical scroll bar I get the expected results.  This _is_ a bug and we
 >> should fix it.
 >
 > I don't think it's a bug, because window-pixel-height includes the
 > fringes, according to its doc string.  Try
 >
 >    (posn-area (posn-at-x-y
 > 	      (- (window-pixel-width) 9)
 > 	      (1- (window-pixel-height))))

The mode line doesn't have any fringes either.

martin