bug#15783: 24.3; posn-at-x-y is relative to the buffer area, posn-at-point is relative to the text area.

bug#15783: 24.3; posn-at-x-y is relative to the buffer area, posn-at-point is relative to the text area.

[椎野 裕樹]

On Emacs 24 with non-nil header-line-format, y offset returned by
`posn-at-point' is relative to the text area *not including the header
line*.  However, `posn-at-x-y' takes y offset relative to the buffer
area *including the header line*.

Is this by design?

When I get the position by `posn-at-point', and then convert the
position to the point by `posn-at-x-y', the resulting point does not
match the point given to `posn-at-point'.

This seems to me a serious inconsistency.

On Emacs 22 and 23, both functions are relative to the buffer area
including the header line.  It's fine.

Cheers,
Yuki Shiino

bug#15783: 24.3; posn-at-x-y is relative to the buffer area, posn-at-point is relative to the text area.

[Eli Zaretskii]

> From: 椎野 裕樹
> 	<shiino.yuki@gmail.com>
> Date: Fri, 1 Nov 2013 23:49:00 +0900
> 
> On Emacs 24 with non-nil header-line-format, y offset returned by
> `posn-at-point' is relative to the text area *not including the header
> line*.  However, `posn-at-x-y' takes y offset relative to the buffer
> area *including the header line*.
> 
> Is this by design?

Yes.

> When I get the position by `posn-at-point', and then convert the
> position to the point by `posn-at-x-y', the resulting point does not
> match the point given to `posn-at-point'.

You should test for non-nil header-line-format, and adjust the value
accordingly.

> On Emacs 22 and 23, both functions are relative to the buffer area
> including the header line.  It's fine.

That caused much more grave bugs.

bug#15783: 24.3; posn-at-x-y is relative to the buffer area, posn-at-point is relative to the text area.

[Alex]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: 椎野 裕樹
>> 	<shiino.yuki@gmail.com>
>> Date: Fri, 1 Nov 2013 23:49:00 +0900
>> 
>> On Emacs 24 with non-nil header-line-format, y offset returned by
>> `posn-at-point' is relative to the text area *not including the header
>> line*.  However, `posn-at-x-y' takes y offset relative to the buffer
>> area *including the header line*.
>> 
>> Is this by design?
>
> Yes.
>
>> When I get the position by `posn-at-point', and then convert the
>> position to the point by `posn-at-x-y', the resulting point does not
>> match the point given to `posn-at-point'.
>
> You should test for non-nil header-line-format, and adjust the value
> accordingly.
>
>> On Emacs 22 and 23, both functions are relative to the buffer area
>> including the header line.  It's fine.
>
> That caused much more grave bugs.

What kind of bugs did it cause?  It's pretty jarring to have the
following return nil when a header-line is present:

  (let ((x-y (posn-x-y (posn-at-point))))
    (equal x-y (posn-x-y (posn-at-x-y (car x-y)
                                      (cdr x-y)))))

I noticed this since an Emacs package uses posn-x-y and posn-at-x-y to
try to preserve the point's screen coordinates before and after a
scroll.  With a header-line, this puts the point one row too high.

It should probably just let-bind `scroll-preserve-screen-position',
right?  Still, the above is pretty counterintuitive. If nothing else, I
think this incompatibility should be highlighted.

P.S. I noticed that the manual for `posn-at-x-y' states that the WHOLE
argument affects both X and Y, but the docstring only states that X is
affected.

bug#15783: 24.3; posn-at-x-y is relative to the buffer area, posn-at-point is relative to the text area.

[Eli Zaretskii]

> From: Alex <agrambot@gmail.com>
> Cc: 椎野 裕樹 <shiino.yuki@gmail.com>,
>   15783@debbugs.gnu.org
> Date: Fri, 17 Nov 2017 01:46:36 -0600
> 
> >> On Emacs 22 and 23, both functions are relative to the buffer area
> >> including the header line.  It's fine.
> >
> > That caused much more grave bugs.
> 
> What kind of bugs did it cause?

See bug#7390, for example.

> It's pretty jarring to have the following return nil when a
> header-line is present:
> 
>   (let ((x-y (posn-x-y (posn-at-point))))
>     (equal x-y (posn-x-y (posn-at-x-y (car x-y)
>                                       (cdr x-y)))))

It's less than ideal, but the alternatives are worse.

> I noticed this since an Emacs package uses posn-x-y and posn-at-x-y to
> try to preserve the point's screen coordinates before and after a
> scroll.  With a header-line, this puts the point one row too high.
> 
> It should probably just let-bind `scroll-preserve-screen-position',
> right?

Probably.

> Still, the above is pretty counterintuitive. If nothing else, I
> think this incompatibility should be highlighted.

Not sure what "highlighted" means in this context.  What are you
suggesting in practice?

> P.S. I noticed that the manual for `posn-at-x-y' states that the WHOLE
> argument affects both X and Y, but the docstring only states that X is
> affected.

Feel free to fix such discrepancies without any discussion, and
thanks.

bug#15783: 24.3; posn-at-x-y is relative to the buffer area, posn-at-point is relative to the text area.

[Alex]

Eli Zaretskii <eliz@gnu.org> writes:

>> What kind of bugs did it cause?
>
> See bug#7390, for example.
>
>> It's pretty jarring to have the following return nil when a
>> header-line is present:
>> 
>>   (let ((x-y (posn-x-y (posn-at-point))))
>>     (equal x-y (posn-x-y (posn-at-x-y (car x-y)
>>                                       (cdr x-y)))))
>
> It's less than ideal, but the alternatives are worse.

I don't understand.  The bug above seems to be about posn-col-row rather
than posn(-at)-x-y.  What alternatives were considered, and how were
they worse?

>> Still, the above is pretty counterintuitive. If nothing else, I
>> think this incompatibility should be highlighted.
>
> Not sure what "highlighted" means in this context.  What are you
> suggesting in practice?

I was thinking about mentioning that you can't expect Elisp code such as
the above to always be true.

>> P.S. I noticed that the manual for `posn-at-x-y' states that the WHOLE
>> argument affects both X and Y, but the docstring only states that X is
>> affected.
>
> Feel free to fix such discrepancies without any discussion, and
> thanks.

Hmm, it appears that they're both wrong.  I believe this is the
breakdown:

  1. Manual:
     WHOLE is nil:     X and Y are relative to the text area
     WHOLE is non-nil: X and Y are relative to the window area

  2. Docstring:
     WHOLE is nil:     X and Y are relative to the text area
     WHOLE is non-nil: X is relative to window area; Y to text area

  3. Actual:
     WHOLE is nil:     X is relative to text area; Y to window area
     WHOLE is non-nil: X and Y are relative to the window area

The behaviour in #1 makes the most sense to me, and I believe that was
the behaviour before Emacs 24.

Any change of reverting to the behaviour to #1, or should the
documentation just be updated to #3?

bug#15783: 24.3; posn-at-x-y is relative to the buffer area, posn-at-point is relative to the text area.

[Eli Zaretskii]

> From: Alex <agrambot@gmail.com>
> Cc: shiino.yuki@gmail.com,  15783@debbugs.gnu.org
> Date: Sat, 18 Nov 2017 00:35:27 -0600
> 
> >> It's pretty jarring to have the following return nil when a
> >> header-line is present:
> >> 
> >>   (let ((x-y (posn-x-y (posn-at-point))))
> >>     (equal x-y (posn-x-y (posn-at-x-y (car x-y)
> >>                                       (cdr x-y)))))
> >
> > It's less than ideal, but the alternatives are worse.
> 
> I don't understand.  The bug above seems to be about posn-col-row rather
> than posn(-at)-x-y.  What alternatives were considered, and how were
> they worse?

I think you didn't read all of the discussion that followed.  The
original bug report was about posn-col-row, but the underlying basic
issue is deeper.  The basic issue is the origin from which posn-at-x-y
and posn-at-point count their values.  There are two classes of
callers for these functions: those which start with mouse clicks, and
those which start with buffer positions.  They have basically
different needs, but both classes don't want to have special
application-level code that accounts for the header line, if it is
present.  So we changed the primitives to cater to each class in the
respective function: posn-at-x-y caters to those which deal with mouse
clicks, and posn-at-point caters to the other kind.  For the details,
see commit 9173a8f.

> >> Still, the above is pretty counterintuitive. If nothing else, I
> >> think this incompatibility should be highlighted.
> >
> > Not sure what "highlighted" means in this context.  What are you
> > suggesting in practice?
> 
> I was thinking about mentioning that you can't expect Elisp code such as
> the above to always be true.

Fine with me, but can you propose a patch for the documentation to do
that?

> Hmm, it appears that they're both wrong.  I believe this is the
> breakdown:
> 
>   1. Manual:
>      WHOLE is nil:     X and Y are relative to the text area
>      WHOLE is non-nil: X and Y are relative to the window area
> 
>   2. Docstring:
>      WHOLE is nil:     X and Y are relative to the text area
>      WHOLE is non-nil: X is relative to window area; Y to text area
> 
>   3. Actual:
>      WHOLE is nil:     X is relative to text area; Y to window area
>      WHOLE is non-nil: X and Y are relative to the window area

This is the same confusion about what "text area" means that is
present in many of our conversations.  The manual says:

  ‘Text Area’
       The “text area” of a frame is a somewhat fictitious area that can
       be embedded in the native frame.  Its position is unspecified.  Its
       width can be obtained by removing from that of the native width the
       widths of the internal border, one vertical scroll bar, and one
       left and one right fringe if they are specified for this frame, see
       *note Layout Parameters::.  Its height can be obtained by removing
       from that of the native height the widths of the internal border
       and the heights of the frame’s internal menu and tool bars and one
       horizontal scroll bar if specified for this frame.

This means the "text area" _includes_ the header line.  So when you
get this result in "emacs -Q":

  M-: (setq header-line-format "Hi There") RET
  M-: (posn-at-x-y 0 0)
   => (#<window 3 on *scratch*> header-line (8 . 0) 0 ("Hi There" . 1) nil (1 . -1) nil (0 . 0) (8 . 16))

it tells you that the origin is the left corner of the text area,
because the X coordinate is 8, not zero, and the position in the
header-line string is 1, not zero.  If, OTOH, you invoke posn-at-x-y
with last argument non-nil, you will get zero for X, which means the
origin is the left corner of the window, which is at the left edge of
the left fringe.

Therefore, the doc string is accurately describing the actual behavior.
Do you agree now?

> The behaviour in #1 makes the most sense to me, and I believe that was
> the behaviour before Emacs 24.

That behavior required features that used mouse clicks as input and
then looked for the corresponding buffer positions to account for the
header line in application code, which caused the bugs discussed in
the thread I pointed to.

> Any change of reverting to the behaviour to #1, or should the
> documentation just be updated to #3?

We are not going back.  The current behavior, while less than ideal,
caused zero bugs since it was introduced, AFAIK.  The documentation
should be updated, of course (I see that the above-mentioned commit
didn't do that, which is probably why the old description survived to
this day).

Thanks.

bug#15783: 24.3; posn-at-x-y is relative to the buffer area, posn-at-point is relative to the text area.

[Stefan Kangas]

[-- Attachment #1: Type: text/plain, Size: 2250 bytes --]

tags 15783 + patch
thanks

Eli Zaretskii <eliz@gnu.org> writes:

>>   3. Actual:
>>      WHOLE is nil:     X is relative to text area; Y to window area
>>      WHOLE is non-nil: X and Y are relative to the window area
>
> This is the same confusion about what "text area" means that is
> present in many of our conversations.  The manual says:
>
>   ‘Text Area’
>        The “text area” of a frame is a somewhat fictitious area that can
>        be embedded in the native frame.  Its position is unspecified.  Its
>        width can be obtained by removing from that of the native width the
>        widths of the internal border, one vertical scroll bar, and one
>        left and one right fringe if they are specified for this frame, see
>        *note Layout Parameters::.  Its height can be obtained by removing
>        from that of the native height the widths of the internal border
>        and the heights of the frame’s internal menu and tool bars and one
>        horizontal scroll bar if specified for this frame.
>
> This means the "text area" _includes_ the header line.  So when you
> get this result in "emacs -Q":
>
>   M-: (setq header-line-format "Hi There") RET
>   M-: (posn-at-x-y 0 0)
>    => (#<window 3 on *scratch*> header-line (8 . 0) 0 ("Hi There" . 1) nil (1 . -1) nil (0 . 0) (8 . 16))
>
> it tells you that the origin is the left corner of the text area,
> because the X coordinate is 8, not zero, and the position in the
> header-line string is 1, not zero.  If, OTOH, you invoke posn-at-x-y
> with last argument non-nil, you will get zero for X, which means the
> origin is the left corner of the window, which is at the left edge of
> the left fringe.
>
> Therefore, the doc string is accurately describing the actual behavior.
> Do you agree now?

I looked into this for a bit, and as far as I understand, Eli is
correct.

> We are not going back.  The current behavior, while less than ideal,
> caused zero bugs since it was introduced, AFAIK.  The documentation
> should be updated, of course (I see that the above-mentioned commit
> didn't do that, which is probably why the old description survived to
> this day).

How does the attached patch look?

[-- Attachment #2: 0001-Fix-documentation-of-posn-at-x-y.patch --]
[-- Type: text/x-diff, Size: 1312 bytes --]

From 20d555bff78de98a4cde7c04fa24b688b2a1d5cd Mon Sep 17 00:00:00 2001
From: Stefan Kangas <stefan@marxist.se>
Date: Sat, 23 Oct 2021 01:19:48 +0200
Subject: [PATCH] Fix documentation of posn-at-x-y

* doc/lispref/commands.texi (Accessing Mouse): Fix documentation of
'posn-at-x-y' to match docstring.  (Bug#15783)
---
 doc/lispref/commands.texi | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/doc/lispref/commands.texi b/doc/lispref/commands.texi
index 3425880fec..6c853a3d23 100644
--- a/doc/lispref/commands.texi
+++ b/doc/lispref/commands.texi
@@ -2354,10 +2354,11 @@ Accessing Mouse
 coordinates @var{x} and @var{y} in a specified frame or window,
 @var{frame-or-window}, which defaults to the selected window.
 The coordinates @var{x} and @var{y} are relative to the
-frame or window used.
-If @var{whole} is @code{nil}, the coordinates are relative
-to the window text area, otherwise they are relative to
-the entire window area including scroll bars, margins and fringes.
+text area of the selected window.
+If @var{whole} is @code{nil}, the @var{x} coordinate is relative to
+the entire window area including scroll bars, margins and fringes,
+while the @var{y} coordinate remains unchanged and relative to the
+text area.
 @end defun
 
 @node Accessing Scroll
-- 
2.30.2

bug#15783: 24.3; posn-at-x-y is relative to the buffer area, posn-at-point is relative to the text area.

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Fri, 22 Oct 2021 19:27:38 -0400
> Cc: Alex <agrambot@gmail.com>, shiino.yuki@gmail.com, 15783@debbugs.gnu.org
> 
> > Therefore, the doc string is accurately describing the actual behavior.
> > Do you agree now?
> 
> I looked into this for a bit, and as far as I understand, Eli is
> correct.
> 
> > We are not going back.  The current behavior, while less than ideal,
> > caused zero bugs since it was introduced, AFAIK.  The documentation
> > should be updated, of course (I see that the above-mentioned commit
> > didn't do that, which is probably why the old description survived to
> > this day).
> 
> How does the attached patch look?

Yes, but with one correction:

> --- a/doc/lispref/commands.texi
> +++ b/doc/lispref/commands.texi
> @@ -2354,10 +2354,11 @@ Accessing Mouse
>  coordinates @var{x} and @var{y} in a specified frame or window,
>  @var{frame-or-window}, which defaults to the selected window.
>  The coordinates @var{x} and @var{y} are relative to the
> -frame or window used.
> -If @var{whole} is @code{nil}, the coordinates are relative
> -to the window text area, otherwise they are relative to
> -the entire window area including scroll bars, margins and fringes.
> +text area of the selected window.
> +If @var{whole} is @code{nil}, the @var{x} coordinate is relative to
      ^^^^^^^^^^^^^^^^^^^^^^^^^
You mean, non-nil, right?

> +the entire window area including scroll bars, margins and fringes,
> +while the @var{y} coordinate remains unchanged and relative to the
> +text area.

And I wouldn't mention what WHOLE does to Y, because it just confuses
things.  Y remains unchanged simply because there are no window
decorations that affect Y coordinates, but there's no need to tell
that in a doc string; if we say nothing about Y, it still tells that Y
is unchanged.

bug#15783: 24.3; posn-at-x-y is relative to the buffer area, posn-at-point is relative to the text area.

[Stefan Kangas]

close 15783 28.1
thanks

Eli Zaretskii <eliz@gnu.org> writes:

>> How does the attached patch look?
>
> Yes, but with one correction:
>
>> --- a/doc/lispref/commands.texi
>> +++ b/doc/lispref/commands.texi
>> @@ -2354,10 +2354,11 @@ Accessing Mouse
>>  coordinates @var{x} and @var{y} in a specified frame or window,
>>  @var{frame-or-window}, which defaults to the selected window.
>>  The coordinates @var{x} and @var{y} are relative to the
>> -frame or window used.
>> -If @var{whole} is @code{nil}, the coordinates are relative
>> -to the window text area, otherwise they are relative to
>> -the entire window area including scroll bars, margins and fringes.
>> +text area of the selected window.
>> +If @var{whole} is @code{nil}, the @var{x} coordinate is relative to
>       ^^^^^^^^^^^^^^^^^^^^^^^^^
> You mean, non-nil, right?

Yes, good spot.

>> +the entire window area including scroll bars, margins and fringes,
>> +while the @var{y} coordinate remains unchanged and relative to the
>> +text area.
>
> And I wouldn't mention what WHOLE does to Y, because it just confuses
> things.  Y remains unchanged simply because there are no window
> decorations that affect Y coordinates, but there's no need to tell
> that in a doc string; if we say nothing about Y, it still tells that Y
> is unchanged.

Thanks, I removed that and pushed it to emacs-28 as commit cdbd03345d.

I'm therefore closing this bug report.