Redisplay resets vscroll when window start changes

Redisplay resets vscroll when window start changes

[Po Lu]

I was implementing pixel-based scrolling based on XInput 2 pixel scroll
events earlier, and i noticed this line in xdisp.c:

 /* Handle case where place to start displaying has been specified,
    unless the specified location is outside the accessible range.  */
  if (w->force_start)
    {
      /* We set this later on if we have to adjust point.  */
      int new_vpos = -1;

      w->force_start = false;
-->   w->vscroll = 0;
      w->window_end_valid = false;

It resets the vscroll whenever window start changes, which is annoying
if you, for example, recenter the window during pixel scroll.

Is it OK to control whether or not the vscroll is reset there based on a
variable or a window parameter?  It would be very convenient to have
such a feature.

Thanks.

Re: Redisplay resets vscroll when window start changes

[Po Lu]

Po Lu <luangruo@yahoo.com> writes:

> Is it OK to control whether or not the vscroll is reset there based on a
> variable or a window parameter?  It would be very convenient to have
> such a feature.

I was just told that Carbon Emacs also has this feature, and that it
uses this feature in its implementation of pixel-based scrolling.

I don't really know how effective it is, though, as I have never used
Carbon Emacs.

Thanks.

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Date: Mon, 15 Nov 2021 20:39:50 +0800
> 
> I was implementing pixel-based scrolling based on XInput 2 pixel scroll
> events earlier, and i noticed this line in xdisp.c:
> 
>  /* Handle case where place to start displaying has been specified,
>     unless the specified location is outside the accessible range.  */
>   if (w->force_start)
>     {
>       /* We set this later on if we have to adjust point.  */
>       int new_vpos = -1;
> 
>       w->force_start = false;
> -->   w->vscroll = 0;
>       w->window_end_valid = false;
> 
> It resets the vscroll whenever window start changes, which is annoying
> if you, for example, recenter the window during pixel scroll.

Yes.  This is the basis of how scrolling commands work in Emacs: they
set the window-start point.  When that happens, vscroll must be reset.

> Is it OK to control whether or not the vscroll is reset there based on a
> variable or a window parameter?  It would be very convenient to have
> such a feature.

That'd open a Pandora box of endless hacks and bug reports, so I'd
prefer not to allow that.  It is not how the present display code was
designed.  Forcing window-start and setting vscroll contradict each
other, because forcing window-start means we want the user to see the
stuff at that buffer position.

Are you using scroll commands to implement this?  If so, don't: they
are not the right way of having pixel-wise scrolling in Emacs.
Instead, scroll the display by controlling the non-zero vscroll,
without forcing window-start.

Re: Redisplay resets vscroll when window start changes

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

>> It resets the vscroll whenever window start changes, which is annoying
>> if you, for example, recenter the window during pixel scroll.

> Yes.  This is the basis of how scrolling commands work in Emacs: they
> set the window-start point.  When that happens, vscroll must be reset.

I understand that, thanks.

>> Is it OK to control whether or not the vscroll is reset there based on a
>> variable or a window parameter?  It would be very convenient to have
>> such a feature.

> Are you using scroll commands to implement this?  If so, don't: they
> are not the right way of having pixel-wise scrolling in Emacs.

Uh, no, but I would like to preserve the vscroll across scrolling
commands, as that's how pixel-wise scrolling works in other editors.

Perhaps some Carbon Emacs people can chime in at this point, as Carbon
Emacs does have this option.

> Instead, scroll the display by controlling the non-zero vscroll,
> without forcing window-start.

Yes, that's what I'm doing.

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 16 Nov 2021 08:02:18 +0800
> 
> > Are you using scroll commands to implement this?  If so, don't: they
> > are not the right way of having pixel-wise scrolling in Emacs.
> 
> Uh, no, but I would like to preserve the vscroll across scrolling
> commands, as that's how pixel-wise scrolling works in other editors.

I don't think I understand.  Can you tell more, and perhaps give an
example?  Why would we want to preserve the vscroll after a real
scroll, when scrolling means you see some completely different text?

Re: Redisplay resets vscroll when window start changes

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> I don't think I understand.  Can you tell more, and perhaps give an
> example?  Why would we want to preserve the vscroll after a real
> scroll, when scrolling means you see some completely different text?

Nevermind that, I found that the variable `auto-window-vscroll' is
actually what I'm looking for.

Thanks, and sorry for the noise.

Re: Redisplay resets vscroll when window start changes

[Po Lu]

Po Lu <luangruo@yahoo.com> writes:

> Thanks, and sorry for the noise.

Actually no, there should also be an option for `line-move' to avoid
setting the vscroll to 0, otherwise the vscroll becomes reset every time
`line-move' is changed, regardless of whether or not the window was
actually scrolled.  Thanks.

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 16 Nov 2021 21:01:07 +0800
> 
> Po Lu <luangruo@yahoo.com> writes:
> 
> > Thanks, and sorry for the noise.
> 
> Actually no, there should also be an option for `line-move' to avoid
> setting the vscroll to 0, otherwise the vscroll becomes reset every time
> `line-move' is changed, regardless of whether or not the window was
> actually scrolled.  Thanks.

If you are moving to another line, how would it make sense to leave
vscroll alone?  Or what am I missing?

Re: Redisplay resets vscroll when window start changes

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> If you are moving to another line, how would it make sense to leave
> vscroll alone?  Or what am I missing?

Basically, if someone has the following content in a window:

AAAA
BBBB
CCCC
DDDD
EEEE
FFFF
GGGG
HHHH

And point is on the first "G", while vscroll is set so that part of
"AAAA" is obscured, he will expect moving up so that point is on the
first "F" to not reset the vscroll.

I don't know why, but that's how every other program works, so people
always expect that and get very confused if a program behaves otherwise.

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 16 Nov 2021 21:38:57 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > If you are moving to another line, how would it make sense to leave
> > vscroll alone?  Or what am I missing?
> 
> Basically, if someone has the following content in a window:
> 
> AAAA
> BBBB
> CCCC
> DDDD
> EEEE
> FFFF
> GGGG
> HHHH
> 
> And point is on the first "G", while vscroll is set so that part of
> "AAAA" is obscured, he will expect moving up so that point is on the
> first "F" to not reset the vscroll.

Moving up pixelwise? that should gradually show more and more of AAAA
with each pixel movement.  How is vscroll involved here?

Or are you talking about some other kind of "moving up?

Re: Redisplay resets vscroll when window start changes

[Michael Welsh Duggan]

[-- Attachment #1: Type: text/plain, Size: 851 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Po Lu <luangruo@yahoo.com>
>> Cc: emacs-devel@gnu.org
>> Date: Tue, 16 Nov 2021 21:38:57 +0800
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > If you are moving to another line, how would it make sense to leave
>> > vscroll alone?  Or what am I missing?
>> 
>> Basically, if someone has the following content in a window:
>> 
>> AAAA
>> BBBB
>> CCCC
>> DDDD
>> EEEE
>> FFFF
>> GGGG
>> HHHH
>> 
>> And point is on the first "G", while vscroll is set so that part of
>> "AAAA" is obscured, he will expect moving up so that point is on the
>> first "F" to not reset the vscroll.
>
> Moving up pixelwise? that should gradually show more and more of AAAA
> with each pixel movement.  How is vscroll involved here?
>
> Or are you talking about some other kind of "moving up?

Given the following buffer:


[-- Attachment #2: Screenshot from 2021-11-16 12-06-46.png --]
[-- Type: image/png, Size: 7622 bytes --]

[-- Attachment #3: Type: text/plain, Size: 44 bytes --]


If I type `C-x C-e` at point, I get this:


[-- Attachment #4: Screenshot from 2021-11-16 12-07-41.png --]
[-- Type: image/png, Size: 7608 bytes --]

[-- Attachment #5: Type: text/plain, Size: 52 bytes --]


At this point, if I type, say, `C-n', I get this:


[-- Attachment #6: Screenshot from 2021-11-16 12-08-51.png --]
[-- Type: image/png, Size: 7488 bytes --]

[-- Attachment #7: Type: text/plain, Size: 361 bytes --]


As you can see, the vscroll has reset, despite simply doing
cursor-movement within the visible reason.  This is the behavior that I
believe Po Lu is complaining about.  Why should cursor movement that
wouldn't cause a scrolling event under normal circumstances cause one
when vertical fractional scrolling is present?

-- 
Michael Welsh Duggan
(md5i@md5i.com)

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

> From: Michael Welsh Duggan <mwd@md5i.com>
> Cc: Po Lu <luangruo@yahoo.com>,  emacs-devel@gnu.org
> Date: Tue, 16 Nov 2021 12:12:57 -0500
> 
> As you can see, the vscroll has reset, despite simply doing
> cursor-movement within the visible reason.  This is the behavior that I
> believe Po Lu is complaining about.  Why should cursor movement that
> wouldn't cause a scrolling event under normal circumstances cause one
> when vertical fractional scrolling is present?

Because the default C-n is not designed to support use cases like this
one, where the vscrolled line is not tall enough.  What you want is to
rebind C-n to a different command (or have a different key sequence
for a different command), because the one C-n runs now will bite you
all the way in such situation; the fact that it resets vscroll is just
the tip of a very large iceberg.  There are 3 functions there that
work in tight cooperation, each one with non-trivial logic, and all of
them implicitly assuming that they are dealing with lines
significantly taller than the default face's font.  That it seems to
"almost" work is an illusion.

Re: Redisplay resets vscroll when window start changes

[Stefan Monnier]

> That it seems to "almost" work is an illusion.

The flip side is that if it currently doesn't really work anyway, then
while not resetting vscroll might change its brokenness it won't
necessarily make it worse overall.
;-)


        Stefan

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Michael Welsh Duggan <mwd@md5i.com>,  luangruo@yahoo.com,
>   emacs-devel@gnu.org
> Date: Tue, 16 Nov 2021 22:24:08 -0500
> 
> > That it seems to "almost" work is an illusion.
> 
> The flip side is that if it currently doesn't really work anyway, then
> while not resetting vscroll might change its brokenness it won't
> necessarily make it worse overall.
> ;-)

It does work (and works quite well, thank you), just not with the
use-cases described in this discussion.

Re: Redisplay resets vscroll when window start changes

[Po Lu]

Michael Welsh Duggan <mwd@md5i.com> writes:

> As you can see, the vscroll has reset, despite simply doing
> cursor-movement within the visible reason.  This is the behavior that I
> believe Po Lu is complaining about.  Why should cursor movement that
          ^^^^^^^^^^^^^^^^^^^^^^^^^^

Minor correction: I wasn't the source of this complaint, some other
people were.

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Wed, 17 Nov 2021 08:26:52 +0800
> 
> Michael Welsh Duggan <mwd@md5i.com> writes:
> 
> > As you can see, the vscroll has reset, despite simply doing
> > cursor-movement within the visible reason.  This is the behavior that I
> > believe Po Lu is complaining about.  Why should cursor movement that
>           ^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> Minor correction: I wasn't the source of this complaint, some other
> people were.

My suggestion is to implement equivalents of C-n/C-p that do what you
want, instead of asking for previous/next-line to be able to preserve
the vscroll, because those commands aren't supposed to support the use
case you want to support.

Re: Redisplay resets vscroll when window start changes

[Stefan Monnier]

Eli Zaretskii [2021-11-17 05:31:00] wrote:
> My suggestion is to implement equivalents of C-n/C-p that do what you
> want, instead of asking for previous/next-line to be able to preserve
> the vscroll, because those commands aren't supposed to support the use
> case you want to support.

[ I feel somewhat responsible for this mess, since IIRC I was the one
  who first tried to write a line-move that preserves the pixel-column,
  by using the redisplay iterator code, without understand very much of
  what it does or how it works.  ]

I'm still not sure how/why line-move should affect vscroll.
AFAIK while line-move does use some of the redisplay code, in theory at
least its effect should only ever be to move point (i.e. when called
within a `save-excursion` it should end up having no effect at all).

Would there be a way to distinguish the case where line-move can return
a correct result without changing vscroll, from the case where it does
require changing vscroll in order to find its answer?


        Stefan

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Po Lu <luangruo@yahoo.com>,  mwd@md5i.com,  emacs-devel@gnu.org
> Date: Tue, 16 Nov 2021 22:40:35 -0500
> 
> I'm still not sure how/why line-move should affect vscroll.

Because it needs to handle the case of scrolling through a very tall
image.  When that scrolling shows the last (bottom-most) portion of
the image, the next C-n should go to the next screen line, and that
means zero out the vscroll.  Conversely, when we show only the upper
portion of the image, the next C-n should increase vscroll to show its
lower portions.

> AFAIK while line-move does use some of the redisplay code, in theory at
> least its effect should only ever be to move point (i.e. when called
> within a `save-excursion` it should end up having no effect at all).

If you only move point, but leave vscroll at zero, you will never be
able to show the parts of the image that exceed the window's height.

> Would there be a way to distinguish the case where line-move can return
> a correct result without changing vscroll, from the case where it does
> require changing vscroll in order to find its answer?

I don't think I understand the question.  It might be worthwhile to
keep in mind that the display engine (almost) never changes vscroll,
so if you want it to show a large object at some position, you must
manipulate vscroll from Lisp.

Re: Redisplay resets vscroll when window start changes

[Stefan Monnier]

>> I'm still not sure how/why line-move should affect vscroll.
> Because it needs to handle the case of scrolling through a very tall
> image.
[...]
>> Would there be a way to distinguish the case where line-move can return
>> a correct result without changing vscroll, from the case where it does
>> require changing vscroll in order to find its answer?
> I don't think I understand the question.

The question can be rephrased as: could `line-move` only reset vscroll
in the case where we're scrolling through a very tall image?


        Stefan

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: luangruo@yahoo.com,  mwd@md5i.com,  emacs-devel@gnu.org
> Date: Wed, 17 Nov 2021 15:40:39 -0500
> 
> >> Would there be a way to distinguish the case where line-move can return
> >> a correct result without changing vscroll, from the case where it does
> >> require changing vscroll in order to find its answer?
> > I don't think I understand the question.
> 
> The question can be rephrased as: could `line-move` only reset vscroll
> in the case where we're scrolling through a very tall image?

It currently handles the case of a tall screen line, which could be
due to an image, an xwidget, or simply very large font.

If you want to limit it only to those cases, then perhaps the answer
is yes, but the problem with those functions is that they try to
handle many use cases that are not completely described anywhere, so
testing whether a particular change could break any of them is nigh
impossible in practice.  And the code does work for the use cases for
which it was written.  Which is why I suggested a completely new
command for the purpose under discussion.

Re: Redisplay resets vscroll when window start changes

[Stefan Monnier]

>> >> Would there be a way to distinguish the case where line-move can return
>> >> a correct result without changing vscroll, from the case where it does
>> >> require changing vscroll in order to find its answer?
>> > I don't think I understand the question.
>> 
>> The question can be rephrased as: could `line-move` only reset vscroll
>> in the case where we're scrolling through a very tall image?
>
> It currently handles the case of a tall screen line, which could be
> due to an image, an xwidget, or simply very large font.
>
> If you want to limit it only to those cases, then perhaps the answer
> is yes, but the problem with those functions is that they try to
> handle many use cases that are not completely described anywhere, so
> testing whether a particular change could break any of them is nigh
> impossible in practice.  And the code does work for the use cases for
> which it was written.  Which is why I suggested a completely new
> command for the purpose under discussion.

I must be missing something: in the test case posted here, AFAICT we
just have a call to `previous-line` which doesn't involve any tall
screen line, yet the result is still that vscroll is reset.


        Stefan

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: luangruo@yahoo.com,  mwd@md5i.com,  emacs-devel@gnu.org
> Date: Thu, 18 Nov 2021 22:17:41 -0500
> 
> > If you want to limit it only to those cases, then perhaps the answer
> > is yes, but the problem with those functions is that they try to
> > handle many use cases that are not completely described anywhere, so
> > testing whether a particular change could break any of them is nigh
> > impossible in practice.  And the code does work for the use cases for
> > which it was written.  Which is why I suggested a completely new
> > command for the purpose under discussion.
> 
> I must be missing something: in the test case posted here, AFAICT we
> just have a call to `previous-line` which doesn't involve any tall
> screen line, yet the result is still that vscroll is reset.

The current code is not prepared to deal with test cases like that
one.  The current Emacs code never sets vscroll unless we are moving
through a tall line, so resetting vscroll as part of previous-line is
a no-op when there's no tall lines in sight, and is not doing any
harm.  The test case was produced by a feature not yet in Emacs, and
so I'm saying that such a feature needs its own variant of
previous-line.

Re: Redisplay resets vscroll when window start changes

[Stefan Monnier]

Eli Zaretskii [2021-11-19 09:13:53] wrote:
> The current code is not prepared to deal with test cases like that
> one.  The current Emacs code never sets vscroll unless we are moving
> through a tall line, so resetting vscroll as part of previous-line is
> a no-op when there's no tall lines in sight, and is not doing any
> harm.  The test case was produced by a feature not yet in Emacs, and
> so I'm saying that such a feature needs its own variant of
> previous-line.

But to the extent than this feature does not intend to change the
behavior of `previous-line`, this new variant of `previous-line` should
ideally do everything exactly like the current `previous-line`, except
be more careful not to reset vscroll, right?

So, ideally this new version could also be used when that new feature
isn't used?


        Stefan

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: luangruo@yahoo.com,  mwd@md5i.com,  emacs-devel@gnu.org
> Date: Fri, 19 Nov 2021 13:07:32 -0500
> 
> Eli Zaretskii [2021-11-19 09:13:53] wrote:
> > The current code is not prepared to deal with test cases like that
> > one.  The current Emacs code never sets vscroll unless we are moving
> > through a tall line, so resetting vscroll as part of previous-line is
> > a no-op when there's no tall lines in sight, and is not doing any
> > harm.  The test case was produced by a feature not yet in Emacs, and
> > so I'm saying that such a feature needs its own variant of
> > previous-line.
> 
> But to the extent than this feature does not intend to change the
> behavior of `previous-line`, this new variant of `previous-line` should
> ideally do everything exactly like the current `previous-line`, except
> be more careful not to reset vscroll, right?

No, not at all.  The current code consults the height of the line
through which it scrolls many times and for various purposes.  All of
those places can potentially do the wrong thing in the use case
described here.

Re: Redisplay resets vscroll when window start changes

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> My suggestion is to implement equivalents of C-n/C-p that do what you
> want, instead of asking for previous/next-line to be able to preserve
> the vscroll, because those commands aren't supposed to support the use
> case you want to support.

Yes, I understand that.  But the problem is any command that can call
`line-move' will end up resetting the vscroll, which most people don't
expect.  Since making `line-move' not reset the vscroll is not an
option, a better solution will have to be found, which replacing
previous-line/next-line is not.

I know of an external package named `good-scroll' that tries to solve
this problem.  I'll let you know of my findings.

Thanks in advance.

Re: Redisplay resets vscroll when window start changes

[Eli Zaretskii]

On November 17, 2021 6:31:32 AM GMT+02:00, Po Lu <luangruo@yahoo.com> wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > My suggestion is to implement equivalents of C-n/C-p that do what you
> > want, instead of asking for previous/next-line to be able to preserve
> > the vscroll, because those commands aren't supposed to support the use
> > case you want to support.
> 
> Yes, I understand that.  But the problem is any command that can call
> `line-move' will end up resetting the vscroll, which most people don't
> expect.  Since making `line-move' not reset the vscroll is not an
> option, a better solution will have to be found, which replacing
> previous-line/next-line is not.


My point is precisely that line-move should not be used for this purpose, because it and the other 2 functions it calls don't support this use case, and OTOH include a lot of code logic which implicitly assumes a very different use case.  It would be much easier and cleaner to implement what you need from scratch.

Re: Redisplay resets vscroll when window start changes

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

>> Basically, if someone has the following content in a window:
>> 
>> AAAA
>> BBBB
>> CCCC
>> DDDD
>> EEEE
>> FFFF
>> GGGG
>> HHHH
>> 
>> And point is on the first "G", while vscroll is set so that part of
>> "AAAA" is obscured, he will expect moving up so that point is on the
>> first "F" to not reset the vscroll.

> Moving up pixelwise? that should gradually show more and more of AAAA
> with each pixel movement.  How is vscroll involved here?

> Or are you talking about some other kind of "moving up?

Yes, I meant "moving up" as in `previous-line' and other such commands
that call `line-move'.

Thanks.