The future of Follow Mode - a proposal.

The future of Follow Mode - a proposal.

[Alan Mackenzie]

Hello, Anders and Emacs.

Although Follow Mode has served extremely well for several decades, it has
several shortcomings which make it less good than it might be:
(i) It is not particularly fast - synchronising the several pertinent windows
  can take several tenths of a second, even on modern hardware.  One reason
  for this is that Follow Mode and the display engine fight eachother, and
  several strategems which are necessary for FM to win (such as explicitly
  calling `redisplay' from the post-command-hook) are nevertheless suboptimal.
(ii) Where adjacent windows are of unequal width (this is the normal case), a
  continued line is displayed incorrectly when it straddles two windows:
  either one or more characters are displayed twice, or they are not displayed
  at all (depending on which window is the wider one).  This is
  extraordinarily difficult to fix with the current display engine and
  follow.el.
(iii) Each Follow Mode window has its own individual mode line, although
  Follow Mode's aim is to create the illusion of a single multi-column window.
  Often important info is truncated, due to the narrowness of a typical FM
  window.  It would be better to have a single mode line stretching across the
  entire screen.

I propose moving much of Follow Mode's mechanism into our C code, in
particular, into window.c and xdisp.c.

In window.c/h would appear a new `struct wgroup' which would administer Follow
Mode window groups.  It would contain details of the buffer it's
administering, the frame, and pointers to the first and last of the FM
windows.  `struct window' would be amended to contain forward and backward
links, allowing us to create a chain of the FM windows, also to contain a
pointer to the wgroup.  In addition, `struct wgroup' would note which of its
windows is the current/active one.

Also in window.c would come infrastructure for creating and administering
`struct wgroups'.  Such functions as `pop-to-buffer' would typically need to
remove a window from the `struct wgroup' before repurposing it for the new
buffer.

The first shock: `select-window' would be amended, such that attempting to
select any window in a wgroup would result in the current/active window in the
group actually being selected instead.  This would go a long way towards
maintaining the illusion of a single multi-column window.  An additional
&optional parameter would be given to `select-window' meaning "select THIS
window, no matter what", for the use of lower level routines.

`vertical-motion' would need a substantial rethink, since it would have to
work in a "window" whose width varies.  There will likely be several other
such functions needing amendment.  But not many.

Most of the difficult work would be within xdisp.c.  Any form of scrolling is
going to be tricky to implement, as indeed it is in follow.el.  Moving over
the boundary of two windows will mean (?)xdisp.c having to select-window where
it currently scrolls a window.  More generally, the scrolling/point-moving
bits would need to be amended.

Creating a unified mode line seems tricky, given the way it's currently done
in xdisp.c.  (It merely reserves the final line of the window, and writes
there.)  There may be better ways to achieve the single mode line, but one way
would be to implement mode lines as windows in their own right, thus giving
them scope to be somewhere else other than squashed up into a window's last
line.  This would also enable several FM windows to share a single mode line.

Comments?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> Date: Thu, 18 Feb 2016 19:56:30 +0000
> From: Alan Mackenzie <acm@muc.de>
> 
> I propose moving much of Follow Mode's mechanism into our C code, in
> particular, into window.c and xdisp.c.

IMO, moving the code to C will solve only the marginal aspects of
this.  The main problem -- the fact that the current display engine
doesn't support windows of unequal width -- cannot be solved without
deep changes.  We need to at least design these changes first, so that
we have a clear idea how to solve these issues.  FWIW, I thought about
this for a while, and didn't see any easy way of doing it.

If we abandon the design goal of supporting windows of unequal width,
the problem becomes much easier.

I suggest to talk about these main issues first, and only move further
with any kind of implementation once we've resolved this.  The
decision of what to keep in Lisp and what to move to C is also
meaningful only after resolving these main issues.

Re: The future of Follow Mode - a proposal.

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 793 bytes --]

On Thu, Feb 18, 2016 at 2:56 PM, Alan Mackenzie <acm@muc.de> wrote:

> There may be better ways to achieve the single mode line, but one way
> would be to implement mode lines as windows in their own right, thus giving
> them scope to be somewhere else other than squashed up into a window's last
> line.


I have yet to use FM.  OTOH I have often wanted to put the mode line at
of my windows.  There are some hacks out on the web that approximate
such behavior.  They do so by usurping the header line, thereby breaking
various modes.

Putting the mode line at the top of the window is especially desirable on
large, high resolution screens.  When a buffer's contents does not fill
a vertical window there can be a large expanse of blank space between
EOB and that window's mode line.

/john

[-- Attachment #2: Type: text/html, Size: 1225 bytes --]

Re: The future of Follow Mode - a proposal.

[Alan Mackenzie]

Hello, Eli.

On Thu, Feb 18, 2016 at 10:24:10PM +0200, Eli Zaretskii wrote:
> > Date: Thu, 18 Feb 2016 19:56:30 +0000
> > From: Alan Mackenzie <acm@muc.de>
> > 
> > I propose moving much of Follow Mode's mechanism into our C code, in
> > particular, into window.c and xdisp.c.

> IMO, moving the code to C will solve only the marginal aspects of
> this.

It would certainly help with the speed, and likely help get a uniform
mode line.

> The main problem -- the fact that the current display engine doesn't
> support windows of unequal width -- cannot be solved without deep
> changes.  We need to at least design these changes first, so that we
> have a clear idea how to solve these issues.  FWIW, I thought about
> this for a while, and didn't see any easy way of doing it.

The display engine currently works on each window as an independent
entity.  It needs to understand "window groups".  Several things it now
does on windows (scrolling, moving point, ...) will need to be extended
to work on window groups.

Where and why do you see the need for deep changes?

> If we abandon the design goal of supporting windows of unequal width,
> the problem becomes much easier.

Yes, but that's not realistic, I think.  Follow Mode windows _are_ of
unequal width.

> I suggest to talk about these main issues first, and only move further
> with any kind of implementation once we've resolved this.  The
> decision of what to keep in Lisp and what to move to C is also
> meaningful only after resolving these main issues.

Again, could you list the issues in somewhat more detail, please?  A
trouble with projects like this one is that they can hover unendlessly
at a very high level of abstraction, too high for much progress to be
made.  This is why I sketched out in some detail an approach to getting
things done.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The future of Follow Mode - a proposal.

[martin rudalics]

 > Yes, but that's not realistic, I think.  Follow Mode windows _are_ of
 > unequal width.

We can easily make the text area of these windows the same width.
Simply add any remaining pixel to the divider.

martin

Re: The future of Follow Mode - a proposal.

[Anders Lindgren]

[-- Attachment #1: Type: text/plain, Size: 1616 bytes --]

>
> IMO, moving the code to C will solve only the marginal aspects of
> this.  The main problem -- the fact that the current display engine
> doesn't support windows of unequal width -- cannot be solved without
> deep changes.  We need to at least design these changes first, so that
> we have a clear idea how to solve these issues.  FWIW, I thought about
> this for a while, and didn't see any easy way of doing it.
>
> If we abandon the design goal of supporting windows of unequal width,
> the problem becomes much easier.


Windows of unequal width is something that we need to live with, at least
if follow-mode is something that a user can enable without affecting the
layout of a frame, like today.

Anyway, wouldn't it be possible for the display engine to do the layout of
the leftmost window (of a follow mode window group) using the normal system
for finding a suitable start location on even line boundaries. The rest of
the windows in the window group could then use the end position of the
previous window.

In fact, I think it could be supported today from lisp if only it was
possible to disable the part of the display engine that ensures that the
window start position is an even multiple of the window width, on a
window-by-window basis. In the case of follow-mode, the leftmost window
should keep it enabled, whereas the rest should disable this feature.

(The only thing that wouldn't work is if a multi-width character (like
control-l "^L") is partially visible. It will continue to work like today,
with "^" visible at the end of one window and "^L" at the start of the
next.)

    -- Anders

[-- Attachment #2: Type: text/html, Size: 2021 bytes --]

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> Date: Fri, 19 Feb 2016 14:25:23 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> Hello, Eli.
> 
> On Thu, Feb 18, 2016 at 10:24:10PM +0200, Eli Zaretskii wrote:
> > > Date: Thu, 18 Feb 2016 19:56:30 +0000
> > > From: Alan Mackenzie <acm@muc.de>
> > > 
> > > I propose moving much of Follow Mode's mechanism into our C code, in
> > > particular, into window.c and xdisp.c.
> 
> > IMO, moving the code to C will solve only the marginal aspects of
> > this.
> 
> It would certainly help with the speed, and likely help get a uniform
> mode line.

Speed is one of the marginal aspects of this, from my POV.  Follow
mode is not terribly slow as it is.

> > The main problem -- the fact that the current display engine doesn't
> > support windows of unequal width -- cannot be solved without deep
> > changes.  We need to at least design these changes first, so that we
> > have a clear idea how to solve these issues.  FWIW, I thought about
> > this for a while, and didn't see any easy way of doing it.
> 
> The display engine currently works on each window as an independent
> entity.  It needs to understand "window groups".  Several things it now
> does on windows (scrolling, moving point, ...) will need to be extended
> to work on window groups.
> 
> Where and why do you see the need for deep changes?

I already explained this above: "the fact that the current display
engine doesn't support windows of unequal width".  If you want to lift
this limitation, the move_it_* family of functions, which simulate
redisplay, and are the workhorse of every decision Emacs makes that
concerns layout, cannot switch windows in their inner loops.  Their
model is: initialize the iterator object with the metrics of the
window, then use that for the entire iteration.  There's only one copy
of window metrics in the iterator object.  They also assume that text
above the window start and below its end would have been laid out
using the same window width, something that is required when the
display engine considers whether to scroll the window.

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> Date: Fri, 19 Feb 2016 15:34:13 +0100
> From: martin rudalics <rudalics@gmx.at>
> Cc: emacs-devel@gnu.org
> 
>  > Yes, but that's not realistic, I think.  Follow Mode windows _are_ of
>  > unequal width.
> 
> We can easily make the text area of these windows the same width.
> Simply add any remaining pixel to the divider.

My suggestion would be to forcibly equalize the windows.  Then, even
if we cannot do that exactly (e.g., because the number of pixels is
odd), the difference is a single pixel, and that _can_ be handled by
adding the pixel to the divider.  By contrast, a divider that is many
pixels wider than normal will look odd if not buggy.

I fail to see why users of Follow mode won't agree to having
equal-width windows.  After all, this mode is supposed to create an
illusion of one long window, so its parts should have the same width,
IMO.  However, I don't use the mode, so perhaps I'm missing something.

Re: The future of Follow Mode - a proposal.

[Alan Mackenzie]

Hello, John.

On Thu, Feb 18, 2016 at 03:41:22PM -0500, John Yates wrote:
> On Thu, Feb 18, 2016 at 2:56 PM, Alan Mackenzie <acm@muc.de> wrote:

> > There may be better ways to achieve the single mode line, but one way
> > would be to implement mode lines as windows in their own right, thus giving
> > them scope to be somewhere else other than squashed up into a window's last
> > line.


> I have yet to use FM.  OTOH I have often wanted to put the mode line
> at [the top] of my windows.  There are some hacks out on the web that
> approximate such behavior.  They do so by usurping the header line,
> thereby breaking various modes.

> Putting the mode line at the top of the window is especially desirable on
> large, high resolution screens.  When a buffer's contents does not fill
> a vertical window there can be a large expanse of blank space between
> EOB and that window's mode line.

I don't think this would be too difficult to implement, as a hack, and
would be a good learning exercise for the display code.  One problem I
foresee is that there would no longer be anything to separate the main
window area from the minibuffer.  Have you any thoughts on how this
would be handled?

> /john

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> Date: Fri, 19 Feb 2016 15:56:22 +0100
> From: Anders Lindgren <andlind@gmail.com>
> Cc: Alan Mackenzie <acm@muc.de>, emacs-devel <emacs-devel@gnu.org>
> 
> Windows of unequal width is something that we need to live with, at least if follow-mode is something that a
> user can enable without affecting the layout of a frame, like today.

My suggestion is that enabling the mode will equalize the width of all
the windows under Follow mode.

> Anyway, wouldn't it be possible for the display engine to do the layout of the leftmost window (of a follow mode
> window group) using the normal system for finding a suitable start location on even line boundaries. The rest
> of the windows in the window group could then use the end position of the previous window.

This would mean the order in which we display windows will have to be
forced to follow the Follow mode order.  Moreover, it would mean that
the decisions of whether to redisplay a certain window will now have
to consider other windows, something that doesn't happen today.

Moreover, the starting point of the window is just the tip of the
iceberg.  The display engine uses the move_it_* family of functions to
decide where the window starts and ends, when to scroll the window,
whether point is visible or not, etc.  We also use these functions for
user-level window scrolling and in vertical-motion.  All of that code
assumes that the window width doesn't change beyond window edges.
Invalidating this assumption wreaks havoc in many places, so these
functions will have to be rewritten to take the fact that text in the
next window uses a different width.  See the discussion of
vertical-motion we had with Alan several months ago -- while working
on the patch I presented there, I realized that this feature is
unworkable as long as the display engine was not taught to handle
unequal-width windows.

> In fact, I think it could be supported today from lisp if only it was possible to disable the part of the display
> engine that ensures that the window start position is an even multiple of the window width, on a
> window-by-window basis. In the case of follow-mode, the leftmost window should keep it enabled, whereas
> the rest should disable this feature.

The current display engine likes the window start point to be at the
beginning of a physical line.  Lifting this restriction also requires
changes, because currently, even if you trick the display into using a
starting point in the middle of a line, it is likely to reset that to
the line beginning on the next redisplay cycle.

IOW, without changes in the display engine, we will be making a series
of questionable kludges one upon the other.  That's hardly the way to
develop Emacs.

I understand the desire to avoid significant changes in the design of
the display engine, but I see no way this could work reliably, unless
we force all the Follow-mode windows to be of equal width (in
character units).

> (The only thing that wouldn't work is if a multi-width character (like control-l "^L") is partially visible. It will
> continue to work like today, with "^" visible at the end of one window and "^L" at the start of the next.)

Control characters are not the only ones which will bump into this.

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> Date: Fri, 19 Feb 2016 16:21:59 +0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: Emacs developers <emacs-devel@gnu.org>
> 
> One problem I foresee is that there would no longer be anything to
> separate the main window area from the minibuffer.  Have you any
> thoughts on how this would be handled?

You can try this today, it's already supported, AFAIR.

Re: The future of Follow Mode - a proposal.

[Alan Mackenzie]

Hello, Eli.

On Fri, Feb 19, 2016 at 06:08:35PM +0200, Eli Zaretskii wrote:
> > Date: Fri, 19 Feb 2016 14:25:23 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > IMO, moving the code to C will solve only the marginal aspects of
> > > this.

> > It would certainly help with the speed, and likely help get a uniform
> > mode line.

> Speed is one of the marginal aspects of this, from my POV.  Follow
> mode is not terribly slow as it is.

For me, there's a noticeable delay between, say scrolling one window and
the resulting scrolling in the other windows happening.  There are
several distinct redisplays happening, rather than the ideal of just one.
But I work mostly on a Linux VT.  Perhaps the delay's are less noticeable
on X or win32.

> > > The main problem -- the fact that the current display engine doesn't
> > > support windows of unequal width -- cannot be solved without deep
> > > changes.  We need to at least design these changes first, so that we
> > > have a clear idea how to solve these issues.  FWIW, I thought about
> > > this for a while, and didn't see any easy way of doing it.

> > The display engine currently works on each window as an independent
> > entity.  It needs to understand "window groups".  Several things it now
> > does on windows (scrolling, moving point, ...) will need to be extended
> > to work on window groups.

> > Where and why do you see the need for deep changes?

> I already explained this above: "the fact that the current display
> engine doesn't support windows of unequal width".  If you want to lift
> this limitation, the move_it_* family of functions, which simulate
> redisplay, and are the workhorse of every decision Emacs makes that
> concerns layout, cannot switch windows in their inner loops.

I envisage reinitialising the iterator structure as necessary when
passing bewteen windows.  The change in width would be handled at a
relatively high level.  The window start position is known, the window
end position could be calculated as we progress.

> Their model is: initialize the iterator object with the metrics of the
> window, then use that for the entire iteration.  There's only one copy
> of window metrics in the iterator object.  They also assume that text
> above the window start and below its end would have been laid out using
> the same window width, something that is required when the display
> engine considers whether to scroll the window.

As in Fvertical_motion.  Sometimes it will be appropriate to keep the
dimensions of the "old" window (for example, when scrolling by doing
`vertical-motion' followed by `set-window-start'), sometimes it won't be
(for example, in `previous-line' when `line-mode-visual' is set).

Fvertical_motion is difficult because it's function hasn't been
unambiguously tied down (there's never been any need before).  It would
be necessary to classify each use of it as being either "movement within
the physical window structure" or "movement within an ideal infinite
height window".

As an example, `compute_window_start_on_continuation_line' would have to
use the dimensions of the previous window to determine the window-start.
Jiggling the various windows around after text changes or scrolling is
going to be the hard part of the coding.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The future of Follow Mode - a proposal.

[Alan Mackenzie]

Hello, Eli.

On Fri, Feb 19, 2016 at 06:30:01PM +0200, Eli Zaretskii wrote:
> > Date: Fri, 19 Feb 2016 15:56:22 +0100
> > From: Anders Lindgren <andlind@gmail.com>
> > Cc: Alan Mackenzie <acm@muc.de>, emacs-devel <emacs-devel@gnu.org>

> > Windows of unequal width is something that we need to live with, at
> > least if follow-mode is something that a user can enable without
> > affecting the layout of a frame, like today.

> My suggestion is that enabling the mode will equalize the width of all
> the windows under Follow mode.

This would sort of work in the usual case, but in the general case,
arbitrary windows, not necessarily covering a nice rectangular area would
be involved.  It is not simple to adjust such an arbitrary set to make
them all the same width.

> > Anyway, wouldn't it be possible for the display engine to do the
> > layout of the leftmost window (of a follow mode window group) using
> > the normal system for finding a suitable start location on even line
> > boundaries. The rest of the windows in the window group could then
> > use the end position of the previous window.

> This would mean the order in which we display windows will have to be
> forced to follow the Follow mode order.  Moreover, it would mean that
> the decisions of whether to redisplay a certain window will now have
> to consider other windows, something that doesn't happen today.

It doesn't happen in the display engine, no, but it happens at the Lisp
level in Follow Mode.  It would be more efficient in every sense if it
were actually done by the display engine.

> Moreover, the starting point of the window is just the tip of the
> iceberg.  The display engine uses the move_it_* family of functions to
> decide where the window starts and ends, when to scroll the window,
> whether point is visible or not, etc.  We also use these functions for
> user-level window scrolling and in vertical-motion.  All of that code
> assumes that the window width doesn't change beyond window edges.
> Invalidating this assumption wreaks havoc in many places, so these
> functions will have to be rewritten to take the fact that text in the
> next window uses a different width.  See the discussion of
> vertical-motion we had with Alan several months ago --

:-)  I've still got the code somewhere which allows an arbitrary
window-start to be set.  That wasn't the difficult bit.

> while working on the patch I presented there, I realized that this
> feature is unworkable as long as the display engine was not taught to
> handle unequal-width windows.

The display engine would need to handle window _groups_.  The variable
width part would be part of this.

> > In fact, I think it could be supported today from lisp if only it was
> > possible to disable the part of the display engine that ensures that
> > the window start position is an even multiple of the window width, on
> > a window-by-window basis. In the case of follow-mode, the leftmost
> > window should keep it enabled, whereas the rest should disable this
> > feature.

I tried that last autumn.  It was more complicated than I'd imagined.

> The current display engine likes the window start point to be at the
> beginning of a physical line.  Lifting this restriction also requires
> changes, because currently, even if you trick the display into using a
> starting point in the middle of a line, it is likely to reset that to
> the line beginning on the next redisplay cycle.

This will cease to be a problem if the display engine can handle window
groups.  The display engine already allows windows to start in
continuation lines.  It's just how you calculate the start of such a
line.

> IOW, without changes in the display engine, we will be making a series
> of questionable kludges one upon the other.  That's hardly the way to
> develop Emacs.

> I understand the desire to avoid significant changes in the design of
> the display engine, but I see no way this could work reliably, unless
> we force all the Follow-mode windows to be of equal width (in
> character units).

I agree that the display engine would need significant modification.

> > (The only thing that wouldn't work is if a multi-width character
> > (like control-l "^L") is partially visible. It will continue to work
> > like today, with "^" visible at the end of one window and "^L" at the
> > start of the next.)

> Control characters are not the only ones which will bump into this.

This surely occurs already, and might not be relevant to the exercise.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> Date: Fri, 19 Feb 2016 18:18:34 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > > Where and why do you see the need for deep changes?
> 
> > I already explained this above: "the fact that the current display
> > engine doesn't support windows of unequal width".  If you want to lift
> > this limitation, the move_it_* family of functions, which simulate
> > redisplay, and are the workhorse of every decision Emacs makes that
> > concerns layout, cannot switch windows in their inner loops.
> 
> I envisage reinitialising the iterator structure as necessary when
> passing bewteen windows.  The change in width would be handled at a
> relatively high level.  The window start position is known, the window
> end position could be calculated as we progress.

The functions we talk about currently don't know what they are invoked
for.  Your envisioned changes imply that they should behave
differently depending on whether the results will be used for layout
of the current window or the next/previous window in a group.  That's
part of the changes I had in mind.  They are not trivial.  But without
them, what you want to do will not work reliably.

> As an example, `compute_window_start_on_continuation_line' would have to
> use the dimensions of the previous window to determine the window-start.
> Jiggling the various windows around after text changes or scrolling is
> going to be the hard part of the coding.

Yes, and the result will be non-trivial changes in the overall logic,
because redisplaying a window will no longer be independent of other
windows.

It's all doable, of course, but I suggest taking a good look at the
use cases for each of these functions, before you design the way they
should work to support windows of unequal width.

Re: The future of Follow Mode - a proposal.

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 1226 bytes --]

On Fri, Feb 19, 2016 at 11:21 AM, Alan Mackenzie <acm@muc.de> wrote:

> One problem I
> foresee is that there would no longer be anything to separate the main
> window area from the minibuffer.  Have you any thoughts on how this
> would be handled?


This may seem heretical but I would move the mini-buffer to the topof
the frame.  I can already approximate such a layout using a separate
mini-buffer frame and the _NET_WM_STRUT_PARTIAL property.

Ideally there would be a frame parameter that would allow me to specify
that the mini-buffer should be positioned at the top of the frame.

My motivation here is that I have a single, large, very high resolution
screen.  I maintain a set of tall, horizontally arrayed windows.  I advise
split-window and delete-window to keep all windows balanced.  Since
these windows are so tall very often they are only partially filled.  The
net effect is that interesting content is concentrated at the top of my
screen.  That being the case needing to look to the bottom of the
screen to inspect the mode-line and/or mini-buffer only slows me down.
I would be much happier if I could have:
    * the mini-buffer at the top of its frame
    * the mode-line at the top of its window

/john

[-- Attachment #2: Type: text/html, Size: 2151 bytes --]

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> Date: Fri, 19 Feb 2016 14:25:10 -0500
> From: John Yates <john@yates-sheets.org>
> Cc: Emacs developers <emacs-devel@gnu.org>
> 
> This may seem heretical but I would move the mini-buffer to the topof
> the frame. I can already approximate such a layout using a separate
> mini-buffer frame and the _NET_WM_STRUT_PARTIAL property.
> 
> Ideally there would be a frame parameter that would allow me to specify
> that the mini-buffer should be positioned at the top of the frame.
> 
> My motivation here is that I have a single, large, very high resolution
> screen. I maintain a set of tall, horizontally arrayed windows. I advise
> split-window and delete-window to keep all windows balanced. Since
> these windows are so tall very often they are only partially filled. The
> net effect is that interesting content is concentrated at the top of my
> screen. That being the case needing to look to the bottom of the
> screen to inspect the mode-line and/or mini-buffer only slows me down.
> I would be much happier if I could have:
> * the mini-buffer at the top of its frame
> * the mode-line at the top of its window

It might be much easier to make text display at the bottom of a
window.  Have you considered that possibility?

Re: The future of Follow Mode - a proposal.

[Alan Mackenzie]

Hello, Eli.

On Fri, Feb 19, 2016 at 08:45:39PM +0200, Eli Zaretskii wrote:
> > Date: Fri, 19 Feb 2016 18:18:34 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > I already explained this above: "the fact that the current display
> > > engine doesn't support windows of unequal width".  If you want to lift
> > > this limitation, the move_it_* family of functions, which simulate
> > > redisplay, and are the workhorse of every decision Emacs makes that
> > > concerns layout, cannot switch windows in their inner loops.

> > I envisage reinitialising the iterator structure as necessary when
> > passing bewteen windows.  The change in width would be handled at a
> > relatively high level.

Or, maybe not, se below.

> > The window start position is known, the window end position could be
> > calculated as we progress.

> The functions we talk about currently don't know what they are invoked
> for.  Your envisioned changes imply that they should behave
> differently depending on whether the results will be used for layout
> of the current window or the next/previous window in a group.  That's
> part of the changes I had in mind.  They are not trivial.  But without
> them, what you want to do will not work reliably.

How about adding an extra boolean parameter to the move_it_* functions,
perhaps called `physical', which when set would mean the function would
have to adjust its iterator when crossing a window boundary, when not set
would work the same way as it currently does?  `vertical-motion' would
also need this extra &optional parameter, possibly a few other defuns,
too.

There are around 150 calls to move_it_*.  I'm guessing that most of these
would set `physical' to false, perhaps more of the ones in window.c would
use true.

> > As an example, `compute_window_start_on_continuation_line' would have to
> > use the dimensions of the previous window to determine the window-start.
> > Jiggling the various windows around after text changes or scrolling is
> > going to be the hard part of the coding.

> Yes, and the result will be non-trivial changes in the overall logic,
> because redisplaying a window will no longer be independent of other
> windows.

Yes.  This is what is currently implemented in Follow Mode.

> It's all doable, of course, but I suggest taking a good look at the
> use cases for each of these functions, before you design the way they
> should work to support windows of unequal width.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> Date: Sat, 20 Feb 2016 12:44:15 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > The functions we talk about currently don't know what they are invoked
> > for.  Your envisioned changes imply that they should behave
> > differently depending on whether the results will be used for layout
> > of the current window or the next/previous window in a group.  That's
> > part of the changes I had in mind.  They are not trivial.  But without
> > them, what you want to do will not work reliably.
> 
> How about adding an extra boolean parameter to the move_it_* functions,
> perhaps called `physical', which when set would mean the function would
> have to adjust its iterator when crossing a window boundary, when not set
> would work the same way as it currently does?  `vertical-motion' would
> also need this extra &optional parameter, possibly a few other defuns,
> too.

I don't think it's a boolean.  It should be the buffer position where
the window should be switched.  So probably 2 parameters, for the
beginning and end of the window.  Maybe also the window to switch to.

And then you need to implement the handling of these new arguments.

> There are around 150 calls to move_it_*.  I'm guessing that most of these
> would set `physical' to false, perhaps more of the ones in window.c would
> use true.

Maybe, I don't know.  The problem is that these functions are called
in several layers, and all of them will have to know about these new
arguments.  Also, some utility functions, like pos_visible_p, are
called in different situations, so the caller will have to supply
those arguments as required in each case.

> > > As an example, `compute_window_start_on_continuation_line' would have to
> > > use the dimensions of the previous window to determine the window-start.
> > > Jiggling the various windows around after text changes or scrolling is
> > > going to be the hard part of the coding.
> 
> > Yes, and the result will be non-trivial changes in the overall logic,
> > because redisplaying a window will no longer be independent of other
> > windows.
> 
> Yes.  This is what is currently implemented in Follow Mode.

No, I mean that redisplay of all the windows in a group will have to
be done in one go, not one window at a time.

Re: The future of Follow Mode - a proposal.

[Alan Mackenzie]

Hello, Eli.

On Sat, Feb 20, 2016 at 03:05:16PM +0200, Eli Zaretskii wrote:
> > Date: Sat, 20 Feb 2016 12:44:15 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > The functions we talk about currently don't know what they are invoked
> > > for.  Your envisioned changes imply that they should behave
> > > differently depending on whether the results will be used for layout
> > > of the current window or the next/previous window in a group.  That's
> > > part of the changes I had in mind.  They are not trivial.  But without
> > > them, what you want to do will not work reliably.

> > How about adding an extra boolean parameter to the move_it_* functions,
> > perhaps called `physical', which when set would mean the function would
> > have to adjust its iterator when crossing a window boundary, when not set
> > would work the same way as it currently does?  `vertical-motion' would
> > also need this extra &optional parameter, possibly a few other defuns,
> > too.

> I don't think it's a boolean.  It should be the buffer position where
> the window should be switched.  So probably 2 parameters, for the
> beginning and end of the window.  Maybe also the window to switch to.

> And then you need to implement the handling of these new arguments.

An alternative approach, which would be slightly less disruptive, would
be to add in new functions which would work on window groups.  I
anticipate needing group versions of `move_it_by_lines', `move_it_to',
`move_it_vertically', and possibly `move_it_vertically_backward'.  struct
window would be augmented with the following three fields:

    /* The next and previous windows in any window group.  */
    Lisp_Object group_next;
    Lisp_Object group_prev;
    /* The window group this window belongs to, or nil.  */
    Lisp_Object window_group;

. A first draft of `move_it_by_lines_in_group' follows.  It compiles, but
hasn't been tested in the slightest.  If it moves off the boundary of a
window, the `struct it' is reinitialised in the adjoining window.  If it
goes off the top of the first window or the bottom of the last, it
behaves as currently.  Comments on the general approach would be welcome.


/* Move IT by a specified number DVPOS of screen lines down.  DVPOS
   negative means move up.  DVPOS == 0 means move to the start of the
   screen line.

   If IT moves off of the window top or bottom boundary, reinitialize IT
   for the next window in the window group (if any) and carry on there,
   and so on, until we've moved DVPOS screen lines.  */

void
move_it_by_lines_in_group (struct it *it, ptrdiff_t dvpos)
{
  struct window *w = it->w;

  if (dvpos <= 0)
    {
      while (!NILP (w->group_prev)
             && -dvpos > it->vpos - it->first_vpos)
        {
          dvpos += it->vpos - it->first_vpos;
          move_it_by_lines (it, it->first_vpos - it->vpos);
          w = XWINDOW (w->group_prev);
          /* Set the iterator to just off the bottom of the new window. */
          init_iterator (it, w, IT_CHARPOS (*it), IT_BYTEPOS (*it),
                         w->desired_matrix->rows + w->total_lines
                         - WINDOW_WANTS_MODELINE_P (w),
                         it->base_face_id);
          it->first_vpos = WINDOW_WANTS_HEADER_LINE_P (w);
          it->vpos = w->total_lines - WINDOW_WANTS_MODELINE_P (w);
          it->current_y = w->pixel_height
            - WINDOW_WANTS_MODELINE_P (w) * w->mode_line_height;
          it->hpos = 0;
          it->current_x = 0;
        }
      if (dvpos)
        move_it_by_lines (it, dvpos);
    }
  else
    {
      while (!NILP (w->group_next)
             && dvpos >= w->total_lines - WINDOW_WANTS_MODELINE_P (w)
             - it->vpos)
        {
          dvpos -= w->total_lines - WINDOW_WANTS_MODELINE_P (w) - it->vpos;
          move_it_by_lines (it, w->total_lines - WINDOW_WANTS_MODELINE_P (w)
                            - it->vpos);
          w = XWINDOW (w->group_next);
          /* Set the iterator to the top of the new window. */
          init_iterator (it, w, IT_CHARPOS (*it), IT_BYTEPOS (*it),
                         w->desired_matrix->rows
                         + WINDOW_WANTS_HEADER_LINE_P (w),
                         it->base_face_id);
          it->first_vpos = it->vpos = WINDOW_WANTS_HEADER_LINE_P (w);
          it->current_y = WINDOW_WANTS_HEADER_LINE_P (w)
            * w->header_line_height;
          it->hpos = 0;
          it->current_x = 0;
        }
      if (dvpos)
        move_it_by_lines (it, dvpos);
    }
}




> > There are around 150 calls to move_it_*.  I'm guessing that most of these
> > would set `physical' to false, perhaps more of the ones in window.c would
> > use true.

40 move_it_by_lines + 46 move_it_to + 20 move_it_vertically\(_backward\)? = 106.

> Maybe, I don't know.  The problem is that these functions are called
> in several layers, and all of them will have to know about these new
> arguments.  Also, some utility functions, like pos_visible_p, are
> called in different situations, so the caller will have to supply
> those arguments as required in each case.

Yes.  There's goint to be some complication here.

> > > > As an example, `compute_window_start_on_continuation_line' would have to
> > > > use the dimensions of the previous window to determine the window-start.
> > > > Jiggling the various windows around after text changes or scrolling is
> > > > going to be the hard part of the coding.

> > > Yes, and the result will be non-trivial changes in the overall logic,
> > > because redisplaying a window will no longer be independent of other
> > > windows.

> > Yes.  This is what is currently implemented in Follow Mode.

> No, I mean that redisplay of all the windows in a group will have to
> be done in one go, not one window at a time.

Indeed.  But often (say, a single character insertion), only one window
of a group will need redisplay.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The future of Follow Mode - a proposal.

[Stefan Monnier]

> An alternative approach, which would be slightly less disruptive, would
> be to add in new functions which would work on window groups.

As mentioned last time this follow-mode discussion came up, I think
a better approach (maybe not easier, but more generally useful) would be
to add an Elisp hook that can control the order in which
windows are redisplayed.


        Stefan

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 23 Feb 2016 22:57:41 -0500
> 
> As mentioned last time this follow-mode discussion came up, I think
> a better approach (maybe not easier, but more generally useful) would be
> to add an Elisp hook that can control the order in which
> windows are redisplayed.

But that's orthogonal to the issues described by Alan, right?

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> Date: Tue, 23 Feb 2016 23:11:56 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > I don't think it's a boolean.  It should be the buffer position where
> > the window should be switched.  So probably 2 parameters, for the
> > beginning and end of the window.  Maybe also the window to switch to.
> 
> > And then you need to implement the handling of these new arguments.
> 
> An alternative approach, which would be slightly less disruptive, would
> be to add in new functions which would work on window groups.  I
> anticipate needing group versions of `move_it_by_lines', `move_it_to',
> `move_it_vertically', and possibly `move_it_vertically_backward'.

You will still face the same problem: when to call these new functions
and when the old ones.

> struct window would be augmented with the following three fields:
> 
>     /* The next and previous windows in any window group.  */
>     Lisp_Object group_next;
>     Lisp_Object group_prev;
>     /* The window group this window belongs to, or nil.  */
>     Lisp_Object window_group;

(Not sure why these are Lisp objects.)

>   if (dvpos <= 0)
>     {
>       while (!NILP (w->group_prev)
>              && -dvpos > it->vpos - it->first_vpos)
>         {
>           dvpos += it->vpos - it->first_vpos;
>           move_it_by_lines (it, it->first_vpos - it->vpos);

This call to move_it_by_lines is a waste of cycles: you know you are
going to the window start point, so just go there.  The move_it_*
family of functions cannot go back, only forward, so what actually
happens here is that the call will go back one line more than its
argument, and then come back forward by slow iteration.  You don't
want that.

>           w = XWINDOW (w->group_prev);
>           /* Set the iterator to just off the bottom of the new window. */
>           init_iterator (it, w, IT_CHARPOS (*it), IT_BYTEPOS (*it),
>                          w->desired_matrix->rows + w->total_lines
>                          - WINDOW_WANTS_MODELINE_P (w),
>                          it->base_face_id);
>           it->first_vpos = WINDOW_WANTS_HEADER_LINE_P (w);
>           it->vpos = w->total_lines - WINDOW_WANTS_MODELINE_P (w);

You cannot add vpos and w->total_lines: they are measured in different
units.  The former is the (zero-based) number of a screen line, the
latter is in units of frame's canonical character height.  Going to
the end of the window will not in general give vpos the value of
total_lines.

>   else
>     {
>       while (!NILP (w->group_next)
>              && dvpos >= w->total_lines - WINDOW_WANTS_MODELINE_P (w)
>              - it->vpos)
>         {
>           dvpos -= w->total_lines - WINDOW_WANTS_MODELINE_P (w) - it->vpos;
>           move_it_by_lines (it, w->total_lines - WINDOW_WANTS_MODELINE_P (w)
>                             - it->vpos);
>           w = XWINDOW (w->group_next);
>           /* Set the iterator to the top of the new window. */
>           init_iterator (it, w, IT_CHARPOS (*it), IT_BYTEPOS (*it),
>                          w->desired_matrix->rows
>                          + WINDOW_WANTS_HEADER_LINE_P (w),
>                          it->base_face_id);

You want to avoid calling init_iterator here, since this loses the
bidi context.  E.g., the paragraph direction is lost, and must be
recomputed, which might be expensive.  You want to have a more
light-weight function for recomputing the fields of the iterator
object that depend on the window metrics.

> > > There are around 150 calls to move_it_*.  I'm guessing that most of these
> > > would set `physical' to false, perhaps more of the ones in window.c would
> > > use true.
> 
> 40 move_it_by_lines + 46 move_it_to + 20 move_it_vertically\(_backward\)? = 106.

I think you forgot move_it_in_display_line and
move_it_in_display_line_to.

> > > > Yes, and the result will be non-trivial changes in the overall logic,
> > > > because redisplaying a window will no longer be independent of other
> > > > windows.
> 
> > > Yes.  This is what is currently implemented in Follow Mode.
> 
> > No, I mean that redisplay of all the windows in a group will have to
> > be done in one go, not one window at a time.
> 
> Indeed.  But often (say, a single character insertion), only one window
> of a group will need redisplay.

You won't know that until you actually redisplay that one window and
then check some conditions.

Re: The future of Follow Mode - a proposal.

[Stefan Monnier]

>> As mentioned last time this follow-mode discussion came up, I think
>> a better approach (maybe not easier, but more generally useful) would be
>> to add an Elisp hook that can control the order in which
>> windows are redisplayed.

> But that's orthogonal to the issues described by Alan, right?

Hmm... unless I misunderstood his suggestion, no.  Because "my" Elisp hook
would let you do what he wants directly in Elisp without any changes
in C.


        Stefan

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: emacs-devel@gnu.org
> Date: Wed, 24 Feb 2016 13:57:32 -0500
> 
> >> As mentioned last time this follow-mode discussion came up, I think
> >> a better approach (maybe not easier, but more generally useful) would be
> >> to add an Elisp hook that can control the order in which
> >> windows are redisplayed.
> 
> > But that's orthogonal to the issues described by Alan, right?
> 
> Hmm... unless I misunderstood his suggestion, no.  Because "my" Elisp hook
> would let you do what he wants directly in Elisp without any changes
> in C.

I don't see how that could be possible.  The problem is not (only) the
order of displaying windows, the problem is that displaying each
window many times invokes the move_it_* functions that go beyond
window boundaries, and make decisions based on that.  These decisions
end up incorrect when the text beyond the window is displayed in a
window of a different width.

Re: The future of Follow Mode - a proposal.

[Stefan Monnier]

> window boundaries, and make decisions based on that.  These decisions
> end up incorrect when the text beyond the window is displayed in a
> window of a different width.

Hmm... can you give an example?


        Stefan

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: emacs-devel@gnu.org
> Date: Wed, 24 Feb 2016 15:10:28 -0500
> 
> > window boundaries, and make decisions based on that.  These decisions
> > end up incorrect when the text beyond the window is displayed in a
> > window of a different width.
> 
> Hmm... can you give an example?

The most striking one is vertical-motion (which doesn't belong to
redisplay at all, and thus will not benefit from your suggestion).

Redisplay proper does similar stuff at places as part of its decision
where to place point and whether/by how much to scroll the window in
order to bring point back into view.

Re: The future of Follow Mode - a proposal.

[Stefan Monnier]

> The most striking one is vertical-motion (which doesn't belong to
> redisplay at all, and thus will not benefit from your suggestion).
> Redisplay proper does similar stuff at places as part of its decision
> where to place point and whether/by how much to scroll the window in
> order to bring point back into view.

I see, thanks.  Indeed both examples fall outside the scope of
"redisplay" that could be covered by the kind of hook I have in mind.

I live in a world where wrapped lines are sufficiently rare that I don't
really care and rarely think about what can happen in those cases.

But now that I think about it: I'm not sure how vertical-motion could
handle a "multiple-window" case where the windows don't have the same
width (same thing in other similar cases, such as with overlays with
a `window' property that make them only apply to some of the multiple
windows, or when the windows aren't all in the same frame and don't use
the same font).  The desired semantic seems undefined except for the
case where the vertical motion is applied to the "currently displayed
state" (so we know when to use which window data).


        Stefan

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Wed, 24 Feb 2016 19:30:55 -0500
> 
> I live in a world where wrapped lines are sufficiently rare that I don't
> really care and rarely think about what can happen in those cases.
> 
> But now that I think about it: I'm not sure how vertical-motion could
> handle a "multiple-window" case where the windows don't have the same
> width (same thing in other similar cases, such as with overlays with
> a `window' property that make them only apply to some of the multiple
> windows, or when the windows aren't all in the same frame and don't use
> the same font).  The desired semantic seems undefined except for the
> case where the vertical motion is applied to the "currently displayed
> state" (so we know when to use which window data).

Actually, vertical-motion completely breaks in that case.  I don't
even see a way that will allow to solve that situation in principle,
except in some very specific and restricted use cases.  Which is why I
strongly suggest to change Follow Mode so that it forces all of its
windows be of the same width.  With the current pixelwise control of
window dimensions, this is easy.  I think this will magically remove
many of the problems that currently plague Follow Mode, and leave us
with something we can reasonably easy to solve and maintain.

However, Alan disagrees, and insists on supporting windows of unequal
width.

Re: The future of Follow Mode - a proposal.

[Stefan Monnier]

> Actually, vertical-motion completely breaks in that case.  I don't
> even see a way that will allow to solve that situation in principle,
> except in some very specific and restricted use cases.

Exactly.  E.g. the behavior of vertical-motion could be "made to work" in
the case where the motion is within the existing glyph matrices of the
relevant windows, tho even in that case there could be several possible
desirable behaviors depending on the intention behind the use of
vertical-motion.

> Which is why I strongly suggest to change Follow Mode so that it
> forces all of its windows be of the same width.

And in that case my Elisp hook approach should be usable (assuming it's
implementable ;-).

> With the current pixelwise control of window dimensions, this is easy.

Not sure about "easy" but yes.

> I think this will magically remove many of the problems that currently
> plague Follow Mode, and leave us with something we can reasonably easy
> to solve and maintain.

Sounds fine (and doable in Elisp).

> However, Alan disagrees, and insists on supporting windows of unequal
> width.

I think it might makes sense to allow the user to disable the "make all
follow-mode windows same-width" and just live with the corresponding
quirks, but I suspect that the amount of work needed to make follow-mode
work "100% correctly" for the mixed-width case is really large (it will
involve defining new primitives to replace vertical-motion, then
changing all users accordingly).  What's the expected use-case that
would justify such an effort?


        Stefan

Re: The future of Follow Mode - a proposal.

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Thu, 25 Feb 2016 11:46:58 -0500
> 
> > Actually, vertical-motion completely breaks in that case.  I don't
> > even see a way that will allow to solve that situation in principle,
> > except in some very specific and restricted use cases.
> 
> Exactly.  E.g. the behavior of vertical-motion could be "made to work" in
> the case where the motion is within the existing glyph matrices of the
> relevant windows, tho even in that case there could be several possible
> desirable behaviors depending on the intention behind the use of
> vertical-motion.

Basically, it can be made to work when the text around the boundary
uses the same font.

> > Which is why I strongly suggest to change Follow Mode so that it
> > forces all of its windows be of the same width.
> 
> And in that case my Elisp hook approach should be usable (assuming it's
> implementable ;-).

Could very well be, I didn't think seriously about such a situation
because Alan opposes to solve things like that.

> > With the current pixelwise control of window dimensions, this is easy.
> 
> Not sure about "easy" but yes.

I meant making the windows of equal width is easy.

> > However, Alan disagrees, and insists on supporting windows of unequal
> > width.
> 
> I think it might makes sense to allow the user to disable the "make all
> follow-mode windows same-width" and just live with the corresponding
> quirks, but I suspect that the amount of work needed to make follow-mode
> work "100% correctly" for the mixed-width case is really large (it will
> involve defining new primitives to replace vertical-motion, then
> changing all users accordingly).  What's the expected use-case that
> would justify such an effort?

I hope Alan has a good answer to that question.

Re: The future of Follow Mode - a proposal.

[Alan Mackenzie]

Hello, Eli.

On Wed, Feb 24, 2016 at 08:34:37PM +0200, Eli Zaretskii wrote:
> > Date: Tue, 23 Feb 2016 23:11:56 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > I don't think it's a boolean.  It should be the buffer position where
> > > the window should be switched.  So probably 2 parameters, for the
> > > beginning and end of the window.  Maybe also the window to switch to.

> > > And then you need to implement the handling of these new arguments.

> > An alternative approach, which would be slightly less disruptive, would
> > be to add in new functions which would work on window groups.  I
> > anticipate needing group versions of `move_it_by_lines', `move_it_to',
> > `move_it_vertically', and possibly `move_it_vertically_backward'.

> You will still face the same problem: when to call these new functions
> and when the old ones.

OK.  I'm not sure it'll be all that big a problem, but we'll see.

> > struct window would be augmented with the following three fields:

> >     /* The next and previous windows in any window group.  */
> >     Lisp_Object group_next;
> >     Lisp_Object group_prev;
> >     /* The window group this window belongs to, or nil.  */
> >     Lisp_Object window_group;

> (Not sure why these are Lisp objects.)

So as to be able to put one or more of them in a saved window
configuration, without garbage collection messing up.  I don't see these
types ever being user visible, so perhaps one of these PSEUDOVECTOR
thingies would do instead.

> >   if (dvpos <= 0)
> >     {
> >       while (!NILP (w->group_prev)
> >              && -dvpos > it->vpos - it->first_vpos)
> >         {
> >           dvpos += it->vpos - it->first_vpos;
> >           move_it_by_lines (it, it->first_vpos - it->vpos);

> This call to move_it_by_lines is a waste of cycles: you know you are
> going to the window start point, so just go there.  The move_it_*
> family of functions cannot go back, only forward, so what actually
> happens here is that the call will go back one line more than its
> argument, and then come back forward by slow iteration.  You don't
> want that.

Er, oh!  :-)

> >           w = XWINDOW (w->group_prev);
> >           /* Set the iterator to just off the bottom of the new window. */
> >           init_iterator (it, w, IT_CHARPOS (*it), IT_BYTEPOS (*it),
> >                          w->desired_matrix->rows + w->total_lines
> >                          - WINDOW_WANTS_MODELINE_P (w),
> >                          it->base_face_id);
> >           it->first_vpos = WINDOW_WANTS_HEADER_LINE_P (w);
> >           it->vpos = w->total_lines - WINDOW_WANTS_MODELINE_P (w);

> You cannot add vpos and w->total_lines: they are measured in different
> units.  The former is the (zero-based) number of a screen line, the
> latter is in units of frame's canonical character height.  Going to
> the end of the window will not in general give vpos the value of
> total_lines.

There seem to be three ways of meausuring vertical distance: by pixel,
by screen line number, and by lines as displayed.  Or something like
that.  Because lines as displayed can be taller than standard lines
(from which w->total_lines is calculated), counting screen lines isn't a
good way of judging the position on a screen.  I really want pixel
positions for that.  move_it_by_lines uses screen line numbers, and
move_it_vertically uses pixel measurements.  So when I'm wanting to find
the top of a left hand windows, having just scrolled the middle window
(a long way), I'll really be wanting to use
move_it_vertically\(_backwards\)?, I think.

> >   else
> >     {
> >       while (!NILP (w->group_next)
> >              && dvpos >= w->total_lines - WINDOW_WANTS_MODELINE_P (w)
> >              - it->vpos)
> >         {
> >           dvpos -= w->total_lines - WINDOW_WANTS_MODELINE_P (w) - it->vpos;
> >           move_it_by_lines (it, w->total_lines - WINDOW_WANTS_MODELINE_P (w)
> >                             - it->vpos);
> >           w = XWINDOW (w->group_next);
> >           /* Set the iterator to the top of the new window. */
> >           init_iterator (it, w, IT_CHARPOS (*it), IT_BYTEPOS (*it),
> >                          w->desired_matrix->rows
> >                          + WINDOW_WANTS_HEADER_LINE_P (w),
> >                          it->base_face_id);

> You want to avoid calling init_iterator here, since this loses the
> bidi context.  E.g., the paragraph direction is lost, and must be
> recomputed, which might be expensive.

There's a standard way of preserving the bidi stuff, is there not?

> You want to have a more light-weight function for recomputing the
> fields of the iterator object that depend on the window metrics.

That's easy enough once I've identified them.  :-)  But I take the
point.

> > > > There are around 150 calls to move_it_*.  I'm guessing that most of these
> > > > would set `physical' to false, perhaps more of the ones in window.c would
> > > > use true.

> > 40 move_it_by_lines + 46 move_it_to + 20 move_it_vertically\(_backward\)? = 106.

> I think you forgot move_it_in_display_line and
> move_it_in_display_line_to.

These two are restricted to the current line, hence can't move out of
the current window.  So their callers would have set the iterator
correctly for the current window.

> > > > > Yes, and the result will be non-trivial changes in the overall logic,
> > > > > because redisplaying a window will no longer be independent of other
> > > > > windows.

> > > > Yes.  This is what is currently implemented in Follow Mode.

> > > No, I mean that redisplay of all the windows in a group will have to
> > > be done in one go, not one window at a time.

> > Indeed.  But often (say, a single character insertion), only one window
> > of a group will need redisplay.

> You won't know that until you actually redisplay that one window and
> then check some conditions.

I anticipate redisplay_window will cope with this, either triggering
redisplay of the other windows in the group immediately, or somehow
marking them for this.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The future of Follow Mode - a proposal.

[Alan Mackenzie]

Hello, Eli.

On Thu, Feb 25, 2016 at 06:28:18PM +0200, Eli Zaretskii wrote:
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Cc: emacs-devel@gnu.org
> > Date: Wed, 24 Feb 2016 19:30:55 -0500

> > I live in a world where wrapped lines are sufficiently rare that I don't
> > really care and rarely think about what can happen in those cases.

> > But now that I think about it: I'm not sure how vertical-motion could
> > handle a "multiple-window" case where the windows don't have the same
> > width (same thing in other similar cases, such as with overlays with
> > a `window' property that make them only apply to some of the multiple
> > windows, or when the windows aren't all in the same frame and don't use
> > the same font).  The desired semantic seems undefined except for the
> > case where the vertical motion is applied to the "currently displayed
> > state" (so we know when to use which window data).

> Actually, vertical-motion completely breaks in that case.  I don't
> even see a way that will allow to solve that situation in principle,
> except in some very specific and restricted use cases.  Which is why I
> strongly suggest to change Follow Mode so that it forces all of its
> windows be of the same width.  With the current pixelwise control of
> window dimensions, this is easy.

On a tty, a pixel has the thickness of a character.  It could easily
happen that to make all the windows of equal width, it's necessary to
put a "terminator" vertical line on the RH window, or even double width
window separators in.  This would be ugly.

> I think this will magically remove many of the problems that currently
> plague Follow Mode, and leave us with something we can reasonably easy
> to solve and maintain.

> However, Alan disagrees, and insists on supporting windows of unequal
> width.

If it can be done without undue difficulty, then yes.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The future of Follow Mode - a proposal.

[Alan Mackenzie]

Hello, Stefan.

On Wed, Feb 24, 2016 at 01:57:32PM -0500, Stefan Monnier wrote:
> >> As mentioned last time this follow-mode discussion came up, I think
> >> a better approach (maybe not easier, but more generally useful) would be
> >> to add an Elisp hook that can control the order in which
> >> windows are redisplayed.

> > But that's orthogonal to the issues described by Alan, right?

> Hmm... unless I misunderstood his suggestion, no.  Because "my" Elisp hook
> would let you do what he wants directly in Elisp without any changes
> in C.

What would this lisp hook do, and where?

Merely controlling the order of window display is no great feat.
Determining window starts for these Follow Mode windows is the difficult
bit.  Currently Follow Mode fights with the display engine to get what
it needs, often resulting in delays of several tenths of a second in the
subsequent redisplay operations after the first one.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The future of Follow Mode - a proposal.

[Stefan Monnier]

> Merely controlling the order of window display is no great feat.
> Determining window starts for these Follow Mode windows is the difficult
> bit.

If you con control the order, you can make sure the first follow-mode
window is rendered first, so you can set the window-start of the second
window before it gets rendered, etc...
IOW, controlling the order of window display lets you solve the issue of
determining the window-start for subsequent windows.

> Currently Follow Mode fights with the display engine to get what
> it needs, often resulting in delays of several tenths of a second in the
> subsequent redisplay operations after the first one.

That's why I'd hope that if you add a hook so follow-mode can tweak the
redisplay from inside, it should work much better.


        Stefan

Re: The future of Follow Mode - a proposal.

[Alan Mackenzie]

Hello, Stefan.

On Thu, Feb 25, 2016 at 04:10:22PM -0500, Stefan Monnier wrote:
> > Merely controlling the order of window display is no great feat.
> > Determining window starts for these Follow Mode windows is the difficult
> > bit.

> If you con control the order, you can make sure the first follow-mode
> window is rendered first, so you can set the window-start of the second
> window before it gets rendered, etc...
> IOW, controlling the order of window display lets you solve the issue of
> determining the window-start for subsequent windows.

But often it is the second or subsequent window that is to be fixed in
place (say, after a scroll down operation in that window).  The
difficult bit is to determine the starting position of the first window
which will result in the subsequent window being correctly placed when
there is to be no gap, and no overlap between windows.

Follow Mode currently does a lot of backwards and forwards motion (e.g.
with `vertical-motion') to get this right.  The display engine could do
this much more efficiently - Lisp has access only to a small part of the
functionality in C.

> > Currently Follow Mode fights with the display engine to get what
> > it needs, often resulting in delays of several tenths of a second in the
> > subsequent redisplay operations after the first one.

> That's why I'd hope that if you add a hook so follow-mode can tweak the
> redisplay from inside, it should work much better.

Possibly.  What would this hook do?  Calculating the start position of
an earlier window is probably best done in C (for speed).

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: The future of Follow Mode - a proposal.

[Stefan Monnier]

> Possibly.  What would this hook do?  Calculating the start position of
> an earlier window is probably best done in C (for speed).

Yes, as it is now, isn't it?


        Stefan