bug#33532: 26.1; set-window-configuration does not restore display start

bug#33532: 26.1; set-window-configuration does not restore display start

[Markus Triska]

In "emacs -Q", please evaluate the following form:

    (let ((buf (get-buffer-create "buf")))
      (switch-to-buffer buf)
      (erase-buffer)
      (dotimes (x 10)
        (insert (format "x is %s\n" x)))
      (recenter 3)
      (sit-for 1)
      (let ((conf (current-window-configuration))
            (ws (window-start))
            (str (with-current-buffer buf (buffer-string))))
        (with-current-buffer buf
          (erase-buffer)
          (insert str))
        (set-window-configuration conf)
        (sit-for 1)
        (message "window-start: %s vs. %s" ws (window-start))))

This shows that after set-window-configuration, even though the buffer
content is the same as at the time the configuration was obtained, the
value of (window-start) differs from before: After the window
configuration is set, (window-start) is 1. I expect it to be 50 (i.e.,
the value at the time the configuration was obtained), because the
documentation of current-window-configuration states:

    This describes ... for each ... buffer, where display starts ...

If possible, could you please consider restoring window-start to the
exact same position? Thank you very much!

In GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw scroll bars)
 of 2018-11-18 built on debian
Repository revision: 07f8f9bc5a51f5aa94eb099f3e15fbe0c20ea1ea
Windowing system distributor 'The X.Org Foundation', version 11.0.11902000
System Description:	Debian GNU/Linux 9.6 (stretch)

bug#33532: 26.1; set-window-configuration does not restore display start

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Date: Tue, 27 Nov 2018 23:11:21 +0100
> 
>     (let ((buf (get-buffer-create "buf")))
>       (switch-to-buffer buf)
>       (erase-buffer)
>       (dotimes (x 10)
>         (insert (format "x is %s\n" x)))
>       (recenter 3)
>       (sit-for 1)
>       (let ((conf (current-window-configuration))
>             (ws (window-start))
>             (str (with-current-buffer buf (buffer-string))))
>         (with-current-buffer buf
>           (erase-buffer)
>           (insert str))
>         (set-window-configuration conf)
>         (sit-for 1)
>         (message "window-start: %s vs. %s" ws (window-start))))
> 
> This shows that after set-window-configuration, even though the buffer
> content is the same as at the time the configuration was obtained, the
> value of (window-start) differs from before: After the window
> configuration is set, (window-start) is 1. I expect it to be 50 (i.e.,
> the value at the time the configuration was obtained), because the
> documentation of current-window-configuration states:

The window-start point is a marker, and is saved as such.  Your code
replaces the entire text of the buffer with something new, so all the
markers are invalidated.

How important is the use case of replacing buffer text with identical
text?  Because replacing with a different text will make preserving
window-start position based on its numerical value not really a
sensible thing, would it?  It could even be impossible, e.g. if the
same position happens to be in the middle of a sequence of composed
characters.

bug#33532: 26.1; set-window-configuration does not restore display start

[Markus Triska]

Eli Zaretskii <eliz@gnu.org> writes:

> How important is the use case of replacing buffer text with identical
> text?  Because replacing with a different text will make preserving
> window-start position based on its numerical value not really a
> sensible thing, would it?  It could even be impossible, e.g. if the
> same position happens to be in the middle of a sequence of composed
> characters.

Please consider a use case that is described in the Emacs manual:

    In order to store a window configuration on disk and
    read it back in another Emacs session ...

In such use cases, I want to restore the exact same text in all buffers,
and then restore the window configuration, including display starts.

In fact, the snippet illustrates an even simpler situation: It stores
the configuration in a variable, and the session keeps running.  I would
like to obtain the configuration, then close all buffers, and later
restore their contents (for example, from files), and then use
set-window-configuration to restore the original window configuration,
including all window-starts.

Could set-window-configuration be generalized to reliably restore the
window-starts also in such cases? If not, please consider adjusting the
documentation to make clear what this actually guarantees. In
particular, when are window-starts reliably restored, and when not?

Thank you and all the best,
Markus

bug#33532: 26.1; set-window-configuration does not restore display start

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Cc: 33532@debbugs.gnu.org
> Date: Wed, 28 Nov 2018 18:13:57 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > How important is the use case of replacing buffer text with identical
> > text?  Because replacing with a different text will make preserving
> > window-start position based on its numerical value not really a
> > sensible thing, would it?  It could even be impossible, e.g. if the
> > same position happens to be in the middle of a sequence of composed
> > characters.
> 
> Please consider a use case that is described in the Emacs manual:
> 
>     In order to store a window configuration on disk and
>     read it back in another Emacs session ...
> 
> In such use cases, I want to restore the exact same text in all buffers,
> and then restore the window configuration, including display starts.

Please propose how to do that.  If it can be done ion a useful manner,
I don't see why we shouldn't.

> In fact, the snippet illustrates an even simpler situation: It stores
> the configuration in a variable, and the session keeps running.  I would
> like to obtain the configuration, then close all buffers, and later
> restore their contents (for example, from files), and then use
> set-window-configuration to restore the original window configuration,
> including all window-starts.

Well, you get everything _excluding_ the window-start.  I think it's
still a bargain.

> Could set-window-configuration be generalized to reliably restore the
> window-starts also in such cases? If not, please consider adjusting the
> documentation to make clear what this actually guarantees. In
> particular, when are window-starts reliably restored, and when not?

As long as the buffer text exists, you will get the best approximation
to restoring window-start.

bug#33532: 26.1; set-window-configuration does not restore display start

[Markus Triska]

Eli Zaretskii <eliz@gnu.org> writes:

>> In such use cases, I want to restore the exact same text in all buffers,
>> and then restore the window configuration, including display starts.
>
> Please propose how to do that.  If it can be done ion a useful manner,
> I don't see why we shouldn't.

In my opinion, one way to do it is to have current-window-configuration
store the window-start internally as an integer (instead of a marker),
denoting the position of the display start.

Then, set-window-configuration could set the window-start to that
position.

Would that work? At least that is how I currently work around the issue.

> Well, you get everything _excluding_ the window-start.  I think it's
> still a bargain.

It is! However, the documentation suggests that it is even better, when
it's currently not.

All the best,
Markus

bug#33532: 26.1; set-window-configuration does not restore display start

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Cc: 33532@debbugs.gnu.org
> Date: Wed, 28 Nov 2018 18:58:30 +0100
> 
> In my opinion, one way to do it is to have current-window-configuration
> store the window-start internally as an integer (instead of a marker),
> denoting the position of the display start.
> 
> Then, set-window-configuration could set the window-start to that
> position.

How is this useful if the text has changed?  You are just starting
display from some random position, don't you?

> Would that work?

Not in general, no.

> At least that is how I currently work around the issue.

I don't think this could be a general solution.  But I'll let others
chime in and comment on this.

bug#33532: 26.1; set-window-configuration does not restore display start

[martin rudalics]

 > Please consider a use case that is described in the Emacs manual:
 >
 >      In order to store a window configuration on disk and
 >      read it back in another Emacs session ...

But the whole text is

   In order to store a window configuration on
   disk and read it back in another Emacs session, you can use the
   functions described next.

and the functions mentioned there are `window-state-get' and
`window-state-put'.  So you have to tell us what you want to fix: The
behavior of window configurations throughout one and the same Emacs
session (which uses markers to restore positions) or when restoring a
previous session (which relies on positions only).

 > In such use cases, I want to restore the exact same text in all buffers,

What is "to restore the exact same text in all buffers"?  The text at
the time the configuration was stored in a register?  That one might
not be available any more.  Restoring a window configuration (or
state) can only use the buffer text as is and adjust positions
accordingly.

 > and then restore the window configuration, including display starts.
 >
 > In fact, the snippet illustrates an even simpler situation: It stores
 > the configuration in a variable, and the session keeps running.  I would
 > like to obtain the configuration, then close all buffers,

I suppose you mean "kill all buffers" here.  This will implicitly make
the window start and point positions of any window that showed the
buffer previously and is stored in a configuration to point nowhere.
Furthermore, there is no guarantee that the same buffer will be shown
in that window when the configuration is restored.

 > and later
 > restore their contents (for example, from files), and then use
 > set-window-configuration to restore the original window configuration,
 > including all window-starts.

If you use window states instead of configurations then this should
work.  States store positions instead of markers.

 > Could set-window-configuration be generalized to reliably restore the
 > window-starts also in such cases? If not, please consider adjusting the
 > documentation to make clear what this actually guarantees. In
 > particular, when are window-starts reliably restored, and when not?

Emacs tries its best to do that.  But killing and reverting buffers,
among other operations, can defy such attempts.  So you either should
try 'window-state-get' and 'window-state-put' to do what you want or
save any positions separately before killing buffers and restore them
together with their buffers after restoring a configuration.

martin

bug#33532: 26.1; set-window-configuration does not restore display start

[Markus Triska]

martin rudalics <rudalics@gmx.at> writes:

> and the functions mentioned there are `window-state-get' and
> `window-state-put'.

These functions have an analogous issue, which you can see with:

    (let ((buf (get-buffer-create "buf")))
      (switch-to-buffer buf)
      (erase-buffer)
      (dotimes (x 10)
        (insert (format "x is %s\n" x)))
      (recenter 3)
      (sit-for 1)
      (let ((conf (window-state-get))
            (ws (window-start))
            (str (with-current-buffer buf (buffer-string))))
        (with-current-buffer buf
          (erase-buffer)
          (insert str))
        (window-state-put conf)
        (sit-for 1)
        (message "window-start: %s vs. %s" ws (window-start))))

> So you have to tell us what you want to fix: The behavior of window
> configurations throughout one and the same Emacs session (which uses
> markers to restore positions) or when restoring a previous session
> (which relies on positions only).

If possible, please fix both. I filed the simpler case first, but both
functions are broken.  For now, I would like to concentrate on the case
that only involves variables in the same Emacs session, without saving
files to disk. I can file a separate issue for the other function.

> What is "to restore the exact same text in all buffers"?  The text at
> the time the configuration was stored in a register?

It means the buffer text that can be obtained for example with
(buffer-string), and can be restored exactly as it was at any time. The
snippet I posted illustrates this case.

> Restoring a window configuration (or state) can only use the buffer
> text as is and adjust positions accordingly.

This is what I am asking for, and what currently does not work. Please
see the snippet I posted for a reproducible test case: The buffer text
is what it was, but the start position is not restored.

> Furthermore, there is no guarantee that the same buffer will be shown
> in that window when the configuration is restored.

The fact that other guarantees are also broken is definitely
unfortunate. However, improving just the particular aspect I mentioned
would already be extremely helpful for my use case.

> If you use window states instead of configurations then this should
> work.  States store positions instead of markers.

Please see above: The snippet shows that it doesn't work. Also, I am
primarily interested in the window configuration, and the documentation
suggests that the window-start is saved (and restored).

If possible, please consider implementing this, or adjusting the
documentation so that it states what is actually guaranteed.

Thank you and all the best,
Markus

bug#33532: 26.1; set-window-configuration does not restore display start

[martin rudalics]

 > These functions have an analogous issue, which you can see with:
 >
 >      (let ((buf (get-buffer-create "buf")))
 >        (switch-to-buffer buf)
 >        (erase-buffer)
 >        (dotimes (x 10)
 >          (insert (format "x is %s\n" x)))
 >        (recenter 3)
 >        (sit-for 1)
 >        (let ((conf (window-state-get))
 >              (ws (window-start))
 >              (str (with-current-buffer buf (buffer-string))))
 >          (with-current-buffer buf
 >            (erase-buffer)
 >            (insert str))
 >          (window-state-put conf)
 >          (sit-for 1)
 >          (message "window-start: %s vs. %s" ws (window-start))))

You have to call 'window-state-get' with the WRITABLE argument
non-nil, like (window-state-get nil t).  Otherwise, it gets you
markers just like 'current-window-configuration'.

Note that I did not disregard the remainder of your mail but please
let's continue the discussion after you tried my suggestion.

Thanks, martin

bug#33532: 26.1; set-window-configuration does not restore display start

[Markus Triska]

martin rudalics <rudalics@gmx.at> writes:

> You have to call 'window-state-get' with the WRITABLE argument
> non-nil, like (window-state-get nil t).  Otherwise, it gets you
> markers just like 'current-window-configuration'.

Perfect! This is what I need also for current-window-configuration!
Also, could this be made the default? Or is there some advantage in
using markers instead of the actual positions that can be restored?

All the best,
Markus

bug#33532: 26.1; set-window-configuration does not restore display start

[martin rudalics]

 > Perfect! This is what I need also for current-window-configuration!
 > Also, could this be made the default? Or is there some advantage in
 > using markers instead of the actual positions that can be restored?

First of all we need a "reasonable" use case.  The only purpose of a
form like

(let ((str (with-current-buffer buf (buffer-string))))
   (with-current-buffer buf
     (erase-buffer)
     (insert str)))

is to annihilate all marker positions in buf and I have yet to
understand why that could be useful.  I'm sure you have something more
reasonable in mind.  Once we have such a use case we can decide how to
best support it.

AFAICT Emacs' window configurations use window start and point markers
ever since.  And while they interfere with reverting buffers (see
Bug#33458 for a discussion) turning them into pure positions would
break far too many things as to make that the default.

We could store - in configurations and states - both, markers and
their corresponding pure positions.  Then if, when restoring a
configuration or state, a marker's position has dropped to 1 while the
corresponding saved pure position is > 1, use the the pure position
instead.  This should automatically fix your case and not violate any
others.  Juri what do you think?

martin

bug#33532: 26.1; set-window-configuration does not restore display start

[Eli Zaretskii]

> From: Markus Triska <triska@metalevel.at>
> Cc: Eli Zaretskii <eliz@gnu.org>,  33532@debbugs.gnu.org
> Date: Fri, 30 Nov 2018 17:58:39 +0100
> 
> Or is there some advantage in using markers instead of the actual
> positions that can be restored?

Markers are much more useful default, because they preserve position
in text even if text is modified.  So in the usual cases markers are a
better default.

bug#33532: 26.1; set-window-configuration does not restore display start

[Juri Linkov]

>> Perfect! This is what I need also for current-window-configuration!
>> Also, could this be made the default? Or is there some advantage in
>> using markers instead of the actual positions that can be restored?
>
> First of all we need a "reasonable" use case.  The only purpose of a
> form like
>
> (let ((str (with-current-buffer buf (buffer-string))))
>   (with-current-buffer buf
>     (erase-buffer)
>     (insert str)))
>
> is to annihilate all marker positions in buf and I have yet to
> understand why that could be useful.  I'm sure you have something more
> reasonable in mind.  Once we have such a use case we can decide how to
> best support it.
>
> AFAICT Emacs' window configurations use window start and point markers
> ever since.  And while they interfere with reverting buffers (see
> Bug#33458 for a discussion) turning them into pure positions would
> break far too many things as to make that the default.
>
> We could store - in configurations and states - both, markers and
> their corresponding pure positions.  Then if, when restoring a
> configuration or state, a marker's position has dropped to 1 while the
> corresponding saved pure position is > 1, use the the pure position
> instead.  This should automatically fix your case and not violate any
> others.  Juri what do you think?

I see no drawbacks, let's try storing both (maybe together with
metadata like dired-filename).

bug#33532: 26.1; set-window-configuration does not restore display start

[martin rudalics]

 >> We could store - in configurations and states - both, markers and
 >> their corresponding pure positions.  Then if, when restoring a
 >> configuration or state, a marker's position has dropped to 1 while the
 >> corresponding saved pure position is > 1, use the the pure position
 >> instead.  This should automatically fix your case and not violate any
 >> others.  Juri what do you think?
 >
 > I see no drawbacks, let's try storing both (maybe together with
 > metadata like dired-filename).

It will fail when a window is stowed away in a list or configuration
and the entire stretch of the buffer from 'point-min' up to some
position at or after the stored point position is deleted.  In that
case, the point of any window showing the buffer should rightfully
drop to 'point-min' but in our case it would go to the old position
when the buffer is re-shown in the window.  While this might be a rare
use case, it would be confusing nevertheless.

martin

bug#33532: 26.1; set-window-configuration does not restore display start

[Juri Linkov]

>>> We could store - in configurations and states - both, markers and
>>> their corresponding pure positions.  Then if, when restoring a
>>> configuration or state, a marker's position has dropped to 1 while the
>>> corresponding saved pure position is > 1, use the the pure position
>>> instead.  This should automatically fix your case and not violate any
>>> others.  Juri what do you think?
>>
>> I see no drawbacks, let's try storing both (maybe together with
>> metadata like dired-filename).
>
> It will fail when a window is stowed away in a list or configuration
> and the entire stretch of the buffer from 'point-min' up to some
> position at or after the stored point position is deleted.  In that
> case, the point of any window showing the buffer should rightfully
> drop to 'point-min' but in our case it would go to the old position
> when the buffer is re-shown in the window.  While this might be a rare
> use case, it would be confusing nevertheless.

I still think the problem is in markers dropping to 'point-min'.
Shouldn't a marker remember its previous position as a number
and try to restore it when this position becomes available again?

bug#33532: 26.1; set-window-configuration does not restore display start

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 613 bytes --]

 > I still think the problem is in markers dropping to 'point-min'.

Due to the buffer erasing part when reverting the buffer.

 > Shouldn't a marker remember its previous position as a number
 > and try to restore it when this position becomes available again?

I think that Stefan's approach to restore windows' point markers does
that in a more sophisticated way.  I'll attach a patch.

But that patch does not address window configurations and states -
simply because it can't access them.  We need a different solution for
those.  And it obviously does not handle non-default reverters like
'dired'.

martin

[-- Attachment #2: fileio.diff --]
[-- Type: text/plain, Size: 4375 bytes --]

diff --git a/src/fileio.c b/src/fileio.c
index d979571..c5c295e 100644
--- a/src/fileio.c
+++ b/src/fileio.c
@@ -3513,23 +3513,78 @@ CONTEXT should be a list (USER ROLE TYPE RANGE), where the list
 static Lisp_Object
 get_window_points_and_markers (void)
 {
+  Lisp_Object buffer = Fcurrent_buffer ();
   Lisp_Object pt_marker = Fpoint_marker ();
-  Lisp_Object windows
-    = call3 (Qget_buffer_window_list, Fcurrent_buffer (), Qnil, Qt);
-  Lisp_Object window_markers = windows;
-  /* Window markers (and point) are handled specially: rather than move to
-     just before or just after the modified text, we try to keep the
-     markers at the same distance (bug#19161).
-     In general, this is wrong, but for window-markers, this should be harmless
-     and is convenient for the end user when most of the file is unmodified,
-     except for a few minor details near the beginning and near the end.  */
+  Lisp_Object windows = window_list ();
+  Lisp_Object window;
+  Lisp_Object window_markers = Qnil;
+  /* Window markers (and point) are handled specially: rather than
+     move to just before or just after the modified text, we try to
+     keep the markers at the same distance (bug#19161).
+
+     In general, this is wrong, but for window markers, this should be
+     harmless and is convenient for the end user when most of the file
+     is unmodified, except for a few minor details near the beginning
+     and near the end.
+
+     Window point markers now include the window point markers from
+     the lists of each live window's previous and next buffers.  */
   for (; CONSP (windows); windows = XCDR (windows))
-    if (WINDOWP (XCAR (windows)))
+    if (WINDOW_LIVE_P (window = XCAR (windows)))
       {
-	Lisp_Object window_marker = XWINDOW (XCAR (windows))->pointm;
-	XSETCAR (windows,
-		 Fcons (window_marker, Fmarker_position (window_marker)));
+	struct window *w = XWINDOW (window);
+	Lisp_Object prev_buffers = w->prev_buffers;
+	Lisp_Object next_buffers = w->next_buffers;
+
+	/* Look at window's buffer first.  */
+	if (EQ (WINDOW_BUFFER (w), buffer))
+	  {
+	    Lisp_Object window_marker = XWINDOW (XCAR (windows))->pointm;
+
+	    window_markers =
+	      Fcons (Fcons (window_marker, Fmarker_position (window_marker)),
+		     window_markers);
+
+	    /* Skip the lists of previous and next buffers.  */
+	    continue;
+	  }
+
+	/* Scan window's previous buffers.  */
+	for (; CONSP (prev_buffers); prev_buffers = XCDR (prev_buffers))
+	  if (CONSP (XCAR (prev_buffers)))
+	    {
+	      Lisp_Object triple = XCAR (prev_buffers);
+
+	      if (EQ (XCAR (triple), buffer))
+		{
+		  Lisp_Object prev_marker = Fnth (make_fixnum (2), triple);
+
+		  window_markers =
+		    Fcons (Fcons (prev_marker, Fmarker_position (prev_marker)),
+			   window_markers);
+
+		  /* Skip the list of window's next buffers.  */
+		  continue;
+		}
+	    }
+
+	/* Scan window's next buffers.  */
+	for (; CONSP (next_buffers); next_buffers = XCDR (next_buffers))
+	  if (CONSP (XCAR (next_buffers)))
+	    {
+	      Lisp_Object triple = XCAR (next_buffers);
+
+	      if (EQ (XCAR (triple), buffer))
+		{
+		  Lisp_Object next_marker = Fnth (make_fixnum (2), triple);
+
+		  window_markers =
+		    Fcons (Fcons (next_marker, Fmarker_position (next_marker)),
+			   window_markers);
+		}
+	    }
       }
+
   return Fcons (Fcons (pt_marker, Fpoint ()), window_markers);
 }
 
@@ -3543,6 +3598,7 @@ CONTEXT should be a list (USER ROLE TYPE RANGE), where the list
 	Lisp_Object car = XCAR (window_markers);
 	Lisp_Object marker = XCAR (car);
 	Lisp_Object oldpos = XCDR (car);
+
 	if (MARKERP (marker) && FIXNUMP (oldpos)
 	    && XFIXNUM (oldpos) > same_at_start
 	    && XFIXNUM (oldpos) < same_at_end)
@@ -3552,6 +3608,7 @@ CONTEXT should be a list (USER ROLE TYPE RANGE), where the list
 	    double growth = newsize / (double)oldsize;
 	    ptrdiff_t newpos
 	      = same_at_start + growth * (XFIXNUM (oldpos) - same_at_start);
+
 	    Fset_marker (marker, make_fixnum (newpos), Qnil);
 	  }
       }
@@ -6285,7 +6342,6 @@ current when building the annotations (i.e., at least once), with that
   DEFSYM (Qdelete_directory, "delete-directory");
 
   DEFSYM (Qsubstitute_env_in_file_name, "substitute-env-in-file-name");
-  DEFSYM (Qget_buffer_window_list, "get-buffer-window-list");
 
   DEFSYM (Qstdin, "stdin");
   DEFSYM (Qstdout, "stdout");

bug#33532: 26.1; set-window-configuration does not restore display start

[Juri Linkov]

> But that patch does not address window configurations and states -
> simply because it can't access them.  We need a different solution for
> those.  And it obviously does not handle non-default reverters like
> 'dired'.

What do you think about implementing the same behavior for
markers like it's implemented by 'comint-move-point-for-output'?
I.e. the same way as 'comint-move-point-for-output' moves point
to the end of the output, after erasing the buffer markers could
move their positions towards their previous valid position until
there is enough reverted text that they reach the old position.

This is straightforward to implement.  I see print.c has a question
in the comments:

    case PVEC_MARKER:
      print_c_string ("#<marker ", printcharfun);
      /* Do you think this is necessary?  */
      if (XMARKER (obj)->insertion_type != 0)
	print_c_string ("(moves after insertion) ", printcharfun);

I think this is necessary.  And this 'insertion_type' could also
move after insertion until it reaches its old position in the
reverted buffer.

bug#33532: 26.1; set-window-configuration does not restore display start

[Eli Zaretskii]

> From: Juri Linkov <juri@jurta.org>
> Cc: Markus Triska <triska@metalevel.at>,  Eli Zaretskii <eliz@gnu.org>,  33532@debbugs.gnu.org
> Date: Tue, 04 Dec 2018 00:59:29 +0200
> 
> What do you think about implementing the same behavior for
> markers like it's implemented by 'comint-move-point-for-output'?
> I.e. the same way as 'comint-move-point-for-output' moves point
> to the end of the output, after erasing the buffer markers could
> move their positions towards their previous valid position until
> there is enough reverted text that they reach the old position.
> 
> This is straightforward to implement.

Is it?  Then perhaps I don't understand what "this" is, because
support for keeping the markers' positions is implemented on the C
level, and every insertion/deletion primitive runs the low-level code
which implements that.  I wouldn't call changes in that area
"straightforward" at all.

Please also keep in mind that markers are the basis for other
important features, like overlays and conversion of character to byte
positions and vice versa.  We need to consider the implications on all
those features.

bug#33532: 26.1; set-window-configuration does not restore display start

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 1785 bytes --]

 > What do you think about implementing the same behavior for
 > markers like it's implemented by 'comint-move-point-for-output'?
 > I.e. the same way as 'comint-move-point-for-output' moves point
 > to the end of the output, after erasing the buffer markers could
 > move their positions towards their previous valid position until
 > there is enough reverted text that they reach the old position.
 >
 > This is straightforward to implement.  I see print.c has a question
 > in the comments:
 >
 >      case PVEC_MARKER:
 >        print_c_string ("#<marker ", printcharfun);
 >        /* Do you think this is necessary?  */
 >        if (XMARKER (obj)->insertion_type != 0)
 > 	print_c_string ("(moves after insertion) ", printcharfun);
 >
 > I think this is necessary.  And this 'insertion_type' could also
 > move after insertion until it reaches its old position in the
 > reverted buffer.

'auto-revert-tail-mode' already has

       (when buffer-file-name
         (setq eob (eobp))
         (walk-windows
          (lambda (window)
            (and (eq (window-buffer window) buffer)
                 (= (window-point window) (point-max))
                 (push window eoblist)))
          'no-mini t))

plus

       (when buffer-file-name
         (when eob (goto-char (point-max)))
         (dolist (window eoblist)
           (set-window-point window (point-max)))))

without changing the markers' point insertion types.  We can easily
extend that to handle a window's previous buffers' points.  But
changing the insertion type of markers is far too delicate with all
the implications Eli cited.  We should avoid that wherever possible.

BTW, the last patch I sent was needlessly complicated - a window's
next buffers don't have any markers.  I attach a better one.

martin

[-- Attachment #2: fileio.diff --]
[-- Type: text/plain, Size: 3814 bytes --]

diff --git a/src/fileio.c b/src/fileio.c
index d979571..b3fdc60 100644
--- a/src/fileio.c
+++ b/src/fileio.c
@@ -3513,23 +3513,59 @@ CONTEXT should be a list (USER ROLE TYPE RANGE), where the list
 static Lisp_Object
 get_window_points_and_markers (void)
 {
+  Lisp_Object buffer = Fcurrent_buffer ();
   Lisp_Object pt_marker = Fpoint_marker ();
-  Lisp_Object windows
-    = call3 (Qget_buffer_window_list, Fcurrent_buffer (), Qnil, Qt);
-  Lisp_Object window_markers = windows;
-  /* Window markers (and point) are handled specially: rather than move to
-     just before or just after the modified text, we try to keep the
-     markers at the same distance (bug#19161).
-     In general, this is wrong, but for window-markers, this should be harmless
-     and is convenient for the end user when most of the file is unmodified,
-     except for a few minor details near the beginning and near the end.  */
+  Lisp_Object windows = window_list ();
+  Lisp_Object window;
+  Lisp_Object window_markers = Qnil;
+  /* Window markers (and point) are handled specially: rather than
+     move to just before or just after the modified text, we try to
+     keep the markers at the same distance (bug#19161).
+
+     In general, this is wrong, but for window markers, this should be
+     harmless and is convenient for the end user when most of the file
+     is unmodified, except for a few minor details near the beginning
+     and near the end.
+
+     Window point markers now include the window point markers from
+     the lists of each live window's previous buffers.  */
   for (; CONSP (windows); windows = XCDR (windows))
-    if (WINDOWP (XCAR (windows)))
+    if (WINDOW_LIVE_P (window = XCAR (windows)))
       {
-	Lisp_Object window_marker = XWINDOW (XCAR (windows))->pointm;
-	XSETCAR (windows,
-		 Fcons (window_marker, Fmarker_position (window_marker)));
+	struct window *w = XWINDOW (window);
+	Lisp_Object prev_buffers = w->prev_buffers;
+
+	/* Look at window's buffer first.  */
+	if (EQ (WINDOW_BUFFER (w), buffer))
+	  {
+	    Lisp_Object window_marker = XWINDOW (XCAR (windows))->pointm;
+
+	    window_markers =
+	      Fcons (Fcons (window_marker, Fmarker_position (window_marker)),
+		     window_markers);
+
+	    /* Skip the list of previous buffers.  */
+	    continue;
+	  }
+
+	/* Scan window's previous buffers.  */
+	for (; CONSP (prev_buffers); prev_buffers = XCDR (prev_buffers))
+	  if (CONSP (XCAR (prev_buffers)))
+	    {
+	      Lisp_Object triple = XCAR (prev_buffers);
+
+	      if (EQ (XCAR (triple), buffer))
+		{
+		  Lisp_Object prev_marker = Fnth (make_fixnum (2), triple);
+
+		  window_markers =
+		    Fcons (Fcons (prev_marker, Fmarker_position (prev_marker)),
+			   window_markers);
+		  break;
+		}
+	    }
       }
+
   return Fcons (Fcons (pt_marker, Fpoint ()), window_markers);
 }
 
@@ -3543,6 +3579,7 @@ CONTEXT should be a list (USER ROLE TYPE RANGE), where the list
 	Lisp_Object car = XCAR (window_markers);
 	Lisp_Object marker = XCAR (car);
 	Lisp_Object oldpos = XCDR (car);
+
 	if (MARKERP (marker) && FIXNUMP (oldpos)
 	    && XFIXNUM (oldpos) > same_at_start
 	    && XFIXNUM (oldpos) < same_at_end)
@@ -3552,6 +3589,7 @@ CONTEXT should be a list (USER ROLE TYPE RANGE), where the list
 	    double growth = newsize / (double)oldsize;
 	    ptrdiff_t newpos
 	      = same_at_start + growth * (XFIXNUM (oldpos) - same_at_start);
+
 	    Fset_marker (marker, make_fixnum (newpos), Qnil);
 	  }
       }
@@ -6285,7 +6323,6 @@ current when building the annotations (i.e., at least once), with that
   DEFSYM (Qdelete_directory, "delete-directory");
 
   DEFSYM (Qsubstitute_env_in_file_name, "substitute-env-in-file-name");
-  DEFSYM (Qget_buffer_window_list, "get-buffer-window-list");
 
   DEFSYM (Qstdin, "stdin");
   DEFSYM (Qstdout, "stdout");

bug#33532: 26.1; set-window-configuration does not restore display start

[Juri Linkov]

>> What do you think about implementing the same behavior for
>> markers like it's implemented by 'comint-move-point-for-output'?
>> I.e. the same way as 'comint-move-point-for-output' moves point
>> to the end of the output, after erasing the buffer markers could
>> move their positions towards their previous valid position until
>> there is enough reverted text that they reach the old position.
>> 
>> This is straightforward to implement.
>
> Is it?  Then perhaps I don't understand what "this" is, because
> support for keeping the markers' positions is implemented on the C
> level, and every insertion/deletion primitive runs the low-level code
> which implements that.  I wouldn't call changes in that area
> "straightforward" at all.

Straightforward because I thought it could reuse the existing
implementation of `insertion_type' that moves the marker
after text insertion.  Maybe I'm mistaken.

bug#33532: 26.1; set-window-configuration does not restore display start

[Juri Linkov]

> 'auto-revert-tail-mode' already has
>
>       (when buffer-file-name
>         (setq eob (eobp))
>         (walk-windows
>          (lambda (window)
>            (and (eq (window-buffer window) buffer)
>                 (= (window-point window) (point-max))
>                 (push window eoblist)))
>          'no-mini t))
>
> plus
>
>       (when buffer-file-name
>         (when eob (goto-char (point-max)))
>         (dolist (window eoblist)
>           (set-window-point window (point-max)))))
>
> without changing the markers' point insertion types.  We can easily
> extend that to handle a window's previous buffers' points.

Again, no support for window configurations and non-writable window
states.  But maybe it's possible to access all buffer's windows,
even windows in window configurations/states, and changes their
window points, including previous buffers' points in window
configurations/states.

BTW, I understand why reverting a dired buffer moves its point-marker to
the beginning of the buffer in the selected window window restored from
a window configuration.  But I don't understand why point-marker is
moved to the end of the dired buffer in non-selected window restored
from a window configuration.

bug#33532: 26.1; set-window-configuration does not restore display start

[martin rudalics]

 > Again, no support for window configurations and non-writable window
 > states.  But maybe it's possible to access all buffer's windows,
 > even windows in window configurations/states, and changes their
 > window points, including previous buffers' points in window
 > configurations/states.

How would we handle window configurations?  We could add a new
variable say 'window-configuration-list' and make sure that
configurations accessible from that variable are scanned by revert
buffer functions.  Applications and users would be responsible for
removing a configuration from that list when it's no more needed.

But we also need accessor/setter functions for Lisp slots like the
windows stored in the configuration and for any such window its list
of previous and next buffers (where "any such window" could be a
deleted one).  Doable but certainly non-trivial.

Window states would have to be handled in a similar way.  But there
the task would be simpler: Add a variable, say 'window-states-list'
and if a state is on that list, reverting a buffer could scan it right
away.  Doable but still requires some work.

 > BTW, I understand why reverting a dired buffer moves its point-marker to
 > the beginning of the buffer in the selected window window restored from
 > a window configuration.  But I don't understand why point-marker is
 > moved to the end of the dired buffer in non-selected window restored
 > from a window configuration.

I have no idea either.

martin

bug#33532: 26.1; set-window-configuration does not restore display start

[Juri Linkov]

>> Again, no support for window configurations and non-writable window
>> states.  But maybe it's possible to access all buffer's windows,
>> even windows in window configurations/states, and changes their
>> window points, including previous buffers' points in window
>> configurations/states.
>
> How would we handle window configurations?  We could add a new
> variable say 'window-configuration-list' and make sure that
> configurations accessible from that variable are scanned by revert
> buffer functions.  Applications and users would be responsible for
> removing a configuration from that list when it's no more needed.
>
> But we also need accessor/setter functions for Lisp slots like the
> windows stored in the configuration and for any such window its list
> of previous and next buffers (where "any such window" could be a
> deleted one).  Doable but certainly non-trivial.
>
> Window states would have to be handled in a similar way.  But there
> the task would be simpler: Add a variable, say 'window-states-list'
> and if a state is on that list, reverting a buffer could scan it right
> away.  Doable but still requires some work.

Then maybe instead of immediately modifying values in windows (like
e.g. comint-postoutput-scroll-to-bottom uses get-buffer-window-list
and comint-adjust-window-point) and instead of trying to do the same
in window configurations/states, better would be to modify these values
when windows are restored from window configurations/states/prev-next-buffers?

bug#33532: 26.1; set-window-configuration does not restore display start

[martin rudalics]

 > Then maybe instead of immediately modifying values in windows (like
 > e.g. comint-postoutput-scroll-to-bottom uses get-buffer-window-list
 > and comint-adjust-window-point) and instead of trying to do the same
 > in window configurations/states, better would be to modify these values
 > when windows are restored from window configurations/states/prev-next-buffers?

That was my idea when I proposed to go to the buffer's point when
restoring a configuration/state and the buffer had been reverted since
the configuration was saved.  Note that a configuration already has to
display some live buffer when the one it stored got killed since it was
saved.  Still, it might not be what users want, especially if they use
configurations to show the same buffer in multiple windows.

Note in this context that the collector does not scan window
configurations specially (or at least I cannot see where it does that
or whether any pointers in a configuration are weak).  So if you save
many configurations (or states) and kill buffers often, the dead
buffer objects and the markers these configurations save - including
those from the windows' previous and next buffers - will steadily pile
up.  If someone knows better, please correct me.

martin

bug#33532: 26.1; set-window-configuration does not restore display start

[Juri Linkov]

>> Then maybe instead of immediately modifying values in windows (like
>> e.g. comint-postoutput-scroll-to-bottom uses get-buffer-window-list
>> and comint-adjust-window-point) and instead of trying to do the same
>> in window configurations/states, better would be to modify these values
>> when windows are restored from window configurations/states/prev-next-buffers?
>
> That was my idea when I proposed to go to the buffer's point when
> restoring a configuration/state and the buffer had been reverted since
> the configuration was saved.  Note that a configuration already has to
> display some live buffer when the one it stored got killed since it was
> saved.  Still, it might not be what users want, especially if they use
> configurations to show the same buffer in multiple windows.

Also writable states need better point restoring logic, so maybe
window configurations and writable/non-writable states should support
a hook-like functions that will return the window point when
the buffer is re-displayed in the window, so e.g. dired could
set window point to the dired-filename that was displayed in
that window previously.

bug#33532: 26.1; set-window-configuration does not restore display start

[Juri Linkov]

>> BTW, I understand why reverting a dired buffer moves its point-marker to
>> the beginning of the buffer in the selected window restored from
>> a window configuration.  But I don't understand why point-marker is
>> moved to the end of the dired buffer in non-selected window restored
>> from a window configuration.
>
> I have no idea either.

Please see a complete test case in bug#33871.