bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Keith David Bershatsky]

At the present time, I do not believe that Emacs has the ability to calculate a new `window-start` and a new `window-end` without a redisplay occurring, because it needs to take into account things like the new cursor position, `visual-line-mode`, `linenum-mode`, etc.

Functions assigned to the `post-command-hook` use the `window-start` and `window-end` positions that were previously visible before a new redisplay occurs.

Example:

* Long buffer of text that exceeds the `window-height`.

* Jump interactively from `beginning-of-buffer` to `ending-of-buffer`, or visa-versa.

* In the `post-command-hook`, have the following two elements:

  ** Remove overlays xyz between the *old* `window-start` and *old* `window-end`.

  ** Add overlays xyz between the *new* `window-start` and *new* `window-end`.

The final component above (i.e., ** Add overlays xyz between the *new* . . .) cannot be done correctly without a forced `redisplay` occurring.  The result of a forced redisplay is that the user will see a momentary visual representation of the buffer *without* overlays xyz.  The forced redisplay permits Emacs to correctly calculate the *new* `window-start` and *new* `window-end`.

In a stackoverflow thread regarding this issue, @Stefan suggested that I take a look at the `jit-lock-register`.  After combing through the code in `jit-lock.el` and trying some experiments, I believe that `jit-lock.el` is more suited to a situation when buffer *modification* occurs -- it is also suited for situations where one may wish to incrementially act upon an entire buffer in small chunks at a time to increase efficiency of avaialble resources.

http://stackoverflow.com/questions/23923371/emacs-calculating-new-window-start-end-without-redisplay

I believe that the solution may be to have a `sneak-preview-redisplay` whereby the *new* `window-start` and *new* `window-end` could be correctly calculated without the visual buffer actually being redisplayed.

Thanks,

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

In GNU Emacs 24.4.50.1 (x86_64-apple-darwin10.8.0, NS appkit-1038.36 Version 10.6.8 (Build 10K549))
 of 2014-06-01 on MP.local
Repository revision: 117215 lekktu@gmail.com-20140601162519-4dacx2w0ak528z2r
Windowing system distributor `Apple', version 10.3.1038
Configured using:
 `configure --with-ns'

Configured features:
ACL LIBXML2 ZLIB

Important settings:
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  lawlist-scroll-bar-mode: t
  lawlist-delete-selection-mode: t
  lawlist-linum-mode: t
  lawlist-visual-line-mode: t
  vr-mode: t
  hr-mode: t
  flyspell-mode: t
  desktop-save-mode: t
  tabbar-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  transient-mark-mode: t

Recent input:
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <M-right> <M-right> <M-right> <M-right> <M-right> 
<M-right> <M-right> <M-right> <M-right> <M-right> <M-right> 
<M-right> <M-right> <M-right> <M-right> <M-right> <M-right> 
<M-right> <M-right> <M-right> <M-right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <M-right> <M-right> 
<M-right> <M-right> <M-right> <M-right> <M-right> <M-right> 
<M-right> <s-up> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <up> <M-right> <M-right> <M-right> 
<M-right> <M-right> <M-right> <M-right> <M-right> <M-right> 
<M-right> <M-right> <M-right> <M-right> <M-right> <M-right> 
<M-right> <M-right> <M-right> <M-right> <M-right> <M-right> 
<M-right> <M-right> <M-right> <M-right> <M-right> <M-right> 
<M-right> <M-right> <M-right> <M-right> <M-right> <M-right> 
<M-right> <M-right> <M-right> <M-right> <M-right> <M-left> 
<M-left> <M-left> <M-left> <M-left> <M-left> <M-left> 
<M-left> <M-left> <M-left> <M-left> <M-left> <M-left> 
<M-left> <M-left> <M-left> <M-left> <M-left> <M-left> 
<M-left> <M-left> <M-left> <M-left> <M-left> <M-left> 
<M-left> <wheel-down> <double-wheel-down> <triple-wheel-down> 
<wheel-down> <double-wheel-down> <triple-wheel-down> 
<wheel-down> <double-wheel-down> <triple-wheel-down> 
<wheel-down> <double-wheel-down> <triple-wheel-down> 
<triple-wheel-down> <wheel-down> <double-wheel-down> 
<triple-wheel-down> <wheel-down> <double-wheel-down> 
<triple-wheel-down> <wheel-down> <double-wheel-down> 
<triple-wheel-down> <wheel-up> <double-wheel-up> <triple-wheel-up> 
<wheel-up> <wheel-up> <double-wheel-up> <triple-wheel-up> 
<help-echo> <down-mouse-1> <mouse-1> <wheel-up> <double-wheel-up> 
<triple-wheel-up> <triple-wheel-up> <wheel-up> <double-wheel-up> 
<triple-wheel-up> <down> <down> <down> <down> <down> 
<up> <up> <up> <up> <up> <s-up> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <s-right> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> s-N 
s-w k <left> <left> <left> <left> <left> <left> <left> 
<left> <left> s-N s-w k <escape> x e m a c s - b u 
g <return> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
r e p o r t - e m a c s - b u g <return>

Recent messages:
Saving...done
Mark set
[k]ill or [s]ave draft?
Loading msgdb for +/Users/HOME/.0.data/.0.emacs/.0.mail/msgdb/imap/mail.lawlist.com/lawlist/INBOX.Drafts/spool...done
Type C-c C-x C-z to exit MIME mode, and type C-c C-x ? to get help.
Saving...done
Mark set
[k]ill or [s]ave draft?
Loading msgdb for +/Users/HOME/.0.data/.0.emacs/.0.mail/msgdb/imap/mail.lawlist.com/lawlist/INBOX.Drafts/spool...done
*beep*

Load-path shadows:
/Users/HOME/.0.data/.0.emacs/.0.flim/md4 hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/md4
/Users/HOME/.0.data/.0.emacs/.0.flim/hex-util hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/hex-util
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/net/sasl
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl-ntlm hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/net/sasl-ntlm
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl-digest hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/net/sasl-digest
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl-cram hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/net/sasl-cram
/Users/HOME/.0.data/.0.emacs/.0.flim/ntlm hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/net/ntlm
/Users/HOME/.0.data/.0.emacs/.0.flim/hmac-md5 hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/net/hmac-md5
/Users/HOME/.0.data/.0.emacs/.0.flim/hmac-def hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/net/hmac-def
/Users/HOME/.0.data/.0.emacs/.0.wl/rfc2368 hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/mail/rfc2368
/Users/HOME/.0.data/.0.emacs/.0.wl/utf7 hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/gnus/utf7
/Users/HOME/.0.data/.0.emacs/.0.simi/smime hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/gnus/smime
/Users/HOME/.emacs.d/elpa/tabulated-list-20120406.1351/tabulated-list hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/emacs-lisp/tabulated-list
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/obsolete/pgg
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-pgp5 hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/obsolete/pgg-pgp5
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-pgp hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/obsolete/pgg-pgp
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-parse hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/obsolete/pgg-parse
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-gpg hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/obsolete/pgg-gpg
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-def hides /Users/HOME/.0.data/.0.emacs/Emacs_06_01_2014.app/Contents/Resources/lisp/obsolete/pgg-def

Features:
(shadow emacsbug modb-legacy wl-mime mime-edit pgg-parse pccl pccl-20
signature mime-setup mail-mime-setup semi-setup mime-pgp pgg-def
mime-play filename mime-image modb-standard elmo-imap4 disp-table
bbdb-autoloads lawlist-wl wl-demo wl-draft eword-encode wl-template
sendmail elmo-net elmo-cache elmo-map elmo-dop wl-news wl-address
wl-thread wl-folder wl wl-e21 wl-spam wl-action wl-summary wl-refile
wl-util pp elmo-flag elmo-localdir wl-message elmo-mime mmelmo-buffer
mmelmo-imap mime-view mime-conf calist semi-def mmimap mime-parse
mmbuffer mmgeneric elmo-multi elmo-spam elsp-header elsp-generic elmo
elmo-signal wl-highlight wl-vars wl-version elmo-msgdb modb
modb-generic modb-entity luna mime elmo-util emu invisible inv-23 poem
poem-e20 poem-e20_3 eword-decode std11 elmo-date elmo-vars
elmo-version w3m-load mime-w3m w3m browse-url doc-view jka-compr
image-mode w3m-hist w3m-fb bookmark-w3m w3m-ems w3m-ccl ccl
w3m-favicon w3m-image w3m-proc w3m-util smiley gnus-art mm-uu mml2015
mm-view mml-smime smime savehist lawlist-vr-hr lawlist-whitespace
lawlist-github conf-mode log-edit add-log find-lisp package esh-var
esh-io esh-cmd esh-opt esh-ext esh-proc esh-arg eldoc esh-groups
eshell esh-module esh-mode esh-util dired-x view tramp tramp-compat
tramp-loaddefs trampver server grep epa epg epg-config diff-mode
autorevert filenotify log-view pcvs-util ido time-stamp vc-git vc
vc-dispatcher ediff-merg ediff-wind ediff-diff ediff-mult ediff-help
ediff-init ediff-util ediff rx ert ewoc debug eieio-base
lawlist-calculator ps-print ps-def lpr flyspell ispell bbdb timezone
find-func dired-aux lawlist-yasnippet help-mode multiple-cursors
mc-separate-operations rectangular-region-mode mc-mark-more thingatpt
mc-cycle-cursors mc-edit-lines multiple-cursors-core rect saveplace
lawlist-tex-mode pcase compile shell pcomplete comint ansi-color ring
skeleton compare-w desktop frameset lawlist-tabbar lawlist-org
lawlist-calendar edmacro kmacro derived lawlist-toodledo advice
url-http url-auth url-gw url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util url-parse
auth-source eieio byte-opt bytecomp byte-compile cconv eieio-core
password-cache url-vars mailcap json xml noutline outline easy-mmode
gnus-sum gnus-group gnus-undo gnus-start gnus-cloud nnimap nnmail
mail-source tls utf7 mel path-util mime-def alist mcharset mcs-20
mcs-e20 pcustom pces pces-e20 pces-20 broken poe pym static apel-ver
product netrc nnoo parse-time gnus-spec gnus-int gnus-range message
cl-macs dired format-spec rfc822 mml easymenu mml-sec mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums
mailabbrev gmm-utils mailheader gnus-win gnus gnus-ems nnheader
gnus-util mail-utils mm-util help-fns mail-prsvr wid-edit cl gv
cl-loaddefs cl-lib time-date tooltip electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel ns-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
cocoa ns multi-tty emacs)

Memory information:
((conses 16 736782 78749)
 (symbols 48 55597 0)
 (miscs 40 501 2390)
 (strings 32 116279 16804)
 (string-bytes 1 3864862)
 (vectors 16 42181)
 (vector-slots 8 741196 80204)
 (floats 8 1011 262)
 (intervals 56 778 234)
 (buffers 960 13))

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Stefan Monnier]

> In a stackoverflow thread regarding this issue, @Stefan suggested that
> I take a look at the `jit-lock-register`.  After combing through the
> code in `jit-lock.el` and trying some experiments, I believe that
> `jit-lock.el` is more suited to a situation when buffer *modification*
> occurs -- it is also suited for situations where one may wish to
> incrementially act upon an entire buffer in small chunks at a time to
> increase efficiency of avaialble resources.

Indeed, jit-lock is designed for features that depend on the buffer's
*contents* but not on things like the value of `point'.

> I believe that the solution may be to have a `sneak-preview-redisplay`
> whereby the *new* `window-start` and *new* `window-end` could be
> correctly calculated without the visual buffer actually
> being redisplayed.

Of course, there's a circularity problem, here: if you need window-start
(you can already compute window-end without an actual redisplay) in
order to then install things like text-properties properties and
overlays that affect the display, then those added properties may end up
requiring changing window-start again (because they end up pushing point
outside of the window and require a scroll).

This said, the request is reasonable, and seems related to the needs of
follow-mode as well.

IIUC your particular use case (drawing crosshairs to show where the
cursor is located) is one where what you really need is to know when
window-start changes (just like follow-mode).

So, a good feature might be one that runs a hook when redisplay decides
that window-start needs to be reset (i.e. that point moved out of the
window and we need to scroll).


        Stefan

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 11 Jun 2014 14:00:39 -0400
> Cc: 17678@debbugs.gnu.org
> 
> > I believe that the solution may be to have a `sneak-preview-redisplay`
> > whereby the *new* `window-start` and *new* `window-end` could be
> > correctly calculated without the visual buffer actually
> > being redisplayed.
> 
> Of course, there's a circularity problem, here: if you need window-start
> (you can already compute window-end without an actual redisplay) in
> order to then install things like text-properties properties and
> overlays that affect the display, then those added properties may end up
> requiring changing window-start again (because they end up pushing point
> outside of the window and require a scroll).

Indeed.  But if there's a hook that is forbidden from making such
changes to buffer text, then this is doable.  Otherwise, it's a waste
of time having such a hook, since it will be unreliable.

> IIUC your particular use case (drawing crosshairs to show where the
> cursor is located) is one where what you really need is to know when
> window-start changes (just like follow-mode).
> 
> So, a good feature might be one that runs a hook when redisplay decides
> that window-start needs to be reset (i.e. that point moved out of the
> window and we need to scroll).

The display engine changes window-start for several reasons, only one
of them being that point moved off screen.

I think follow-mode also needs to know when the window-end changes
(which could happen with no changes in window-start).  This is
normally known only at the end of window redisplay, IIRC.

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Stefan Monnier]

> The display engine changes window-start for several reasons, only one
> of them being that point moved off screen.

Is there a comment somewhere that documents the cases where it can happen?

> I think follow-mode also needs to know when the window-end changes
> (which could happen with no changes in window-start).  This is
> normally known only at the end of window redisplay, IIRC.

In the case of follow-mode, I think the best way to handle that is to
make it possible for follow-mode to force redisplay to go through the
windows in a particular order.


        Stefan

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: esq@lawlist.com, 17678@debbugs.gnu.org
> Date: Wed, 11 Jun 2014 17:58:11 -0400
> 
> > The display engine changes window-start for several reasons, only one
> > of them being that point moved off screen.
> 
> Is there a comment somewhere that documents the cases where it can happen?

Not that I know of.

In general, there are functions that set the window-start forcibly.

> > I think follow-mode also needs to know when the window-end changes
> > (which could happen with no changes in window-start).  This is
> > normally known only at the end of window redisplay, IIRC.
> 
> In the case of follow-mode, I think the best way to handle that is to
> make it possible for follow-mode to force redisplay to go through the
> windows in a particular order.

That'd require some infrastructure that doesn't exist.

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Stefan Monnier]

>> > The display engine changes window-start for several reasons, only one
>> > of them being that point moved off screen.
>> Is there a comment somewhere that documents the cases where it can happen?
> Not that I know of.
> In general, there are functions that set the window-start forcibly.

I know of code that does that from outside of redisplay, but within
redisplay I only know of the "move point back into view".

>> In the case of follow-mode, I think the best way to handle that is to
>> make it possible for follow-mode to force redisplay to go through the
>> windows in a particular order.
> That'd require some infrastructure that doesn't exist.

Yes, that's indeed the topic of this thread.


        Stefan

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: esq@lawlist.com,  17678@debbugs.gnu.org
> Date: Thu, 12 Jun 2014 22:11:39 -0400
> 
> >> > The display engine changes window-start for several reasons, only one
> >> > of them being that point moved off screen.
> >> Is there a comment somewhere that documents the cases where it can happen?
> > Not that I know of.
> > In general, there are functions that set the window-start forcibly.
> 
> I know of code that does that from outside of redisplay, but within
> redisplay I only know of the "move point back into view".

Why does it matter if the trigger comes from outside redisplay or as
part of redisplay?

Anyway, one situation of the latter kind is when the text at
window-start changed.  Another one is when the window-start was in a
continuation line, and the window width changed, see
compute_window_start_on_continuation_line and its callers.

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Stefan Monnier]

>> I know of code that does that from outside of redisplay, but within
>> redisplay I only know of the "move point back into view".
> Why does it matter if the trigger comes from outside redisplay or as
> part of redisplay?

If it's done outside of redisplay, then pre-redisplay-function already
gets the right window-start and the problem is already solved.
If OTOH redisplay decides to scroll, it won't re-execute
pre-redisplay-function, so Elisp doesn't get a chance to react to this
new window-start.

> Anyway, one situation of the latter kind is when the text at
> window-start changed.  Another one is when the window-start was in a
> continuation line, and the window width changed, see
> compute_window_start_on_continuation_line and its callers.

Ah, right, that makes sense.


        Stefan

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: esq@lawlist.com,  17678@debbugs.gnu.org
> Date: Fri, 13 Jun 2014 08:34:59 -0400
> 
> >> I know of code that does that from outside of redisplay, but within
> >> redisplay I only know of the "move point back into view".
> > Why does it matter if the trigger comes from outside redisplay or as
> > part of redisplay?
> 
> If it's done outside of redisplay, then pre-redisplay-function already
> gets the right window-start and the problem is already solved.

Not necessarily: there are the w->optional_new_start and
w->force_start flags, which determine what redisplay does with
window-start in these cases.

Also, the window-start could be set to a value that leaves point out
of the displayed area, in which case it won't be in effect.

> If OTOH redisplay decides to scroll, it won't re-execute
> pre-redisplay-function, so Elisp doesn't get a chance to react to this
> new window-start.

??? Then what are those calls to run_window_scroll_functions that
redisplay issues?

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Stefan Monnier]

> Not necessarily: there are the w->optional_new_start and
> w-> force_start flags, which determine what redisplay does with
> window-start in these cases.
> Also, the window-start could be set to a value that leaves point out
> of the displayed area, in which case it won't be in effect.

Right, but these fall back into the case where redisplay performs
a second pass trough the window/buffer, so it's similar to the scrolling
case, right?

>> If OTOH redisplay decides to scroll, it won't re-execute
>> pre-redisplay-function, so Elisp doesn't get a chance to react to this
>> new window-start.
> ??? Then what are those calls to run_window_scroll_functions that
> redisplay issues?

Ah, right, I forgot about this.  So maybe window-scroll-functions is the
answer to the OP's needs!


        Stefan

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: esq@lawlist.com,  17678@debbugs.gnu.org
> Date: Fri, 13 Jun 2014 10:14:03 -0400
> 
> > Not necessarily: there are the w->optional_new_start and
> > w-> force_start flags, which determine what redisplay does with
> > window-start in these cases.
> > Also, the window-start could be set to a value that leaves point out
> > of the displayed area, in which case it won't be in effect.
> 
> Right, but these fall back into the case where redisplay performs
> a second pass trough the window/buffer, so it's similar to the scrolling
> case, right?

Not necessarily, AFAIR.  Sometimes these situations are detected and
handled on the fly.

> >> If OTOH redisplay decides to scroll, it won't re-execute
> >> pre-redisplay-function, so Elisp doesn't get a chance to react to this
> >> new window-start.
> > ??? Then what are those calls to run_window_scroll_functions that
> > redisplay issues?
> 
> Ah, right, I forgot about this.  So maybe window-scroll-functions is the
> answer to the OP's needs!

Or maybe we should call them in more places during redisplay.

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Keith David Bershatsky]

The `window-scroll-functions` hook would be more useful if the value of `window-start` could be obtained *before* running the `post-command-hook`.  At the present time, it would appear that the `post-command-hook` runs *before* the `window-scroll-functions` hook.  Therefore, the `post-command-hook` cannot presently use the `window-start` value produced from the `window-scroll-functions` hook (which occurs *after* the `post-command-hook` has already finished).

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

At Fri, 13 Jun 2014 10:14:03 -0400,
Stefan Monnier wrote:

* * *

> >> If OTOH redisplay decides to scroll, it won't re-execute
> >> pre-redisplay-function, so Elisp doesn't get a chance to react to this
> >> new window-start.
> > ??? Then what are those calls to run_window_scroll_functions that
> > redisplay issues?
> 
> Ah, right, I forgot about this.  So maybe window-scroll-functions is the
> answer to the OP's needs!
> 
> 
>         Stefan

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Eli Zaretskii]

> Date:  Fri, 13 Jun 2014 09:22:07 -0700
> From:  Keith David Bershatsky <esq@lawlist.com>
> Cc:  17678@debbugs.gnu.org
> 
> The `window-scroll-functions` hook would be more useful if the value of `window-start` could be obtained *before* running the `post-command-hook`.  At the present time, it would appear that the `post-command-hook` runs *before* the `window-scroll-functions` hook.  Therefore, the `post-command-hook` cannot presently use the `window-start` value produced from the `window-scroll-functions` hook (which occurs *after* the `post-command-hook` has already finished).

Yes, post-command-hook runs before window-scroll-functions.  That's
because post-command-hook runs immediately after a command execution
is finished and Emacs is about to return to the command loop, while
window-scroll-functions run as part of redisplay, which is entered
whenever Emacs is idle, i.e., after the return to the command loop.

Unfortunately, what you are asking for is not possible with the
current design of Emacs, because window-start is computed as part of
redisplay (it is one of the most important functions of redisplay to
compute that).

OTOH, I don't understand why do you need what you ask for.  For
example, why can't you do whatever you need from
window-scroll-functions, instead of doing some of it in
post-command-hook?  Perhaps you should describe the problem you are
trying to solve in more detail.

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Keith David Bershatsky]

I believe splitting up the work between the two hooks may be possible -- I will need to revise the conditions once I identify additional situations.  As far as I can tell, the `window-scroll-functions` hook is NOT triggered when `point` STAYS between *old* `window-start` and *old* `window-end`.  So when `point` STAYS between *old* `window-start` and *old* `window-end`, I will need to use the `post-command-hook`.  When point moves BEYOND *old* `window-start` or `*old* `window-end`, then the `window-scroll-functions` hook can take over -- with a forced new `(window-end nil t)`.

`window-scroll-functions`:

      (or
        (< (point) (window-start)) ;; good for things like `beginning-of-buffer`
        (> (point) (window-end)) ;; good for things like `end-of-buffer`
        (> (point) (window-end nil t))) ;; good for things like `yank`

`post-comand-hook`

    (and
      (not (< (point) (window-start)))
      (not (> (point) (window-end)))
      (not (> (point) (window-end nil t))))

I haven't tried it yet, but it appears to make sense based on what I saw this morning with some `messages` that I set up using each hook.

Keith




---------------------------------------

On Jun 13, 2014, at 10:55 AM, Eli Zaretskii wrote:

>> Date:  Fri, 13 Jun 2014 09:22:07 -0700
>> From:  Keith David Bershatsky <esq@lawlist.com>
>> Cc:  17678@debbugs.gnu.org
>> 
>> The `window-scroll-functions` hook would be more useful if the value of `window-start` could be obtained *before* running the `post-command-hook`.  At the present time, it would appear that the `post-command-hook` runs *before* the `window-scroll-functions` hook.  Therefore, the `post-command-hook` cannot presently use the `window-start` value produced from the `window-scroll-functions` hook (which occurs *after* the `post-command-hook` has already finished).
> 
> Yes, post-command-hook runs before window-scroll-functions.  That's
> because post-command-hook runs immediately after a command execution
> is finished and Emacs is about to return to the command loop, while
> window-scroll-functions run as part of redisplay, which is entered
> whenever Emacs is idle, i.e., after the return to the command loop.
> 
> Unfortunately, what you are asking for is not possible with the
> current design of Emacs, because window-start is computed as part of
> redisplay (it is one of the most important functions of redisplay to
> compute that).
> 
> OTOH, I don't understand why do you need what you ask for.  For
> example, why can't you do whatever you need from
> window-scroll-functions, instead of doing some of it in
> post-command-hook?  Perhaps you should describe the problem you are
> trying to solve in more detail.

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Stefan Monnier]

>> > Not necessarily: there are the w->optional_new_start and
>> > w-> force_start flags, which determine what redisplay does with
>> > window-start in these cases.
>> > Also, the window-start could be set to a value that leaves point out
>> > of the displayed area, in which case it won't be in effect.
>> Right, but these fall back into the case where redisplay performs
>> a second pass trough the window/buffer, so it's similar to the scrolling
>> case, right?
> Not necessarily, AFAIR.  Sometimes these situations are detected and
> handled on the fly.

Aha!  Could you add comment somewhere in xdisp.c discussing the above
issues about how/when is window-start obeyed and when it's not and
when that causes a second pass and when that's handled on the fly?
It doesn't have to be exhaustive, but at least mentioning the known
cases, and ideally pointing to the corresponding code.

>> Ah, right, I forgot about this.  So maybe window-scroll-functions is the
>> answer to the OP's needs!
> Or maybe we should call them in more places during redisplay.

Could be.  But I already find it hard to know what can be done with
window-scroll-functions because it's not clear exactly when it's called,
in 2 sense:
- it's not clear exactly when it is called and when it is not.
- it's not clear exactly at which stage of redisplay it is called
  (e.g. what has already been processed, what hasn't; will any and all
  modifications caused by window-scroll-functions be reflected on screen
  at the end of the current redisplay, or will some of it only be
  handled by the next redisplay?  If so, which do and which don't?)


-- Stefan

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Keith David Bershatsky]

Here is a first draft, which appears to correctly handle common movement through the buffer using `beginning-of-buffer`, `end-of-buffer`, and custom paragraph forward / backward function.

    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
    
    (defvar old-window-start nil
    "This local variable is set within the `post-command-hook`; and,
    is also used by the `window-scroll-functions` hook.")
    (make-variable-buffer-local 'old-window-start)
    
    (defvar old-window-end nil
    "This local variable is set within the `post-command-hook`; and,
    is also used by the `window-scroll-functions` hook.")
    (make-variable-buffer-local 'old-window-end)
    
    (defvar old-window-end-forced nil
    "This local variable is set within the `post-command-hook`; and,
    is also used by the `window-scroll-functions` hook.")
    (make-variable-buffer-local 'old-window-end-forced)
    
    (defun test-post-command-hook ()
      (when
          (and
            (not (minibufferp))
            (window-live-p (get-buffer-window (current-buffer))))
        (setq old-window-start (window-start))
        (setq old-window-end (window-end))
        (setq old-window-end-forced (window-end nil t))
        (when
            (and
              (not (< (point) old-window-start))
              (not (> (point) old-window-end))
              (not (> (point) old-window-end-forced)))
          (message (concat
            "P.C.H. -- `point`: %s | "
            "`old-window-start`: %s | "
            "`old-window-end`: %s | "
            "`old-window-end-forced`: %s")
              (point)
              old-window-start
              old-window-end
              old-window-end-forced))))
    
    (defun test-window-scroll-functions (win _start)
      "Good for things like: `beginning-of-buffer`; `end-of-buffer`; `yank`; etc"
      (when
          (and
            (not (minibufferp))
            (window-live-p (get-buffer-window (current-buffer))))
        (when
            (or
              (< (point) old-window-start)
              (> (point) old-window-end)
              (> (point) old-window-end-forced))
          (message (concat
            "W.S.F. -- `point`: %s | "
            "*new* window-start: %s | "
            "*new* window-end: %s")
              (point)
              _start
              (window-end win t)))))
    
    (define-minor-mode test-mode
    "A minor-mode for testing `window-start` and `window-end`."
      :init-value nil
      :lighter " 𝓣𝓔𝓢𝓣"
      :keymap nil
      :global nil
      (cond
        (test-mode
          (condition-case error
            (progn
              (setq scroll-conservatively 101)
              (setq scroll-margin 0)
              (add-hook 'post-command-hook 'test-post-command-hook nil t)
              (add-hook 'window-scroll-functions 'test-window-scroll-functions nil t)
              (message "Turned ON `test-mode`."))
            (error
             (test-mode 0)
             (signal (car error) (cdr error)))))
        ((not test-mode)
          (remove-hook 'post-command-hook 'test-post-command-hook t)
          (remove-hook 'window-scroll-functions 'test-window-scroll-functions t)
          (message "Turned OFF `test-mode`.") )))
    
    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Eli Zaretskii]

> From: Keith David Bershatsky <esq@lawlist.com>
> Date: Fri, 13 Jun 2014 11:24:01 -0700
> Cc: 17678@debbugs.gnu.org
> 
> I believe splitting up the work between the two hooks may be possible -- I will need to revise the conditions once I identify additional situations.  As far as I can tell, the `window-scroll-functions` hook is NOT triggered when `point` STAYS between *old* `window-start` and *old* `window-end`.  So when `point` STAYS between *old* `window-start` and *old* `window-end`, I will need to use the `post-command-hook`.  When point moves BEYOND *old* `window-start` or `*old* `window-end`, then the `window-scroll-functions` hook can take over -- with a forced new `(window-end nil t)`.

Why do you care about the situation where point stays inside the same
window limits?

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: esq@lawlist.com,  17678@debbugs.gnu.org
> Date: Fri, 13 Jun 2014 14:47:05 -0400
> 
> >> > Not necessarily: there are the w->optional_new_start and
> >> > w-> force_start flags, which determine what redisplay does with
> >> > window-start in these cases.
> >> > Also, the window-start could be set to a value that leaves point out
> >> > of the displayed area, in which case it won't be in effect.
> >> Right, but these fall back into the case where redisplay performs
> >> a second pass trough the window/buffer, so it's similar to the scrolling
> >> case, right?
> > Not necessarily, AFAIR.  Sometimes these situations are detected and
> > handled on the fly.
> 
> Aha!  Could you add comment somewhere in xdisp.c discussing the above
> issues about how/when is window-start obeyed and when it's not and
> when that causes a second pass and when that's handled on the fly?

I will see what I can do.

> Could be.  But I already find it hard to know what can be done with
> window-scroll-functions because it's not clear exactly when it's called,
> in 2 sense:
> - it's not clear exactly when it is called and when it is not.
> - it's not clear exactly at which stage of redisplay it is called
>   (e.g. what has already been processed, what hasn't; will any and all
>   modifications caused by window-scroll-functions be reflected on screen
>   at the end of the current redisplay, or will some of it only be
>   handled by the next redisplay?  If so, which do and which don't?)

I'm not sure I follow.  Redisplay, at the level we are talking, has no
stages.  It goes through all the windows on every frame, and does for
each window what it thinks has to be done in that window.  A process
of redisplaying a window is done in one go, there are no stages or
phases in it.  The window-scroll-functions are called when redisplay
thinks it will scroll the window in order to redisplay it.

Which modifications in window-scroll-functions did you have in mind?

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Keith David Bershatsky]

The custom minor-mode that I am working places overlays between `window-start` and `window-end`, and is triggered upon a variety of occurrences.  The three general categories that trigger removal / placement of overlays are:  (1) buffer modification; (2) window modification; (3) cursor movement.  The overlays draw three categories:  (1) end of line indicators (e.g., pilcrow, or single-angle [for cursor eol]); (2) a horizontal line at the current cursor position that spans the entire window-width (excluding the line numbers and fringes); and, (3) a vertical line aligned with the cursor stretching from the top to bottom of the window (excluding the headline where I have Tabbar). To make the minor mode as efficient as possible (in terms of time needed to remove / place the overlays), I am limiting the area to only the visible window.

The goal is to have the new overlays placed before the redisplay occurs -- this avoids a brief glimpse of the naked buffer once the prior overlays have been removed (and before the new overlays are laid).  Prior to the existence of the `test-mode` that I sent over a short while ago (based on the help that you and Stefan have so graciously provided), I was forcing a redisplay (whenever point moved outside of the *old* visible window limits) in order to obtain the *new* `window-start` and *new* `window-end`.  It looks as if the `test-mode` concept will resolve the issue by handling the two different conditions separately -- i.e., point inside the *old* window limits, versus point outside thereof.

---------------------------------------

On Jun 13, 2014, at 1:54 PM, Eli Zaretskii wrote:

>> From: Keith David Bershatsky <esq@lawlist.com>
>> Date: Fri, 13 Jun 2014 11:24:01 -0700
>> Cc: 17678@debbugs.gnu.org
>> 
>> I believe splitting up the work between the two hooks may be possible -- I will need to revise the conditions once I identify additional situations.  As far as I can tell, the `window-scroll-functions` hook is NOT triggered when `point` STAYS between *old* `window-start` and *old* `window-end`.  So when `point` STAYS between *old* `window-start` and *old* `window-end`, I will need to use the `post-command-hook`.  When point moves BEYOND *old* `window-start` or `*old* `window-end`, then the `window-scroll-functions` hook can take over -- with a forced new `(window-end nil t)`.
> 
> Why do you care about the situation where point stays inside the same
> window limits?

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Eli Zaretskii]

> Date: Fri, 13 Jun 2014 23:59:33 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: esq@lawlist.com, 17678@debbugs.gnu.org
> 
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Cc: esq@lawlist.com,  17678@debbugs.gnu.org
> > Date: Fri, 13 Jun 2014 14:47:05 -0400
> > 
> > >> > Not necessarily: there are the w->optional_new_start and
> > >> > w-> force_start flags, which determine what redisplay does with
> > >> > window-start in these cases.
> > >> > Also, the window-start could be set to a value that leaves point out
> > >> > of the displayed area, in which case it won't be in effect.
> > >> Right, but these fall back into the case where redisplay performs
> > >> a second pass trough the window/buffer, so it's similar to the scrolling
> > >> case, right?
> > > Not necessarily, AFAIR.  Sometimes these situations are detected and
> > > handled on the fly.
> > 
> > Aha!  Could you add comment somewhere in xdisp.c discussing the above
> > issues about how/when is window-start obeyed and when it's not and
> > when that causes a second pass and when that's handled on the fly?
> 
> I will see what I can do.

Done in emacs-24 revision 117239.  Please see if it's good enough.

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Keith David Bershatsky]

Thank you both, Eli and Stefan, for helping me to resolve this issue.  The test minor mode concept is working well, with just a few minor revisions.  I posted the test minor mode as an answer on Stackoverflow (giving credit to both of you for the key ingredient), and I'll probably revise it a bit over the next few days.  That may help future internet searchers looking for a solution to similar issues.

The custom minor mode project I described (with cross-hairs in the form of thin lines, and end of line indicators) is now shaping up quite nicely.  Your help is greatly appreciated!

Keith

bug#17678: 24.4.50; Feature Request -- calculate new `window-start` & `window-end` before visual redisplay

[Stefan Monnier]

>> Aha!  Could you add comment somewhere in xdisp.c discussing the above
>> issues about how/when is window-start obeyed and when it's not and
>> when that causes a second pass and when that's handled on the fly?
> I will see what I can do.
> Done in emacs-24 revision 117239.  Please see if it's good enough.

Looks great, thank you very much.

> I'm not sure I follow.  Redisplay, at the level we are talking, has no
> stages.  It goes through all the windows on every frame, and does for
> each window what it thinks has to be done in that window.  A process
> of redisplaying a window is done in one go, there are no stages or
> phases in it.

AFAIK window-scroll-functions (as well as jit-lock) is run in the
middle of redisplay redisplay, so they get to see "the inside" and so
they break the illusion that redisplay happens "in one go".

> The window-scroll-functions are called when redisplay
> thinks it will scroll the window in order to redisplay it.
> Which modifications in window-scroll-functions did you have in mind?

For example:
- if code in window-scroll-functions affects some data which affects
  the mode-line, will the mode-line correctly reflect the state after
  running window-scroll-functions?
- if code in window-scroll-functions affects some data which influences
  the display of other windows (e.g. adds overlays, changes
  window-start, you name it), will those changes be reflected in the
  other windows at the end of this redisplay cycle?
- if point needs to be changed by redisplay (because window-start was
  set far away), will window-scroll-functions see the "old" point value
  or the "new" point value?


-- Stefan