bug#18618: 25.0.50; `window-end win t` produces erroenous result with `window-scroll-functions` hook.

bug#18618: 25.0.50; `window-end win t` produces erroenous result with `window-scroll-functions` hook.

[Keith David Bershatsky]

Steps to reproduce the issue.

1.  Create a function that reports (e.g., a message) the value of `(window-end win t)` and attach that function to the `window-scroll-functions` hook.

2.  Open a long file in either fundamental-mode or text-mode.

3.  M-x end-of-buffer

4.  M-x beginning-of-buffer

The result of step 4 reports an erroneous window-end value that is at the very end of the buffer, instead of the correct window-end (i.e., which is much closer to the beginning of the buffer).

This makes it impossible to correctly draw overlays between window-start and window-end, because Emacs thinks the entire buffer should be used following an interactive use of `beginning-of-buffer`.

Please feel free to use my test-mode, which is a minor mode for testing window-start and window-end.

Thanks,

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; test-mode

(defvar test-old-window-start nil
"This local variable is set within the `post-command-hook`; and,
is also used by the `window-scroll-functions` hook.")
(make-variable-buffer-local 'test-old-window-start)

(defvar test-old-window-end nil
"This local variable is set within the `post-command-hook`; and,
is also used by the `window-scroll-functions` hook.")
(make-variable-buffer-local 'test-old-window-end)

(defvar test-old-window-end-forced nil
"This local variable is set within the `post-command-hook`; and,
is also used by the `window-scroll-functions` hook.")
(make-variable-buffer-local 'test-old-window-end-forced)

(defvar test-new-window-start nil
"This local variable is set within the `window-scroll-functions`.")
(make-variable-buffer-local 'test-new-window-start)

(defvar test-new-window-end nil
"This local variable is set within the `window-scroll-functions`.")
(make-variable-buffer-local 'test-new-window-end)

(defun test-post-command-hook ()
"NOT good for things like: `beginning-of-buffer`; `end-of-buffer`; `yank`; etc.
NOTE:  When using `this-command` in conjunction with the `post-command-hook`,
the `window-scroll-functions` hook would need to use `last-command`."
  (when
      (and
        (not (minibufferp))
        (window-live-p (get-buffer-window (current-buffer))))
    (setq test-old-window-start (window-start))
    (setq test-old-window-end (window-end))
    (setq test-old-window-end-forced (window-end nil t))
    ;; DEBUGGING TEST
    ;;  (message "pt: %s | ows: %s | owe: %s | owe-f: %s"
    ;;    (point)
    ;;    test-old-window-start
    ;;    test-old-window-end
    ;;    test-old-window-end-forced)
    (when
        (or
          (and
            (not (< (point) test-old-window-start))
            (pos-visible-in-window-p (point)
              (get-buffer-window (current-buffer) (selected-frame)))
            (not (> (point) test-old-window-end))
            (not (> (point) test-old-window-end-forced)))
          ;; special situation when deleting region greater than size of window.
          (and
            (region-active-p)
            (< test-old-window-end 0))
          ;; special situation when deleting region greater than size of window.
          (and
            (region-active-p)
            (> (point) test-old-window-start)
            (> (point) test-old-window-end)
            (< (point) test-old-window-end-forced)) )
      (test-mode-demonstration
        test-old-window-start
        test-old-window-end
        test-old-window-end-forced
        nil
        nil))))

(defun test-window-scroll-functions (win _start)
"Good for things like: `beginning-of-buffer`; `end-of-buffer`; `yank`; etc.
NOTE:  When using `this-command` in conjunction with the `post-command-hook`,
the `window-scroll-functions` hook would need to use `last-command`."
  (when
      (and
        test-old-window-start
        test-old-window-end
        test-old-window-end-forced
        (not (minibufferp))
        (window-live-p (get-buffer-window (current-buffer))))
    ;; DEBUGGING TEST
    ;; (message "nws: %s | nwe-f: %s" _start (window-end nil t))
    (when
        (or
          (not (= _start test-old-window-start))
          (not (pos-visible-in-window-p (point)
            (get-buffer-window (current-buffer) (selected-frame))))
          (< (point) test-old-window-start)
          (> (point) test-old-window-end)
          (> (point) test-old-window-end-forced))
      (setq test-new-window-start _start)
      (setq test-new-window-end (window-end win t))
      ;; FIX-ME -- special circumstance when jumping paragraph down.
      ;; (when (> (point) test-new-window-end)
      ;; (setq test-new-window-end . . .
      (test-mode-demonstration
        nil
        nil
        nil
        test-new-window-start
        test-new-window-end)
      (setq test-old-window-start nil)
      (setq test-old-window-end nil)
      (setq test-old-window-end-forced nil))))

(defun test-mode-demonstration
  (&optional
    test-old-window-start
    test-old-window-end
    test-old-window-end-forced
    test-new-window-start
    test-new-window-end)
"This is a test-mode demonstration function."
  (let* (
      (window-start
        (cond
          (test-old-window-start
            test-old-window-start)
          (test-new-window-start
            test-new-window-start)
          (t (window-start))))
      (window-end
        (cond
          ((and
              test-old-window-end
              test-old-window-end-forced
              (= test-old-window-end test-old-window-end-forced))
            test-old-window-end)
          ((and
              test-old-window-end
              test-old-window-end-forced
              (> test-old-window-end-forced test-old-window-end))
            test-old-window-end-forced)
          (test-new-window-end
            test-new-window-end)
          (t (window-end (selected-window) t)))) )
    (cond
      ((and
          test-old-window-start
          test-old-window-end
          test-old-window-end-forced)
        (message (concat
        "P.C.H. -- `point`: %s | "
        "`test-old-window-start`: %s | "
        "`test-old-window-end`: %s | "
        "`test-old-window-end-forced`: %s")
          (point)
          test-old-window-start
          test-old-window-end
          test-old-window-end-forced))
      ((and
          test-new-window-start
          test-new-window-end)
        (message (concat
          "W.S.F. -- `point`: %s | "
          "`test-new-window-start`: %s | "
          "`test-new-window-end`: %s")
            (point)
            test-new-window-start
            test-new-window-end))) ))

(define-minor-mode test-mode
"A minor-mode for testing `window-start` / `window-end` BEFORE visual redisplay."
  :init-value nil
  :lighter " TEST"
  :keymap nil
  :global nil
  :group nil
  (cond
    (test-mode
      (condition-case error
        (progn
          (setq scroll-conservatively 101)
          (setq max-mini-window-height 2)
          (add-hook 'post-command-hook 'test-post-command-hook nil t)
          (add-hook 'window-scroll-functions 'test-window-scroll-functions nil t)
          (when (called-interactively-p 'any)
            (message "Turned ON `test-mode`.")))
        (error
         (test-mode 0)
         (signal (car error) (cdr error)))))
    ((not test-mode)
      (setq max-mini-window-height 0.25)
      (remove-hook 'post-command-hook 'test-post-command-hook t)
      (remove-hook 'window-scroll-functions 'test-window-scroll-functions t)
      (when (called-interactively-p 'any)
        (message "Turned OFF `test-mode`.") ))))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;


In GNU Emacs 25.0.50.1 (x86_64-apple-darwin10.8.0, NS appkit-1038.36 Version 10.6.8 (Build 10K549))
 of 2014-10-01 on MP.local
Repository revision: 117996 dmantipov@yandex.ru-20141001132108-zdsxru2390mqyjlu
Windowing system distributor `Apple', version 10.3.1038
Configured using:
 `configure --with-ns'

Configured features:
ACL LIBXML2 ZLIB

Important settings:
  locale-coding-system: utf-8-unix

Major mode: Text

Minor modes in effect:
  sd-mode: t
  test-mode: t
  sb-mode: t
  tb-mode: t
  shell-dirtrack-mode: t
  cm-mode: t
  bc-mode: t
  as-mode: t
  ds-mode: t
  ml-mode: t

Recent input:

Recent messages:
Mark set
W.S.F. -- `point`: 1 | `test-new-window-start`: 1 | `test-new-window-end`: 339823
Turned ON `test-mode`.
P.C.H. -- `point`: 1191 | `test-old-window-start`: 1 | `test-old-window-end`: 5438 | `test-old-window-end-forced`: 5438
W.S.F. -- `point`: 339823 | `test-new-window-start`: 339179 | `test-new-window-end`: 339823
Mark set
W.S.F. -- `point`: 1 | `test-new-window-start`: 1 | `test-new-window-end`: 339823
W.S.F. -- `point`: 339823 | `test-new-window-start`: 339179 | `test-new-window-end`: 339823
Mark set
W.S.F. -- `point`: 1 | `test-new-window-start`: 1 | `test-new-window-end`: 339823

Load-path shadows:
/Users/HOME/.0.data/.0.emacs/.0.flim/md4 hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/md4
/Users/HOME/.0.data/.0.emacs/.0.flim/hex-util hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/hex-util
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/net/sasl
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl-ntlm hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/net/sasl-ntlm
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl-digest hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/net/sasl-digest
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl-cram hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/net/sasl-cram
/Users/HOME/.0.data/.0.emacs/.0.flim/ntlm hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/net/ntlm
/Users/HOME/.0.data/.0.emacs/.0.flim/hmac-md5 hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/net/hmac-md5
/Users/HOME/.0.data/.0.emacs/.0.flim/hmac-def hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/net/hmac-def
/Users/HOME/.0.data/.0.emacs/.0.wl/rfc2368 hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/mail/rfc2368
/Users/HOME/.0.data/.0.emacs/.0.wl/utf7 hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/gnus/utf7
/Users/HOME/.0.data/.0.emacs/.0.simi/smime hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/gnus/smime
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/obsolete/pgg
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-pgp5 hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/obsolete/pgg-pgp5
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-pgp hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/obsolete/pgg-pgp
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-parse hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/obsolete/pgg-parse
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-gpg hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/obsolete/pgg-gpg
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-def hides /Users/HOME/.0.data/.0.emacs/Emacs_10_01_2014.app/Contents/Resources/lisp/obsolete/pgg-def

Features:
(shadow emacsbug modb-legacy mime-setup mail-mime-setup semi-setup
mime-image modb-standard elmo-imap4 eieio-opt speedbar sb-image
ezimage dframe lawlist-desktop frameset lawlist-dv lawlist-mc rect
lawlist-ztree lawlist-wl elmo-nntp wl-demo wl-news wl-address
wl-thread wl wl-e21 wl-draft elmo-pop3 wl-template elmo-net elmo-cache
elmo-map elmo-dop wl-folder wl-spam wl-action wl-summary wl-refile
wl-message wl-mime pgg mime-pgp wl-util pp elmo-flag elmo-localdir
mime-play filename mime-edit eword-encode pgg-parse pccl pccl-20
pgg-def signature sendmail elmo-mime mmelmo-buffer mmelmo-imap
mime-view mime-conf calist semi-def mmimap mime-parse mmbuffer
mmgeneric elmo-filter elmo-multi elmo-spam elsp-header elsp-generic
elmo elmo-signal wl-highlight wl-vars wl-version elmo-msgdb modb
modb-generic modb-entity luna mime elmo-util emu invisible inv-23 poem
poem-e20 poem-e20_3 eword-decode std11 elmo-date elmo-vars
elmo-version w3m-load mime-w3m w3m browse-url doc-view jka-compr
image-mode w3m-hist w3m-fb bookmark-w3m w3m-ems w3m-ccl ccl
w3m-favicon w3m-image w3m-proc w3m-util lawlist-dired dired-aux
lawlist-vr-hr lawlist-ws disp-table lawlist-calculator
lawlist-flyspell bbdb-autoloads bbdb lawlist-yasnippet
lawlist-tex-mode skeleton compare-w lawlist-text-mode lawlist-tabbar
lawlist-github ido view tramp tramp-compat tramp-loaddefs trampver
shell pcomplete help-mode grep compile comint epa epg epg-config
diff-mode autorevert filenotify ansi-color find-lisp log-edit ring
add-log thingatpt log-view pcvs-util conf-mode time-stamp vc-git vc
vc-dispatcher ediff-merg ediff-wind ediff-diff ediff-mult ediff-help
ediff-init ediff-util ediff rx ert ewoc debug timezone eieio-base
lawlist-toodledo url-http url-auth url-gw url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
url-parse auth-source eieio eieio-core password-cache url-vars mailcap
json xml lawlist-org lawlist-calendar byte-opt bytecomp byte-compile
cconv derived noutline outline gnus-sum gnus-group gnus-undo
gnus-start gnus-cloud nnimap nnmail mail-source tls utf7 mel path-util
mime-def alist mcharset mcs-20 mcs-e20 pcustom pces pces-e20 pces-20
broken poe pym static apel-ver product netrc nnoo parse-time gnus-spec
gnus-int gnus-range message dired format-spec rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus gnus-ems
nnheader gnus-util mail-utils mm-util mail-prsvr wid-edit cl
lawlist-frame lawlist-init pcase cl-macs gv advice help-fns easy-mmode
edmacro kmacro cl-loaddefs cl-lib savehist server ps-print ps-def lpr
find-func saveplace easymenu time-date tooltip electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel ns-win tool-bar dnd fontset
image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
cocoa ns multi-tty emacs)

Memory information:
((conses 16 893343 108197)
 (symbols 48 57109 0)
 (miscs 40 115 576)
 (strings 32 115451 12362)
 (string-bytes 1 3835645)
 (vectors 16 43897)
 (vector-slots 8 1477759 244392)
 (floats 8 977 154)
 (intervals 56 4347 92)
 (buffers 976 16))

bug#18618: `window-end win t` produces erroenous result with `window-scroll-functions` hook.

[Keith David Bershatsky]

Upon some further testing, I see that `transient-mark-mode` interferes with properly calculating the new window-end when using the window-scroll-functions hook (e.g., going from the end of the buffer the `beginning-of-buffer`).  When `transient-mark-mode` is enabled, the wrong window end is reported.  When `transient-mark-mode` is deactivated `(transient-mark-mode -1)`, the correct window end is reported.

Keith

bug#18618: 25.0.50; `window-end win t` produces erroenous result with `window-scroll-functions` hook.

[Lars Ingebrigtsen]

Keith David Bershatsky <esq@lawlist.com> writes:

> Steps to reproduce the issue.
>
> 1.  Create a function that reports (e.g., a message) the value of `(window-end win t)` and attach that function to the `window-scroll-functions` hook.
>
> 2.  Open a long file in either fundamental-mode or text-mode.
>
> 3.  M-x end-of-buffer
>
> 4.  M-x beginning-of-buffer
>
> The result of step 4 reports an erroneous window-end value that is at
> the very end of the buffer, instead of the correct window-end (i.e.,
> which is much closer to the beginning of the buffer).

(I'm going through old bug reports that unfortunately got no response at
the time.)

This problem is still present in Emacs 28.  Here's an easier test case:

(defun foo (win _)
  (message "End: %s" (window-end win t))
  nil)
(push 'foo window-scroll-functions)

This reports the same number in both 3) and 4) when transient-mark-mode
is switched on, but not when it's off.  It's also correct if that mode
is on, and the region is active.

I haven't tried to debug further -- perhaps it's immediately obvious to
somebody what could be causing this glitch?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#18618: 25.0.50; `window-end win t` produces erroenous result with `window-scroll-functions` hook.

[Keith David Bershatsky]

Lars:

I haven't played in depth with the Emacs internals for a couple of years or so.  In working on my own feature requests (back in the day), I learned that window-start/end cannot be accurately ascertained with 100% certainty until the tail end of redisplay ...  If a user were to add/remove something with Lisp, then redisplay would need to recalculate to take that modification into consideration -- necessitating further recalculation, which may alter window-start/end.

In terms of my own feature requests, I opted to use `update_window` in dispnew.c to add visual modifications to the glass that did not alter any text or points in the buffer.  That way, no further calculation was needed as to window-start/end points -- as said points were "final" calculations at that late stage of redisplay.  The `window-scroll-functions` hook only operates under certain criteria, but not all the time.  As of my last look a couple of years ago, there was no hook that operated towards the latter part of redisplay with 100% certainty.  For a few years before using `update_window`, I used a combination of the `post-command-hook` and the `window-scroll-functions` hook to try and catch the majority of situations to ascertain window-start/end, but there were always several situa
 tions where the two hooks where insufficient ...  An example of what users were required to do back in the day can be seen by examining libraries such as the deprecated linum-mode, which used both o
 f the aforementioned hooks because there is/was no sole hook that could accurately predict window-start/end with 100% certainty.

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

> Date: [05-28-2021 20:37:18] <29 May 2021 05:37:18 +0200>
> From: Lars Ingebrigtsen <larsi@gnus.org>
> To: Keith David Bershatsky <esq@lawlist.com>
> Cc: 18618@debbugs.gnu.org
> Subject: Re: bug#18618: 25.0.50; `window-end win t` produces erroenous result with `window-scroll-functions` hook.
> 
> Keith David Bershatsky <esq@lawlist.com> writes:
> 
> > Steps to reproduce the issue.
> >
> > 1.  Create a function that reports (e.g., a message) the value of `(window-end win t)` and attach that function to the `window-scroll-functions` hook.
> >
> > 2.  Open a long file in either fundamental-mode or text-mode.
> >
> > 3.  M-x end-of-buffer
> >
> > 4.  M-x beginning-of-buffer
> >
> > The result of step 4 reports an erroneous window-end value that is at
> > the very end of the buffer, instead of the correct window-end (i.e.,
> > which is much closer to the beginning of the buffer).
> 
> (I'm going through old bug reports that unfortunately got no response at
> the time.)
> 
> This problem is still present in Emacs 28.  Here's an easier test case:
> 
> (defun foo (win _)
>   (message "End: %s" (window-end win t))
>   nil)
> (push 'foo window-scroll-functions)
> 
> This reports the same number in both 3) and 4) when transient-mark-mode
> is switched on, but not when it's off.  It's also correct if that mode
> is on, and the region is active.
> 
> I haven't tried to debug further -- perhaps it's immediately obvious to
> somebody what could be causing this glitch?
> 
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no

bug#18618: 25.0.50; `window-end win t` produces erroenous result with `window-scroll-functions` hook.

[Eli Zaretskii]

tags 18618 notabug
close 18618
thanks

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 29 May 2021 05:37:18 +0200
> Cc: 18618@debbugs.gnu.org
> 
> Keith David Bershatsky <esq@lawlist.com> writes:
> 
> > Steps to reproduce the issue.
> >
> > 1.  Create a function that reports (e.g., a message) the value of `(window-end win t)` and attach that function to the `window-scroll-functions` hook.
> >
> > 2.  Open a long file in either fundamental-mode or text-mode.
> >
> > 3.  M-x end-of-buffer
> >
> > 4.  M-x beginning-of-buffer
> >
> > The result of step 4 reports an erroneous window-end value that is at
> > the very end of the buffer, instead of the correct window-end (i.e.,
> > which is much closer to the beginning of the buffer).
> 
> (I'm going through old bug reports that unfortunately got no response at
> the time.)
> 
> This problem is still present in Emacs 28.  Here's an easier test case:
> 
> (defun foo (win _)
>   (message "End: %s" (window-end win t))
>   nil)
> (push 'foo window-scroll-functions)
> 
> This reports the same number in both 3) and 4) when transient-mark-mode
> is switched on, but not when it's off.  It's also correct if that mode
> is on, and the region is active.
> 
> I haven't tried to debug further -- perhaps it's immediately obvious to
> somebody what could be causing this glitch?

This isn't supposed to work.  The doc string of window-end says:

  Return position at which display currently ends in WINDOW.
  WINDOW must be a live window and defaults to the selected one.
  This is updated by redisplay, when it runs to completion.
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

By contrast, window-scroll-functions is a hook run by the display
engine in the middle of redisplaying a window, when the display engine
concludes that it is about to scroll the window.  At that point, the
window's redisplay is by definition not complete yet, so this can only
work by chance.  Which is why window-scroll-functions' doc string says
explicitly this doesn't work:

  Note that the value of ‘window-end’ is not valid when these functions are
  called.

So Emacs behaves here as designed and as documented, and I'm therefore
closing this bug.