bug#73041: 30.0.90; track-changes-mode logs warnings (#70541 regression? not actually fixed?)

bug#73041: 30.0.90; track-changes-mode logs warnings (#70541 regression? not actually fixed?)

[epg]

I wish I could describe exactly what happened.  I found
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70541 which sounds
very much like what I'm seeing, but I don't use any input method.

What I can say:

- I use rust-mode (not rust-ts-mode)
- I use eglot (config below)
- I use rust-analyzer 1.80.1 (3f5fd8d 2024-08-06)
- The problem seems to happen deep into a work day, after many
  commits, reverts, etc.

My eglot configuration:

#+begin_src elisp
  (setq
   eglot-extend-to-xref t
   eglot-ignored-server-capabilities
   '(
     :inlayHintProvider
     :documentOnTypeFormattingProvider
     )
   )
#+end_src

Eventually, this pops into *Messages*:

#+begin_quote
cl--assertion-failed: Assertion failed: (or track-changes--before-no (<= track-changes--before-beg (track-changes--state-beg track-changes--state) beg end (track-changes--state-end track-changes--state) track-changes--before-end))
#+end_quote

And this in *Warnings*:

#+begin_quote
⛔ Warning (emacs): Missing/incorrect calls to ‘before/after-change-functions’!!
Details logged to ‘track-changes--error-log’
#+end_quote

Over time, they just keep repeating, although I think killing the
buffer and reopening the file makes it stop.

If I have inspected ‘track-changes--error-log’ correctly, this is
what it contains (looks like some binary, hope it comes through
in some useful fashion or another):

#+begin_quote
(("lib.rs" (buffer-size 5254 5218) ((t track-changes--recover-from-error ... nil) (t track-changes-fetch ... nil) (t eglot--track-changes-fetch ... nil) (t eglot--signal-textDocument/didChange nil nil) (t eglot--signal-textDocument/didSave nil nil) (t run-hooks ... nil) (t basic-save-buffer ... nil) (t save-buffer ... nil) (t funcall-interactively ... nil) (t call-interactively ... nil) (t command-execute ... nil)) [111 (nil . other-window) 103 (nil . recompile) 121 (nil . undefined) 24 96 (nil . next-error) 1 (nil . move-beginning-of-line) 11 ...]) ("lib.rs" (buffer-size 5256 5254) ((t track-changes--recover-from-error ... nil) (t track-changes-fetch ... nil) (t eglot--track-changes-fetch ... nil) (t eglot--signal-textDocument/didChange nil nil) (t eglot--signal-textDocument/didSave nil nil) (t run-hooks ... nil) (t basic-save-buffer ... nil) (t save-buffer nil nil) (t #[257 "r\211q\210\300 )\207" [save-buffer] 2 ...] ... nil) (t map-y-or-n-p ... nil) (t save-some-buffers ... nil) (t recompile ... nil) ...) [19 (nil . save-buffer) 24 24 (nil . exchange-point-and-mark) 23 (nil . kill-region) 24 19 (nil . save-buffer) backspace (nil . delete-backward-char) ...]) ("lib.rs" (buffer-size 5278 5256) ((t track-changes--recover-from-error ... nil) (t track-changes-fetch ... nil) (t eglot--track-changes-fetch ... nil) (t eglot--signal-textDocument/didChange nil nil) (t eglot--signal-textDocument/didSave nil nil) (t run-hooks ... nil) (t basic-save-buffer ... nil) (t save-buffer ... nil) (t funcall-interactively ... nil) (t call-interactively ... nil) (t command-execute ... nil)) [(nil . backward-word) 2 (nil . backward-char) 67108896 (nil . set-mark-command) 134217734 (nil . forward-sexp) 23 (nil . kill-region) 24 19 (nil . save-buffer) ...]) ("lib.rs" (buffer-size 5324 5278) ((t track-changes--recover-from-error ... nil) (t track-changes-fetch ... nil) (t eglot--track-changes-fetch ... nil) (t eglot--track-changes-signal ... nil) (t #[771 "\211^BZ\211^HG^E]	
#+end_quote

I wish I could reliably repeat it.  Given that it occurs after
many hours of work, `emacs -Q` isn't an option.

Thanks!




In GNU Emacs 30.0.90 (build 9, x86_64-pc-linux-gnu, cairo version
 1.18.0) of 2024-09-03 built on dogato
Repository revision: ae2463796f236b8ee2cef3b5e38bffa13abd2233
Repository branch: emacs-30
Windowing system distributor 'The X.Org Foundation', version 11.0.12101012
System Description: openSUSE Tumbleweed

Configured using:
 'configure -C --prefix=/opt/emacs-30.0.90.5.ae2463796f2
 --disable-silent-rules --with-x-toolkit=no --without-gsettings
 --with-native-compilation=no'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS HARFBUZZ JPEG LCMS2
LIBSELINUX LIBXML2 MODULES NOTIFY INOTIFY OLDXMENU PDUMPER PNG
RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TREE_SITTER WEBP X11 XDBE
XIM XINPUT2 XPM ZLIB

Important settings:
  value of $LC_CTYPE: en_US.UTF-8
  value of $XMODIFIERS: @im=local
  locale-coding-system: utf-8-unix

Major mode: ELisp/d

Minor modes in effect:
  server-mode: t
  editorconfig-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr tabify cus-edit cus-start cus-load
cl-print help-fns radix-tree mh-utils emacsbug conf-mode
smerge-mode whitespace time-stamp perl-mode bug-reference
log-edit mule-util display-line-numbers dired-aux view pulse
color vc-annotate oc-basic org-element org-persist org-id
org-refile org-element-ast inline avl-tree generator ol-eww eww
xdg url-queue mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus
nnselect gnus-art mm-uu mml2015 gnus-sum shr pixel-fill kinsoku
url-file svg dom gnus-group gnus-undo gnus-start gnus-dbus dbus
xml gnus-cloud nnimap nnmail browse-url mail-source utf7 nnoo
parse-time gnus-spec gnus-int gnus-range message yank-media
rfc822 mailabbrev gmm-utils mailheader gnus-win gnus nnheader
range wid-edit ol-docview doc-view jka-compr image-mode exif
dired dired-loaddefs ol-bibtex bibtex iso8601 ol-bbdb ol-w3m
ol-doi org-link-doi org org-macro org-pcomplete org-list
org-footnote org-faces org-entities noutline outline org-version
ob-sql ob-emacs-lisp ob-C ob ob-tangle org-src sh-script smie
treesit executable ob-ref ob-lob ob-table ob-exp ob-comint
ob-core ob-eval cc-mode cc-fonts cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine cc-vars cc-defs org-cycle org-table
ol org-fold org-fold-core org-keys oc org-loaddefs cal-menu
calendar cal-loaddefs org-compat org-macs format-spec misearch
multi-isearch add-log cl-extra server shell pcomplete files-x
grep vc-cvs vc-rcs log-view pcvs-util vc-dir vc rust-utils
rust-mode rust-rustfmt rust-playpen rust-compile rust-cargo rx
eglot external-completion jsonrpc xref flymake thingatpt project
compat diff ert pp ewoc debug backtrace help-mode find-func
filenotify warnings compile comint ansi-osc ansi-color ring pcase
vc-git diff-mode track-changes easy-mmode vc-dispatcher url-cache
url url-proxy url-privacy url-expand url-methods url-history
url-cookie generate-lisp-file url-domsuf url-util url-parse
auth-source eieio eieio-core cl-macs icons json map byte-opt gv
bytecomp byte-compile url-vars mh-acros mh-folder which-func
imenu mh-comp sendmail mail-utils mh-scan mh-gnus mml mm-view
mml-smime mml-sec epa derived epg rfc6068 epg-config smime cl-seq
password-cache gnutls puny dig mailcap mm-decode mm-bodies
mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr gnus-util text-property-search time-date subr-x mh-e
mh-buffers mh-loaddefs editorconfig editorconfig-core
editorconfig-core-handle editorconfig-fnmatch cl-loaddefs cl-lib
rmc iso-transl tooltip cconv eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win
x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse
jit-lock font-lock syntax font-core term/tty-colors frame
minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting
font-render-setting cairo xinput2 x multi-tty move-toolbar
make-network-process emacs)

Memory information:
((conses 16 407345 108770) (symbols 48 32008 1) (strings 32 102422 6760)
 (string-bytes 1 2997195) (vectors 16 63439) (vector-slots 8 1388223 211290)
 (floats 8 478 2399) (intervals 56 29367 817) (buffers 992 73))

bug#73041: 30.0.90; track-changes-mode logs warnings (#70541 regression? not actually fixed?)

[Eli Zaretskii]

> From: epg@pretzelnet.org
> Date: Thu, 05 Sep 2024 08:28:53 -0500
> 
> I wish I could describe exactly what happened.  I found
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70541 which sounds
> very much like what I'm seeing, but I don't use any input method.
> 
> What I can say:
> 
> - I use rust-mode (not rust-ts-mode)
> - I use eglot (config below)
> - I use rust-analyzer 1.80.1 (3f5fd8d 2024-08-06)
> - The problem seems to happen deep into a work day, after many
>   commits, reverts, etc.
> 
> My eglot configuration:
> 
> #+begin_src elisp
>   (setq
>    eglot-extend-to-xref t
>    eglot-ignored-server-capabilities
>    '(
>      :inlayHintProvider
>      :documentOnTypeFormattingProvider
>      )
>    )
> #+end_src
> 
> Eventually, this pops into *Messages*:
> 
> #+begin_quote
> cl--assertion-failed: Assertion failed: (or track-changes--before-no (<= track-changes--before-beg (track-changes--state-beg track-changes--state) beg end (track-changes--state-end track-changes--state) track-changes--before-end))
> #+end_quote
> 
> And this in *Warnings*:
> 
> #+begin_quote
> ⛔ Warning (emacs): Missing/incorrect calls to ‘before/after-change-functions’!!
> Details logged to ‘track-changes--error-log’
> #+end_quote
> 
> Over time, they just keep repeating, although I think killing the
> buffer and reopening the file makes it stop.
> 
> If I have inspected ‘track-changes--error-log’ correctly, this is
> what it contains (looks like some binary, hope it comes through
> in some useful fashion or another):
> 
> #+begin_quote
> (("lib.rs" (buffer-size 5254 5218) ((t track-changes--recover-from-error ... nil) (t track-changes-fetch ... nil) (t eglot--track-changes-fetch ... nil) (t eglot--signal-textDocument/didChange nil nil) (t eglot--signal-textDocument/didSave nil nil) (t run-hooks ... nil) (t basic-save-buffer ... nil) (t save-buffer ... nil) (t funcall-interactively ... nil) (t call-interactively ... nil) (t command-execute ... nil)) [111 (nil . other-window) 103 (nil . recompile) 121 (nil . undefined) 24 96 (nil . next-error) 1 (nil . move-beginning-of-line) 11 ...]) ("lib.rs" (buffer-size 5256 5254) ((t track-changes--recover-from-error ... nil) (t track-changes-fetch ... nil) (t eglot--track-changes-fetch ... nil) (t eglot--signal-textDocument/didChange nil nil) (t eglot--signal-textDocument/didSave nil nil) (t run-hooks ... nil) (t basic-save-buffer ... nil) (t save-buffer nil nil) (t #[257 "r\211q\210\300 )\207" [save-buffer] 2 ...] ... nil) (t map-y-or-n-p ... nil) (t save-some-buffers ... nil) (t recompile ... nil) ...) [19 (nil . save-buffer) 24 24 (nil . exchange-point-and-mark) 23 (nil . kill-region) 24 19 (nil . save-buffer) backspace (nil . delete-backward-char) ...]) ("lib.rs" (buffer-size 5278 5256) ((t track-changes--recover-from-error ... nil) (t track-changes-fetch ... nil) (t eglot--track-changes-fetch ... nil) (t eglot--signal-textDocument/didChange nil nil) (t eglot--signal-textDocument/didSave nil nil) (t run-hooks ... nil) (t basic-save-buffer ... nil) (t save-buffer ... nil) (t funcall-interactively ... nil) (t call-interactively ... nil) (t command-execute ... nil)) [(nil . backward-word) 2 (nil . backward-char) 67108896 (nil . set-mark-command) 134217734 (nil . forward-sexp) 23 (nil . kill-region) 24 19 (nil . save-buffer) ...]) ("lib.rs" (buffer-size 5324 5278) ((t track-changes--recover-from-error ... nil) (t track-changes-fetch ... nil) (t eglot--track-changes-fetch ... nil) (t eglot--track-changes-signal ... nil) (t #[771 "\211^BZ\211^HG^E]	
> #+end_quote
> 
> I wish I could reliably repeat it.  Given that it occurs after
> many hours of work, `emacs -Q` isn't an option.

Adding Stefan.

bug#73041: 30.0.90; track-changes-mode logs warnings (#70541 regression? not actually fixed?)

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 5268 bytes --]

>> I wish I could describe exactly what happened.  I found
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70541 which sounds
>> very much like what I'm seeing, but I don't use any input method.

Thanks for your description of the problem.  A reproducible recipe is of
course better, but sadly, we can't always have what we want.
Let's hope we find that eventually we'll get what we need.

>> Eventually, this pops into *Messages*:
>> 
>> #+begin_quote
>> cl--assertion-failed: Assertion failed: (or track-changes--before-no (<=
>> track-changes--before-beg (track-changes--state-beg track-changes--state)
>> beg end (track-changes--state-end track-changes--state)
>> track-changes--before-end))
>> #+end_quote

Hmm... this clearly points to a bug in track-changes.

>> #+begin_quote
>> ⛔ Warning (emacs): Missing/incorrect calls to ‘before/after-change-functions’!!
>> Details logged to ‘track-changes--error-log’
>> #+end_quote

Barring bugs in track-changes, this points to a bug in some other part
of Emacs, which prevents track-changes from getting the info it needs.
Track-changes (and Eglot) are supposedly equipped to deal with such
problems, because we know we still have many of them.

>> Over time, they just keep repeating,

Hmm... `track-changes--recover-from-error` should (as the name suggests)
"recover" so they shouldn't just keep repeating, unless the source of
the problem keeps repeating as well.  Or unless there's a bug in
track-changes.

>> although I think killing the
>> buffer and reopening the file makes it stop.
>>
>> If I have inspected ‘track-changes--error-log’ correctly, this is
>> what it contains (looks like some binary, hope it comes through
>> in some useful fashion or another):

It's a list of debug info, one per "recovery".  Each debug info is made
of the buffer name, the inconsistency encountered, then a backtrace, and
an extract of (raw) `view-lossage`.  Here's your data reformatted:

    (("lib.rs" (buffer-size 5254 5218)
      ((t track-changes--recover-from-error ... nil)
       (t track-changes-fetch ... nil)
       (t eglot--track-changes-fetch ... nil)
       (t eglot--signal-textDocument/didChange nil nil)
       (t eglot--signal-textDocument/didSave nil nil)
       (t run-hooks ... nil)
       (t basic-save-buffer ... nil)
       (t save-buffer ... nil)
       (t funcall-interactively ... nil)
       (t call-interactively ... nil)
       (t command-execute ... nil))
      [111 (nil . other-window)
       103 (nil . recompile)
       121 (nil . undefined)
       24 96 (nil . next-error)
       1 (nil . move-beginning-of-line)
       11 ...])
     ("lib.rs" (buffer-size 5256 5254)
      ((t track-changes--recover-from-error ... nil)
       (t track-changes-fetch ... nil)
       (t eglot--track-changes-fetch ... nil)
       (t eglot--signal-textDocument/didChange nil nil)
       (t eglot--signal-textDocument/didSave nil nil)
       (t run-hooks ... nil)
       (t basic-save-buffer ... nil)
       (t save-buffer nil nil)
       (t #[257 "r\211q\210\300 )\207" [save-buffer] 2 ...]
       (t map-y-or-n-p ... nil)
       (t save-some-buffers ... nil)
       (t recompile ... nil)
       ...)
      [19 (nil . save-buffer)
       24 24 (nil . exchange-point-and-mark)
       23 (nil . kill-region)
       24 19 (nil . save-buffer)
       backspace (nil . delete-backward-char) ...])
     ("lib.rs" (buffer-size 5278 5256)
      ((t track-changes--recover-from-error ... nil)
       (t track-changes-fetch ... nil)
       (t eglot--track-changes-fetch ... nil)
       (t eglot--signal-textDocument/didChange nil nil)
       (t eglot--signal-textDocument/didSave nil nil)
       (t run-hooks ... nil)
       (t basic-save-buffer ... nil)
       (t save-buffer ... nil)
       (t funcall-interactively ... nil)
       (t call-interactively ... nil)
       (t command-execute ... nil))
      [(nil . backward-word)
       2 (nil . backward-char)
       67108896 (nil . set-mark-command)
       134217734 (nil . forward-sexp)
       23 (nil . kill-region)
       24 19 (nil . save-buffer)
       ...])
     ("lib.rs" (buffer-size 5324 5278)
      ((t track-changes--recover-from-error ... nil)
       (t track-changes-fetch ... nil)
       (t eglot--track-changes-fetch ... nil)
       (t eglot--track-changes-signal ... nil)
       (t #[771 "\211^BZ\211^HG^E] ...)
       ...)
      ...)
     ...)

This suggests the error is always detected "late" when we find that the
buffer-size is not the one we expected, so some buffer change presumably
happened without informing track-changes.
That doesn't explain the assertion failure.

Also, I don't see anything "unusual" in the view-lossage.

>> I wish I could reliably repeat it.  Given that it occurs after
>> many hours of work, `emacs -Q` isn't an option.

Thanks.
I stared at the code for a bit trying to see how that assertion failure
can occur, but nothing came to mind yet.  I think we'll need more data.

Can you try to install the patch below, then run with
`debug-on-error` enabled?
Hopefully when the error occurs, this will give us more info (ideally
including a backtrace).


        Stefan

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: track-changes.patch --]
[-- Type: text/x-diff, Size: 5150 bytes --]

diff --git a/lisp/emacs-lisp/track-changes.el b/lisp/emacs-lisp/track-changes.el
index 92d14959763..325a92317ca 100644
--- a/lisp/emacs-lisp/track-changes.el
+++ b/lisp/emacs-lisp/track-changes.el
@@ -229,6 +229,7 @@ track-changes-register
     (push tracker track-changes--clean-trackers)
     (when disjoint
       (push tracker track-changes--disjoint-trackers))
+    (cl-assert (track-changes--sane-state-p))
     tracker))
 
 (defun track-changes-unregister (id)
@@ -253,7 +254,8 @@ track-changes-unregister
             track-changes--buffer-size
             track-changes--before-clean
             track-changes--state))
-    (remove-hook 'after-change-functions  #'track-changes--after  t)))
+    (remove-hook 'after-change-functions  #'track-changes--after  t))
+  (cl-assert (track-changes--sane-state-p)))
 
 (defun track-changes-fetch (id func)
   "Fetch the pending changes for tracker ID pass them to FUNC.
@@ -372,7 +374,8 @@ track-changes-fetch
           (funcall func beg end (or before lenbefore)))
       ;; Re-enable the tracker's signal only after running `func', so
       ;; as to avoid nested invocations.
-      (cl-pushnew id track-changes--clean-trackers))))
+      (cl-pushnew id track-changes--clean-trackers)
+      (cl-assert (track-changes--sane-state-p)))))
 
 (defun track-changes-inconsistent-state-p ()
   "Return whether the current buffer is in an inconsistent state.
@@ -387,6 +390,19 @@ track-changes-inconsistent-state-p
 
 ;;;; Auxiliary functions.
 
+(defun track-changes--sane-state-p ()
+  (and (equal track-changes--buffer-size (buffer-size))
+       (or track-changes--before-no
+           (pcase track-changes--before-clean
+             ('unset t)
+             ('set (<= track-changes--before-beg track-changes--before-end))
+             ('nil
+              (<= track-changes--before-beg
+                  (track-changes--state-beg track-changes--state)
+                  (track-changes--state-end track-changes--state)
+                  track-changes--before-end))
+             (_ nil)))))
+
 (defun track-changes--clean-state ()
   (cond
    ((null track-changes--state)
@@ -551,7 +567,8 @@ track-changes--before
                                      (length track-changes--before-string))))))
               (setf track-changes--before-end new-bend)
               (cl-callf concat track-changes--before-string
-                (buffer-substring-no-properties old-bend new-bend)))))))))
+                (buffer-substring-no-properties old-bend new-bend))))))))
+  (cl-assert (track-changes--sane-state-p)))
 
 (defun track-changes--after (beg end len)
   (cl-assert track-changes--state)
@@ -576,10 +593,26 @@ track-changes--after
         ;; BEG..END is not covered by previous `before-change-functions'!!
         (track-changes--recover-from-error `(unexpected-after ,beg ,end ,len))
       ;; Note the new changes.
+      (let ((orig-beg (track-changes--state-beg track-changes--state))
+            (orig-end (track-changes--state-end track-changes--state)))
       (when (< beg (track-changes--state-beg track-changes--state))
         (setf (track-changes--state-beg track-changes--state) beg))
       (cl-callf (lambda (old-end) (max end (+ old-end offset)))
           (track-changes--state-end track-changes--state))
+      (cl-assert (or track-changes--before-no
+                     (<= track-changes--before-beg
+                         (track-changes--state-beg track-changes--state)))
+                 nil "<=? %S %S (was %S)"
+                 track-changes--before-beg
+                 (track-changes--state-beg track-changes--state)
+                 orig-beg)
+      (cl-assert (or track-changes--before-no
+                     (<= (track-changes--state-end track-changes--state)
+                         track-changes--before-end))
+                 nil "<=? %S %S (was %S)"
+                 (track-changes--state-end track-changes--state)
+                 track-changes--before-end
+                 orig-end))
       (cl-assert (or track-changes--before-no
                      (<= track-changes--before-beg
                          (track-changes--state-beg track-changes--state)
@@ -591,7 +624,8 @@ track-changes--after
       (if (track-changes--tracker-immediate tracker)
           (funcall (track-changes--tracker-signal tracker) tracker)
         (run-with-timer 0 nil #'track-changes--call-signal
-                        (current-buffer) tracker)))))
+         (current-buffer) tracker))))
+  (cl-assert (track-changes--sane-state-p)))
 
 (defun track-changes--call-signal (buf tracker)
   (when (buffer-live-p buf)
diff --git a/lisp/progmodes/eglot.el b/lisp/progmodes/eglot.el
index 82e99a2c920..a2c9f73fc73 100644
--- a/lisp/progmodes/eglot.el
+++ b/lisp/progmodes/eglot.el
@@ -2813,6 +2813,8 @@ eglot--signal-textDocument/didChange
 
 (defun eglot--signal-textDocument/didOpen ()
   "Send textDocument/didOpen to server."
+  ;; Flush any potential pending change.
+  (eglot--track-changes-fetch eglot--track-changes)
   (setq eglot--recent-changes nil
         eglot--versioned-identifier 0
         eglot--TextDocumentIdentifier-cache nil)

bug#73041: 30.0.90; track-changes-mode logs warnings (#70541 regression? not actually fixed?)

[Eli Zaretskii]

Ping! epg, could you please try Stefan's patches and report back?

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: epg@pretzelnet.org,  73041@debbugs.gnu.org
> Date: Sun, 08 Sep 2024 19:06:09 -0400
> 
> 
> >> I wish I could describe exactly what happened.  I found
> >> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70541 which sounds
> >> very much like what I'm seeing, but I don't use any input method.
> 
> Thanks for your description of the problem.  A reproducible recipe is of
> course better, but sadly, we can't always have what we want.
> Let's hope we find that eventually we'll get what we need.
> 
> >> Eventually, this pops into *Messages*:
> >> 
> >> #+begin_quote
> >> cl--assertion-failed: Assertion failed: (or track-changes--before-no (<=
> >> track-changes--before-beg (track-changes--state-beg track-changes--state)
> >> beg end (track-changes--state-end track-changes--state)
> >> track-changes--before-end))
> >> #+end_quote
> 
> Hmm... this clearly points to a bug in track-changes.
> 
> >> #+begin_quote
> >> ⛔ Warning (emacs): Missing/incorrect calls to ‘before/after-change-functions’!!
> >> Details logged to ‘track-changes--error-log’
> >> #+end_quote
> 
> Barring bugs in track-changes, this points to a bug in some other part
> of Emacs, which prevents track-changes from getting the info it needs.
> Track-changes (and Eglot) are supposedly equipped to deal with such
> problems, because we know we still have many of them.
> 
> >> Over time, they just keep repeating,
> 
> Hmm... `track-changes--recover-from-error` should (as the name suggests)
> "recover" so they shouldn't just keep repeating, unless the source of
> the problem keeps repeating as well.  Or unless there's a bug in
> track-changes.
> 
> >> although I think killing the
> >> buffer and reopening the file makes it stop.
> >>
> >> If I have inspected ‘track-changes--error-log’ correctly, this is
> >> what it contains (looks like some binary, hope it comes through
> >> in some useful fashion or another):
> 
> It's a list of debug info, one per "recovery".  Each debug info is made
> of the buffer name, the inconsistency encountered, then a backtrace, and
> an extract of (raw) `view-lossage`.  Here's your data reformatted:
> 
>     (("lib.rs" (buffer-size 5254 5218)
>       ((t track-changes--recover-from-error ... nil)
>        (t track-changes-fetch ... nil)
>        (t eglot--track-changes-fetch ... nil)
>        (t eglot--signal-textDocument/didChange nil nil)
>        (t eglot--signal-textDocument/didSave nil nil)
>        (t run-hooks ... nil)
>        (t basic-save-buffer ... nil)
>        (t save-buffer ... nil)
>        (t funcall-interactively ... nil)
>        (t call-interactively ... nil)
>        (t command-execute ... nil))
>       [111 (nil . other-window)
>        103 (nil . recompile)
>        121 (nil . undefined)
>        24 96 (nil . next-error)
>        1 (nil . move-beginning-of-line)
>        11 ...])
>      ("lib.rs" (buffer-size 5256 5254)
>       ((t track-changes--recover-from-error ... nil)
>        (t track-changes-fetch ... nil)
>        (t eglot--track-changes-fetch ... nil)
>        (t eglot--signal-textDocument/didChange nil nil)
>        (t eglot--signal-textDocument/didSave nil nil)
>        (t run-hooks ... nil)
>        (t basic-save-buffer ... nil)
>        (t save-buffer nil nil)
>        (t #[257 "r\211q\210\300 )\207" [save-buffer] 2 ...]
>        (t map-y-or-n-p ... nil)
>        (t save-some-buffers ... nil)
>        (t recompile ... nil)
>        ...)
>       [19 (nil . save-buffer)
>        24 24 (nil . exchange-point-and-mark)
>        23 (nil . kill-region)
>        24 19 (nil . save-buffer)
>        backspace (nil . delete-backward-char) ...])
>      ("lib.rs" (buffer-size 5278 5256)
>       ((t track-changes--recover-from-error ... nil)
>        (t track-changes-fetch ... nil)
>        (t eglot--track-changes-fetch ... nil)
>        (t eglot--signal-textDocument/didChange nil nil)
>        (t eglot--signal-textDocument/didSave nil nil)
>        (t run-hooks ... nil)
>        (t basic-save-buffer ... nil)
>        (t save-buffer ... nil)
>        (t funcall-interactively ... nil)
>        (t call-interactively ... nil)
>        (t command-execute ... nil))
>       [(nil . backward-word)
>        2 (nil . backward-char)
>        67108896 (nil . set-mark-command)
>        134217734 (nil . forward-sexp)
>        23 (nil . kill-region)
>        24 19 (nil . save-buffer)
>        ...])
>      ("lib.rs" (buffer-size 5324 5278)
>       ((t track-changes--recover-from-error ... nil)
>        (t track-changes-fetch ... nil)
>        (t eglot--track-changes-fetch ... nil)
>        (t eglot--track-changes-signal ... nil)
>        (t #[771 "\211^BZ\211^HG^E] ...)
>        ...)
>       ...)
>      ...)
> 
> This suggests the error is always detected "late" when we find that the
> buffer-size is not the one we expected, so some buffer change presumably
> happened without informing track-changes.
> That doesn't explain the assertion failure.
> 
> Also, I don't see anything "unusual" in the view-lossage.
> 
> >> I wish I could reliably repeat it.  Given that it occurs after
> >> many hours of work, `emacs -Q` isn't an option.
> 
> Thanks.
> I stared at the code for a bit trying to see how that assertion failure
> can occur, but nothing came to mind yet.  I think we'll need more data.
> 
> Can you try to install the patch below, then run with
> `debug-on-error` enabled?
> Hopefully when the error occurs, this will give us more info (ideally
> including a backtrace).
> 
> 
>         Stefan
> 
> diff --git a/lisp/emacs-lisp/track-changes.el b/lisp/emacs-lisp/track-changes.el
> index 92d14959763..325a92317ca 100644
> --- a/lisp/emacs-lisp/track-changes.el
> +++ b/lisp/emacs-lisp/track-changes.el
> @@ -229,6 +229,7 @@ track-changes-register
>      (push tracker track-changes--clean-trackers)
>      (when disjoint
>        (push tracker track-changes--disjoint-trackers))
> +    (cl-assert (track-changes--sane-state-p))
>      tracker))
>  
>  (defun track-changes-unregister (id)
> @@ -253,7 +254,8 @@ track-changes-unregister
>              track-changes--buffer-size
>              track-changes--before-clean
>              track-changes--state))
> -    (remove-hook 'after-change-functions  #'track-changes--after  t)))
> +    (remove-hook 'after-change-functions  #'track-changes--after  t))
> +  (cl-assert (track-changes--sane-state-p)))
>  
>  (defun track-changes-fetch (id func)
>    "Fetch the pending changes for tracker ID pass them to FUNC.
> @@ -372,7 +374,8 @@ track-changes-fetch
>            (funcall func beg end (or before lenbefore)))
>        ;; Re-enable the tracker's signal only after running `func', so
>        ;; as to avoid nested invocations.
> -      (cl-pushnew id track-changes--clean-trackers))))
> +      (cl-pushnew id track-changes--clean-trackers)
> +      (cl-assert (track-changes--sane-state-p)))))
>  
>  (defun track-changes-inconsistent-state-p ()
>    "Return whether the current buffer is in an inconsistent state.
> @@ -387,6 +390,19 @@ track-changes-inconsistent-state-p
>  
>  ;;;; Auxiliary functions.
>  
> +(defun track-changes--sane-state-p ()
> +  (and (equal track-changes--buffer-size (buffer-size))
> +       (or track-changes--before-no
> +           (pcase track-changes--before-clean
> +             ('unset t)
> +             ('set (<= track-changes--before-beg track-changes--before-end))
> +             ('nil
> +              (<= track-changes--before-beg
> +                  (track-changes--state-beg track-changes--state)
> +                  (track-changes--state-end track-changes--state)
> +                  track-changes--before-end))
> +             (_ nil)))))
> +
>  (defun track-changes--clean-state ()
>    (cond
>     ((null track-changes--state)
> @@ -551,7 +567,8 @@ track-changes--before
>                                       (length track-changes--before-string))))))
>                (setf track-changes--before-end new-bend)
>                (cl-callf concat track-changes--before-string
> -                (buffer-substring-no-properties old-bend new-bend)))))))))
> +                (buffer-substring-no-properties old-bend new-bend))))))))
> +  (cl-assert (track-changes--sane-state-p)))
>  
>  (defun track-changes--after (beg end len)
>    (cl-assert track-changes--state)
> @@ -576,10 +593,26 @@ track-changes--after
>          ;; BEG..END is not covered by previous `before-change-functions'!!
>          (track-changes--recover-from-error `(unexpected-after ,beg ,end ,len))
>        ;; Note the new changes.
> +      (let ((orig-beg (track-changes--state-beg track-changes--state))
> +            (orig-end (track-changes--state-end track-changes--state)))
>        (when (< beg (track-changes--state-beg track-changes--state))
>          (setf (track-changes--state-beg track-changes--state) beg))
>        (cl-callf (lambda (old-end) (max end (+ old-end offset)))
>            (track-changes--state-end track-changes--state))
> +      (cl-assert (or track-changes--before-no
> +                     (<= track-changes--before-beg
> +                         (track-changes--state-beg track-changes--state)))
> +                 nil "<=? %S %S (was %S)"
> +                 track-changes--before-beg
> +                 (track-changes--state-beg track-changes--state)
> +                 orig-beg)
> +      (cl-assert (or track-changes--before-no
> +                     (<= (track-changes--state-end track-changes--state)
> +                         track-changes--before-end))
> +                 nil "<=? %S %S (was %S)"
> +                 (track-changes--state-end track-changes--state)
> +                 track-changes--before-end
> +                 orig-end))
>        (cl-assert (or track-changes--before-no
>                       (<= track-changes--before-beg
>                           (track-changes--state-beg track-changes--state)
> @@ -591,7 +624,8 @@ track-changes--after
>        (if (track-changes--tracker-immediate tracker)
>            (funcall (track-changes--tracker-signal tracker) tracker)
>          (run-with-timer 0 nil #'track-changes--call-signal
> -                        (current-buffer) tracker)))))
> +         (current-buffer) tracker))))
> +  (cl-assert (track-changes--sane-state-p)))
>  
>  (defun track-changes--call-signal (buf tracker)
>    (when (buffer-live-p buf)
> diff --git a/lisp/progmodes/eglot.el b/lisp/progmodes/eglot.el
> index 82e99a2c920..a2c9f73fc73 100644
> --- a/lisp/progmodes/eglot.el
> +++ b/lisp/progmodes/eglot.el
> @@ -2813,6 +2813,8 @@ eglot--signal-textDocument/didChange
>  
>  (defun eglot--signal-textDocument/didOpen ()
>    "Send textDocument/didOpen to server."
> +  ;; Flush any potential pending change.
> +  (eglot--track-changes-fetch eglot--track-changes)
>    (setq eglot--recent-changes nil
>          eglot--versioned-identifier 0
>          eglot--TextDocumentIdentifier-cache nil)

bug#73041: 30.0.90; track-changes-mode logs warnings (#70541 regression? not actually fixed?)

[Eric Gillespie]

Eli Zaretskii <eliz@gnu.org> writes:

> Ping! epg, could you please try Stefan's patches and report back?

Sorry for the delay.  I applied that patch the next time I built,
and it definitely had an impact, but not what I expected.

I now have a trivial -Q test case for a bug, though I fear it may
not be the bug I originally reported here.

1. Run 'cargo new foo' to create a new cargo project
2. emacs -Q foo/src/main.rs
3. M-x rust-ts-mode RET
4. M-x eglot RET
5. C-x k RET

Expected:
Buffer main.rs is killed.

Actual:
(cl-assert (track-changes--sane-state-p)) assertion triggered.

Who knows, maybe these two bugs share a root cause.

> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Hmm... this clearly points to a bug in track-changes.

In fact, I was able to trace this kill-buffer behavior all the
way back to the birth of track-changes in commit
d7a83e23d47ca9e3e6ca70078e956e31301e5e6d (the patch needed a tiny
bit of conflict resolution to apply that far back, but it was no
problem).

Thanks!

-- 
Eric Gillespie <*> epg@pretzelnet.org

bug#73041: 30.0.90; track-changes-mode logs warnings (#70541 regression? not actually fixed?)

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 319 bytes --]

> 1. Run 'cargo new foo' to create a new cargo project
> 2. emacs -Q foo/src/main.rs
> 3. M-x rust-ts-mode RET
> 4. M-x eglot RET
> 5. C-x k RET
>
> Expected:
> Buffer main.rs is killed.
>
> Actual:
> (cl-assert (track-changes--sane-state-p)) assertion triggered.

Duh, sorry.  Try this patch instead.


        Stefan

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: track-changes.patch --]
[-- Type: text/x-diff, Size: 5856 bytes --]

diff --git a/lisp/emacs-lisp/track-changes.el b/lisp/emacs-lisp/track-changes.el
index 92d14959763..24df0aac6bb 100644
--- a/lisp/emacs-lisp/track-changes.el
+++ b/lisp/emacs-lisp/track-changes.el
@@ -213,6 +213,7 @@ track-changes-register
 that may block, do as little work as possible, ...
 When IMMEDIATE is non-nil, the SIGNAL should probably not always call
 `track-changes-fetch', since that would defeat the purpose of this library."
+  (cl-assert (track-changes--sane-state-p))
   (when (and nobefore disjoint)
     ;; FIXME: Without `before-change-functions', we can discover
     ;; a disjoint change only after the fact, which is not good enough.
@@ -229,6 +230,7 @@ track-changes-register
     (push tracker track-changes--clean-trackers)
     (when disjoint
       (push tracker track-changes--disjoint-trackers))
+    (cl-assert (track-changes--sane-state-p))
     tracker))
 
 (defun track-changes-unregister (id)
@@ -236,6 +238,7 @@ track-changes-unregister
 Trackers can consume resources (especially if `track-changes-fetch' is
 not called), so it is good practice to unregister them when you don't
 need them any more."
+  (cl-assert (track-changes--sane-state-p))
   (unless (memq id track-changes--trackers)
     (error "Unregistering a non-registered tracker: %S" id))
   (setq track-changes--trackers (delq id track-changes--trackers))
@@ -253,7 +256,8 @@ track-changes-unregister
             track-changes--buffer-size
             track-changes--before-clean
             track-changes--state))
-    (remove-hook 'after-change-functions  #'track-changes--after  t)))
+    (remove-hook 'after-change-functions  #'track-changes--after  t))
+  (cl-assert (track-changes--sane-state-p)))
 
 (defun track-changes-fetch (id func)
   "Fetch the pending changes for tracker ID pass them to FUNC.
@@ -372,7 +376,8 @@ track-changes-fetch
           (funcall func beg end (or before lenbefore)))
       ;; Re-enable the tracker's signal only after running `func', so
       ;; as to avoid nested invocations.
-      (cl-pushnew id track-changes--clean-trackers))))
+      (cl-pushnew id track-changes--clean-trackers)
+      (cl-assert (track-changes--sane-state-p)))))
 
 (defun track-changes-inconsistent-state-p ()
   "Return whether the current buffer is in an inconsistent state.
@@ -387,6 +392,26 @@ track-changes-inconsistent-state-p
 
 ;;;; Auxiliary functions.
 
+(defun track-changes--sane-state-p ()
+  (if (null track-changes--trackers)
+      (and (zerop track-changes--before-beg)
+           (zerop track-changes--before-end)
+           (equal track-changes--before-string "")
+           (null track-changes--buffer-size)
+           (eq track-changes--before-clean 'unset)
+           (null track-changes--state))
+    (and (equal track-changes--buffer-size (buffer-size))
+         (or track-changes--before-no
+             (pcase track-changes--before-clean
+               ('unset t)
+               ('set (<= track-changes--before-beg track-changes--before-end))
+               ('nil
+                (<= track-changes--before-beg
+                    (track-changes--state-beg track-changes--state)
+                    (track-changes--state-end track-changes--state)
+                    track-changes--before-end))
+               (_ nil))))))
+
 (defun track-changes--clean-state ()
   (cond
    ((null track-changes--state)
@@ -551,7 +576,8 @@ track-changes--before
                                      (length track-changes--before-string))))))
               (setf track-changes--before-end new-bend)
               (cl-callf concat track-changes--before-string
-                (buffer-substring-no-properties old-bend new-bend)))))))))
+                (buffer-substring-no-properties old-bend new-bend))))))))
+  (cl-assert (track-changes--sane-state-p)))
 
 (defun track-changes--after (beg end len)
   (cl-assert track-changes--state)
@@ -576,10 +602,26 @@ track-changes--after
         ;; BEG..END is not covered by previous `before-change-functions'!!
         (track-changes--recover-from-error `(unexpected-after ,beg ,end ,len))
       ;; Note the new changes.
+      (let ((orig-beg (track-changes--state-beg track-changes--state))
+            (orig-end (track-changes--state-end track-changes--state)))
       (when (< beg (track-changes--state-beg track-changes--state))
         (setf (track-changes--state-beg track-changes--state) beg))
       (cl-callf (lambda (old-end) (max end (+ old-end offset)))
           (track-changes--state-end track-changes--state))
+      (cl-assert (or track-changes--before-no
+                     (<= track-changes--before-beg
+                         (track-changes--state-beg track-changes--state)))
+                 nil "<=? %S %S (was %S)"
+                 track-changes--before-beg
+                 (track-changes--state-beg track-changes--state)
+                 orig-beg)
+      (cl-assert (or track-changes--before-no
+                     (<= (track-changes--state-end track-changes--state)
+                         track-changes--before-end))
+                 nil "<=? %S %S (was %S)"
+                 (track-changes--state-end track-changes--state)
+                 track-changes--before-end
+                 orig-end))
       (cl-assert (or track-changes--before-no
                      (<= track-changes--before-beg
                          (track-changes--state-beg track-changes--state)
@@ -591,7 +633,8 @@ track-changes--after
       (if (track-changes--tracker-immediate tracker)
           (funcall (track-changes--tracker-signal tracker) tracker)
         (run-with-timer 0 nil #'track-changes--call-signal
-                        (current-buffer) tracker)))))
+         (current-buffer) tracker))))
+  (cl-assert (track-changes--sane-state-p)))
 
 (defun track-changes--call-signal (buf tracker)
   (when (buffer-live-p buf)

bug#73041: 30.0.90; track-changes-mode logs warnings (#70541 regression? not actually fixed?)

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 500 bytes --]

Hi Eric,

> Duh, sorry.  Try this patch instead.

I pushed to Emacs `master` a patch to `track-changes.el` which doesn't
fix any bug but allows it to record more information to help find the
root cause of errors.

If you're able to try Emacs `master` (which would be helpful), then the
patch I sent doesn't apply to it (any more), so please use the patch
below instead and add `(setq track-changes-record-errors 'trace)` to your
init file so as to activate the additional recording.


        Stefan

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: track-changes.patch --]
[-- Type: text/x-diff, Size: 5836 bytes --]

diff --git a/lisp/emacs-lisp/track-changes.el b/lisp/emacs-lisp/track-changes.el
index 1b0f64f544d..245b7b89ae5 100644
--- a/lisp/emacs-lisp/track-changes.el
+++ b/lisp/emacs-lisp/track-changes.el
@@ -219,6 +219,7 @@ track-changes-register
 When IMMEDIATE is non-nil, the SIGNAL should probably not always call
 `track-changes-fetch', since that would defeat the purpose of this library."
   (track-changes--trace)
+  (cl-assert (track-changes--sane-state-p))
   (when (and nobefore disjoint)
     ;; FIXME: Without `before-change-functions', we can discover
     ;; a disjoint change only after the fact, which is not good enough.
@@ -235,6 +236,7 @@ track-changes-register
     (push tracker track-changes--clean-trackers)
     (when disjoint
       (push tracker track-changes--disjoint-trackers))
+    (cl-assert (track-changes--sane-state-p))
     tracker))
 
 (defun track-changes-unregister (id)
@@ -243,6 +245,7 @@ track-changes-unregister
 not called), so it is good practice to unregister them when you don't
 need them any more."
   (track-changes--trace)
+  (cl-assert (track-changes--sane-state-p))
   (unless (memq id track-changes--trackers)
     (error "Unregistering a non-registered tracker: %S" id))
   (setq track-changes--trackers (delq id track-changes--trackers))
@@ -260,7 +263,8 @@ track-changes-unregister
             track-changes--buffer-size
             track-changes--before-clean
             track-changes--state))
-    (remove-hook 'after-change-functions  #'track-changes--after  t)))
+    (remove-hook 'after-change-functions  #'track-changes--after  t))
+  (cl-assert (track-changes--sane-state-p)))
 
 (defun track-changes-fetch (id func)
   "Fetch the pending changes for tracker ID pass them to FUNC.
@@ -380,7 +384,8 @@ track-changes-fetch
           (funcall func beg end (or before lenbefore)))
       ;; Re-enable the tracker's signal only after running `func', so
       ;; as to avoid nested invocations.
-      (cl-pushnew id track-changes--clean-trackers))))
+      (cl-pushnew id track-changes--clean-trackers)
+      (cl-assert (track-changes--sane-state-p)))))
 
 (defun track-changes-inconsistent-state-p ()
   "Return whether the current buffer is in an inconsistent state.
@@ -418,6 +423,26 @@ track-changes--trace
                        (track-change--backtrace
                         10 #'track-changes--trace)))))
 
+(defun track-changes--sane-state-p ()
+  (if (null track-changes--trackers)
+      (and (zerop track-changes--before-beg)
+           (zerop track-changes--before-end)
+           (equal track-changes--before-string "")
+           (null track-changes--buffer-size)
+           (eq track-changes--before-clean 'unset)
+           (null track-changes--state))
+    (and (equal track-changes--buffer-size (buffer-size))
+         (or track-changes--before-no
+             (pcase track-changes--before-clean
+               ('unset t)
+               ('set (<= track-changes--before-beg track-changes--before-end))
+               ('nil
+                (<= track-changes--before-beg
+                    (track-changes--state-beg track-changes--state)
+                    (track-changes--state-end track-changes--state)
+                    track-changes--before-end))
+               (_ nil))))))
+
 (defun track-changes--clean-state ()
   (cond
    ((null track-changes--state)
@@ -586,7 +611,8 @@ track-changes--before
                                      (length track-changes--before-string))))))
               (setf track-changes--before-end new-bend)
               (cl-callf concat track-changes--before-string
-                (buffer-substring-no-properties old-bend new-bend)))))))))
+                (buffer-substring-no-properties old-bend new-bend))))))))
+  (cl-assert (track-changes--sane-state-p)))
 
 (defun track-changes--after (beg end len)
   (track-changes--trace)
@@ -612,10 +638,26 @@ track-changes--after
         ;; BEG..END is not covered by previous `before-change-functions'!!
         (track-changes--recover-from-error `(unexpected-after ,beg ,end ,len))
       ;; Note the new changes.
+      (let ((orig-beg (track-changes--state-beg track-changes--state))
+            (orig-end (track-changes--state-end track-changes--state)))
       (when (< beg (track-changes--state-beg track-changes--state))
         (setf (track-changes--state-beg track-changes--state) beg))
       (cl-callf (lambda (old-end) (max end (+ old-end offset)))
           (track-changes--state-end track-changes--state))
+      (cl-assert (or track-changes--before-no
+                     (<= track-changes--before-beg
+                         (track-changes--state-beg track-changes--state)))
+                 nil "<=? %S %S (was %S)"
+                 track-changes--before-beg
+                 (track-changes--state-beg track-changes--state)
+                 orig-beg)
+      (cl-assert (or track-changes--before-no
+                     (<= (track-changes--state-end track-changes--state)
+                         track-changes--before-end))
+                 nil "<=? %S %S (was %S)"
+                 (track-changes--state-end track-changes--state)
+                 track-changes--before-end
+                 orig-end))
       (cl-assert (or track-changes--before-no
                      (<= track-changes--before-beg
                          (track-changes--state-beg track-changes--state)
@@ -627,7 +669,8 @@ track-changes--after
       (if (track-changes--tracker-immediate tracker)
           (funcall (track-changes--tracker-signal tracker) tracker)
         (run-with-timer 0 nil #'track-changes--call-signal
-                        (current-buffer) tracker)))))
+         (current-buffer) tracker))))
+  (cl-assert (track-changes--sane-state-p)))
 
 (defun track-changes--call-signal (buf tracker)
   (when (buffer-live-p buf)

bug#73041: 30.0.90; track-changes-mode logs warnings (#70541 regression? not actually fixed?)

[Eric Gillespie]

Hi Stefan!

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> If you're able to try Emacs `master` (which would be helpful), then the
> patch I sent doesn't apply to it (any more), so please use the patch
> below instead and add `(setq track-changes-record-errors 'trace)` to your
> init file so as to activate the additional recording.

I applied that patch to master and have been using it every day.
This morning I FINALLY hit it...  on a NEW FILE, where I'm
switching in and out of a branch with a new file.  I'm not sure
all the previous occurrences were on new files, but I could
believe they were.

Anyway, with that clue in hand, I was able to narrow it down to a
simple reproduction recipe.

I'm running emacs built from this commit to master:

commit ef587bf6b46b2ea3ef91b260ac2542666081260d
Author: Stefan Monnier <monnier@iro.umontreal.ca>
Date:   Thu Oct 3 13:32:09 2024

With your cl-assert patch applied on top of that.

GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, cairo version 1.18.2) of 2024-10-04

(We won't use git in the recipe, but eglot requires project-root
which I can't get to work without git (which also seems like a
bug, albeit unrelated...))

 1. cargo new --vcs git foo
 2. echo 'mod foo;' >> foo/src/main.rs
 3. emacs -Q foo/src/foo.rs
    (foo.rs DOES NOT EXIST YET -- that is part of the bug)
 4. M-x rust-ts-mode RET
 5. (progn (setq track-changes-record-errors 'trace) (setq debug-on-error t))
 6. C-x C-f main.rs RET
 7. M-x eglot RET
 8. echo 'struct foo;' > foo/src/foo.rs
 9. C-x C-f foo.rs RET
10. yes RET

BOOM TODAY

Note that #8 must be an edit behind emacs's back.  In my daily
life, what's happening there is I had the buffer 'foo.rs' open
earlier while working on the branch where I added it, I'd
switched to a branch without it, and then I switch back, and
everything fell apart when I reverted it.

Debugger entered--Lisp error: (cl-assertion-failed ((track-changes--sane-state-p) nil))
  cl--assertion-failed((track-changes--sane-state-p))
  track-changes--before(1 13)
  track-changes--after(1 13 0)
  insert-file-contents("/home/epg/tmp/emacsbug73041/foo/src/foo.rs" t nil nil if-regular)
  revert-buffer-insert-file-contents--default-function("/home/epg/tmp/emacsbug73041/foo/src/foo.rs" nil)
  revert-buffer--default(t t)
  revert-buffer(t t)
  find-file-noselect("~/tmp/emacsbug73041/foo/src/foo.rs" nil nil t)
  find-file("~/tmp/emacsbug73041/foo/src/foo.rs" t)
  funcall-interactively(find-file "~/tmp/emacsbug73041/foo/src/foo.rs" t)
  call-interactively(find-file nil nil)
  command-execute(find-file)

I hope this is enough to go on now.

Thanks!

-- 
Eric Gillespie <*> epg@pretzelnet.org

bug#73041: 30.0.90; track-changes-mode logs warnings (#70541 regression? not actually fixed?)

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 1083 bytes --]

>  1. cargo new --vcs git foo
>  2. echo 'mod foo;' >> foo/src/main.rs
>  3. emacs -Q foo/src/foo.rs
>     (foo.rs DOES NOT EXIST YET -- that is part of the bug)
>  4. M-x rust-ts-mode RET
>  5. (progn (setq track-changes-record-errors 'trace) (setq debug-on-error t))
>  6. C-x C-f main.rs RET
>  7. M-x eglot RET
>  8. echo 'struct foo;' > foo/src/foo.rs
>  9. C-x C-f foo.rs RET
> 10. yes RET

Thanks, I managed to reproduce it locally with:

    rm foo.c
    emacs -Q foo.c -f eglot
    echo 'struct foo;' > foo.c
    M-x revert-buffer RET

and that pointed to a bug in `track-changes.el` which could explain the
problem you reported originally.

So, I pushed the patch below to `emacs-30`.

It seems part of the problem also comes from `insert-file-contents`
which apparently fails to run the `before-change-functions` in the above
recipe, but I haven't investigated this any further yet, and
track-changes *should* tolerate this (potentially emitting a warning
along the way), and with the patch, the above recipe now works for me
without error nor warning.


        Stefan



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: track-changes.patch --]
[-- Type: text/x-diff, Size: 1402 bytes --]

diff --git a/lisp/emacs-lisp/track-changes.el b/lisp/emacs-lisp/track-changes.el
index 92d14959763..1d5c5e9917a 100644
--- a/lisp/emacs-lisp/track-changes.el
+++ b/lisp/emacs-lisp/track-changes.el
@@ -555,16 +555,16 @@ track-changes--before
 
 (defun track-changes--after (beg end len)
   (cl-assert track-changes--state)
-  (and (eq track-changes--before-clean 'unset)
-       (not track-changes--before-no)
-       ;; This can be a sign that a `before-change-functions' went missing,
-       ;; or that we called `track-changes--clean-state' between
-       ;; a `before-change-functions' and `after-change-functions'.
-       (track-changes--before beg end))
-  (setq track-changes--before-clean nil)
   (let ((offset (- (- end beg) len)))
-    (cl-incf track-changes--before-end offset)
     (cl-incf track-changes--buffer-size offset)
+    (if (and (eq track-changes--before-clean 'unset)
+             (not track-changes--before-no))
+         ;; This can be a sign that a `before-change-functions' went missing,
+         ;; or that we called `track-changes--clean-state' between
+         ;; a `before-change-functions' and `after-change-functions'.
+         (track-changes--before beg end)
+      (cl-incf track-changes--before-end offset))
+    (setq track-changes--before-clean nil)
     (if (not (or track-changes--before-no
                  (save-restriction
                    (widen)