bug#64321: 29.0.92; Tree-Sitter/which-function Narrow/Widen causes modified buffer

bug#64321: 29.0.92; Tree-Sitter/which-function Narrow/Widen causes modified buffer

[Troy Brown]

This problem seems to manifest with multiple tree-sitter modes.  It
appears there is an interaction with which-function-mode and tree-sitter
which causes the buffer to be considered modified either after a narrow
or widen of the buffer.

To reproduce this, use a "hello_world.c" as follows:

--8<---------------cut here---------------start------------->8---
#include <stdio.h>

int main (void)
{
  printf("Hello, world!\n");
  return 0;
}
--8<---------------cut here---------------end--------------->8---

Running "emacs -Q" with the corresponding tree-sitter grammar library
installed.  Move into the main function and narrow-to-defun.  In some
modes, I've seen this cause the buffer to show as modified, however with
this example, it seems to occur after widening.

The following sequence of commands can be used to reproduce the issue:

C-x C-f ~/hello_world.c
M-x c-ts-mode
M-x which-function-mode
C-x n d
C-x n w

At this point, the modeline shows that the buffer is modified, and
running "M-: (buffer-modified-p)" indicates "t".  As previously
mentioned, this appears to only happen with tree-sitter modes when
which-function-mode is enabled in the buffer.

I then repeated the same but also triggered a backtrace on the first
change as follows:

M-: (add-to-list 'first-change-hook #'backtrace)

With that in place, the following is the generated traceback:

  backtrace()
  treesit--font-lock-notifier(((1 . 21)) #<treesit-parser for c>)
  treesit-buffer-root-node(c)
  treesit-node-at(68)
  treesit--things-around(68
"\\(?:class_specifier\\|enum_specifier\\|function_defi..."
c-ts-mode--defun-valid-p)
  treesit-thing-at-point(("\\(?:class_specifier\\|enum_specifier\\|function_defi..."
. c-ts-mode--defun-valid-p) top-level)
  treesit-defun-at-point()
  treesit-add-log-current-defun()
  c-ts-mode--emacs-current-defun-name()
  add-log-current-defun()
  which-function()
  which-func-update-1(#<window 3 on hello_world.c>)
  which-func-update()
  apply(which-func-update nil)
  timer-event-handler([t 0 0 500000 t which-func-update nil idle 0 nil])

bug#64321: 29.0.92; Tree-Sitter/which-function Narrow/Widen causes modified buffer

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 2881 bytes --]



> On Jun 27, 2023, at 7:23 PM, Troy Brown <brownts@troybrown.dev> wrote:
> 
> This problem seems to manifest with multiple tree-sitter modes.  It
> appears there is an interaction with which-function-mode and tree-sitter
> which causes the buffer to be considered modified either after a narrow
> or widen of the buffer.
> 
> To reproduce this, use a "hello_world.c" as follows:
> 
> --8<---------------cut here---------------start------------->8---
> #include <stdio.h>
> 
> int main (void)
> {
>  printf("Hello, world!\n");
>  return 0;
> }
> --8<---------------cut here---------------end--------------->8---
> 
> Running "emacs -Q" with the corresponding tree-sitter grammar library
> installed.  Move into the main function and narrow-to-defun.  In some
> modes, I've seen this cause the buffer to show as modified, however with
> this example, it seems to occur after widening.
> 
> The following sequence of commands can be used to reproduce the issue:
> 
> C-x C-f ~/hello_world.c
> M-x c-ts-mode
> M-x which-function-mode
> C-x n d
> C-x n w
> 
> At this point, the modeline shows that the buffer is modified, and
> running "M-: (buffer-modified-p)" indicates "t".  As previously
> mentioned, this appears to only happen with tree-sitter modes when
> which-function-mode is enabled in the buffer.
> 
> I then repeated the same but also triggered a backtrace on the first
> change as follows:
> 
> M-: (add-to-list 'first-change-hook #'backtrace)
> 
> With that in place, the following is the generated traceback:
> 
>  backtrace()
>  treesit--font-lock-notifier(((1 . 21)) #<treesit-parser for c>)
>  treesit-buffer-root-node(c)
>  treesit-node-at(68)
>  treesit--things-around(68
> "\\(?:class_specifier\\|enum_specifier\\|function_defi..."
> c-ts-mode--defun-valid-p)
>  treesit-thing-at-point(("\\(?:class_specifier\\|enum_specifier\\|function_defi..."
> . c-ts-mode--defun-valid-p) top-level)
>  treesit-defun-at-point()
>  treesit-add-log-current-defun()
>  c-ts-mode--emacs-current-defun-name()
>  add-log-current-defun()
>  which-function()
>  which-func-update-1(#<window 3 on hello_world.c>)
>  which-func-update()
>  apply(which-func-update nil)
>  timer-event-handler([t 0 0 500000 t which-func-update nil idle 0 nil])

Thanks for the backtrace, it’s very helpful.

Narrowing and widening are transparent to tree-sitter, so when the buffer widens, from tree-sitter’s POV, text are added to the buffer. Which-func calling the tree-sitter current-defund function ultimately leads to tree-sitter calling treesit—font-lock-notifier which puts text properties on the “added” portion of the buffer, which changed the modified status.

treesit—font-lock-notifier should put text properties without changing modified status, like what jit-lock does. This patch should fix it.

Yuan


[-- Attachment #2: silent.patch --]
[-- Type: application/octet-stream, Size: 1026 bytes --]

From 2c90ade09a4d52a583158cb9cacf665ac11e8387 Mon Sep 17 00:00:00 2001
From: Yuan Fu <casouri@gmail.com>
Date: Tue, 27 Jun 2023 20:58:34 -0700
Subject: [PATCH] Tree-sitter use with-silent-modifications like jit-lock
 (bug#64321)

* lisp/treesit.el (treesit--font-lock-notifier): Use
with-silent-modifications when marking modified text to be fontified
by jit-lock.  This is what jit-lock itself does.
---
 lisp/treesit.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/treesit.el b/lisp/treesit.el
index 0e1d7931d49..04d460fdea4 100644
--- a/lisp/treesit.el
+++ b/lisp/treesit.el
@@ -1072,7 +1072,8 @@ treesit--font-lock-notifier
       (when treesit--font-lock-verbose
         (message "Notifier received range: %s-%s"
                  (car range) (cdr range)))
-      (put-text-property (car range) (cdr range) 'fontified nil))))
+      (with-silent-modifications
+        (put-text-property (car range) (cdr range) 'fontified nil)))))
 
 ;;; Indent
 
-- 
2.33.1


[-- Attachment #3: Type: text/plain, Size: 2 bytes --]

bug#64321: 29.0.92; Tree-Sitter/which-function Narrow/Widen causes modified buffer

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Tue, 27 Jun 2023 21:04:22 -0700
> Cc: 64321@debbugs.gnu.org,
>  Eli Zaretskii <eliz@gnu.org>
> 
> Thanks for the backtrace, it’s very helpful.
> 
> Narrowing and widening are transparent to tree-sitter, so when the buffer widens, from tree-sitter’s POV, text are added to the buffer. Which-func calling the tree-sitter current-defund function ultimately leads to tree-sitter calling treesit—font-lock-notifier which puts text properties on the “added” portion of the buffer, which changed the modified status.
> 
> treesit—font-lock-notifier should put text properties without changing modified status, like what jit-lock does. This patch should fix it.

Thanks.  If this solves the problem, please install in the emacs-29
release branch.

bug#64321: 29.0.92; Tree-Sitter/which-function Narrow/Widen causes modified buffer

[Troy Brown]

On Wed, Jun 28, 2023 at 12:04 AM Yuan Fu <casouri@gmail.com> wrote:
>
> Narrowing and widening are transparent to tree-sitter, so when the buffer widens, from tree-sitter’s POV, text are added to the buffer. Which-func calling the tree-sitter current-defund function ultimately leads to tree-sitter calling treesit—font-lock-notifier which puts text properties on the “added” portion of the buffer, which changed the modified status.
>
> treesit—font-lock-notifier should put text properties without changing modified status, like what jit-lock does. This patch should fix it.
>

Thanks for the explanation and the supplied patch.  I can confirm that
this patch fixes the issue.

bug#64321: 29.0.92; Tree-Sitter/which-function Narrow/Widen causes modified buffer

[Yuan Fu]

> On Jun 28, 2023, at 9:25 AM, Troy Brown <brownts@troybrown.dev> wrote:
> 
> On Wed, Jun 28, 2023 at 12:04 AM Yuan Fu <casouri@gmail.com> wrote:
>> 
>> Narrowing and widening are transparent to tree-sitter, so when the buffer widens, from tree-sitter’s POV, text are added to the buffer. Which-func calling the tree-sitter current-defund function ultimately leads to tree-sitter calling treesit—font-lock-notifier which puts text properties on the “added” portion of the buffer, which changed the modified status.
>> 
>> treesit—font-lock-notifier should put text properties without changing modified status, like what jit-lock does. This patch should fix it.
>> 
> 
> Thanks for the explanation and the supplied patch.  I can confirm that
> this patch fixes the issue.

Cool, thanks, pushed.

Yuan