bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably

[Eli Zaretskii <eliz@gnu.org>]

> Date: Sun, 10 Dec 2023 20:16:37 -0800
> Cc: dmitry@gutov.dev, 66732@debbugs.gnu.org, dominik@honnef.co
> From: Yuan Fu <casouri@gmail.com>
> 
> (defun treesit--font-lock-notifier (ranges parser)
>    "Ensures updated parts of the parse-tree are refontified.
> RANGES is a list of (BEG . END) ranges, PARSER is the tree-sitter
> parser notifying of the change."
>    (with-current-buffer (treesit-parser-buffer parser)
>      (dolist (range ranges)
>        (when treesit--font-lock-verbose
>          (message "Notifier received range: %s-%s"
>                   (car range) (cdr range)))
>        (with-silent-modifications
>          (put-text-property (car range) (cdr range) 'fontified nil)))))
> 
> This notifier function will be called during redisplay [1]. I suspect 
> that because of this timing, redisplay doesn't refontify the marked 
> region immediately. So I added a timer, I think that ensures we mark the 
> affected region in the next command loop?
> 
> (defun treesit--font-lock-notifier (ranges parser)
>    "Ensures updated parts of the parse-tree are refontified.
> RANGES is a list of (BEG . END) ranges, PARSER is the tree-sitter
> parser notifying of the change."
>    (with-current-buffer (treesit-parser-buffer parser)
>      (dolist (range ranges)
>        (when treesit--font-lock-verbose
>          (message "Notifier received range: %s-%s"
>                   (car range) (cdr range)))
>        (run-with-timer
>         0 nil
>         (lambda ()
>           (with-silent-modifications
>             (put-text-property (car range) (cdr range)
>                                'fontified nil)))))))
> 
> This seems to work. Eli, do you see any problem using run-with-timer 
> this way?

Why do you ask? is there something in this function that bothers you?

What's the correct way to mark some region unfontified?

> [1] The chain of events if roughly: user types the last "/" -> redisplay 
> -> fontify that character -> access parser -> parser reparses -> calls 
> notifier.

The way to mark region of text as not fontified is to set its
'fontified' property nil.

Instead of a timer, did you try to use pre-redisplay-function?

I added Stefan, in case he has some suggestions and ideas.