bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably

[Yuan Fu <casouri@gmail.com>]

> On Dec 13, 2023, at 5:43 PM, Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
> On 13/12/2023 05:28, Yuan Fu wrote:
>>>> c-ts-mode--emacs-set-ranges is registered as a range rule, so many tree-sitter function calls it before doing anything to make sure range is up-to-date. treesit-font-lock-fontify-region calls treesit-update-ranges at the beginning of its body, and treesit-update-ranges calls c-ts-mode--emacs-set-ranges.
>>> 
>>> That seems to mean that any feature accessing the parse tree should call treesit-update-ranges first. Including syntax-propertize-functions and *-extend-region-functions, which we currently don't do.
>>> 
>>> But which boundaries is it supposed to use? Should treesit--syntax-extend-region call treesit-update-ranges, when wait for the parser updates, then possibly call treesit-update-ranges again on the extended boundaries, and so on, until the parser stops sending notifications? Will it stop?
>> It'll stop. If the ranges don't change, no reparse will happen. And only buffer content change can cause range change. Reparse itself can't. So at most you'll see reparse -> update ranges -> reparse.
>>> Anyway, could you try my patch? Like I said, I'm not sure if the insufficient fontification I'm observing in c-ts-mode is due to the problem with the solution, or due to the other redisplay-related problems on my system.
>> Yeah, it doesn't solve the problem in c-ts-mode regarding block comments.
>> We might need to run (progn (force-parse) (update-ranges) (force-parse)) before jit-lock-fontify-now and sytax-ppss.
> 
> Well... I've replaced
> 
>  (treesit-buffer-root-node (treesit-language-at (point)))
> 
> in treesit--syntax-extend-region with
> 
>  (treesit-buffer-root-node (treesit-language-at (point)))
>  (treesit-update-ranges beg end)
>  (treesit-buffer-root-node (treesit-language-at (point)))
> 
> and even tried commenting out the call to 'treesit-update-ranges' inside treesit-font-lock-fontify-region, but the result looks unchanged. And the region does get extended, according to my print-debugging inside font-lock-default-fontify-region.
> 
> Could you check if you're seeing the same?

It should be

(treesit-buffer-root-node 'emacs-c)
(treesit-update-ranges)
(treesit-buffer-root-node ‘c)

I tried, and it doesn’t make a difference. But I must admit I’m not very clear on what your patch tries to do. Does it try to extend the to-be-fontified region before jit-lock runs?

Yuan