bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably

[Yuan Fu <casouri@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2360 bytes --]



> On Dec 19, 2023, at 5:52 PM, Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
> On 19/12/2023 05:12, Yuan Fu wrote:
>>> On Dec 17, 2023, at 10:32 AM, Dmitry Gutov<dmitry@gutov.dev>  wrote:
>>> 
>>> On 17/12/2023 03:16, Yuan Fu wrote:
>>>> +(defun treesit--pre-syntax-ppss (&rest _)
>>>> +  "Force reparse and consequently run all notifiers.
>>>> +
>>>> +Similar to font-lock, we want to update the `syntax' text
>>>> +property before `syntax-ppss' starts working on the text."
>>>> +  (treesit--pre-redisplay)
>>>> +  nil)
>>> This callback might be the best place to inform syntax-propertize about the extended bounds which need to be re-parsed, so I'm not sure about Stefan's suggestion to return nil here.
>>> 
>>> Even calling syntax-ppss-flush-cache wouldn't be a full solution, because that wouldn't update the value of START in syntax-propertize.
>> Makes sense. Then I think a treeist function that returns the latest affected regions is in order?
> 
> But latest compared to what? To the previous invocation of the parser?
> 
> What if the current caller uses this function and expects to see changes since time X, but some other feature already instantiated the parser sometime between X and NOW? The callbacks which would run now would send notifications compared to that other previous (unknown) invocation. There could be some system which keeps checkpoints that the callers could reference, but I'm not sure what's the best shape. And how the older history, not needed by any callers anymore, would be evicted (some weak hash map could work, but any caller that's not careful could create leaks...).
> 
> And without the above, we'll need a separate callback which would call syntax-ppss-flush-cache when the parser sends notifcations while invoked outside of syntax-propertize-extend-region-functions. And then its code will have to care about context anyway (inside s-p-e-r-f or not).

Ok, then how about we set a variable (say, treesit--syntax-ppss-lower-bound) to the lower-bound (aka, start) of the affected region in notifiers (so multiple invocation of the notifier will result in a lowest lower-bound), and in syntax-propertize-extend-region-functions, we extend the region to start from treesit--syntax-ppss-lower-bound, if it’s non-nil; after syntax-ppss, we set treesit--syntax-ppss-lower-bound to nil?

Yuan

[-- Attachment #2: Type: text/html, Size: 6632 bytes --]