bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably

[Dmitry Gutov <dmitry@gutov.dev>]

On 16/12/2023 19:11, Stefan Monnier wrote:
>> Hmm... This `put-text-property` can't be right.
>> Can you trace `font-lock-fontify-region` and
>> `font-lock-default-fontify-region` to check that they correctly return
>> the proper (extended) bounds of the region they (re)fontified?
> BTW, using the extend-region-function isn't ideal: this is meant for
> things which can't be refontified separately, so if a change in the
> buffer causes notification to make changes over a large portion of the
> buffer we'll end up (re)fontified*right away*  that whole large
> portion even if only a small part (or even no part at all) is displayed.

BTW, do you have an example of a "thing which can't be refontified 
separately"?

Otherwise, I think a reorganization would be possible where when the 
region is extended to more than the visible area of the buffer, some 
parts of the buffer are marked as unfontified, but don't force their 
highlighting in the same cycle. Basically, what the 
pre-redisplay-functions patch does, but using the 
*-extend-region-functions protocol.

Might not be trivial code-wise. I guess it also depends on how much we 
consider the current approach of requesting an additional display 
iteration, to be a problem (e.g. does that create any real difference 
performance-wise).

The use of pre-redisplay-functions avoid that question, but asks the 
programmer to work on a lower level of abstraction (i.e. know about 
'fontified' => nil), as well as possibly apply this change to a bigger 
region than necessary (again, not sure about the performance impact).