bug#66732: tree-sitter fontification doesn't update multi-line syntax reliably

[Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

> diff --git a/lisp/treesit.el b/lisp/treesit.el
> index 8a07f5023a9..a6ca6334133 100644
> --- a/lisp/treesit.el
> +++ b/lisp/treesit.el
> @@ -1080,13 +1080,39 @@ treesit--font-lock-notifier
>    "Ensures updated parts of the parse-tree are refontified.
>  RANGES is a list of (BEG . END) ranges, PARSER is the tree-sitter
>  parser notifying of the change."
> -  (with-current-buffer (treesit-parser-buffer parser)
> -    (dolist (range ranges)
> -      (when treesit--font-lock-verbose
> -        (message "Notifier received range: %s-%s"
> -                 (car range) (cdr range)))
> -      (with-silent-modifications
> -        (put-text-property (car range) (cdr range) 'fontified nil)))))
> +  ;; (with-current-buffer (treesit-parser-buffer parser)
> +  ;;   (dolist (range ranges)
> +  ;;     (when treesit--font-lock-verbose
> +  ;;       (message "Notifier received range: %s-%s"
> +  ;;                (car range) (cdr range)))
> +  ;;     (with-silent-modifications
> +  ;;       (put-text-property (car range) (cdr range) 'fontified nil))))
> +  )
> +
> +(defun treesit--font-lock-extend-region ()
> +  (treesit-update-ranges)
> +  (dolist (parser (treesit-parser-list nil nil t))
> +    (treesit-parser-root-node parser))
> +  (let ((old-beg font-lock-beg)
> +        (old-end font-lock-end))
> +    (dolist (parser (treesit-parser-list nil nil t))
> +      (when-let ((table treesit--parser-updated-ranges)
> +                 (ranges (gethash parser table)))
> +        (dolist (range ranges)
> +          (setq font-lock-beg (max (min font-lock-beg (car range)) (point-min)))
> +          (setq font-lock-end (min (max font-lock-end (cdr range)) (point-max))))))
> +    (put-text-property font-lock-beg font-lock-end 'fontified nil)
> +    (message "extend region: %s-%s %s-%s this-command: %s" old-beg old-end
> +             font-lock-beg font-lock-end this-command)))

Hmm... This `put-text-property` can't be right.
Can you trace `font-lock-fontify-region` and
`font-lock-default-fontify-region` to check that they correctly return
the proper (extended) bounds of the region they (re)fontified?


        Stefan