bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode

[Dmitry Gutov <dgutov@yandex.ru>]

Yuan? Just making sure you got this message.

On 10/01/2023 16:10, Dmitry Gutov wrote:
> Perhaps Yuan has some further ideas. There are some strong oddities here:
> 
> - Some time into debugging and repeating the benchmark again and again, 
> I get the "Pure Lisp storage overflowed" message. Just once per Emacs 
> session. It doesn't seem to change much, so it might be unimportant.
> 
> - The profiler output looks like this:
> 
>    18050  75%                    - font-lock-fontify-syntactically-region
>    15686  65%                     - treesit-font-lock-fontify-region
>     3738  15% treesit--children-covering-range-recurse
>      188   0%                        treesit-fontify-with-override
> 
> - When running the benchmark for the first time in a buffer (such as 
> ruby.rb), the variable treesit--font-lock-fast-mode is usually changed 
> to t. In one Emacs session, after I changed it to nil and re-ran the 
> benchmark, the variable stayed nil, and the benchmark ran much faster 
> (like 10s vs 36s).
> 
> In the next session, after I restarted Emacs, that didn't happen: it 
> always stayed at t, even if I reset it to nil between runs. But if I 
> comment out the block in treesit-font-lock-fontify-region that uses it
> 
>      ;; (when treesit--font-lock-fast-mode
>      ;;   (setq nodes (treesit--children-covering-range-recurse
>      ;;                (car nodes) start end (* 4 jit-lock-chunk-size))))
> 
> and evaluate the defun, the benchmark runs much faster again: 11s.
> 
> (But then I brought it all back, and re-ran the tests, and the variable 
> stayed nil that time around; to sum up: the way it's turned on is 
> unstable.)
> 
> Should treesit--font-lock-fast-mode be locally bound inside that 
> function, so that it's reset between chunks? Or maybe the condition for 
> its enabling should be tweaked? E.g. I don't think there are any 
> particularly large or deep nodes in ruby.rb's parse tree. It's a very 
> shallow file.