bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode

[Juri Linkov <juri@linkov.net>]

>> After more rules were added recently to ruby-ts--font-lock-settings,
>> font-lock became slow even on very small files.  Some measurements:
>
> If you saw a particular commit that made things slower, did you try
> reverting it? What was the performance after?

No particular commit, just adding more rules degrades performance
gradually.

>> M-: (benchmark-run 1000 (progn (font-lock-mode -1) (font-lock-mode 1) (font-lock-ensure)))
>> M-x ruby-mode
>> (1.3564674989999999 0 0.0)
>> M-x ruby-ts-mode
>> (8.349582391999999 2 6.489918534000001)
>
> I have tried this scenario (which, to be frank, is pretty artificial, given
> that fontification is usually performed in chunks, not over the whole
> buffer).
>
> Perhaps the results depend on a particular file. The ones I have tried
> (ruby.rb and ruby-after-operator-indent.rb) show only 2x difference (or
> less). The difference was in favor of ruby-mode, but given the difference
> in approaches I wouldn't be surprised if ruby-ts-mode incurs a fixed
> overhead somewhere.

On test/lisp/progmodes/ruby-mode-resources/ruby.rb I see these numbers:

ruby-mode
(8.701560543000001 95 1.045961102)

ruby-ts-mode
(34.653148898000005 1464 16.904981779)

>> This is not a problem when files are visited infrequently, but
>> becomes a problem for diff-syntax fontification that wants to
>> highlight simultaneously many files from git logs.
>> So a temporary measure would be not to enable ruby-ts-mode
>> in internal buffers:
>
> Is it common to try to highlight 1000 or even 100 files in one diff?

100 is rare, but tens is pretty common, so this problem affects
only this specific case.

>> (add-hook 'find-file-hook
>>            (lambda ()
>>              (when (and (eq major-mode 'ruby-mode)
>>                         ;; Only when not internal as from diff-syntax
>>                         (not (string-prefix-p " " (buffer-name))))
>>                (ruby-ts-mode))))
>
> Have you tried similar tests with other -ts- modes? Ones with complex
> font-lock rules in particular.

I tried with c-ts-mode, and it's very fast.

> I've tried commenting out different rules in ruby-ts--font-lock-settings,
> but none of them seem to have particularly outsides impact. Performance
> seems, roughly, inversely proportional to the number of separate
> "features".

Indeed, this is what I see - no particular rule, only their number
affects performance.

> And if all ts modes turn out to have this problem, perhaps the place to
> improve this is inside some common code.

I noticed that while most library files are small, e.g.
libtree-sitter-c.so is 401,528 bytes,
libtree-sitter-ruby.so is 2,130,616 bytes
that means that it has more complex logic
that might explain its performance.

In this case, when nothing could be done to improve performance,
please close this request.