From: Dmitry Gutov <dgutov@yandex.ru>
To: Juri Linkov <juri@linkov.net>, Yuan Fu <casouri@gmail.com>
Cc: 60691@debbugs.gnu.org
Subject: bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode
Date: Tue, 10 Jan 2023 16:10:49 +0200 [thread overview]
Message-ID: <4ed5051c-ea5a-91b1-6b8c-5349a3495a16@yandex.ru> (raw)
In-Reply-To: <86leman71u.fsf@mail.linkov.net>
On 10/01/2023 10:10, Juri Linkov wrote:
>>> After more rules were added recently to ruby-ts--font-lock-settings,
>>> font-lock became slow even on very small files. Some measurements:
>>
>> If you saw a particular commit that made things slower, did you try
>> reverting it? What was the performance after?
>
> No particular commit, just adding more rules degrades performance
> gradually.
But I don't think I added that many rules recently. No more than a
quarter anyway.
>>> M-: (benchmark-run 1000 (progn (font-lock-mode -1) (font-lock-mode 1) (font-lock-ensure)))
>>> M-x ruby-mode
>>> (1.3564674989999999 0 0.0)
>>> M-x ruby-ts-mode
>>> (8.349582391999999 2 6.489918534000001)
>>
>> I have tried this scenario (which, to be frank, is pretty artificial, given
>> that fontification is usually performed in chunks, not over the whole
>> buffer).
>>
>> Perhaps the results depend on a particular file. The ones I have tried
>> (ruby.rb and ruby-after-operator-indent.rb) show only 2x difference (or
>> less). The difference was in favor of ruby-mode, but given the difference
>> in approaches I wouldn't be surprised if ruby-ts-mode incurs a fixed
>> overhead somewhere.
>
> On test/lisp/progmodes/ruby-mode-resources/ruby.rb I see these numbers:
>
> ruby-mode
> (8.701560543000001 95 1.045961102)
>
> ruby-ts-mode
> (34.653148898000005 1464 16.904981779)
Interesting. It's 12s vs 36s for me, as I've retested now.
>>> This is not a problem when files are visited infrequently, but
>>> becomes a problem for diff-syntax fontification that wants to
>>> highlight simultaneously many files from git logs.
>>> So a temporary measure would be not to enable ruby-ts-mode
>>> in internal buffers:
>>
>> Is it common to try to highlight 1000 or even 100 files in one diff?
>
> 100 is rare, but tens is pretty common, so this problem affects
> only this specific case.
So it's a 0,8-3s delay in those cases? That's not ideal.
>>> (add-hook 'find-file-hook
>>> (lambda ()
>>> (when (and (eq major-mode 'ruby-mode)
>>> ;; Only when not internal as from diff-syntax
>>> (not (string-prefix-p " " (buffer-name))))
>>> (ruby-ts-mode))))
>>
>> Have you tried similar tests with other -ts- modes? Ones with complex
>> font-lock rules in particular.
>
> I tried with c-ts-mode, and it's very fast.
Just how fast is it? The number of font-lock features is has is
comparable (though a little smaller).
I've tried the same benchmark for it in admin/alloc-colors.c, and it
comes out to
(3.2004193190000003 30 0.9609690980000067)
Which seems comparable.
Not sure how to directly test the modes against each other, but if I
enable ruby-ts-mode in the same file, the benchmark comes to 1s.
Or if I enable c-ts-mode in ruby.rb -- 16s.
>> I've tried commenting out different rules in ruby-ts--font-lock-settings,
>> but none of them seem to have particularly outsides impact. Performance
>> seems, roughly, inversely proportional to the number of separate
>> "features".
>
> Indeed, this is what I see - no particular rule, only their number
> affects performance.
>
>> And if all ts modes turn out to have this problem, perhaps the place to
>> improve this is inside some common code.
>
> I noticed that while most library files are small, e.g.
> libtree-sitter-c.so is 401,528 bytes,
> libtree-sitter-ruby.so is 2,130,616 bytes
> that means that it has more complex logic
> that might explain its performance.
ruby is indeed one of the larger ones. Among the ones I have here
compiled, it's exceeded only by cpp. 2.29 MB vs 2.12 MB.
But testing admin/alloc-colors.c with c++-ts-mode vs c-ts-mode gives
very similar performance, so it's unlikely that the complexity of the
grammar is directly responsible.
> In this case, when nothing could be done to improve performance,
> please close this request.
Perhaps Yuan has some further ideas. There are some strong oddities here:
- Some time into debugging and repeating the benchmark again and again,
I get the "Pure Lisp storage overflowed" message. Just once per Emacs
session. It doesn't seem to change much, so it might be unimportant.
- The profiler output looks like this:
18050 75% - font-lock-fontify-syntactically-region
15686 65% - treesit-font-lock-fontify-region
3738 15%
treesit--children-covering-range-recurse
188 0% treesit-fontify-with-override
- When running the benchmark for the first time in a buffer (such as
ruby.rb), the variable treesit--font-lock-fast-mode is usually changed
to t. In one Emacs session, after I changed it to nil and re-ran the
benchmark, the variable stayed nil, and the benchmark ran much faster
(like 10s vs 36s).
In the next session, after I restarted Emacs, that didn't happen: it
always stayed at t, even if I reset it to nil between runs. But if I
comment out the block in treesit-font-lock-fontify-region that uses it
;; (when treesit--font-lock-fast-mode
;; (setq nodes (treesit--children-covering-range-recurse
;; (car nodes) start end (* 4 jit-lock-chunk-size))))
and evaluate the defun, the benchmark runs much faster again: 11s.
(But then I brought it all back, and re-ran the tests, and the variable
stayed nil that time around; to sum up: the way it's turned on is unstable.)
Should treesit--font-lock-fast-mode be locally bound inside that
function, so that it's reset between chunks? Or maybe the condition for
its enabling should be tweaked? E.g. I don't think there are any
particularly large or deep nodes in ruby.rb's parse tree. It's a very
shallow file.
next prev parent reply other threads:[~2023-01-10 14:10 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-09 17:16 bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode Juri Linkov
2023-01-09 22:33 ` Dmitry Gutov
2023-01-10 8:10 ` Juri Linkov
2023-01-10 14:10 ` Dmitry Gutov [this message]
2023-01-10 17:50 ` Juri Linkov
2023-01-11 12:12 ` Dmitry Gutov
2023-01-11 12:12 ` Dmitry Gutov
2023-01-12 21:58 ` Yuan Fu
2023-01-12 23:40 ` Dmitry Gutov
2023-01-13 7:57 ` Eli Zaretskii
2023-01-13 9:15 ` Yuan Fu
2023-01-13 11:51 ` Eli Zaretskii
2023-01-14 3:48 ` Yuan Fu
2023-01-14 7:29 ` Eli Zaretskii
2023-01-14 7:51 ` Yuan Fu
2023-01-14 8:01 ` Eli Zaretskii
2023-01-14 8:46 ` Andreas Schwab
2023-01-14 23:03 ` Yuan Fu
2023-01-18 6:50 ` Yuan Fu
2023-01-19 18:28 ` Dmitry Gutov
2023-01-20 22:24 ` Yuan Fu
2023-01-22 2:01 ` Dmitry Gutov
2023-01-29 8:25 ` Yuan Fu
2023-01-29 23:07 ` Dmitry Gutov
2023-01-29 23:23 ` Yuan Fu
2023-01-30 0:15 ` Dmitry Gutov
2023-02-01 5:26 ` Yuan Fu
2023-02-01 15:11 ` Dmitry Gutov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4ed5051c-ea5a-91b1-6b8c-5349a3495a16@yandex.ru \
--to=dgutov@yandex.ru \
--cc=60691@debbugs.gnu.org \
--cc=casouri@gmail.com \
--cc=juri@linkov.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.