bug#60691: 29.0.60; Slow tree-sitter font-lock in ruby-ts-mode

[Dmitry Gutov <dgutov@yandex.ru>]

On 01/02/2023 07:26, Yuan Fu wrote:
>>> I should mention this in the comments, but the fast mode is only for very rare cases, where the file is mechanically generated and has some peculiarities that causes tree-sitter to work poorly. If the file is hand-written and “normal”, even huge files like xdisp.c is well below the bar. Therefore I don’t think “crossing the line” will realistically happen when editing source files.
>>> Here is the stats of two “problematic files”, named packet and dec_mask, comparing to xdisp.c:
>>> ;;           max-depth max-width count
>>> ;; cut-off   100       4000
>>> ;; packet   (98159     46581 1895137)
>>> ;; dec mask (3         64301 283995)
>>> ;; xdisp.c  (29        985   218971)
>>> I’d say that any regular source file, even mechanically generated, wouldn’t go beyond ~50 levels in depth, and hand-written files should never has a node that has 4000+ direct children in the parse tree.
>> Oh, thanks for the explanation. Then the current strategy makes sense.
>>
>> Is xdisp.c absolutely the largest C file in your experience?
>>
>> According to the above numbers, a file that's only 4x as large could hit our current cutoff.
> I don’t think these stats increase linearly as the file size increases. Even if there is a file that has a node with 3999 direct children, and the developer adds another one, I’d say it’s better not to turn on “fast mode” immediately.

I see your point.

In the previous message I was talking about a different scenario: when a 
project has a file 4x the size of xdisp.c, and the user just opens it. I 
suspect it's not great to have "fast mode" enabled in that case? Like, 
false positive.

Anyway, this is a very theoretical concern on my part.