Re: treesitter local parser: huge slowdown and memory usage in a long file

[Vincenzo Pupillo <v.pupillo@gmail.com>]

I don't know if this is a stupid idea or not, but I'll try to explain it. 
My other php-ts-mode (the one without a tree-sitter parser for php) does these things:
there is a "treesit-font-lock-rules" to capture a comment node,
this rule calls a function that tries to figure out if it is a comment block in PHP. If it is a comment block, it uses 
some regular expression for the font-locking, otherwise use treesit-fontify-with-override for the entire comment.
Treesit "knows" the intervals in the file to inject the embedded parser.
Can this information be used for local embedded parsers?

V.

In data martedì 13 febbraio 2024 09:15:49 CET, Yuan Fu ha scritto:
> 
> > On Feb 12, 2024, at 6:09 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> From: Yuan Fu <casouri@gmail.com>
> >> Date: Sun, 11 Feb 2024 20:16:11 -0800
> >> Cc: "Ergus via Emacs development discussions." <emacs-devel@gnu.org>,
> >> Eli Zaretskii <eliz@gnu.org>
> >> 
> >> Thanks, the culprit is the call to treesit-update-ranges in treesit--pre-redisplay, where we don’t pass it any specific range, so it updates the range for the whole buffer. Eli, is there any way to get a rough estimate the range that redisplay is refreshing? Do you think something like this would work?
> >> 
> >>    (treesit-update-ranges
> >>     (max (point-min) (- (window-start) 1000)) ; BEG
> >>     (min (point-max) (+ (or (window-end) (+ (window-start) 4000)) 1000))) ; END
> >> 
> >> I guess the window-start would be outdated in pre-redisplay-function...
> > 
> > The problem is that window-start is not guaranteed to be up-to-date
> > when pre-redisplay-function is called: the window-start is updated by
> > redisplay, and pre-redisplay-function is called before the update.
> > Moreover, pre-redisplay-function could be called either once or twice
> > in a redisplay cycle, and window-start is up-to-date only for the
> > second call.
> > 
> > The window-end point is basically never up-to-date during redisplay,
> > only at its very end.
> > 
> > So my suggestion would be to define the range from position of point,
> > using the window dimensions; see get_narrowed_width for ideas.  This
> > could lose if the buffer has a lot of invisible text, so I suggest to
> > check for invisible properties, and if they are present in the buffer,
> > punt and use the whole accessible portion of the buffer (I don't
> > expect PHP buffers, or any buffers in programming-language modes, to
> > have invisible text).
> 
> Ah, clever :-) Programming language buffers could have invisible text when the user uses hideshow, or folded some section of code using outline-minor-mode :-(
> 
> But as I said in the reply to Dmitry, we might need some better design for updating parser ranges than the current one. I’ll just fix V’s problem for now by updating the range around point, and ignore invisible text for now.
> 
> Yuan
>