bug#67262: python-ts-mode cannot identify triple-quoted-strings

[Eli Zaretskii <eliz@gnu.org>]

How should we make progress with this bug?  I'd like to fix it before
Emacs 29.2 is released.

> Date: Tue, 12 Dec 2023 23:55:43 +0200
> Cc: 67262@debbugs.gnu.org
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> On 12/12/2023 10:32, Yuan Fu wrote:
> 
> >> syntax-ppss-flush-cache is called by edits (and by the re-parse). It 
> >> seems like it will be odd to have execution the other way around 
> >> and/or add some hook into it which would call the re-parse and extend 
> >> the region to be invalidated.
> >>
> >> syntax-propertize could have another hook added, yes. Or an advice.
> >>
> >> But it seems better to reuse some of the existing hooks, such as 
> >> syntax-propertize-extend-region-functions. It treesit.c provided a way 
> >> to fetch the newly-invalidated region, the treesit-major-mode-setup 
> >> could add a new function to syntax-propertize-extend-region-functions 
> >> which would invoke that feature. But even now it can instantiate the 
> >> parse, which would call treesit-force-reparse internally, and then 
> >> collect the info from the callbacks.
> > syntax-propertize-extend-region-functions looks perfect. We just need to 
> > force a reparse in it and the notifier will do the rest.
> >>
> >> And yet another way - is to extend the region to be propertized from 
> >> inside the major mode's syntax-propertize-function, invalidating some 
> >> earlier entries too. The main problem with that, I think, is that 
> >> every ts mode will have to repeat that trick. And that authors would 
> >> have to know to do that. How to make that easier and more obvious, is 
> >> a question.
> >>
> >> Finally, if I'm right that bug#66732 has a similar cause, then a 
> >> shared solution that can be reused by syntax and font-lock (or 
> >> preferably just fix both in the same place) would be ideal.
> >>
> >>> I'm not quite sure about how do we cause this re-parse. The 
> >>> straightforward approach would be calling treesit-force-reparse[1] in 
> >>> syntax-propertize/syntax-ppss-flush-cache. But ideally I'd like to 
> >>> keep tree-sitter transparent for syntax.el. Maybe we can add a hook 
> >>> in syntax-propertize/syntax-ppss-flush-cache.
> >>>
> >>> [1] This function doesn't exist yet, but it's easy to define in lisp.
> >>
> >> treesit-parser-root-node calls it anyway and does little else, so we 
> >> could get by with just using it.
> >>
> > Yep.
> 
> For the casual observer: I posted a solution using the above method in 
> comments to bug#66732 yesterday.
>