bug#62951: 29.0.90; c-ts-mode: Incorrect fontification due to FOR_EACH_TAIL_SAFE

[Dmitry Gutov <dmitry@gutov.dev>]

On 27/04/2023 01:19, Yuan Fu wrote:
> 
> 
>> On Apr 23, 2023, at 2:04 PM, Dmitry Gutov <dmitry@gutov.dev> wrote:
>>
>> On 23/04/2023 03:28, Yuan Fu wrote:
>>> What do you think of extending the parser to support these macros instead? (So we fork tree-sitter-c.) If we can fix the parser, we don’t need to retrofit hacks onto font-lock, indent, etc, separately, and it truly fixes the problem. The downside is compiling from grammar source to grammar.c needs rust and node tools. But I guess depending on the grammar maintained by tree-sitter’s author isn’t too much different from depending on the grammar maintained by another individual (ie, me)?
>>
>> We had also talked at some point about replacing the actual text that the parser sees with something else.
>>
>> If this can be done in a straightforward way (with tracking the subsequent correspondence of "real" text back to the buffer for syntax highlighting), that might be the perfect solution: we'd have a defcustom which would hold a list of macros used in the current codebase in the form of templates, and we'd set a bunch of them in emacs/.dir-locals.el.
>>
>> I'm not sure how difficult this is to implement and maintain, but it's probably going to be less work to maintain than a fork of the grammar.
> 
> Sounds to me a bit difficult to write. Eg, translating between tree-sitter position and buffer position efficiently isn’t too easy. Now plus narrowing, and what if the narrowing boundary is in the middle of a replace region?

When the match isn't full, the replacement wouldn't be performed. Same 
as with macro name that isn't fully typed out yet.

Yeah, it does seem like a lot of work, but the result might be that 
everybody's macros could be supported.

I'm definitely not volunteering, though, so please take this as just a 
suggestion.

> My idea right now is to use the range feature in tree-sitter. Since the “body” of FOR_EACH_TAIL is valid C, I can either set the ranges for the parser so it ignores FOR_EACH_TAIL, or I can add another parser that only parses the body of FOR_EACH_TAIL.

Sounds good. Especially for Emacs 29 (maybe).