Re: tree-sitter: conceptional problem solvable at Emacs' level?

[Eli Zaretskii <eliz@gnu.org>]

> From: Yuan Fu <casouri@gmail.com>
> Date: Thu, 9 Feb 2023 23:33:10 -0800
> Cc: Holger Schurig <holgerschurig@gmail.com>,
>  Emacs-devel@gnu.org
> 
> > Parsers (without a full C preprocessor inside) can only work for
> > languages like Python, which cannot be enhanced with syntax-modifying
> > macros.
> 
> Right. Our best hope is for someone to try extend the current tree-sitter-c grammar, but I don’t know how feasible it is. Emacs can also do some limited workaround, but the potential in that department is slim.

I think we still have a way to go before we reach the above
conclusions (which basically mean we give up on improving the
situation with C/C++ macros).  We should explore other approaches.

One such approach would be to perform our own analysis when the parser
returns an error node due to macros.

Another possibility is to complicate the function we pass to
tree-sitter with which to read buffer text, in a way that replaces the
text of a macro with something else (in the simplest case, just space
characters), so as to avoid errors in the parser, and again analyze
the macros in our own code.

And I'm sure there are other alternatives.  This issue is not unique
to Emacs, so studying how other IDEs deal with it could also yield
ideas.

Volunteers interested in improving support for C/C++ based on
tree-sitter are very welcome to step forward and work on these issues.

Thanks.