bug#64017: Wrong conversion from Emacs to Tree-sitter S-expression syntax

[Yuan Fu <casouri@gmail.com>]

Thanks for catching this.

> On Jun 12, 2023, at 7:14 AM, Mattias Engdegård <mattias.engdegard@gmail.com> wrote:
> 
> `treesit-pattern-expand` converts a query pattern into tree-sitter S-expression syntax, as a string. The conversion mainly converts certain keywords but the main problem is that it prints strings in Emacs syntax which differs from that of tree-sitter.
> 
> As a consequence, :match regexps cannot contain newlines:
> 
> (treesit-query-capture
> 'java
> '(((identifier) @font-lock-constant-face
>    (:match "hello\n" @font-lock-constant-face))))
> 
> signals a syntax error.
> 
> As far as I can tell the tree-sitter string syntax allows for the escape sequences:
> 
> \n = LF
> \r = CR
> \t = TAB
> \0 = NUL  (only a single 0 -- no octal escapes!)
> \X = the character X itself
> 
> Unescape newlines result in a syntax error as seen in the example above. NULs don't seem to go well either.
> 
> At the very least, the conversion should avoid literal newlines and NULs in the result (and probably CR and TAB). This cannot be done with a straight prin1-to-string.
> 
> (By the way, why is the conversion written in C? Was Lisp too slow?)

Because I wasn't sure if it’s ok for C functions to rely on Lisp functions, plus the function is simple enough. Right now if one doesn’t load treesit.el, all the C functions work fine.

> 
> Ideally we should not need to expose the tree-sitter s-exp query syntax at all. Surely Emacs s-exps should be preferable in every case?
> 

It shouldn’t hurt to expose the tree-sitter sexp. Other editors mainly use the string syntax.

Yuan