Re: New package emacs-parser-generator

[Yuan Fu <casouri@gmail.com>]

The tree-sitter integration doesn’t provide anything to define a grammar nor generating a parser. It merely uses pre-defined grammar (written by tree-sitter community) to parse buffers and produce an AST.

To write an tree-sitter grammar definition you need to write the grammar in JavaScript and pass it to tree-sitter’s parser generator. The generator spits out a grammar definition encoded in a C source file (in the form of a C struct). To use this grammar definition, we compile it to a library, load it at runtime, pass the struct to tree-sitter library, and tree-sitter can now parse according to this grammar definition.

The tree-sitter integration provides 1) Lisp wrappers for tree-sitter’s C API; 2) some convenient functions built on the C API; 3) integration with font-lock and indentation. I didn’t add any “common API”, I simply used existing frameworks in Emacs: for font-lock I used font-lock-fontify-region-function, for indent I used indent-line-function. If in the future a common API for parses is desirable, tree-sitter can easily comply.

IOW, this is what tree-sitter integration currently does:

font-lock                                indent
      |                                     |
font-lock-fontify-region-function        indent-line-function
      |                                     |
      |                                     |
tree-sitter-font-lock-fontify-region     tree-sitter-indent-function

This is what could happen if we want a common API:

font-lock                                indent
      |                                     |
font-lock-fontify-region-function        indent-line-function
      |                                     |
      |                                     |
     common  API  --------------------------+
     /         \
    /           \
   /             \
tree-sitter    other parser

Yuan