Re: Questions about tree-sitter

[Lynn Winebarger <owinebar@gmail.com>]

On Thu, Sep 7, 2023 at 7:42 PM Yuan Fu <casouri@gmail.com> wrote:
> > On Sep 6, 2023, at 9:11 AM, Lynn Winebarger <owinebar@gmail.com> wrote:
> >
> > On Wed, Aug 30, 2023 at 3:03 AM Yuan Fu <casouri@gmail.com> wrote:
> >>> Is it possible to reload a grammar after modifying it?
> >>
> >> No, and it’s probably not easy to implement either, since unloading the grammar would require Emacs to purge/invalid all the node/query/parsers using that grammar.
> >
> > [ ... ]
> > Therefore, given functionality to translate elisp data into the raw C
> > structures, we should be able to dynamically create language data
> > structures to pass to the tree-sitter library to create a library.
> > We would also need a table driven lexer framework in place of the
> > generated lexer in the C file to completely avoid going through a C
> > compiler.
> > The other novel features of tree-sitter parsers appear to be
> > implemented in the parser runtime, not in the table calculation.
> >
> > I've implemented LALR(1) parser generators two or three times in the
> > last couple of decades, this might be a fun project for me while I am
> > unambiguously able to contribute to GNU Emacs.
>
> That’ll be great. But note that the parser structure has scape hatches: certain things can be implemented by arbitrary C function. Also tree-sitter allows grammars to use custom scanners [1].
>
My primary interest is in using the tree-sitter parser framework with
the grammars and lexers constructed for Semantic in elisp.  That's the
strongest use-case.  That can be done by a single library implementing
a generic table-driven scanner function.

For other cases, it's a mixed bag.  If only the grammar changes, and
all C code is fixed, then modifications to the grammar could be
reloaded.  If this feature was really important to the user, they
could probably implement the C code to call Elisp functions that could
be updated dynamically, at least during development.

But you are correct that this will not solve the problem for arbitrary
tree-sitter language definitions with embedded C code.  For use in
emacs, the user might implement any required functions in a dynamic
module that could be loaded and unloaded separately from the
tree-sitter language library.  But that will not happen with the
parser.c produced by the tree-sitter cli tool.

Lynn