Re: New package emacs-parser-generator

[Christian Johansson <christian@cvj.se>]

Ok thanks for the link, I will investigate and see if I can figure it out. 

Read a bit of the research and it seems to to use a state-independent lex-analyzer and do like multiple alternative / nondeterministic error-tolerant parses to make a probalistic best parse which is then merged into the previous AST or something like that..

Regards
Christian

> 1 dec. 2021 kl. 20:25 skrev Yuan Fu <casouri@gmail.com>:
> 
> 
>> 
>>> 
>>> Tree-sitter parses incrementally, I modified primitive insert/delete functions in insdel.c to incrementally parse changed content. There is no need for threads as incremental parsing is extremely fast. We don’t make copies of buffer string, instead, we pass tree-sitter library a function that reads directly from the buffer.
>> 
>> Ok where can I read about this function? How does treesitter handle incremental parses which are state-dependent? For example PHPs lex-analyzer works differently in different states of the grammar, like do you signal to treesitter a point in the buffer so it can backtrack the parsers states in order to correctly perform a incremental parse on the new content?
> 
> I assume you mean the read function? It can be found here:
> 
> https://github.com/casouri/emacs/blob/106d050ad5d02f673f8a089e1f10c1eacfedd124/src/tree-sitter.c#L372
> 
> Tree-sitter only requires to be informed of every change to the buffer, it will read the buffer for itself and update the AST. I have no idea if it backtracks behind the scenes. Presumably tree-sitter can figure out from where to backtrack from the change information we give it.
> 
> Yuan