Re: [SPAM UNSURE] Maybe we're taking a wrong approach towards tree-sitter

["Perry E. Metzger" <perry@piermont.com>]

On 7/30/21 08:06, Arthur Miller wrote:
> I undestand that having specialized regex matcher is more efficient than
> some generalized regular matcher current font-locking in Emacs relies
> upon, but is it *that* more efficient to be worth the extra troubles?

It is not a question of efficiency. You cannot parse a context free 
grammar using regular expressions. The reason that almost all our 
highlight modes produce random garbage throughout is that you cannot 
parse a context free grammar using regular expressions. (For many 
languages, correctness isn't just occasionally violated, it's generally 
violated.)

Reliable highlighting regardless of code formatting, reliable 
indentation assistance, reliable code folding, and other such features 
require that the editor be able to both parse the program being edited 
_and_ that the editor be able to incrementally re-parse it as it changes 
in minimal time.

Other editors now have such features and make good use of them. Highly 
reliable code folding alone is worth the price of admission IMHO. 
(Currently, the best we can do for code folding is assume that the 
indentation is correct.)

LSP has been revolutionary in improving the programmer's experience in 
Emacs. Tree Sitter will provide significant additional improvement.

> TS seem to keep state (a node) for each character typed, that will be a
> lot of memory consumed in some big files.

No one will be forced to turn it on. I, however, almost certainly will. 
My productivity is more important to me than my RAM budget. Those that 
don't like it, though, won't have to pay the RAM tax.

Perry