Re: Why tree-sitter instead of Semantic? (was Re: CC Mode with font-lock-maximum-decoration 2)

[Eric Ludlam <ericludlam@gmail.com>]

On 8/16/22 6:42 AM, Lynn Winebarger wrote:
> On Sun, Aug 14, 2022 at 3:24 PM Eric Ludlam <ericludlam@gmail.com> wrote:
> > I was also frequently surprised by how hard it was to get CEDET to 'just
> > work' well enough for everyone to use it as intended, and how often
> > people just jumped over to simpler one-off external tools because the
> > full suite way CEDET works was too heavy a lift.  That in turn resulted
> > in not a lot of contributors to help support/improve those workflows.
> > Tools like LSP also became good enough where there was no way I could
> > keep up.  I had hoped to pull data from external tools like lsp into the
> > framework CEDET used, but again the simpler one-off tools were too
> > appealing to that audience.
> >
> > Overall, I think that is fine though - having many projects
> > experimenting with different techniques, and having the best solution
> > win is the benefit of free software.  Developing CEDET back when it was
> > the only game it town was a good time with many good people helping, and
> > I am glad to have been a part of that, and I'm glad CEDET is still
> > useful in many cases.
>
> I think there should be a substantive place for such a framework in 
> Emacs, regardless of external tools that can be used to provide some 
> of the analysis.  Even if a mode doesn't use the parser generated by a 
> grammar, the grammar can also provide a description of the syntactic 
> structure that can be used in separating fontification from syntactic 
> analysis.  If I understand it correctly, Semantic provides support for 
> that generic approach and tying the classification to the text through 
> overlays  It should be straightforward for a major mode to create a 
> set of faces that can be applied generically by font-lock based on 
> those overlays instead of via regular expressions on the underlying text.
> I've skimmed lsp-mode, but I can't tell how it attaches the analysis 
> from the server to the text.
> Just looking at tsc-core's GitHub page, I don't see a similar generic 
> approach being provided.  I get the impression there is a lot of 
> dependence on the individual language/mode as to how the information 
> gets incorporated in the fontification.

One of my goals in CEDET/Semantic was to provide distinct interfaces 
between layers.  Thus, there is a parsing/tagging layer, a middle layer 
to manage tagging data, and a few example tools that could take 
advantage of the data.

This mostly worked well.  One parser that didn't make it into Emacs used 
ctags which added support for 8 additional modes such as sh, asm, & 
pascal.  This tool would populate the tag data the same way as other 
semantic parsers, so when you use some of tools that sit on top, such as 
'senator' everything just works.

So indeed, if the middle layer were to be used with more tools, new 
parser/taggers could just fill it in as a way to let those tools work.  
In fact, there's no reason why lsp couldn't populate the tagging layer 
while continuing to do the other things it does well.

Eric