bug#26217: bug#2910: 23.0.60; Shell-script coloring bug

[Dmitry Gutov <dmitry@gutov.dev>]

On 14/10/2023 22:07, Stefan Monnier wrote:
>>> FWIW, it's not clear at all what such a layer would look like, so we're
>>> pretty far from it.  I'd welcome people start thinking about it, maybe
>>> by looking at existing alternatives like our own `wisi` (in GNU ELPA),
>>> SMIE, maybe LSP (assuming there are servers out there which can provide
>>> that kind of functionality), etc...
>> I don't know how feasible that would be,
> 
> That's the wrong way to look at it.  There is no doubt that it
> is feasible.  What is under question is what it would take.

And whether it would be in any way more economical than just waiting for 
the next comparable solution to come along and writing either 
compatibility shims inside each ts mode, or copying code and adapting 
into the next set of *-ts<N>-mode modes.

So for now we could be better off just coming up with an easier way to 
migrate user configurations across major modes for a given language than 
what we have now.

>> given that the ts major modes we write have to reference fairly low
>> level concerns (such as node names, different across all grammars).
> 
> That just means that a given set of highlighting/indentation rules would
> not necessarily work with all possible parser-backends.  But maybe we
> could bridge the gap by allowing some intermediate layer that could do
> things like translate node names (could be useful even within the
> tree-sitter context to deal with evolving grammars).

That's not impossible, but IME the sets of rules are fairly uniformly 
lower-level.

> I'm not saying this is the way to go, mind you.  I don't know how it
> could/should work.
> 
> But I do think we could do worse than start thinking about it, because
> tree-sitter is the kind of technology that's on the "treadmill", whereas
> Emacs' evolution has a different pace: how well will Emacs-29's TS modes
> work with 2028's tree-sitter grammars?

1. That's why I'm sure we'll start using some grammar pinning - either 
to the commit hash, or to the version range.

2. Overall it's a solid point: either tree-sitter "stabilizes" after a 
while, or goes off the map.

>> Maybe porting Lezer (https://lezer.codemirror.net/) could become
> 
> Interesting, thanks.
> 
>> a replacement in such a scenario, but then we're back to maintaining
>> our own grammars again, and with lower performance by an order of
>> a magnitude.
> 
> We could look into Lezer support just to help us guide the design of an
> intermediate layer API.  No need to maintain lots of our own grammars or
> take the performance impact :-)

It's as good an approach as any, though we have no guarantee that the 
"next tree-sitter" will have a similar enough shape to either of TS or 
Lezer, for an easy future migration.