Re: Comments to the new tree sitter implementation

[Yuan Fu <casouri@gmail.com>]

> 
> Hi, there!
> 
>> 
>> Sorry that I can’t get back to you very soon. I’ve been incredibly
>> busy recently. I’ll find time to look at your points and try to debug
>> the problems you mention with typescript-ts.el.
>> 
> 
> No worries.  I'll just note one more thing because I remember it now:
> Why are you calling the font lock default function after tree sitter has
> done its job?  That seems like the wrong thing for several reasons:
> 
> 1. Shouldn't tree sitter be sufficient?
> 2. It makes the default function get higher precedence than tree-sitter,
> causing some troublesome behavior.

That is intended. Maybe I should emphasize in the manual that major modes shouldn’t set regex font-locks if tree-sitter is enabled (added to todo list). I left regex font-lock enabled and to have higher precedence because many Emacs features uses regex font-lock for arbitrary highlighting. One example is highlight-regexp, but also highlights for outline, and packages like hl-todo, rainbow-mode, etc. It makes sense for their highlights to override the “base” highlight applied by tree-sitter.

> 
> Specifically, if you set single-quote as a string in the syntax table,
> to make the major mode understand it as a string delimiter, it will mess
> up comments.  This example snippet will make the rest of the file
> fontlock as a string:
> 
> ```typescript
> function foo() {
>  // don't mark the rest of the file as a string because of this:
>  //    ^
>  const dont = 3;
>  const do = "":
>  const it = {};
> }
> ```
> 
> If the last two lines of `tree-sitter-font-lock-fontify-region` are
> removed, things appear to work as normal.  It also works fine with
> things such as eglot marking a variable as unused in the comment face
> etc.

IIUC this can be solved by disabling the syntax-table-based font-locking by setting KEYWORDS-ONLY in font-lock-defaults.

Thanks,
Yuan