Re: Tree-sitter integration in python.el

[Yuan Fu <casouri@gmail.com>]

> On Oct 11, 2022, at 3:15 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>>>> The complaint for font-lock-maximum-decoration is that it’s obscure and
>>>> too corse-grained. So my idea is for each major mode to provide
>>>> fined-grained controls like
>>>> python-fontify-type/f-string/assignment/built-in/etc. And tree-sitter
>>>> makes it easy to implement this kind of toggle. But I guess a global
>>>> control is also nice, I can make tree-sitter respect
>>>> font-lock-maximum-decoration, in addition to the fined grained
>>>> local-control.
>>> 
>>> I think having tree-sitter respect font-lock-maximum-decoration would
>>> be good, because it allows a major-mode agnostic way of controlling
>>> fontifications.  With tree-sitter in mind, we'd need to agree on what
>>> kind of syntactic entities are included in each level (which is also a
>>> Good Thing, because currently what is level N of font-lock is entirely
>>> up to the major-mode, AFAIU).
>> 
>> I think it is difficult to define syntactic entities for each level such
>> that it is generally enough to include all kinds of major mode out there,
>> and specific enough to be useful. It is easy for common programming
>> languages, but hard for others like html, css, prolog, etc.
> 
> How 'bout:
> 
> - Major modes provide a list of available fontification "features"
>  in a buffer-local variable like `treesit-font-lock-features`
>  e.g. `(type f-string assignment built-in ...)`.
> - the list of features is *ordered* so that
>  the user can control the fontification via a Customizable
>  `treesit-font-lock-level` that can be set between 0 and 100 to
>  select the corresponding proportion of features.
> - Users can additionally enable/disable those features individually via
>  some `treesit-font-lock-control` variable.

One problem I can see is that the same level could give very different busyness across modes. That would defeat the purpose of having a single setting. Say mode 1 has (A B C D E F G H I), where ABC are very basic, DEF moderate, and GHI fancy, mode 2 has (A B C D E F G), A basic, BCDEF moderate, G fancy. Then 80 would give a fancy font-lock in mode 1 and a moderate font-lock in mode 2. You get the idea.

Currently tree-sitter supports both :level and :toggle. When defining queries you can say :toggle python-fontify-types, and python-fontify-types can control this query. :level would be the global rough settings, :toggle would be the local, fine-grained setting. Instead of variables, :toggle could be changed to use symbols as you suggested, so it doesn’t create a gazillion variables. But WDYT of the general design?

Yuan