Sorry for the delayed response.
 
> For many Emacs users, they want a quieter or even minimal fontification.
 
I'm not against it. I just think that highlighting of an interpolation
as a string is wrong. Is it possible to set quiet fontification in
emacs-lisp mode, in such a way that `keywords' in doc-strings were
fontified as a doc-string itself? I think it is similar to
interpolation, it serves the purpose of separating different semantic
elements from each other. IMHO, users who like quiet levels will benefit
from interpolation highlighted differently.
 
> Instead of the third level, the check should use the value
> treesit-font-lock-level. And it should check for each level smaller than
> or equal to treesit-font-lock-level.
 
Done.
 
> Non-essential fontification like "var" shouldn't be
> activated at that level. So I suggest to put it in the variable feature,
> along with many other non-essential fontifications. (Variable feature is
> placed at level 4.)
 
I added a new feature variable-definition for variables defined for local scopes and put it on the 3rd level.
I also added rules to variable-definition feature for variables in list
comprehension ( [var+1 for var in []] ) and as_pattern (with T as var:,
except E as var:, case str() as var:).
 
I've noticed that vars in `for var1, (var2, var3) in []:` are highlighted by the rule from the assignment feature (specifically `pattern_list`, `tuple_pattern`).
It seems easy to fix `pattern_list`, but not so easy for
`tuple_pattern`, since this node may occur recursively.
I didn't touch these rules for now.
 
 
 
Summary for all changes in the patch.
 
New feature variable-definition:
`for var in range(3)`
`[var+1 for var in []]`
`with T as var:`
`except E as var:`
`case str() as var:`
highlight var as font-lock-variable-name-face
 
assignment feature:
var := 3 (named_expression)
var *= 3 (augmented_assignment)
Highlight var as font-lock-variable-name-face.
 
Make list_splat_pattern query more precise.
list_splat_pattern may appear not only in assignments: var, *rest = call(),
but in the parameter list too: def f(*args).
Highlight args only for the first case in assignment feature.
 
 
type feature:
Fontify built-ins (dict,list,etc.) as types when they are used in type hints.
support nested union types, for example `Lvl1 | Lvl2[Lvl3[Lvl3], Lvl2]`.
This structure is represented via nesting binary_operator and subscript nodes in the grammar.
Function python--treesit-fontify-union-types iterates over all children and highlight identifier nodes.
 
Fontify base class names in the class definition: class Temp(Base1, pack0.Base2):
Fontify class patterns in case statement: case [TempC() | bytes(b)]:
Highlight the second argument as a type in isinstance/issubclass call:
isinstance(var2, (str, dict, Type1)); issubclass(var1, int|str)
 
For all dotted names of a type highlight only the last part of the name,
e.g. collections.abc.Iterator.
 
 
decorator feature:
Highlight dotted names: @pytest.mark.skip
Function python--treesit-fontify-dotted-decorator iterates over all nested attribute nodes and highlight identifier nodes.
 
When font-lock-level is set 4, `skip` had function-call face in: @pytest.mark.skip(reason='t')
Add `:override t` to decorator feature to override function-call face.
 
 
string feature:
Enable interpolation highlighting only if string-interpolation is
presented on the enabled levels of treesit-font-lock-feature-list.
Fix fontification of strings inside of f-strings interpolation,
e.g. for f"beg {'nested'}" - 'nested' was not fontified as string.
 
 
function feature:
Do not override the face of builtin functions (all, bytes etc.) with
the function call face
 
 
keyword feature:
Add "is not"  to the `python--treesit-keywords` list.
 
 
 
12.12.2023, 11:24, "Yuan Fu" <casouri@gmail.com>:



On 12/11/23 5:18 PM, Denis Zubarev wrote:

 Yuan and Dmitry, thank you for review and suggestions.
 > Can we do this instead: in python--treesit-fontify-string, we check if
   string-interpolation feature is enabled, if it is, fontify string_start,
   string_content and string_end only; if not, fontify the whole string.
 Done.
 Enable interpolation highlighting only if 'string-interpolation is
 presented on the third level of treesit-font-lock-feature-list.
 Personally, If I saw a f-string with an interpolation fontified as
 string, I would assume that it is bug.
 Clearly it is not a string, so it should be highlighted distinctly.
 But if it is a convention across all languages, we should follow it.


I  encourage everyone to also think in terms of fontification levels, in
addition to features.

For many Emacs users, they want a quieter or even minimal fontification.
Some people only want comment and function names highlighted, and they
can get it by setting the fontification level to 1, because
python-ts-mode only activates the comment and definition feature at
level 1. The string feature is at level 2, this level is still
relatively simplistic. And full string interpolation probably don't
belong at that level. That's why I separated it out into another
feature, and placed string-interpolation at level 3.
 

 > I think "for var in range(3)" should be part of the "definition" feature
   because a variable is defined there. Alongside parameters.
 I added it to definitions.


Again, if we think of fontification levels, the definition feature is
about fontifying the function names of definitions, and it's at a low
level (level 1). Non-essential fontification like "var" shouldn't be
activated at that level. So I suggest to put it in the variable feature,
along with many other non-essential fontifications. (Variable feature is
placed at level 4.)
 

 My thoughts about parameters. I started to extend rules for them since
 they are very limited now.
 But I'm not sure what face to use for them.
 I would like to not use the same face as for assignments, because I'd
 want to highlight them differently.
 It seems that there is no appropriate face in font-lock.el, so I ended
 up creating my own face in my config.
 Does it make sense to add new face for parameters in font-lock.el?
 Or it is too small feature for its own face?
 I also apply this face for keyword argument in function calls.

To be honest, I don't have any good ideas. Perhaps we can add a
parameter face that inherits from variable name face by default, Dmitry,
WDYT?

 Summary for all changes in the patch.
 definition feature:
 `for var in range(3)` highlight var as font-lock-variable-name-face
 assignment feature:
 var := 3 (named_expression)
 var *= 3 (augmented_assignment)
 Highlight var as font-lock-variable-name-face.
 Make list_splat_pattern query more precise.
 list_splat_pattern may appear not only in assignments: var, *rest =
 call(),
 but in the parameter list too: def f(*args).
 Highlight args only for the first case in assignment feature.
 type feature:
 Fontify built-ins (dict,list,etc.) as types when they are used in type
 hints.
 support nested union types, for example `Lvl1 | Lvl2[Lvl3[Lvl3], Lvl2]`.
 This structure is represented via nesting binary_operator and
 subscript nodes in the grammar.
 Function python--treesit-fontify-union-types iterates over all
 children and highlight identifier nodes.
 Fontify base class names in the class definition: class Temp(Base1,
 pack0.Base2):
 Fontify class patterns in case statement: case [TempC() | bytes(b)]:
 Highlight the second argument as a type in isinstance/issubclass call:
 isinstance(var2, (str, dict, Type1)); issubclass(var1, int|str)
 For all dotted names of a type highlight only the last part of the name,
 e.g. collections.abc.Iterator.
 decorator feature:
 Highlight dotted names: @pytest.mark.skip
 Function python--treesit-fontify-dotted-decorator iterates over all
 nested attribute nodes and highlight identifier nodes.
 When font-lock-level is set 4, `skip` had function-call face in:
 @pytest.mark.skip(reason='t')
 Add `:override t` to decorator feature to override function-call face.
 string feature:
 Enable interpolation highlighting only if string-interpolation is
 presented on the third level of treesit-font-lock-feature-list.
 Fix fontification of strings inside of f-strings interpolation,
 e.g. for f"beg {'nested'}" - 'nested' was not fontified as string.

Instead of the third level, the check should use the value
treesit-font-lock-level. And it should check for each level smaller than
or equal to treesit-font-lock-level.

 function feature:
 Do not override the face of builtin functions (all, bytes etc.) with
 the function call face
 keyword feature:
 Add "is not"  to the `python--treesit-keywords` list.
 

Thanks, they look good. The patch is getting rather large, let's focus
on getting the existing changes merged rather than adding new stuff to
it. Though I think your copyright assignment hasn't completed, right?

Yuan