bug#67061: [PATCH] Improve syntax highlighting for python-ts-mode

[Yuan Fu <casouri@gmail.com>]

On 12/12/23 4:44 PM, Dmitry Gutov wrote:
> On 12/12/2023 10:24, Yuan Fu wrote:
>>> > I think "for var in range(3)" should be part of the "definition" 
>>> feature
>>>   because a variable is defined there. Alongside parameters.
>>> I added it to definitions.
>>
>> Again, if we think of fontification levels, the definition feature is 
>> about fontifying the function names of definitions, and it's at a low 
>> level (level 1). Non-essential fontification like "var" shouldn't be 
>> activated at that level. So I suggest to put it in the variable 
>> feature, along with many other non-essential fontifications. 
>> (Variable feature is placed at level 4.)
>
> I disagree: 'var' in this example is not much different from a 
> function parameter. It's a definite place where a variable's name 
> introduced in the current scope.
>
> Python doesn't have special keywords for variable declarations (unlike 
> 'let' in JavaScript or typed declaration in C), so the first time a 
> variable is introduced serves as its declaration. For assignments, we 
> can't easily determine which is the first time for a given scope, but 
> examples like 'for var in ...' or 'except ZeroDivisionError as e:' or 
> '[... for var in ...]' are all unambiguously variable definitions.

Sure, I don't really care too much about which feature should a rule be 
in; what I do care about is to keep first and second fontification level 
relatively quite and minimal, and keep level 3 reasonably conservative. 
And people that want a lot of highlight can turn on level 4.

>
> So I think that:
>
> a) All variable definitions (functions parameters or not) should use 
> font-lock-variable-name-face -- to make it easier to find where a 
> given symbol is introduced.
> b) No font-lock-variable-name-face highlights should be put into the 
> 'variable' feature, which is disabled by default. All of the examples 
> above should either go into 'definition', or if somebody does like 
> that approach, into some new 'variable-declaration' feature (enabled 
> by default). And maybe some into 'assignment', which is on feature 
> level 3.
> c) The 'variable' feature should, at this point, only contain the 
> relatively useless highlights, since we don't track variable lifetimes 
> yet. That's why it's disabled by default.
>
> The current situation across ts modes is that js-ts-mode has variable 
> declarations in the 'definition' feature (and not by my hand, FWIW); 

Gah!

> ruby-ts-mode has a separate 'parameter-definition' feature that 
> encompasses both parameters and other variables; 

> in c-ts-mode highlighting for 'int i = 4' is split between 
> 'definition' and 'assignment' (the latter seemingly redundant); 

Should've been in assignment IMO. I probably overlooked it.

> typescript-ts-mode and rust-ts-mode also follow the principle, more or 
> less.

Well, the only ts-mode that I actually wrote is python-ts-mode. For 
other major modes, I can only suggest. Even for python-ts-mode, I don't 
want to exert my personal opinion onto it too much, except for keeping 
font-lock level 1 and 2 quiet.

>>> My thoughts about parameters. I started to extend rules for them 
>>> since they are very limited now.
>>> But I'm not sure what face to use for them.
>>> I would like to not use the same face as for assignments, because 
>>> I'd want to highlight them differently.
>>> It seems that there is no appropriate face in font-lock.el, so I 
>>> ended up creating my own face in my config.
>>> Does it make sense to add new face for parameters in font-lock.el?
>>> Or it is too small feature for its own face?
>>> I also apply this face for keyword argument in function calls.
>> To be honest, I don't have any good ideas. Perhaps we can add a 
>> parameter face that inherits from variable name face by default, 
>> Dmitry, WDYT?
>
> As per above, parameters don't seem too different from any other 
> variable declarations from my POV. They are similarly useful, so I'd 
> highlight them the same way.
>
> Do we want to have a common face which would inherit from 
> font-lock-variable-name-face and would be used solely for 
> function/methods parameters and nothing else? I don't object, but I 
> don't quite see the point either.

I agree.

Yuan