From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Yuan Fu Newsgroups: gmane.emacs.bugs Subject: bug#67061: [PATCH] Improve syntax highlighting for python-ts-mode Date: Tue, 12 Dec 2023 00:24:41 -0800 Message-ID: <78ffdcf3-e322-49ea-a0d5-d0485ade9e73@gmail.com> References: <8734xdni6y.fsf@yandex.ru> <831qcwycbj.fsf@gnu.org> <83v8a3qh6m.fsf@gnu.org> <834jhadvkt.fsf@gnu.org> <7aee7e42-c07d-9131-18a9-4806f07d4267@gutov.dev> <83a5qw7izt.fsf@gnu.org> <172531702081590@mail.yandex.ru> <212931702208489@mail.yandex.ru> <0c6e2e14-b494-a8cb-3893-ffb39577baf9@gutov.dev> <7b17c99d-6e4b-43b3-af93-993901a3a4ea@gmail.com> <95071702343720@mail.yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="4019"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla Thunderbird Cc: "67061@debbugs.gnu.org" <67061@debbugs.gnu.org> To: Denis Zubarev , Dmitry Gutov , Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Dec 12 09:26:24 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rCy5v-0000iY-0k for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 12 Dec 2023 09:26:23 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rCy5Q-0003H2-SZ; Tue, 12 Dec 2023 03:25:56 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rCy5L-0003GA-H8 for bug-gnu-emacs@gnu.org; Tue, 12 Dec 2023 03:25:47 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rCy5L-0001EJ-8h for bug-gnu-emacs@gnu.org; Tue, 12 Dec 2023 03:25:47 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rCy5a-0007gq-87 for bug-gnu-emacs@gnu.org; Tue, 12 Dec 2023 03:26:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Yuan Fu Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 12 Dec 2023 08:26:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67061 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 67061-submit@debbugs.gnu.org id=B67061.170236951029495 (code B ref 67061); Tue, 12 Dec 2023 08:26:02 +0000 Original-Received: (at 67061) by debbugs.gnu.org; 12 Dec 2023 08:25:10 +0000 Original-Received: from localhost ([127.0.0.1]:55217 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rCy4j-0007ff-SE for submit@debbugs.gnu.org; Tue, 12 Dec 2023 03:25:10 -0500 Original-Received: from mail-oi1-x231.google.com ([2607:f8b0:4864:20::231]:51402) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rCy4e-0007f3-Cy for 67061@debbugs.gnu.org; Tue, 12 Dec 2023 03:25:08 -0500 Original-Received: by mail-oi1-x231.google.com with SMTP id 5614622812f47-3b9e2a014e8so3663488b6e.2 for <67061@debbugs.gnu.org>; Tue, 12 Dec 2023 00:24:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1702369483; x=1702974283; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=8il+Wj6PdqRnvWWfMp/NLw1V5MoOV7hBwCuehu3ldPY=; b=B3VUz9lghzr6vFfsqJ51aHc+gPJ/4uC/+rD5OlEPWNd7n0+FoXsXKxkSiIY7CEm6Vo 2rS9dHKmKRmxJ6Z+pCcytsxsG+dHgl6Gb/+zCcUF5cSCySiX9JkbxDw0qpeJL9NW8nke t4ZvfkquvvFK9mZ612VHWmgMlAqyqxiFcWM6Sov1L/Sm2UJiYfwdEiwNqi1sR+rPvb6r 2/a94IbSeMNnbSd3cJnv2z5QUCTtS8BFJUcS0HBRp0QTaz2EdKg2EdBkWBN54rUsY7nh JMd0TCTKt17fmMm/+mb/jc1cH97M8H7x1/y2vgmrB15W+67DYF+APhoH8kyzRMaxlgmK 7Rqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702369483; x=1702974283; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8il+Wj6PdqRnvWWfMp/NLw1V5MoOV7hBwCuehu3ldPY=; b=coMcXEx4C2jYFTkfbJ8NhXmpNvqMP/7TAG0p3hSdKAom6UU5U49D4riZ/IpEaTOdEy 26y3zmHXQeZl3r+dl95U1t+fNEV4OLCjd6ow5wd5CrqsVAl0mi9k7QCZ/Nw4XUL6Sulv fWiAOAwedbW2KCvha8kg5RvFciQyGJ1WwbjV8LeH4Fr9ajKZzSYj9IKWLPbEBX2gWckk Vi79s6xVEBaZHalHTRNlu+MAoUbBjn8Op4wg5VaNBmmOZYid2K0qq75Fw9x+MIXSzt1y Y+HwcQ3Tez1w57C9YSCF6D0H8J5p8dWW9tSUH+a3wv/L0oo6Jz+BsAJNGvGguRgTmg0J 0jGA== X-Gm-Message-State: AOJu0YxeO6cMgXrQHEjmeD1Sj/Pt1HN7an499dAbT9zdNGKX6x9m840n +ITx4/oxqg9sxLJxw8u2fIA= X-Google-Smtp-Source: AGHT+IG90rYZaKFyJfW0c9pZHBk+dCrtwDEGdD4SEFJeljsZivrt7BkUpPmQcQOhBi838tRNjqFBrA== X-Received: by 2002:a05:6808:2099:b0:3b9:e618:1861 with SMTP id s25-20020a056808209900b003b9e6181861mr7554390oiw.1.1702369483412; Tue, 12 Dec 2023 00:24:43 -0800 (PST) Original-Received: from [192.168.1.7] (172-117-161-177.res.spectrum.com. [172.117.161.177]) by smtp.gmail.com with ESMTPSA id d18-20020aa78692000000b006bdd7cbcf98sm7580963pfo.182.2023.12.12.00.24.42 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 12 Dec 2023 00:24:43 -0800 (PST) Content-Language: en-US In-Reply-To: <95071702343720@mail.yandex.ru> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:276021 Archived-At: On 12/11/23 5:18 PM, Denis Zubarev wrote: > Yuan and Dmitry, thank you for review and suggestions. > > Can we do this instead: in python--treesit-fontify-string, we check if >   string-interpolation feature is enabled, if it is, fontify string_start, >   string_content and string_end only; if not, fontify the whole string. > Done. > Enable interpolation highlighting only if 'string-interpolation is > presented on the third level of treesit-font-lock-feature-list. > Personally, If I saw a f-string with an interpolation fontified as > string, I would assume that it is bug. > Clearly it is not a string, so it should be highlighted distinctly. > But if it is a convention across all languages, we should follow it. I  encourage everyone to also think in terms of fontification levels, in addition to features. For many Emacs users, they want a quieter or even minimal fontification. Some people only want comment and function names highlighted, and they can get it by setting the fontification level to 1, because python-ts-mode only activates the comment and definition feature at level 1. The string feature is at level 2, this level is still relatively simplistic. And full string interpolation probably don't belong at that level. That's why I separated it out into another feature, and placed string-interpolation at level 3. > > I think "for var in range(3)" should be part of the "definition" feature >   because a variable is defined there. Alongside parameters. > I added it to definitions. Again, if we think of fontification levels, the definition feature is about fontifying the function names of definitions, and it's at a low level (level 1). Non-essential fontification like "var" shouldn't be activated at that level. So I suggest to put it in the variable feature, along with many other non-essential fontifications. (Variable feature is placed at level 4.) > My thoughts about parameters. I started to extend rules for them since > they are very limited now. > But I'm not sure what face to use for them. > I would like to not use the same face as for assignments, because I'd > want to highlight them differently. > It seems that there is no appropriate face in font-lock.el, so I ended > up creating my own face in my config. > Does it make sense to add new face for parameters in font-lock.el? > Or it is too small feature for its own face? > I also apply this face for keyword argument in function calls. To be honest, I don't have any good ideas. Perhaps we can add a parameter face that inherits from variable name face by default, Dmitry, WDYT? > Summary for all changes in the patch. > definition feature: > `for var in range(3)` highlight var as font-lock-variable-name-face > assignment feature: > var := 3 (named_expression) > var *= 3 (augmented_assignment) > Highlight var as font-lock-variable-name-face. > Make list_splat_pattern query more precise. > list_splat_pattern may appear not only in assignments: var, *rest = > call(), > but in the parameter list too: def f(*args). > Highlight args only for the first case in assignment feature. > type feature: > Fontify built-ins (dict,list,etc.) as types when they are used in type > hints. > support nested union types, for example `Lvl1 | Lvl2[Lvl3[Lvl3], Lvl2]`. > This structure is represented via nesting binary_operator and > subscript nodes in the grammar. > Function python--treesit-fontify-union-types iterates over all > children and highlight identifier nodes. > Fontify base class names in the class definition: class Temp(Base1, > pack0.Base2): > Fontify class patterns in case statement: case [TempC() | bytes(b)]: > Highlight the second argument as a type in isinstance/issubclass call: > isinstance(var2, (str, dict, Type1)); issubclass(var1, int|str) > For all dotted names of a type highlight only the last part of the name, > e.g. collections.abc.Iterator. > decorator feature: > Highlight dotted names: @pytest.mark.skip > Function python--treesit-fontify-dotted-decorator iterates over all > nested attribute nodes and highlight identifier nodes. > When font-lock-level is set 4, `skip` had function-call face in: > @pytest.mark.skip(reason='t') > Add `:override t` to decorator feature to override function-call face. > string feature: > Enable interpolation highlighting only if string-interpolation is > presented on the third level of treesit-font-lock-feature-list. > Fix fontification of strings inside of f-strings interpolation, > e.g. for f"beg {'nested'}" - 'nested' was not fontified as string. Instead of the third level, the check should use the value treesit-font-lock-level. And it should check for each level smaller than or equal to treesit-font-lock-level. > function feature: > Do not override the face of builtin functions (all, bytes etc.) with > the function call face > keyword feature: > Add "is not"  to the `python--treesit-keywords` list. > Thanks, they look good. The patch is getting rather large, let's focus on getting the existing changes merged rather than adding new stuff to it. Though I think your copyright assignment hasn't completed, right? Yuan