From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.bugs Subject: bug#67061: [PATCH] Improve syntax highlighting for python-ts-mode Date: Thu, 21 Dec 2023 01:34:30 +0200 Message-ID: <16489c3d-bddf-984b-6c98-2a1de9fdb6e8@gutov.dev> References: <8734xdni6y.fsf@yandex.ru> <831qcwycbj.fsf@gnu.org> <83v8a3qh6m.fsf@gnu.org> <834jhadvkt.fsf@gnu.org> <7aee7e42-c07d-9131-18a9-4806f07d4267@gutov.dev> <83a5qw7izt.fsf@gnu.org> <172531702081590@mail.yandex.ru> <212931702208489@mail.yandex.ru> <0c6e2e14-b494-a8cb-3893-ffb39577baf9@gutov.dev> <7b17c99d-6e4b-43b3-af93-993901a3a4ea@gmail.com> <95071702343720@mail.yandex.ru> <78ffdcf3-e322-49ea-a0d5-d0485ade9e73@gmail.com> <7371702772641@mail.yandex.ru> <27502181702944649@mail.yandex.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="14520"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Cc: "67061@debbugs.gnu.org" <67061@debbugs.gnu.org> To: Denis Zubarev , Yuan Fu , Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Thu Dec 21 00:35:21 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rG65u-0003WH-8k for geb-bug-gnu-emacs@m.gmane-mx.org; Thu, 21 Dec 2023 00:35:19 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rG65c-00077D-72; Wed, 20 Dec 2023 18:35:00 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rG65b-000774-4r for bug-gnu-emacs@gnu.org; Wed, 20 Dec 2023 18:34:59 -0500 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rG65a-0005Fc-MV for bug-gnu-emacs@gnu.org; Wed, 20 Dec 2023 18:34:58 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rG65e-0003bO-4n for bug-gnu-emacs@gnu.org; Wed, 20 Dec 2023 18:35:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Dmitry Gutov Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 20 Dec 2023 23:35:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 67061 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 67061-submit@debbugs.gnu.org id=B67061.170311528613821 (code B ref 67061); Wed, 20 Dec 2023 23:35:02 +0000 Original-Received: (at 67061) by debbugs.gnu.org; 20 Dec 2023 23:34:46 +0000 Original-Received: from localhost ([127.0.0.1]:42049 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rG65N-0003ao-Gd for submit@debbugs.gnu.org; Wed, 20 Dec 2023 18:34:46 -0500 Original-Received: from out2-smtp.messagingengine.com ([66.111.4.26]:44249) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rG65K-0003aa-VW for 67061@debbugs.gnu.org; Wed, 20 Dec 2023 18:34:44 -0500 Original-Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 4CD335C05F5; Wed, 20 Dec 2023 18:34:34 -0500 (EST) Original-Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Wed, 20 Dec 2023 18:34:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1703115274; x=1703201674; bh=VvregLDodIDU3sx0jomR28tLOF45AZBemzUNwqVXJjc=; b= oBY7B5MkTFuqU28vp2qoYdmEaf3sQz1zU4p5W9MAPJBVFnXphHvT2yir3c4nMmCP FZ8BJyxf1w8ovQbUFDltpDOXDJ+H9NhWqDf/r/FP/+kcg2j/1FYlijac8ySa7kQN GuqU+WQG4I2J0BEuq/y1+POJtvFpeFTie28n2QDSbXE2QZfYOUTOxq5fr1E36VRk jbWhLgjyt/Hkowa5zkiutG7gsglig8gjatDszwAASiK//lig7PV4tS/zN5oht8D3 OBkNakNyiAPbQaY2TI1fU/HFX1vGJBTmarv8ZhgjrbB2/J2/0RMBbo6hSpT7BrzK CvESpFVsYRtiLtbojo2PiQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1703115274; x= 1703201674; bh=VvregLDodIDU3sx0jomR28tLOF45AZBemzUNwqVXJjc=; b=K +GlU+96nQpF+MGJEjWJ6gJMEwCJT5QNgZKWP5o6bm0RN4jrEbACikpJbsJ7wE2cV bSMF7Y4+4jrvqM+0RP12lqJ2zva29xY+QeKQB+ka1SdCVcRNmBXqJInhB0Y9bOgg YTwYa0FCX6pw90QazaeKte+HWgMV6UiogxjX4JEzJcid8CRuC3SDCcvAvcFDRf+/ 6w2U6kbTQuPWcImlESwJvTBTgXm5ss3o9adgTTzQpeurSXNVqgvjngvu/TCZIZYf qolGOjCcYWI1ew9vHEYzGNSIxOOp1rHsbSCmTWPh4Fk4eHi13AK5NrW0yTFwUNOF V49gMAFmLA3FSHPmdTvMQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvddufedgudefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth htvghrnhephffhleeifffgveevudeugfeifeeuffevgfeutdeitefhiefgtedvheeuvedv vdefnecuffhomhgrihhnpehgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigvpe dtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumhhithhrhiesghhuthhovhdruggvvh X-ME-Proxy: Feedback-ID: i0e71465a:Fastmail Original-Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 20 Dec 2023 18:34:32 -0500 (EST) Content-Language: en-US In-Reply-To: <27502181702944649@mail.yandex.ru> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:276601 Archived-At: On 19/12/2023 02:14, Denis Zubarev wrote: > > If you recall my earlier complaint that these highlightings didn't work > > (and the tests didn't pass), this happened due to an older Python > grammar. > Thank you for investigating this. It seems this commit introduced > changes to type nodes hierarchy > (https://github.com/tree-sitter/tree-sitter-python/commit/bcbf41589f4dc38a98bda4ca4c924eb5cae26f7b). Could be this one, yes. > > The queries didn't lead to errors either (that's a good thing), but maybe > > we'll want to revisit these highlights later to add support for the > > older grammar as well. > It may lead to unnecessarily complex rules. I don't > know is it worth it, since users can easily update grammars. No problem. > > I'm not sure highlighting types based on the caller method and position > > is a good idea. I think that's backward, logically. If one puts a > > non-type value in such argument, and we would highlight it as a type -- > > that seems like the wrong message. > These two functions expect a type (or tuple of types) as the second > argument. To address your concerns about highlighting as a type a > non-type variable, I added regexp python--treesit-type-regex. This regex > matches if text is either built-in type or text starts with capital > letter. I extracted built-in types from the python--treesit-builtins > into its own variable python--treesit-builtin-types. > python--treesit-builtins is now constructing by appending > python--treesit-builtin-types and other built-ins. I hope it is ok. Thank you. I'm actually not sure if we _have to_ check the identifier names in this context (any chance to have a false negative, miss some valid types?), but it probably doesn't hurt either. > > One of the complaints is that "User" is not highlighted as a type when > > used in other, non-built-in methods, which like a reasonable question to > > me. Yes, Python is dynamic, but using CamelCase for types is a fairly > > regular convention, so highlighting such identifiers as types can work. > It is good idea, to highlight some variables as types. But I think it > should be done on the 4th level. One could split the variable feature > into multiple features: variable-type, variable-argument, variable-use, > etc. So for variable-type feature we can use python--treesit-type-regex > and highlight matched identifiers with type face. For now I wanted to > properly highlight types in places where they expected to be. I wouldn't mind the level 4 (after all, python-mode is also conservative here and doesn't add such highlighting), but I'd rather not add the special handling for isinstance/issubclass thing for the reasons previously outlined. Perhaps Yuan will disagree. I'm just here to say that the rest of the patch LGTM.