bug#59977: 29.0.60; python-ts-mode does not fontify function-invocations using font-lock-function-name [PATCH]

bug#59977: 29.0.60; python-ts-mode does not fontify function-invocations using font-lock-function-name [PATCH]

[Jostein Kjønigsen]

[-- Attachment #1.1: Type: text/plain, Size: 10650 bytes --]

Open a python buffer, activate python-ts-mode and write a simple 
statement invoking a function. Example code:

    def somefunc(val)
         return val

    result = someFunc("123");

Observe how function-name is fontified as function name in the function 
definition, but not when invoked.

This is inconsistent with how other major-modes fontify function names.

Attached is a patch which fixes this issue.

--


In GNU Emacs 29.0.60 (build 1, x86_64-pc-linux-gnu, GTK+ Version
  3.24.34, cairo version 1.16.0) of 2022-12-09 built on ThinkPad-T14s
Repository revision: 37f3038d791c1e52b92fbbaf8221092ae8a2f284
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12201003
System Description: Ubuntu 22.10

Configured using:
  'configure --with-json --with-tree-sitter'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
   value of $LC_MONETARY: nb_NO.UTF-8
   value of $LC_NUMERIC: nb_NO.UTF-8
   value of $LC_TIME: nb_NO.UTF-8
   value of $LANG: en_US.UTF-8
   locale-coding-system: utf-8-unix

Major mode: Magit

Minor modes in effect:
   global-git-commit-mode: t
   magit-auto-revert-mode: t
   treemacs-filewatch-mode: t
   treemacs-follow-mode: t
   treemacs-git-mode: t
   treemacs-fringe-indicator-mode: t
   editorconfig-mode: t
   which-function-mode: t
   helm-mode: t
   helm-minibuffer-history-mode: t
   shell-dirtrack-mode: t
   helm--remap-mouse-mode: t
   async-bytecomp-package-mode: t
   delete-selection-mode: t
   global-auto-revert-mode: t
   yas-global-mode: t
   yas-minor-mode: t
   global-nlinum-mode: t
   ido-yes-or-no-mode: t
   override-global-mode: t
   server-mode: t
   global-hl-line-mode: t
   pixel-scroll-precision-mode: t
   doom-modeline-mode: t
   tooltip-mode: t
   global-eldoc-mode: t
   show-paren-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   buffer-read-only: t
   column-number-mode: t
   line-number-mode: t
   transient-mark-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t

Load-path shadows:
/home/jostein/.emacs.d/elpa/transient-20221202.1727/transient hides 
/home/jostein/build/emacs/lisp/transient

Features:
(shadow sort emacsbug mail-extr misearch multi-isearch lsp-diagnostics
lsp-headerline lsp-icons lsp-modeline dap-mouse dap-ui gdb-mi bindat gud
bui bui-list bui-info bui-entry bui-core bui-history bui-button
bui-utils lsp-lens dap-gdb-lldb dap-netcore dap-node dap-utils dap-pwsh
dap-python dap-mode dap-tasks dap-launch posframe dap-overlays view
python skeleton helm-bookmark helm-net helm-adaptive magit-bookmark
treemacs-bookmarks treemacs-tags bookmark help-fns radix-tree apropos
goto-addr helm-command helm-elisp helm-eval edebug debug backtrace
helm-info git-rebase magit-extras flyspell ispell face-remap
magit-submodule magit-obsolete magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log magit-diff smerge-mode
diff git-commit log-edit message sendmail yank-media rfc822 mml mml-sec
epa mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mailabbrev gmm-utils mailheader pcvs-util magit-core
magit-autorevert magit-margin magit-transient magit-process with-editor
magit-mode transient magit-git magit-base magit-section crm compat-27
compat-26 elec-pair toml-ts-mode treesit executable disp-table
bug-reference vc-git diff-mode vc-dispatcher vc-svn winner ffap
tramp-archive tramp-gvfs zeroconf tramp-cache time-stamp dbus add-log
lsp-zig lsp-steep lsp-svelte lsp-sqls lsp-ruby-syntax-tree lsp-yaml
lsp-xml lsp-vimscript lsp-vhdl lsp-volar lsp-vetur lsp-html lsp-verilog
lsp-vala lsp-v lsp-typeprof lsp-ttcn3 lsp-toml lsp-terraform lsp-tex
lsp-sorbet lsp-solargraph lsp-rust lsp-rf lsp-remark lsp-racket lsp-r
lsp-purescript lsp-pylsp lsp-pyls lsp-pwsh lsp-php lsp-pls
lsp-perlnavigator lsp-perl lsp-openscad lsp-ocaml lsp-magik lsp-nix
lsp-nim lsp-nginx lsp-mint lsp-marksman lsp-markdown lsp-lua lsp-kotlin
lsp-json lsp-javascript lsp-idris lsp-haxe lsp-groovy lsp-hack
lsp-graphql lsp-gleam lsp-go lsp-completion lsp-gdscript lsp-fsharp
lsp-fortran lsp-eslint lsp-erlang lsp-emmet lsp-elixir lsp-elm
lsp-dockerfile lsp-docker yaml lsp-dhall lsp-d lsp-css lsp-csharp gnutls
lsp-crystal lsp-cmake lsp-clojure lsp-treemacs lsp-treemacs-generic
lsp-treemacs-themes treemacs-treelib treemacs treemacs-header-line
treemacs-compatibility treemacs-mode treemacs-interface
treemacs-persistence treemacs-filewatch-mode treemacs-follow-mode
treemacs-rendering treemacs-annotations treemacs-async
treemacs-workspaces treemacs-dom treemacs-visuals
treemacs-fringe-indicator treemacs-scope pulse treemacs-faces
treemacs-icons treemacs-themes treemacs-core-utils pfuture
treemacs-logging treemacs-customization treemacs-macros
lsp-semantic-tokens lsp-clangd lsp-beancount lsp-bash lsp-astro
lsp-ansible lsp-angular lsp-ada warnings lsp-actionscript
ido-completing-read+ memoize minibuf-eldef elisp-slime-nav paredit
lsp-mode lsp-protocol tree-widget spinner network-stream nsm
markdown-mode color lv inline ht ewoc epg rfc6068 epg-config
highlight-symbol flycheck editorconfig editorconfig-core
editorconfig-core-handle editorconfig-fnmatch company-oddmuse
company-keywords company-etags etags fileloop generator company-gtags
company-dabbrev-code company-dabbrev company-files company-clang
company-capf company-cmake company-semantic company-template
company-bbdb company which-func hideshow eww url-queue thingatpt shr
pixel-fill kinsoku url-file svg xml dom puny mm-url gnus nnheader
gnus-util mail-utils range mm-util mail-prsvr helm-imenu pcase imenu
ob-plantuml org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro
org-src ob-comint org-pcomplete org-list org-footnote org-faces
org-entities noutline outline ob-emacs-lisp ob-core ob-eval org-cycle
org-table ol org-fold org-fold-core org-keys oc org-loaddefs find-func
cal-menu calendar cal-loaddefs org-version org-compat org-macs helm-mode
helm-misc helm-files image-dired image-dired-tags image-dired-external
image-dired-util xdg image-mode dired dired-loaddefs exif tramp
tramp-loaddefs trampver tramp-integration cus-edit pp cus-load icons
wid-edit files-x tramp-compat shell pcomplete parse-time iso8601
time-date ls-lisp helm-buffers helm-occur helm-tags helm-locate
helm-grep helm-regexp format-spec helm-utils helm-help helm-types helm
helm-global-bindings helm-easymenu helm-core async-bytecomp helm-source
helm-multi-match helm-lib async helm-config delsel autorevert filenotify
yasnippet derived nlinum linum ido-yes-or-no advice ido edmacro kmacro
use-package-bind-key bind-key easy-mmode xref project server hl-line
pixel-scroll cua-base compile-eslint compile comint ansi-osc ansi-color
ring doom-modeline doom-modeline-segments doom-modeline-env
doom-modeline-core all-the-icons all-the-icons-faces data-material
data-weathericons data-octicons data-fileicons data-faicons
data-alltheicons shrink-path rx f f-shortdoc shortdoc
text-property-search s dash compat compat-macs dracula-theme cl-extra
help-mode use-package-ensure use-package-core finder-inf
flycheck-autoloads helm-autoloads helm-core-autoloads
expand-region-autoloads all-the-icons-autoloads bmx-mode-autoloads
doom-modeline-autoloads async-autoloads dracula-theme-autoloads
highlight-symbol-autoloads multiple-cursors-autoloads nlinum-autoloads
toml-mode-autoloads pkg-info-autoloads epl-autoloads undo-tree-autoloads
rust-mode-autoloads editorconfig-autoloads magit-autoloads
yasnippet-autoloads elisp-slime-nav-autoloads crontab-mode-autoloads
ido-yes-or-no-autoloads magit-section-autoloads cargo-autoloads
ido-completing-read+-autoloads memoize-autoloads paredit-autoloads
company-autoloads yaml-mode-autoloads powershell-autoloads
dap-mode-autoloads lsp-docker-autoloads yaml-autoloads
lsp-treemacs-autoloads treemacs-autoloads cfrs-autoloads
posframe-autoloads hydra-autoloads pfuture-autoloads
ace-window-autoloads avy-autoloads bui-autoloads lsp-mode-autoloads
lv-autoloads markdown-mode-autoloads spinner-autoloads ht-autoloads
git-commit-autoloads transient-autoloads compat-autoloads
cmake-mode-autoloads shrink-path-autoloads f-autoloads info
dash-autoloads s-autoloads popup-autoloads queue-autoloads package
browse-url url url-proxy url-privacy url-expand url-methods url-history
url-cookie generate-lisp-file url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs password-cache
json subr-x map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs
cl-lib rmc iso-transl tooltip cconv eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process emacs)

Memory information:
((conses 16 817351 95196)
  (symbols 48 56343 1)
  (strings 32 240462 14017)
  (string-bytes 1 6962558)
  (vectors 16 116127)
  (vector-slots 8 2199610 186656)
  (floats 8 1040 698)
  (intervals 56 9967 4449)
  (buffers 992 53))
-- 
Vennlig hilsen
*Jostein Kjønigsen*

jostein@kjonigsen.net 🍵 jostein@gmail.com
https://jostein.kjønigsen.no <https://jostein.kjønigsen.no>

[-- Attachment #1.2: Type: text/html, Size: 13984 bytes --]

[-- Attachment #2: 0004-python-ts-mode-fontify-function-invocations-using-fo.patch --]
[-- Type: text/x-patch, Size: 1432 bytes --]

From a6e1f611c61d2f8a8ebf663a19982cd25152cf4a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jostein=20Kj=C3=B8nigsen?= <jostein@kjonigsen.net>
Date: Sun, 11 Dec 2022 21:20:27 +0100
Subject: [PATCH 4/4] python-ts-mode: fontify function-invocations using
 font-lock-function-name.

This is consistent with how many other tree-sitter based major-modes
does it, and would improve consistency for fontification in Emacs.
---
 lisp/progmodes/python.el | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index ebee703499a..f86fc4bbeac 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -1089,10 +1089,14 @@ python--treesit-settings
 
    :feature 'definition
    :language 'python
+   :override t
    '((function_definition
       name: (identifier) @font-lock-function-name-face)
      (class_definition
-      name: (identifier) @font-lock-type-face))
+      name: (identifier) @font-lock-type-face)
+     (call function: (identifier) @font-lock-function-name-face)
+     (call function: (attribute
+                      attribute: (identifier) @font-lock-function-name-face)))
 
    :feature 'keyword
    :language 'python
@@ -1158,7 +1162,6 @@ python--treesit-settings
 
    :feature 'property
    :language 'python
-   :override t
    '((attribute
       attribute: (identifier) @font-lock-property-face)
      (class_definition
-- 
2.37.2

bug#59977: 29.0.60; python-ts-mode does not fontify function-invocations using font-lock-function-name [PATCH]

[Yuan Fu]

Jostein Kjønigsen <jostein@secure.kjonigsen.net> writes:

> Open a python buffer, activate python-ts-mode and write a simple statement invoking a function. Example code:
>
>  def somefunc(val)
>      return val
>
>  result = someFunc("123");
>
> Observe how function-name is fontified as function name in the function definition, but not when invoked.
>
> This is inconsistent with how other major-modes fontify function
> names.

Ah, yes. That’s because the "definition" feature, which is by enabled by
default, only highlights definition and ignores invocation. OTOH, the
"function" feature highlights every instance of a function identifier,
but it isn’t enabled by default. Other tree-sitter major modes doesn’t
have "definition" and enable "function" by default, I think.

I’m planning to standardize the features in each major mode according to
the list I showed you earlier. It’d be great if you can have a look at
that list and see if you have any opinions, before we decide on the
final form of it and start to standardize those features in major modes.
(Of course, each language is different, the standard feature list is
just a base upon which major modes will extend/modify.)

Yuan

bug#59977: 29.0.60; python-ts-mode does not fontify function-invocations using font-lock-function-name [PATCH]

[Jostein Kjønigsen]

On 12.12.2022 23:54, Yuan Fu wrote:
> Ah, yes. That’s because the "definition" feature, which is by enabled by
> default, only highlights definition and ignores invocation. OTOH, the
> "function" feature highlights every instance of a function identifier,
> but it isn’t enabled by default.

I've tried latest python-ts-mode from latest emacs-29 branch, with 
fontification set to level 4 and I can still reproduce this error.

Looking at the code, I cannot see any "function" feature defined, and to 
me it seems a patch like the one I added is required to introduce this 
functionality.

> Other tree-sitter major modes doesn’t
> have "definition" and enable "function" by default, I think.

I believe this is standard for js-ts-mode, typescript-ts-mode, 
tsx-ts-mode and csharp-ts-mode. Even on level 3, and if it wasn't, as a 
user of those languages, I would argue in support of having it there :)

I can't "speak" on behalf of the other major-modes though.

> I’m planning to standardize the features in each major mode according to
> the list I showed you earlier. It’d be great if you can have a look at
> that list and see if you have any opinions, before we decide on the
> final form of it and start to standardize those features in major modes.
> (Of course, each language is different, the standard feature list is
> just a base upon which major modes will extend/modify.)
>
> Yuan
I guess that is another discussion entirely (how to implement a given 
set of features).

This bug is about a feature (function-call highlighting)  being fully 
missing :)

--
Jostein

bug#59977: 29.0.60; python-ts-mode does not fontify function-invocations using font-lock-function-name [PATCH]

[Yuan Fu]

Jostein Kjønigsen <jostein@secure.kjonigsen.net> writes:

> On 12.12.2022 23:54, Yuan Fu wrote:
>> Ah, yes. That’s because the "definition" feature, which is by enabled by
>> default, only highlights definition and ignores invocation. OTOH, the
>> "function" feature highlights every instance of a function identifier,
>> but it isn’t enabled by default.
>
> I've tried latest python-ts-mode from latest emacs-29 branch, with
> fontification set to level 4 and I can still reproduce this error.
>
> Looking at the code, I cannot see any "function" feature defined, and
> to me it seems a patch like the one I added is required to introduce
> this functionality.

Yes, you are exactly right. My point is that the change belongs to
"function" feature and not definition. I made a change that added your
rules to a new "function" feature.

>> Other tree-sitter major modes doesn’t
>> have "definition" and enable "function" by default, I think.
>
> I believe this is standard for js-ts-mode, typescript-ts-mode,
> tsx-ts-mode and csharp-ts-mode. Even on level 3, and if it wasn't, as
> a user of those languages, I would argue in support of having it there
> :)
>
> I can't "speak" on behalf of the other major-modes though.

I’m sure a lot of people would prefer that, no doubt. The other modes
currently having them on level 3 is something we should fix (unless we
end up deciding that we want "function" feature on level 3).

>> I’m planning to standardize the features in each major mode according to
>> the list I showed you earlier. It’d be great if you can have a look at
>> that list and see if you have any opinions, before we decide on the
>> final form of it and start to standardize those features in major modes.
>> (Of course, each language is different, the standard feature list is
>> just a base upon which major modes will extend/modify.)
>>
>> Yuan
> I guess that is another discussion entirely (how to implement a given
> set of features).
>
> This bug is about a feature (function-call highlighting)  being fully
> missing :)

Yes, I understand, I’m not against adding this in any way.

Yuan

bug#59977: 29.0.60; python-ts-mode does not fontify function-invocations using font-lock-function-name [PATCH]

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 159 bytes --]

I guess this one can be closed now :)

-- 
*Jostein Kjønigsen*
jostein.kjønigsen.no <https://jostein.kjønigsen.no>
jostein@kjonigsen.net - jostein@gmail.com

[-- Attachment #2: Type: text/html, Size: 599 bytes --]