bug#61303: 29.0.60; rust-ts-mode does not consistently fontify nested macro-invocations

bug#61303: 29.0.60; rust-ts-mode does not consistently fontify nested macro-invocations

[Jostein Kjønigsen]

Hey there.

Thanks for another tree-sitter based mode to hack at :D

I think I've found an error/lacking in rust-ts-mode where we may have to 
move upstream to fix it.

Consider the following contrived (and yes, not very idiomatic) rust-code:

let result = format!("{}{}",
     "Well yes",
     format!("Or {} no?", "possibly"));

With rust-ts-mode first format! invocation will get highlighted as a 
macro-invocation, while the second will not.

Inspecting this using treesit-explore-mode, I get the following tree:

(let_declaration let pattern: (identifier) =
   value:
    (macro_invocation macro: (identifier) !
     (token_tree (
      (string_literal " ")
      (string_literal " ")
      (identifier)
      (token_tree (
       (string_literal " ")
       (string_literal " ")
       ))
      )))
   ;)

Notice the lack of a second/nested "macro_invocation"-node. Instead the 
second macro is just denoted as a "identifier".

To fix this, we'll probably need to upstream some bug-reports and PRs?

--
Jostein

In GNU Emacs 29.0.60 (build 7, x86_64-pc-linux-gnu, GTK+ Version
  3.24.36, cairo version 1.17.6) of 2023-02-05 built on thinkpad-t14s
Repository revision: 088425538f2122d88a4f4e132dbb2f1139648531
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12201007
System Description: Arch Linux

Configured using:
  'configure --with-json --with-tree-sitter'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY INOTIFY
PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
   value of $LANG: en_US.UTF-8
   locale-coding-system: utf-8-unix

Major mode: Rust

Minor modes in effect:
   treesit-explore-mode: t
   electric-pair-mode: t
   highlight-symbol-mode: t
   flycheck-mode: t
   editorconfig-mode: t
   company-mode: t
   which-function-mode: t
   helm-mode: t
   helm-minibuffer-history-mode: t
   shell-dirtrack-mode: t
   helm--remap-mouse-mode: t
   async-bytecomp-package-mode: t
   delete-selection-mode: t
   global-auto-revert-mode: t
   yas-global-mode: t
   yas-minor-mode: t
   global-nlinum-mode: t
   nlinum-mode: t
   ido-yes-or-no-mode: t
   override-global-mode: t
   server-mode: t
   global-hl-line-mode: t
   pixel-scroll-precision-mode: t
   doom-modeline-mode: t
   tooltip-mode: t
   global-eldoc-mode: t
   show-paren-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   column-number-mode: t
   line-number-mode: t
   transient-mark-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   hs-minor-mode: t

Load-path shadows:
/home/jostein/.emacs.d/elpa/transient-20230107.1528/transient hides 
/home/jostein/build/emacs/lisp/transient

Features:
(shadow sort mail-extr emacsbug message yank-media rfc822 mml mml-sec
epa derived epg rfc6068 epg-config mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums bug-reference face-remap cus-start rust-ts-mode
c-ts-common treesit helm-command helm-elisp helm-eval edebug helm-info
vc-hg vc-git diff-mode vc-bzr vc-dispatcher disp-table cargo
cargo-process markdown-mode color elec-pair rust-utils rust-mode
rust-rustfmt rust-playpen rust-compile rust-cargo vc-svn dired-aux
flyspell ispell help-fns radix-tree cl-print tramp-sh image-file
image-converter winner helm-external helm-net tramp-archive tramp-gvfs
tramp-cache time-stamp zeroconf dbus ffap add-log ido-completing-read+
memoize minibuf-eldef elisp-slime-nav paredit highlight-symbol flycheck
editorconfig editorconfig-core editorconfig-core-handle
editorconfig-fnmatch company-oddmuse company-keywords company-etags
etags fileloop generator company-gtags company-dabbrev-code
company-dabbrev company-files company-clang company-capf company-cmake
company-semantic company-template company-bbdb company eglot
external-completion array jsonrpc ert ewoc debug backtrace flymake-proc
flymake warnings which-func hideshow eww url-queue thingatpt shr
pixel-fill kinsoku url-file svg xml dom puny mm-url gnus nnheader
gnus-util mail-utils range mm-util mail-prsvr helm-imenu helm-mode
helm-misc helm-files image-dired image-dired-tags image-dired-external
image-dired-util xdg image-mode dired dired-loaddefs exif tramp
tramp-loaddefs trampver tramp-integration cus-edit pp cus-load wid-edit
files-x tramp-compat shell parse-time iso8601 ls-lisp helm-buffers
helm-occur helm-tags helm-locate helm-grep helm-regexp helm-utils
helm-help helm-types helm helm-global-bindings helm-easymenu helm-core
async-bytecomp helm-source helm-multi-match helm-lib async pcase imenu
ob-plantuml org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro
org-src ob-comint org-pcomplete pcomplete org-list org-footnote
org-faces org-entities time-date noutline outline icons ob-emacs-lisp
ob-core ob-eval org-cycle org-table ol org-fold org-fold-core org-keys
oc org-loaddefs find-func cal-menu calendar cal-loaddefs org-version
org-compat org-macs format-spec delsel autorevert filenotify yasnippet
nlinum linum ido-yes-or-no advice ido edmacro kmacro
use-package-bind-key bind-key easy-mmode xref project server hl-line
pixel-scroll cua-base compile-eslint compile text-property-search comint
ansi-osc ansi-color ring doom-modeline doom-modeline-segments
doom-modeline-env doom-modeline-core all-the-icons all-the-icons-faces
data-material data-weathericons data-octicons data-fileicons
data-faicons data-alltheicons shrink-path rx f f-shortdoc s dash compat
dracula-theme cl-extra help-mode use-package-ensure use-package-core
finder-inf yasnippet-autoloads ido-yes-or-no-autoloads
elisp-slime-nav-autoloads cmake-mode-autoloads flycheck-autoloads
pkg-info-autoloads magit-autoloads all-the-icons-autoloads
crontab-mode-autoloads powershell-autoloads doom-modeline-autoloads
undo-tree-autoloads rust-mode-autoloads magit-section-autoloads
paredit-autoloads dracula-theme-autoloads cargo-autoloads
yaml-mode-autoloads helm-autoloads popup-autoloads queue-autoloads
nlinum-autoloads bmx-mode-autoloads company-autoloads
git-commit-autoloads multiple-cursors-autoloads dap-mode-autoloads
lsp-treemacs-autoloads treemacs-autoloads cfrs-autoloads
posframe-autoloads hydra-autoloads pfuture-autoloads
ace-window-autoloads avy-autoloads bui-autoloads transient-autoloads
ido-completing-read+-autoloads memoize-autoloads with-editor-autoloads
compat-autoloads epl-autoloads lsp-docker-autoloads yaml-autoloads
highlight-symbol-autoloads expand-region-autoloads lsp-mode-autoloads
lv-autoloads markdown-mode-autoloads spinner-autoloads ht-autoloads
shrink-path-autoloads f-autoloads dash-autoloads s-autoloads info
editorconfig-autoloads helm-core-autoloads async-autoloads package
browse-url url url-proxy url-privacy url-expand url-methods url-history
url-cookie generate-lisp-file url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs password-cache
json subr-x map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs
cl-lib rmc iso-transl tooltip cconv eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process emacs)

Memory information:
((conses 16 595433 43945)
  (symbols 48 41127 1)
  (strings 32 186487 7325)
  (string-bytes 1 5326600)
  (vectors 16 70866)
  (vector-slots 8 906939 66598)
  (floats 8 868 339)
  (intervals 56 7370 541)
  (buffers 984 35))

bug#61303: 29.0.60; rust-ts-mode does not consistently fontify nested macro-invocations

[Theodor Thornhill via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Jostein Kjønigsen <jostein@secure.kjonigsen.net> writes:

> Hey there.
>

Hi!

> Thanks for another tree-sitter based mode to hack at :D
>
> I think I've found an error/lacking in rust-ts-mode where we may have to move
> upstream to fix it.
>
> Consider the following contrived (and yes, not very idiomatic) rust-code:
>
> let result = format!("{}{}",
>     "Well yes",
>     format!("Or {} no?", "possibly"));
>
> With rust-ts-mode first format! invocation will get highlighted as a
> macro-invocation, while the second will not.
>
> Inspecting this using treesit-explore-mode, I get the following tree:
>
> (let_declaration let pattern: (identifier) =
>   value:
>    (macro_invocation macro: (identifier) !
>     (token_tree (
>      (string_literal " ")
>      (string_literal " ")
>      (identifier)
>      (token_tree (
>       (string_literal " ")
>       (string_literal " ")
>       ))
>      )))
>   ;)
>
> Notice the lack of a second/nested "macro_invocation"-node. Instead the second
> macro is just denoted as a "identifier".
>
> To fix this, we'll probably need to upstream some bug-reports and PRs?

Sounds like it!  Just in case - the code compiles, right?

Theo

bug#61303: 29.0.60; rust-ts-mode does not consistently fontify nested macro-invocations

[Jostein Kjønigsen]

[-- Attachment #1: Type: text/plain, Size: 1442 bytes --]


On Sun, Feb 5, 2023, at 23:22, Theodor Thornhill wrote:
> Jostein Kjønigsen <jostein@secure.kjonigsen.net> writes:
> 
> > Hey there.
> >
> 
> Hi!
> 
> > Thanks for another tree-sitter based mode to hack at :D
> >
> > I think I've found an error/lacking in rust-ts-mode where we may have to move
> > upstream to fix it.
> >
> > Consider the following contrived (and yes, not very idiomatic) rust-code:
> >
> > let result = format!("{}{}",
> >     "Well yes",
> >     format!("Or {} no?", "possibly"));
> >
> > With rust-ts-mode first format! invocation will get highlighted as a
> > macro-invocation, while the second will not.
> >
> > Inspecting this using treesit-explore-mode, I get the following tree:
> >
> > (let_declaration let pattern: (identifier) =
> >   value:
> >    (macro_invocation macro: (identifier) !
> >     (token_tree (
> >      (string_literal " ")
> >      (string_literal " ")
> >      (identifier)
> >      (token_tree (
> >       (string_literal " ")
> >       (string_literal " ")
> >       ))
> >      )))
> >   ;)
> >
> > Notice the lack of a second/nested "macro_invocation"-node. Instead the second
> > macro is just denoted as a "identifier".
> >
> > To fix this, we'll probably need to upstream some bug-reports and PRs?
> 
> Sounds like it!  Just in case - the code compiles, right?
> 
> Theo
> 

Yes. This is compilable, runnable rust-code. 

—
Jostein

[-- Attachment #2: Type: text/html, Size: 2641 bytes --]