bug#64830: 29.1; C++ treesitter mode no coloration

bug#64830: 29.1; C++ treesitter mode no coloration

[David Come]

[-- Attachment #1: Type: text/plain, Size: 15256 bytes --]



When opening a C++ file with major mode c++-ts-mode, there is not
coloration

In the Messsage buffer, I see

Error during redisplay: (jit-lock-function 14) signaled (treesit-query-error "Node type error at" 99 "(true) @font-lock-constant-face (false) @font-lock-constant-face (null) @font-lock-constant-face (nullptr) @font-lock-constant-face" "Debug the query with `treesit-query-validate'") [2 times]


In GNU Emacs 29.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.20,
cairo version 1.16.0) of 2023-07-24 built on ubuntu
Repository revision: 31cef9a4eac01fff5ff4fcb89d7e2b7815e93bad
Repository branch: HEAD
Windowing system distributor 'The X.Org Foundation', version 11.0.12013000
System Description: Ubuntu 20.04.6 LTS

Configured using:
'configure --prefix /usr/local --with-native-compilation --without-pgtk
--with-json --with-gnutls --with-rsvg --without-xwidgets
--with-toolkit-scroll-bars --without-xaw3d --without-mailutils
--without-pop --with-tree-sitter 'CFLAGS=-O3 -mtune=native
-march=native -fomit-frame-pointer''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XDBE XIM XINPUT2
XPM GTK3 ZLIB

Important settings:
value of $LANG: en_US.UTF-8
locale-coding-system: utf-8-unix

Major mode: Magit

Minor modes in effect:
global-company-mode: t
company-mode: t
which-key-mode: t
counsel-projectile-mode: t
global-treesit-auto-mode: t
recentf-mode: t
winner-mode: t
treemacs-icons-dired-mode: t
dap-tooltip-mode: t
dap-ui-many-windows-mode: t
dap-ui-mode: t
lsp-treemacs-sync-mode: t
treemacs-filewatch-mode: t
treemacs-follow-mode: t
treemacs-git-mode: t
treemacs-fringe-indicator-mode: t
dap-auto-configure-mode: t
dap-mode: t
bury-successful-compilation: t
yas-global-mode: t
yas-minor-mode: t
all-the-icons-ivy-rich-mode: t
ivy-rich-mode: t
projectile-mode: t
counsel-mode: t
ivy-mode: t
gcmh-mode: t
global-whitespace-cleanup-mode: t
engine-mode: t
unkillable-scratch: t
dimmer-mode: t
windmove-mode: t
global-undo-tree-mode: t
undo-tree-mode: t
global-git-commit-mode: t
magit-auto-revert-mode: t
global-auto-revert-mode: t
shell-dirtrack-mode: t
server-mode: t
volatile-highlights-mode: t
override-global-mode: t
tooltip-mode: t
global-eldoc-mode: t
show-paren-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
buffer-read-only: t
column-number-mode: t
line-number-mode: t
indent-tabs-mode: t
transient-mark-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t

Load-path shadows:
/home/dcome/.emacs.d/elpa/transient-20230723.1411/transient hides /home/dcome/source/emacs-29-rc1/lisp/transient

Features:
(shadow sort mail-extr emacsbug magit-extras display-line-numbers
make-mode tramp tramp-loaddefs trampver tramp-integration tramp-compat
mule-util treemacs-bookmarks treemacs-tags magit-bookmark bookmark
lsp-diagnostics company-cmake company-keywords company-dabbrev-code
company-dabbrev company-files company-yasnippet company-capf company
lsp-headerline lsp-icons lsp-modeline astyle reformatter lsp-ui
lsp-ui-flycheck lsp-ui-doc lsp-ui-imenu lsp-ui-peek lsp-ui-sideline
flycheck lsp-ui-util which-key view lsp-zig lsp-tilt lsp-steep
lsp-svelte lsp-sqls lsp-ruby-syntax-tree lsp-ruby-lsp lsp-yaml lsp-xml
lsp-vimscript lsp-vhdl lsp-volar lsp-vetur lsp-html lsp-verilog lsp-vala
lsp-v lsp-typeprof lsp-ttcn3 lsp-toml lsp-terraform lsp-tex lsp-sorbet
lsp-solargraph lsp-semgrep lsp-rust lsp-rf lsp-ruff-lsp lsp-remark
lsp-racket lsp-r lsp-purescript lsp-pylsp lsp-pyls lsp-pwsh lsp-php
lsp-pls lsp-perlnavigator lsp-perl lsp-openscad lsp-ocaml lsp-magik
lsp-nix lsp-nim lsp-nginx lsp-mint lsp-marksman lsp-markdown lsp-lua
lsp-kotlin lsp-json lsp-javascript lsp-idris lsp-haxe lsp-groovy
lsp-hack lsp-graphql lsp-glsl lsp-gleam lsp-go lsp-completion
lsp-gdscript lsp-fsharp lsp-fortran lsp-eslint lsp-erlang lsp-emmet
lsp-elixir lsp-elm lsp-dockerfile lsp-dhall lsp-d lsp-css lsp-csharp
lsp-crystal lsp-credo lsp-cmake lsp-clojure lsp-semantic-tokens
lsp-clangd lsp-beancount lsp-bash lsp-astro lsp-awk lsp-ansible
lsp-angular lsp-ada lsp-actionscript c-ts-mode hideshow
counsel-projectile oc-basic ol-eww eww url-queue mm-url ol-rmail ol-mhe
ol-irc ol-info ol-gnus nnselect gnus-art mm-uu mml2015 mm-view mml-smime
smime dig gnus-sum shr pixel-fill kinsoku url-file svg dom gnus-group
gnus-undo gnus-start gnus-dbus dbus xml gnus-cloud nnimap nnmail
mail-source utf7 nnoo parse-time gnus-spec gnus-int gnus-range gnus-win
gnus nnheader range ol-docview doc-view jka-compr image-mode exif
ol-bibtex bibtex iso8601 ol-bbdb ol-w3m ol-doi org-link-doi org-agenda
time emacsql emacsql-compiler ghub-graphql treepy gsexp ghub url-http
url-gw url-auth let-alist gnutls treesit-auto tree-sitter-langs
tree-sitter-langs-build tar-mode arc-mode archive-mode tree-sitter-hl
tree-sitter tree-sitter-load tree-sitter-cli tsc tsc-dyn tsc-dyn-get
dired-aux tsc-obsolete register-list dockerfile-mode sh-script smie
executable groovy-mode dts-mode afternoon-theme py-autopep8 lsp-haskell
haskell-mode haskell-cabal haskell-utils haskell-font-lock
haskell-indentation haskell-string haskell-sort-imports haskell-lexeme
haskell-align-imports haskell-complete-module haskell-ghc-support
flymake-proc flymake dabbrev haskell-customize dashboard
dashboard-widgets recentf ob-shell ob-org org-bullets
the-org-mode-expansions org-element org-persist org-id org-refile
avl-tree org org-macro org-pcomplete org-list org-footnote org-faces
org-entities ob-emacs-lisp org-table org-loaddefs cal-menu calendar
cal-loaddefs winner move-text helpful cc-langs trace edebug debug
backtrace info-look find-func elisp-refs ivy-avy link-hint ffap
goto-addr ibuffer-vc treemacs-magit treemacs-icons-dired
treemacs-projectile dap-mouse dap-ui lsp-treemacs lsp-treemacs-generic
lsp-treemacs-themes treemacs-treelib treemacs treemacs-header-line
treemacs-compatibility treemacs-mode treemacs-interface
treemacs-persistence treemacs-filewatch-mode treemacs-follow-mode
treemacs-rendering treemacs-annotations treemacs-async
treemacs-workspaces treemacs-dom treemacs-visuals
treemacs-fringe-indicator pulse treemacs-faces treemacs-icons
treemacs-scope treemacs-themes treemacs-core-utils pfuture hl-line
treemacs-logging treemacs-customization treemacs-macros gdb-mi bindat
gud bui bui-list bui-info bui-entry bui-core bui-history bui-button
bui-utils lsp-lens dap-python dap-mode dap-tasks dap-launch lsp-docker
yaml posframe dap-overlays lsp-mode lsp-protocol tree-widget
network-stream nsm inline ht f f-shortdoc ivy-xref cmake-font-lock
cmake-mode rst bury-successful-compilation qml-mode flyspell ispell
ivy-yasnippet auto-yasnippet haskell-snippets yasnippet-snippets
yasnippet all-the-icons-ivy-rich all-the-icons all-the-icons-faces
data-material data-weathericons data-octicons data-fileicons
data-faicons data-alltheicons ivy-rich projectile ido counsel-fd
counsel-test counsel-test-pytest counsel-test-ctest counsel-test-core
counsel xdg swiper ivy delsel ivy-faces ivy-overlay colir async
mermaid-mode protobuf-mode rotate d2-mode ob ob-tangle ol org-src ob-ref
ob-lob ob-table ob-exp ob-comint ob-core org-cycle org-fold
org-fold-core ob-eval org-keys oc org-compat org-version org-macs
major-mode-hydra kill-or-bury-alive gcmh json-mode json-snatcher
js-mode-expansions js c-ts-common treesit cc-mode-expansions cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs dired-subtree dired-hacks-utils rainbow-mode rainbow-delimiters
smart-forward whitespace-cleanup-mode whitespace engine-mode miniedit
expand-region yaml-mode-expansions text-mode-expansions
er-basic-expansions expand-region-core expand-region-custom zetteldeft
ace-window avy deft iedit iedit-lib goto-line-preview unkillable-scratch
markdown-mode noutline outline dimmer face-remap color windmove
sentence-navigation ample-regexps swap-buffers quail cl annotate rg
files-x rg-info-hack rg-menu rg-ibuffer rg-result wgrep-rg wgrep
rg-history rg-header ibuf-ext ibuffer ibuffer-loaddefs grep compile 0xc
quelpa-use-package quelpa lisp-mnt help-fns radix-tree undo-tree queue
git-modes gitignore-mode gitconfig-mode conf-mode gitattributes-mode
xterm-color magit-lfs diff-hl log-view vc-dir ewoc vc git-link thingatpt
vdiff-magit magit-ediff ediff ediff-merg ediff-mult ediff-wind
ediff-diff ediff-help ediff-init ediff-util git-timemachine vc-git
vc-dispatcher magit-submodule magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log which-func imenu
magit-diff git-commit log-edit message sendmail yank-media puny dired
dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068 epg-config
gnus-util text-property-search time-date mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
mailabbrev mail-utils gmm-utils mailheader pcvs-util add-log magit-core
magit-autorevert autorevert filenotify magit-margin magit-transient
magit-process with-editor shell pcomplete comint ansi-osc server
ansi-color magit-mode transient magit-git magit-base magit-section
format-spec cursor-sensor crm compat vdiff smerge-mode diff diff-mode
pretty-hydra s dash repo gitlab-ci-mode yaml-mode use-package-hydra
edmacro kmacro comp comp-cstr warnings etags fileloop generator xref
project advice volatile-highlights paradox paradox-menu
paradox-commit-list hydra ring lv cus-edit pp cus-load icons wid-edit
paradox-execute paradox-github paradox-core spinner cl-extra help-mode
use-package use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core finder-inf
0xc-autoloads afternoon-theme-autoloads all-the-icons-ivy-rich-autoloads
all-the-icons-autoloads annotate-autoloads astyle-autoloads
async-autoloads auto-yasnippet-autoloads bm-autoloads
bury-successful-compilation-autoloads clang-format-autoloads
cmake-font-lock-autoloads cmake-mode-autoloads company-autoloads
counsel-fd-autoloads counsel-projectile-autoloads counsel-autoloads
counsel-test-autoloads d2-mode-autoloads dap-mode-autoloads
bui-autoloads dashboard-autoloads diff-hl-autoloads dimmer-autoloads
dired-subtree-autoloads dired-hacks-utils-autoloads
dockerfile-mode-autoloads dts-mode-autoloads easy-hugo-autoloads
engine-mode-autoloads flycheck-autoloads flyspell-correct-ivy-autoloads
flyspell-correct-autoloads forge-autoloads closql-autoloads
emacsql-autoloads gcmh-autoloads ghub-autoloads git-link-autoloads
git-modes-autoloads git-timemachine-autoloads gitlab-ci-mode-autoloads
goto-line-preview-autoloads groovy-mode-autoloads
haskell-snippets-autoloads helpful-autoloads elisp-refs-autoloads
highlight-indent-guides-autoloads ibuffer-vc-autoloads iedit-autoloads
ivy-avy-autoloads ivy-rich-autoloads ivy-xref-autoloads
ivy-yasnippet-autoloads json-mode-autoloads rx json-snatcher-autoloads
kill-or-bury-alive-autoloads kotlin-mode-autoloads
kotlin-ts-mode-autoloads link-hint-autoloads lsp-docker-autoloads
lsp-haskell-autoloads haskell-mode-autoloads lsp-ivy-autoloads
lsp-treemacs-autoloads lsp-ui-autoloads lsp-mode-autoloads f-autoloads
magit-lfs-autoloads major-mode-hydra-autoloads markdown-mode-autoloads
mermaid-mode-autoloads miniedit-autoloads move-text-autoloads
olivetti-autoloads org-bullets-autoloads paradox-autoloads
pipenv-autoloads load-env-vars-autoloads pkg-info-autoloads
epl-autoloads popup-autoloads pretty-hydra-autoloads
protobuf-mode-autoloads py-autopep8-autoloads pyvenv-autoloads
qml-mode-autoloads quelpa-use-package-autoloads quelpa-autoloads
rainbow-delimiters-autoloads rainbow-mode-autoloads
reformatter-autoloads register-list-autoloads repo-autoloads
request-autoloads rg-autoloads rotate-autoloads
sentence-navigation-autoloads ample-regexps-autoloads
shell-pop-autoloads smart-comment-autoloads smart-forward-autoloads
expand-region-autoloads spinner-autoloads swap-buffers-autoloads
swiper-autoloads ivy-autoloads tree-sitter-langs-autoloads
tree-sitter-autoloads treemacs-icons-dired-autoloads
treemacs-magit-autoloads treemacs-projectile-autoloads
treemacs-autoloads cfrs-autoloads posframe-autoloads ht-autoloads
pfuture-autoloads s-autoloads projectile-autoloads treepy-autoloads
treesit-auto-autoloads tsc-autoloads undo-tree-autoloads queue-autoloads
unkillable-scratch-autoloads use-package-hydra-autoloads
vdiff-magit-autoloads magit-autoloads pcase magit-section-autoloads
git-commit-autoloads transient-autoloads dash-autoloads vdiff-autoloads
hydra-autoloads lv-autoloads visual-regexp-steroids-autoloads
visual-regexp-autoloads volatile-highlights-autoloads wgrep-autoloads
which-key-autoloads whitespace-cleanup-mode-autoloads
with-editor-autoloads info compat-autoloads xterm-color-autoloads
yaml-autoloads yaml-mode-autoloads yasnippet-snippets-autoloads
yasnippet-autoloads zetteldeft-autoloads ace-window-autoloads
avy-autoloads deft-autoloads package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 1459818 925289)
(symbols 48 74217 152)
(strings 32 443096 123894)
(string-bytes 1 15926874)
(vectors 16 156046)
(vector-slots 8 2896484 996131)
(floats 8 2121 6640)
(intervals 56 12743 9883)
(buffers 984 42))



[-- Attachment #2: Type: text/html, Size: 23997 bytes --]

bug#64830: 29.1; C++ treesitter mode no coloration

[Eli Zaretskii]

merge 64830 64818
thanks

> From: David Come <david.come@ageagle.com>
> Date: Mon, 24 Jul 2023 09:13:33 +0000
> msip_labels: 
> 
> When opening a C++ file with major mode c++-ts-mode, there is not 
> coloration 
> 
> In the Messsage buffer, I see 
> 
> Error during redisplay: (jit-lock-function 14) signaled (treesit-query-error "Node type error at" 99 "(true)
> @font-lock-constant-face (false) @font-lock-constant-face (null) @font-lock-constant-face (nullptr)
> @font-lock-constant-face" "Debug the query with `treesit-query-validate'") [2 times] 

Thanks.

This is the same issue as bug#64830; merged.

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Alan Mackenzie]

Hello, Emacs.

In my emacs-30 build (updated earlier today), C++ source in c++-ts-mode
is not getting fontified, except for comments and preprocessor
constructs.

I think I'm using tree-sitter-cpp version 0.22.2 as downloaded from the
Gentoo repository.  Is there an easy way to check this from inside
Emacs?

This would appear to be bug 64818/64830, which are still open in
debbugs.  Shouldn't they be closed before the first/next emacs-30
pretest?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> Date: Wed, 14 Aug 2024 17:35:05 +0000
> From: Alan Mackenzie <acm@muc.de>
> 
> Hello, Emacs.
> 
> In my emacs-30 build (updated earlier today), C++ source in c++-ts-mode
> is not getting fontified, except for comments and preprocessor
> constructs.

I cannot reproduce this, at least not with a random C++ source file I
tried to.  Does this happen for you in "emacs -Q" and for every C++
file?

If this happens with any file and in 'emacs -Q", do you see redisplay
error messages in *Messages*?  If so, what are those messages?

> I think I'm using tree-sitter-cpp version 0.22.2 as downloaded from the
> Gentoo repository.  Is there an easy way to check this from inside
> Emacs?

No, not AFAIK.  And the problem is not the version of tree-sitter, it
is usually the version of the C++ grammar library.  Do you know the
version of that one you have installed?

> This would appear to be bug 64818/64830, which are still open in
> debbugs.  Shouldn't they be closed before the first/next emacs-30
> pretest?

Those bugs were fixed.  The reason they are not closed is that we
raised a more general issue there, and that is not yet resolved.

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Alan Mackenzie]

Hello, Eli.

On Thu, Aug 15, 2024 at 08:25:34 +0300, Eli Zaretskii wrote:
> > Date: Wed, 14 Aug 2024 17:35:05 +0000
> > From: Alan Mackenzie <acm@muc.de>

> > In my emacs-30 build (updated earlier today), C++ source in c++-ts-mode
> > is not getting fontified, except for comments and preprocessor
> > constructs.

> I cannot reproduce this, at least not with a random C++ source file I
> tried to.  Does this happen for you in "emacs -Q" and for every C++
> file?

Yes, and yes.  Here's a recipe:

Create the file ~/class-3.cc with the following contents:
/////////////////////////////////////////////////////////////////////////
// foo
class A
{
    struct B {
        int i;
    };
    struct B {
        int i;
    };
};
/////////////////////////////////////////////////////////////////////////

(i) emacs -Q
(ii) M-x load-library <RET> c-ts-mode <RET>
(iii) C-x C-f ~/class-3.cc <RET>

The buffer, in c++-ts-mode, has no fontification apart from the comment
on its first line.

> If this happens with any file and in 'emacs -Q", do you see redisplay
> error messages in *Messages*?  If so, what are those messages?

Yes, I see a single message after the above recipe.  It's:

Error during redisplay: (jit-lock-function 1) signaled
(treesit-query-error "Node type error at" 677 "[\"_Atomic\" \"break\"
\"case\" \"const\" \"continue\" \"default\" \"do\" \"else\" \"enum\"
\"extern\" \"for\" \"goto\" \"if\" \"inline\" \"register\" \"restrict\"
\"return\" \"sizeof\" \"static\" \"struct\" \"switch\" \"typedef\"
\"union\" \"volatile\" \"while\" \"and\" \"and_eq\" \"bitand\" \"bitor\"
\"catch\" \"class\" \"co_await\" \"co_return\" \"co_yield\" \"compl\"
\"\concept\" \"consteval\" \"constexpr\" \"constinit\" \"decltype\"
\"delete\" \"explicit\" \"final\" \"friend\" \"mutable\" \"namespace\"
\"new\" \"noexcept\" \"not\" \"not_eq\" \"operator\" \"or\" \"or_eq\"
\"override\" \"private\" \"protected\" \"public\" \"requires\"
\"template\" \"throw\" \"try\" \"typename\" \"using\" \"xor\"
\"xor_eq\"] @font-lock-keyword-face (auto) @font-lock-keyword-face
(this) @font-lock-keyword-face (virtual) @font-lock-keyword-face" "Debug
the query with `treesit-query-validate'")

> > I think I'm using tree-sitter-cpp version 0.22.2 as downloaded from the
> > Gentoo repository.  Is there an easy way to check this from inside
> > Emacs?

> No, not AFAIK.  And the problem is not the version of tree-sitter, it
> is usually the version of the C++ grammar library.  Do you know the
> version of that one you have installed?

I have libtree-sitter-cpp.so.0.14, built and installed on 2024-08-04.
The package is (confusingly) called dev-libs/tree-sitter-cpp-0.22.2.

> > This would appear to be bug 64818/64830, which are still open in
> > debbugs.  Shouldn't they be closed before the first/next emacs-30
> > pretest?

> Those bugs were fixed.  The reason they are not closed is that we
> raised a more general issue there, and that is not yet resolved.

OK.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> Date: Fri, 16 Aug 2024 16:44:33 +0000
> Cc: Yuan Fu <casouri@gmail.com>, 64830@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> Create the file ~/class-3.cc with the following contents:
> /////////////////////////////////////////////////////////////////////////
> // foo
> class A
> {
>     struct B {
>         int i;
>     };
>     struct B {
>         int i;
>     };
> };
> /////////////////////////////////////////////////////////////////////////
> 
> (i) emacs -Q
> (ii) M-x load-library <RET> c-ts-mode <RET>
> (iii) C-x C-f ~/class-3.cc <RET>
> 
> The buffer, in c++-ts-mode, has no fontification apart from the comment
> on its first line.
> 
> > If this happens with any file and in 'emacs -Q", do you see redisplay
> > error messages in *Messages*?  If so, what are those messages?
> 
> Yes, I see a single message after the above recipe.  It's:
> 
> Error during redisplay: (jit-lock-function 1) signaled
> (treesit-query-error "Node type error at" 677 "[\"_Atomic\" \"break\"
> \"case\" \"const\" \"continue\" \"default\" \"do\" \"else\" \"enum\"
> \"extern\" \"for\" \"goto\" \"if\" \"inline\" \"register\" \"restrict\"
> \"return\" \"sizeof\" \"static\" \"struct\" \"switch\" \"typedef\"
> \"union\" \"volatile\" \"while\" \"and\" \"and_eq\" \"bitand\" \"bitor\"
> \"catch\" \"class\" \"co_await\" \"co_return\" \"co_yield\" \"compl\"
> \"\concept\" \"consteval\" \"constexpr\" \"constinit\" \"decltype\"
> \"delete\" \"explicit\" \"final\" \"friend\" \"mutable\" \"namespace\"
> \"new\" \"noexcept\" \"not\" \"not_eq\" \"operator\" \"or\" \"or_eq\"
> \"override\" \"private\" \"protected\" \"public\" \"requires\"
> \"template\" \"throw\" \"try\" \"typename\" \"using\" \"xor\"
> \"xor_eq\"] @font-lock-keyword-face (auto) @font-lock-keyword-face
> (this) @font-lock-keyword-face (virtual) @font-lock-keyword-face" "Debug
> the query with `treesit-query-validate'")
> 
> > > I think I'm using tree-sitter-cpp version 0.22.2 as downloaded from the
> > > Gentoo repository.  Is there an easy way to check this from inside
> > > Emacs?
> 
> > No, not AFAIK.  And the problem is not the version of tree-sitter, it
> > is usually the version of the C++ grammar library.  Do you know the
> > version of that one you have installed?
> 
> I have libtree-sitter-cpp.so.0.14, built and installed on 2024-08-04.
> The package is (confusingly) called dev-libs/tree-sitter-cpp-0.22.2.

I can reproduce this in Emacs 29, but not in the current emacs-30
branch.  I think I also have libtree-sitter-cpp version 0.22.2, but I
also tested with 0.22.3 and I see no problem there, either (with Emacs
30).  So I wonder why you see this in Emacs 30.

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> Cc: 64830@debbugs.gnu.org, casouri@gmail.com
> Date: Fri, 16 Aug 2024 20:40:07 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> I can reproduce this in Emacs 29, but not in the current emacs-30
> branch.  I think I also have libtree-sitter-cpp version 0.22.2, but I
> also tested with 0.22.3 and I see no problem there, either (with Emacs
> 30).  So I wonder why you see this in Emacs 30.

Maybe your Emacs 30 build is old?

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Alan Mackenzie]

Hello, Eli.

On Fri, Aug 16, 2024 at 20:45:05 +0300, Eli Zaretskii wrote:
> > Cc: 64830@debbugs.gnu.org, casouri@gmail.com
> > Date: Fri, 16 Aug 2024 20:40:07 +0300
> > From: Eli Zaretskii <eliz@gnu.org>

> > I can reproduce this in Emacs 29, but not in the current emacs-30
> > branch.  I think I also have libtree-sitter-cpp version 0.22.2, but I
> > also tested with 0.22.3 and I see no problem there, either (with Emacs
> > 30).  So I wonder why you see this in Emacs 30.

> Maybe your Emacs 30 build is old?

No.  I updated it on Wednesday, the most recent commit I have being:

commit 9bedb957bebdca99b1bb96f58ea790e20ed48dee (HEAD -> emacs-30,
origin/emacs-30)
Author: Eli Zaretskii <eliz@gnu.org>
Date:   Wed Aug 14 11:35:48 2024 +0300

    Improve documentation of time-parsing functions

..  I will update it right now and retry ....

..... DONE.  It makes no difference.  I don't understand either why I see
this bug and you don't.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> Date: Fri, 16 Aug 2024 18:06:31 +0000
> Cc: 64830@debbugs.gnu.org, casouri@gmail.com, acm@muc.de
> From: Alan Mackenzie <acm@muc.de>
> 
> > Maybe your Emacs 30 build is old?
> 
> No.  I updated it on Wednesday, the most recent commit I have being:
> 
> commit 9bedb957bebdca99b1bb96f58ea790e20ed48dee (HEAD -> emacs-30,
> origin/emacs-30)
> Author: Eli Zaretskii <eliz@gnu.org>
> Date:   Wed Aug 14 11:35:48 2024 +0300
> 
>     Improve documentation of time-parsing functions
> 
> ..  I will update it right now and retry ....
> 
> ..... DONE.  It makes no difference.  I don't understand either why I see
> this bug and you don't.

Maybe try updating the C++ grammar library?

Yuan, any ideas?

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Yuan Fu]

> On Aug 16, 2024, at 11:27 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> Date: Fri, 16 Aug 2024 18:06:31 +0000
>> Cc: 64830@debbugs.gnu.org, casouri@gmail.com, acm@muc.de
>> From: Alan Mackenzie <acm@muc.de>
>> 
>>> Maybe your Emacs 30 build is old?
>> 
>> No.  I updated it on Wednesday, the most recent commit I have being:
>> 
>> commit 9bedb957bebdca99b1bb96f58ea790e20ed48dee (HEAD -> emacs-30,
>> origin/emacs-30)
>> Author: Eli Zaretskii <eliz@gnu.org>
>> Date:   Wed Aug 14 11:35:48 2024 +0300
>> 
>>    Improve documentation of time-parsing functions
>> 
>> ..  I will update it right now and retry ....
>> 
>> ..... DONE.  It makes no difference.  I don't understand either why I see
>> this bug and you don't.
> 
> Maybe try updating the C++ grammar library?
> 
> Yuan, any ideas?

Nothing obviously wrong from a glance. I’m very busy recently but I’ll have some time this week to look into this. Sorry for the delay :-)

Yuan

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 1377 bytes --]



> On Aug 19, 2024, at 8:46 PM, Yuan Fu <casouri@gmail.com> wrote:
> 
> 
> 
>> On Aug 16, 2024, at 11:27 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> 
>>> Date: Fri, 16 Aug 2024 18:06:31 +0000
>>> Cc: 64830@debbugs.gnu.org, casouri@gmail.com, acm@muc.de
>>> From: Alan Mackenzie <acm@muc.de>
>>> 
>>>> Maybe your Emacs 30 build is old?
>>> 
>>> No.  I updated it on Wednesday, the most recent commit I have being:
>>> 
>>> commit 9bedb957bebdca99b1bb96f58ea790e20ed48dee (HEAD -> emacs-30,
>>> origin/emacs-30)
>>> Author: Eli Zaretskii <eliz@gnu.org>
>>> Date:   Wed Aug 14 11:35:48 2024 +0300
>>> 
>>>   Improve documentation of time-parsing functions
>>> 
>>> ..  I will update it right now and retry ....
>>> 
>>> ..... DONE.  It makes no difference.  I don't understand either why I see
>>> this bug and you don't.
>> 
>> Maybe try updating the C++ grammar library?
>> 
>> Yuan, any ideas?
> 
> Nothing obviously wrong from a glance. I’m very busy recently but I’ll have some time this week to look into this. Sorry for the delay :-)
> 
> Yuan

Upon closer inspection, I think this is caused by a recent change in c-ts-mode font-lock rules, in 014aab9847a0d3d898cb8cbc7224143f2d741abb.

Alan, could you do this: don’t upgrade your c++ grammar and try this patch, if I was right it should fix your problem. Thanks!

Yuan


[-- Attachment #2: cpp-font-lock.patch --]
[-- Type: application/octet-stream, Size: 1832 bytes --]

From a171709cadd915f48d6c8ed3f9e1e19f6a4fde19 Mon Sep 17 00:00:00 2001
From: Yuan Fu <casouri@gmail.com>
Date: Sat, 24 Aug 2024 11:32:43 -0700
Subject: [PATCH] Fix c-ts-mode font-lock for older c++ grammar (bug#64830)

* lisp/progmodes/c-ts-mode.el:
(c-ts-mode--virtual-named-p): New function.
(c-ts-mode--font-lock-settings): Use either keyword or named node
depending on grammar version.
---
 lisp/progmodes/c-ts-mode.el | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/lisp/progmodes/c-ts-mode.el b/lisp/progmodes/c-ts-mode.el
index 6c406a8acd9..66e4999677b 100644
--- a/lisp/progmodes/c-ts-mode.el
+++ b/lisp/progmodes/c-ts-mode.el
@@ -537,6 +537,16 @@ c-ts-mode--top-level-label-matcher
        (equal "function_definition"
               (treesit-node-type (treesit-node-parent parent)))))
 
+;;; Grammar checks
+
+;; Ref: https://github.com/tree-sitter/tree-sitter-cpp/commit/c51ffe9d7b67aa9500db01c26d8c075cfe032d0f
+(defun c-ts-mode--virtual-named-p ()
+  "Return t if virtual is a named node instead of a keyword."
+  (ignore-errors
+    (progn
+      (treesit-query-compile 'c++ "(virtual) @cap")
+      t)))
+
 ;;; Font-lock
 
 (defvar c-ts-mode--feature-list
@@ -635,9 +645,11 @@ c-ts-mode--font-lock-settings
    :feature 'keyword
    `([,@(c-ts-mode--keywords mode)] @font-lock-keyword-face
      ,@(when (eq mode 'cpp)
-         '((auto) @font-lock-keyword-face
+         `((auto) @font-lock-keyword-face
            (this) @font-lock-keyword-face
-           (virtual) @font-lock-keyword-face)))
+           ,@(if (c-ts-mode--virtual-named-p)
+                 '((virtual) @font-lock-keyword-face)
+               '("virtual" @font-lock-keyword-face)))))
 
    :language mode
    :feature 'operator
-- 
2.39.5 (Apple Git-151)

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Alan Mackenzie]

Hello, Yuan.

On Sat, Aug 24, 2024 at 11:35:36 -0700, Yuan Fu wrote:


> > On Aug 19, 2024, at 8:46 PM, Yuan Fu <casouri@gmail.com> wrote:



> >> On Aug 16, 2024, at 11:27 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> >>> Date: Fri, 16 Aug 2024 18:06:31 +0000
> >>> Cc: 64830@debbugs.gnu.org, casouri@gmail.com, acm@muc.de
> >>> From: Alan Mackenzie <acm@muc.de>

> >>>> Maybe your Emacs 30 build is old?

> >>> No.  I updated it on Wednesday, the most recent commit I have being:

> >>> commit 9bedb957bebdca99b1bb96f58ea790e20ed48dee (HEAD -> emacs-30,
> >>> origin/emacs-30)
> >>> Author: Eli Zaretskii <eliz@gnu.org>
> >>> Date:   Wed Aug 14 11:35:48 2024 +0300

> >>>   Improve documentation of time-parsing functions

> >>> ..  I will update it right now and retry ....

> >>> ..... DONE.  It makes no difference.  I don't understand either why I see
> >>> this bug and you don't.

> >> Maybe try updating the C++ grammar library?

> >> Yuan, any ideas?

> > Nothing obviously wrong from a glance. I’m very busy recently but I’ll have some time this week to look into this. Sorry for the delay :-)

> > Yuan

> Upon closer inspection, I think this is caused by a recent change in c-ts-mode font-lock rules, in 014aab9847a0d3d898cb8cbc7224143f2d741abb.

> Alan, could you do this: don’t upgrade your c++ grammar and try this patch, if I was right it should fix your problem. Thanks!

I updated my emacs-30 branch, checked that that commit was included, and
rebuilt it.  The problem still exists on my copy of Emacs 30.  :-(

> Yuan

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Yuan Fu]

> On Aug 24, 2024, at 12:38 PM, Alan Mackenzie <acm@muc.de> wrote:
> 
> Hello, Yuan.
> 
> On Sat, Aug 24, 2024 at 11:35:36 -0700, Yuan Fu wrote:
> 
> 
>>> On Aug 19, 2024, at 8:46 PM, Yuan Fu <casouri@gmail.com> wrote:
> 
> 
> 
>>>> On Aug 16, 2024, at 11:27 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>>>>> Date: Fri, 16 Aug 2024 18:06:31 +0000
>>>>> Cc: 64830@debbugs.gnu.org, casouri@gmail.com, acm@muc.de
>>>>> From: Alan Mackenzie <acm@muc.de>
> 
>>>>>> Maybe your Emacs 30 build is old?
> 
>>>>> No.  I updated it on Wednesday, the most recent commit I have being:
> 
>>>>> commit 9bedb957bebdca99b1bb96f58ea790e20ed48dee (HEAD -> emacs-30,
>>>>> origin/emacs-30)
>>>>> Author: Eli Zaretskii <eliz@gnu.org>
>>>>> Date:   Wed Aug 14 11:35:48 2024 +0300
> 
>>>>>  Improve documentation of time-parsing functions
> 
>>>>> ..  I will update it right now and retry ....
> 
>>>>> ..... DONE.  It makes no difference.  I don't understand either why I see
>>>>> this bug and you don't.
> 
>>>> Maybe try updating the C++ grammar library?
> 
>>>> Yuan, any ideas?
> 
>>> Nothing obviously wrong from a glance. I’m very busy recently but I’ll have some time this week to look into this. Sorry for the delay :-)
> 
>>> Yuan
> 
>> Upon closer inspection, I think this is caused by a recent change in c-ts-mode font-lock rules, in 014aab9847a0d3d898cb8cbc7224143f2d741abb.
> 
>> Alan, could you do this: don’t upgrade your c++ grammar and try this patch, if I was right it should fix your problem. Thanks!
> 
> I updated my emacs-30 branch, checked that that commit was included, and
> rebuilt it.  The problem still exists on my copy of Emacs 30.  :-(

No no I mean apply the attached patch and see if it fixes the problem. The commit hash I mentioned is the source of the bug, not the fix ;-)

Yuan

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Alan Mackenzie]

Hello, Yuan.

On Sat, Aug 24, 2024 at 13:43:25 -0700, Yuan Fu wrote:


> > On Aug 24, 2024, at 12:38 PM, Alan Mackenzie <acm@muc.de> wrote:
> > On Sat, Aug 24, 2024 at 11:35:36 -0700, Yuan Fu wrote:
> >>> On Aug 19, 2024, at 8:46 PM, Yuan Fu <casouri@gmail.com> wrote:
> >>>> On Aug 16, 2024, at 11:27 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> >>>>> Date: Fri, 16 Aug 2024 18:06:31 +0000
> >>>>> Cc: 64830@debbugs.gnu.org, casouri@gmail.com, acm@muc.de
> >>>>> From: Alan Mackenzie <acm@muc.de>

> >>>>>> Maybe your Emacs 30 build is old?

> >>>>> No.  I updated it on Wednesday, the most recent commit I have being:

> >>>>> commit 9bedb957bebdca99b1bb96f58ea790e20ed48dee (HEAD -> emacs-30,
> >>>>> origin/emacs-30)
> >>>>> Author: Eli Zaretskii <eliz@gnu.org>
> >>>>> Date:   Wed Aug 14 11:35:48 2024 +0300

> >>>>>  Improve documentation of time-parsing functions

> >>>>> ..  I will update it right now and retry ....

> >>>>> ..... DONE.  It makes no difference.  I don't understand either
> >>>>> why I see this bug and you don't.

> >>>> Maybe try updating the C++ grammar library?

> >>>> Yuan, any ideas?

> >>> Nothing obviously wrong from a glance. I’m very busy recently but
> >>> I’ll have some time this week to look into this. Sorry for the
> >>> delay :-)

> >>> Yuan

> >> Upon closer inspection, I think this is caused by a recent change in
> >> c-ts-mode font-lock rules, in
> >> 014aab9847a0d3d898cb8cbc7224143f2d741abb.

> >> Alan, could you do this: don’t upgrade your c++ grammar and try this
> >> patch, if I was right it should fix your problem. Thanks!

> > I updated my emacs-30 branch, checked that that commit was included,
> > and rebuilt it.  The problem still exists on my copy of Emacs 30.
> > :-(

> No no I mean apply the attached patch and see if it fixes the problem.
> The commit hash I mentioned is the source of the bug, not the fix ;-)

Sorry about the misunderstanding.

I've now applied your patch, the one whose first hunk's header is:

@@ -537,6 +537,16 @@ c-ts-mode--top-level-label-matcher

, but it unfortunately doesn't solve the bug.  On templates-21.cc, one of
the test files from CC Mode, on doing M-x c++-ts-mode, there is no
fontification at all.  In *Messages* we have

Error during redisplay: (jit-lock-function 1) signaled
(treesit-query-error "Node type error at" 677 "[\"_Atomic\" \"break\"
\"case\" \"const\" \"continue\" \"default\" \"do\" \"else\" \"enum\"
\"extern\" \"for\" \"goto\" \"if\" \"inline\" \"register\" \"restrict\"
\"return\" \"sizeof\" \"static\" \"struct\" \"switch\" \"typedef\"
\"union\" \"volatile\" \"while\" \"and\" \"and_eq\" \"bitand\" \"bitor\"
\"catch\" \"class\" \"co_await\" \"co_return\" \"co_yield\" \"compl\"
\"concept\" \"consteval\" \"constexpr\" \"constinit\" \"decltype\"
\"delete\" \"explicit\" \"final\" \"friend\" \"mutable\" \"namespace\"
\"new\" \"noexcept\" \"not\" \"not_eq\" \"operator\" \"or\" \"or_eq\"
\"override\" \"private\" \"protected\" \"public\" \"requires\"
\"template\" \"throw\" \"try\" \"typename\" \"using\" \"xor\" \"xor_eq\"]
@font-lock-keyword-face (auto) @font-lock-keyword-face (this)
@font-lock-keyword-face (virtual) @font-lock-keyword-face" "Debug the
query with `treesit-query-validate'")

That "Node type error at" 677 isn't a buffer position - the buffer is
only 325 characters long.

Is there anything else I could do to help, here?


> Yuan

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> Date: Sun, 25 Aug 2024 02:19:28 +0000
> Cc: Eli Zaretskii <eliz@gnu.org>, 64830@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> I've now applied your patch, the one whose first hunk's header is:
> 
> @@ -537,6 +537,16 @@ c-ts-mode--top-level-label-matcher
> 
> , but it unfortunately doesn't solve the bug.  On templates-21.cc, one of
> the test files from CC Mode, on doing M-x c++-ts-mode, there is no
> fontification at all.  In *Messages* we have
> 
> Error during redisplay: (jit-lock-function 1) signaled
> (treesit-query-error "Node type error at" 677 "[\"_Atomic\" \"break\"
> \"case\" \"const\" \"continue\" \"default\" \"do\" \"else\" \"enum\"
> \"extern\" \"for\" \"goto\" \"if\" \"inline\" \"register\" \"restrict\"
> \"return\" \"sizeof\" \"static\" \"struct\" \"switch\" \"typedef\"
> \"union\" \"volatile\" \"while\" \"and\" \"and_eq\" \"bitand\" \"bitor\"
> \"catch\" \"class\" \"co_await\" \"co_return\" \"co_yield\" \"compl\"
> \"concept\" \"consteval\" \"constexpr\" \"constinit\" \"decltype\"
> \"delete\" \"explicit\" \"final\" \"friend\" \"mutable\" \"namespace\"
> \"new\" \"noexcept\" \"not\" \"not_eq\" \"operator\" \"or\" \"or_eq\"
> \"override\" \"private\" \"protected\" \"public\" \"requires\"
> \"template\" \"throw\" \"try\" \"typename\" \"using\" \"xor\" \"xor_eq\"]
> @font-lock-keyword-face (auto) @font-lock-keyword-face (this)
> @font-lock-keyword-face (virtual) @font-lock-keyword-face" "Debug the
> query with `treesit-query-validate'")
> 
> That "Node type error at" 677 isn't a buffer position - the buffer is
> only 325 characters long.
> 
> Is there anything else I could do to help, here?

AFAIR, you still didn't respond to my questions about the version of
the C++ grammar library you are using, nor tried to upgrade to the
latest version of that grammar library.

FTR, I'm using tree-sitter version 0.20.8 and grammar library from
their Git repository as of June 17 (this is version 0.22.2 of the
grammar library with a couple of changes after that version).  I have
no problems with displaying C++ files whatsoever, and *Messages* is
clean.  I also tried the latest version of the grammar library
(0.22.3), and didn't see any problems there, either.

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Alan Mackenzie]

Hello, Eli.

On Sun, Aug 25, 2024 at 07:54:43 +0300, Eli Zaretskii wrote:
> > Date: Sun, 25 Aug 2024 02:19:28 +0000
> > Cc: Eli Zaretskii <eliz@gnu.org>, 64830@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > I've now applied your patch, the one whose first hunk's header is:

> > @@ -537,6 +537,16 @@ c-ts-mode--top-level-label-matcher

> > , but it unfortunately doesn't solve the bug.  On templates-21.cc, one of
> > the test files from CC Mode, on doing M-x c++-ts-mode, there is no
> > fontification at all.  In *Messages* we have

> > Error during redisplay: (jit-lock-function 1) signaled
> > (treesit-query-error "Node type error at" 677 "[\"_Atomic\" \"break\"
> > \"case\" \"const\" \"continue\" \"default\" \"do\" \"else\" \"enum\"
> > \"extern\" \"for\" \"goto\" \"if\" \"inline\" \"register\" \"restrict\"
> > \"return\" \"sizeof\" \"static\" \"struct\" \"switch\" \"typedef\"
> > \"union\" \"volatile\" \"while\" \"and\" \"and_eq\" \"bitand\" \"bitor\"
> > \"catch\" \"class\" \"co_await\" \"co_return\" \"co_yield\" \"compl\"
> > \"concept\" \"consteval\" \"constexpr\" \"constinit\" \"decltype\"
> > \"delete\" \"explicit\" \"final\" \"friend\" \"mutable\" \"namespace\"
> > \"new\" \"noexcept\" \"not\" \"not_eq\" \"operator\" \"or\" \"or_eq\"
> > \"override\" \"private\" \"protected\" \"public\" \"requires\"
> > \"template\" \"throw\" \"try\" \"typename\" \"using\" \"xor\" \"xor_eq\"]
> > @font-lock-keyword-face (auto) @font-lock-keyword-face (this)
> > @font-lock-keyword-face (virtual) @font-lock-keyword-face" "Debug the
> > query with `treesit-query-validate'")

> > That "Node type error at" 677 isn't a buffer position - the buffer is
> > only 325 characters long.

> > Is there anything else I could do to help, here?

> AFAIR, you still didn't respond to my questions about the version of
> the C++ grammar library you are using, nor tried to upgrade to the
> latest version of that grammar library.

I thought I said that right at the beginning, in my first post on this
thread.  I'm using tree-sitter-cpp version 0.22.2, and version 0.22.6 of
tree-sitter itself.

> FTR, I'm using tree-sitter version 0.20.8 and grammar library from
> their Git repository as of June 17 (this is version 0.22.2 of the
> grammar library with a couple of changes after that version).  I have
> no problems with displaying C++ files whatsoever, and *Messages* is
> clean.  I also tried the latest version of the grammar library
> (0.22.3), and didn't see any problems there, either.

I haven't been able to find a download facility on the tree-sitter-cpp
page at Github.  Did you download a ready built version of
tree-sitter-cpp 0.22.3, or did you have to build it yourself from
sources?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> Date: Sun, 25 Aug 2024 12:08:33 +0000
> Cc: casouri@gmail.com, 64830@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > FTR, I'm using tree-sitter version 0.20.8 and grammar library from
> > their Git repository as of June 17 (this is version 0.22.2 of the
> > grammar library with a couple of changes after that version).  I have
> > no problems with displaying C++ files whatsoever, and *Messages* is
> > clean.  I also tried the latest version of the grammar library
> > (0.22.3), and didn't see any problems there, either.
> 
> I haven't been able to find a download facility on the tree-sitter-cpp
> page at Github.  Did you download a ready built version of
> tree-sitter-cpp 0.22.3, or did you have to build it yourself from
> sources?

The latter.

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Yuan Fu]

> On Aug 24, 2024, at 7:19 PM, Alan Mackenzie <acm@muc.de> wrote:
> 
> Hello, Yuan.
> 
> On Sat, Aug 24, 2024 at 13:43:25 -0700, Yuan Fu wrote:
> 
> 
>>> On Aug 24, 2024, at 12:38 PM, Alan Mackenzie <acm@muc.de> wrote:
>>> On Sat, Aug 24, 2024 at 11:35:36 -0700, Yuan Fu wrote:
>>>>> On Aug 19, 2024, at 8:46 PM, Yuan Fu <casouri@gmail.com> wrote:
>>>>>> On Aug 16, 2024, at 11:27 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>>>>>>> Date: Fri, 16 Aug 2024 18:06:31 +0000
>>>>>>> Cc: 64830@debbugs.gnu.org, casouri@gmail.com, acm@muc.de
>>>>>>> From: Alan Mackenzie <acm@muc.de>
> 
>>>>>>>> Maybe your Emacs 30 build is old?
> 
>>>>>>> No.  I updated it on Wednesday, the most recent commit I have being:
> 
>>>>>>> commit 9bedb957bebdca99b1bb96f58ea790e20ed48dee (HEAD -> emacs-30,
>>>>>>> origin/emacs-30)
>>>>>>> Author: Eli Zaretskii <eliz@gnu.org>
>>>>>>> Date:   Wed Aug 14 11:35:48 2024 +0300
> 
>>>>>>> Improve documentation of time-parsing functions
> 
>>>>>>> ..  I will update it right now and retry ....
> 
>>>>>>> ..... DONE.  It makes no difference.  I don't understand either
>>>>>>> why I see this bug and you don't.
> 
>>>>>> Maybe try updating the C++ grammar library?
> 
>>>>>> Yuan, any ideas?
> 
>>>>> Nothing obviously wrong from a glance. I’m very busy recently but
>>>>> I’ll have some time this week to look into this. Sorry for the
>>>>> delay :-)
> 
>>>>> Yuan
> 
>>>> Upon closer inspection, I think this is caused by a recent change in
>>>> c-ts-mode font-lock rules, in
>>>> 014aab9847a0d3d898cb8cbc7224143f2d741abb.
> 
>>>> Alan, could you do this: don’t upgrade your c++ grammar and try this
>>>> patch, if I was right it should fix your problem. Thanks!
> 
>>> I updated my emacs-30 branch, checked that that commit was included,
>>> and rebuilt it.  The problem still exists on my copy of Emacs 30.
>>> :-(
> 
>> No no I mean apply the attached patch and see if it fixes the problem.
>> The commit hash I mentioned is the source of the bug, not the fix ;-)
> 
> Sorry about the misunderstanding.
> 
> I've now applied your patch, the one whose first hunk's header is:
> 
> @@ -537,6 +537,16 @@ c-ts-mode--top-level-label-matcher
> 
> , but it unfortunately doesn't solve the bug.  On templates-21.cc, one of
> the test files from CC Mode, on doing M-x c++-ts-mode, there is no
> fontification at all.  In *Messages* we have
> 
> Error during redisplay: (jit-lock-function 1) signaled
> (treesit-query-error "Node type error at" 677 "[\"_Atomic\" \"break\"
> \"case\" \"const\" \"continue\" \"default\" \"do\" \"else\" \"enum\"
> \"extern\" \"for\" \"goto\" \"if\" \"inline\" \"register\" \"restrict\"
> \"return\" \"sizeof\" \"static\" \"struct\" \"switch\" \"typedef\"
> \"union\" \"volatile\" \"while\" \"and\" \"and_eq\" \"bitand\" \"bitor\"
> \"catch\" \"class\" \"co_await\" \"co_return\" \"co_yield\" \"compl\"
> \"concept\" \"consteval\" \"constexpr\" \"constinit\" \"decltype\"
> \"delete\" \"explicit\" \"final\" \"friend\" \"mutable\" \"namespace\"
> \"new\" \"noexcept\" \"not\" \"not_eq\" \"operator\" \"or\" \"or_eq\"
> \"override\" \"private\" \"protected\" \"public\" \"requires\"
> \"template\" \"throw\" \"try\" \"typename\" \"using\" \"xor\" \"xor_eq\"]
> @font-lock-keyword-face (auto) @font-lock-keyword-face (this)
> @font-lock-keyword-face (virtual) @font-lock-keyword-face" "Debug the
> query with `treesit-query-validate'")
> 
> That "Node type error at" 677 isn't a buffer position - the buffer is
> only 325 characters long.
> 
> Is there anything else I could do to help, here?

Yes, could you try evaluating this three forms, and tell me their result?

(treesit-query-validate 'cpp '("virtual" @cap))

(treesit-query-validate 'cpp '((virtual) @cap))

(treesit-query-validate 'cpp '((auto) @font-lock-keyword-face
                               (this) @font-lock-keyword-face
                               (virtual) @font-lock-keyword-face))

To give more context, in the error you see, position 677 is a position in the query string, which points to the (virtual) part. That leads me to believe your grammar doesn’t recognize the (virtual) query. So in the patch I provided earlier, I added a function that tests whether the grammar recognizes (virtual), and if not, it uses the “virtual” query instead. The difference between (virtual) and “virtual” in the query is that (virtual) is a named node, and “virtual” is a keyword.

I’m still not sure why you have this problem though, because I built libtree-sitter-cpp v0.22.2 and v0.22.0 and tested them locally, and they recognize (virtual) just fine. Anyway, evaluating those three forms will tell us what’s going on.

And to your question for building tree-sitter grammar, you can use the script at https://github.com/casouri/tree-sitter-module

You can either run ./batch to build all the languages, or run ./build cpp to only build for cpp. Then just copy the grammar file in dist to ~/.emacs.d/tree-sitter.

Yuan

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Alan Mackenzie]

Hello, Yuan.

On Sun, Aug 25, 2024 at 15:40:31 -0700, Yuan Fu wrote:


> > On Aug 24, 2024, at 7:19 PM, Alan Mackenzie <acm@muc.de> wrote:

> > Hello, Yuan.

> > On Sat, Aug 24, 2024 at 13:43:25 -0700, Yuan Fu wrote:


> >>> On Aug 24, 2024, at 12:38 PM, Alan Mackenzie <acm@muc.de> wrote:
> >>> On Sat, Aug 24, 2024 at 11:35:36 -0700, Yuan Fu wrote:
> >>>>> On Aug 19, 2024, at 8:46 PM, Yuan Fu <casouri@gmail.com> wrote:
> >>>>>> On Aug 16, 2024, at 11:27 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> >>>>>>> Date: Fri, 16 Aug 2024 18:06:31 +0000
> >>>>>>> Cc: 64830@debbugs.gnu.org, casouri@gmail.com, acm@muc.de
> >>>>>>> From: Alan Mackenzie <acm@muc.de>

> >>>>>>>> Maybe your Emacs 30 build is old?

> >>>>>>> No.  I updated it on Wednesday, the most recent commit I have being:

> >>>>>>> commit 9bedb957bebdca99b1bb96f58ea790e20ed48dee (HEAD -> emacs-30,
> >>>>>>> origin/emacs-30)
> >>>>>>> Author: Eli Zaretskii <eliz@gnu.org>
> >>>>>>> Date:   Wed Aug 14 11:35:48 2024 +0300

> >>>>>>> Improve documentation of time-parsing functions

> >>>>>>> ..  I will update it right now and retry ....

> >>>>>>> ..... DONE.  It makes no difference.  I don't understand either
> >>>>>>> why I see this bug and you don't.

> >>>>>> Maybe try updating the C++ grammar library?

> >>>>>> Yuan, any ideas?

> >>>>> Nothing obviously wrong from a glance. I’m very busy recently but
> >>>>> I’ll have some time this week to look into this. Sorry for the
> >>>>> delay :-)

> >>>>> Yuan

> >>>> Upon closer inspection, I think this is caused by a recent change in
> >>>> c-ts-mode font-lock rules, in
> >>>> 014aab9847a0d3d898cb8cbc7224143f2d741abb.

> >>>> Alan, could you do this: don’t upgrade your c++ grammar and try this
> >>>> patch, if I was right it should fix your problem. Thanks!

> >>> I updated my emacs-30 branch, checked that that commit was included,
> >>> and rebuilt it.  The problem still exists on my copy of Emacs 30.
> >>> :-(

> >> No no I mean apply the attached patch and see if it fixes the problem.
> >> The commit hash I mentioned is the source of the bug, not the fix ;-)

> > Sorry about the misunderstanding.

> > I've now applied your patch, the one whose first hunk's header is:

> > @@ -537,6 +537,16 @@ c-ts-mode--top-level-label-matcher

> > , but it unfortunately doesn't solve the bug.  On templates-21.cc, one of
> > the test files from CC Mode, on doing M-x c++-ts-mode, there is no
> > fontification at all.  In *Messages* we have

> > Error during redisplay: (jit-lock-function 1) signaled
> > (treesit-query-error "Node type error at" 677 "[\"_Atomic\" \"break\"
> > \"case\" \"const\" \"continue\" \"default\" \"do\" \"else\" \"enum\"
> > \"extern\" \"for\" \"goto\" \"if\" \"inline\" \"register\" \"restrict\"
> > \"return\" \"sizeof\" \"static\" \"struct\" \"switch\" \"typedef\"
> > \"union\" \"volatile\" \"while\" \"and\" \"and_eq\" \"bitand\" \"bitor\"
> > \"catch\" \"class\" \"co_await\" \"co_return\" \"co_yield\" \"compl\"
> > \"concept\" \"consteval\" \"constexpr\" \"constinit\" \"decltype\"
> > \"delete\" \"explicit\" \"final\" \"friend\" \"mutable\" \"namespace\"
> > \"new\" \"noexcept\" \"not\" \"not_eq\" \"operator\" \"or\" \"or_eq\"
> > \"override\" \"private\" \"protected\" \"public\" \"requires\"
> > \"template\" \"throw\" \"try\" \"typename\" \"using\" \"xor\" \"xor_eq\"]
> > @font-lock-keyword-face (auto) @font-lock-keyword-face (this)
> > @font-lock-keyword-face (virtual) @font-lock-keyword-face" "Debug the
> > query with `treesit-query-validate'")

> > That "Node type error at" 677 isn't a buffer position - the buffer is
> > only 325 characters long.

> > Is there anything else I could do to help, here?

> Yes, could you try evaluating this three forms, and tell me their result?

Done.  But see below, first!

> (treesit-query-validate 'cpp '("virtual" @cap))

"QUERY is valid"

> (treesit-query-validate 'cpp '((virtual) @cap))

In *Tree-Sitter check query*, I got:
    Node type error at: 2
    (virtual) @cap
In the reponse area, I got:
    t
..

> (treesit-query-validate 'cpp '((auto) @font-lock-keyword-face
>                                (this) @font-lock-keyword-face
>                                (virtual) @font-lock-keyword-face))

In *Tree-Sitter check query*, I got:
    Node type error at: 64
    (auto) @font-lock-keyword-face (this) @font-lock-keyword-face (virtual) @font-lock-keyword-face
, and again in the response area:
    t
..

> To give more context, in the error you see, position 677 is a position
> in the query string, which points to the (virtual) part. That leads me
> to believe your grammar doesn’t recognize the (virtual) query. So in
> the patch I provided earlier, I added a function that tests whether the
> grammar recognizes (virtual), and if not, it uses the “virtual” query
> instead. The difference between (virtual) and “virtual” in the query is
> that (virtual) is a named node, and “virtual” is a keyword.

> I’m still not sure why you have this problem though, because I built
> libtree-sitter-cpp v0.22.2 and v0.22.0 and tested them locally, and
> they recognize (virtual) just fine. Anyway, evaluating those three
> forms will tell us what’s going on.

I downloaded my libtree-sitter-cpp.so.0.14 from my GNU/Linux
distribution, Gentoo.  It regards itself as dev-libs/tree-sitter-cpp
version 0.22.2.

Might it be worthwhile adding some functionality to Emacs whereby a user
could check which versions of grammar libraries she's got?  Or is there
something like that already?

> And to your question for building tree-sitter grammar, you can use the
> script at https://github.com/casouri/tree-sitter-module

> You can either run ./batch to build all the languages, or run ./build
> cpp to only build for cpp. Then just copy the grammar file in dist to
> ~/.emacs.d/tree-sitter.

Ah, that's it!  I didn't know the grammar library in use was the one in
~/.emacs/tree-sitter.  There, I've got 

    -rwxr-xr-x 1 acm acm 2319888 Jan 19  2023 libtree-sitter-cpp.so

, which is very old.  In my /usr/lib64, I've got

    lrwxrwxrwx 1 root root      26 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so.0 -> libtree-sitter-cpp.so.0.14
    lrwxrwxrwx 1 root root      23 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so -> libtree-sitter-cpp.so.0
    -rwxr-xr-x 1 root root 3175712 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so.0.14

Why do we have this extra complication with ~/.emacs.d/tree-sitter as a
location for libraries?  In GNU/Linux distributions (well, in Gentoo at
any rate), newly downloaded libraries are put into /usr/lib64, so that
anybody naively updating a grammar library is going to have it masked out
by the old one still in ~/.emacs.d/tree-sitter.  Maybe I don't understand
the library mechanism as well as I ought to.

> Yuan

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> Date: Mon, 26 Aug 2024 17:25:34 +0000
> Cc: Eli Zaretskii <eliz@gnu.org>, 64830@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> Ah, that's it!  I didn't know the grammar library in use was the one in
> ~/.emacs/tree-sitter.  There, I've got 
> 
>     -rwxr-xr-x 1 acm acm 2319888 Jan 19  2023 libtree-sitter-cpp.so
> 
> , which is very old.  In my /usr/lib64, I've got
> 
>     lrwxrwxrwx 1 root root      26 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so.0 -> libtree-sitter-cpp.so.0.14
>     lrwxrwxrwx 1 root root      23 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so -> libtree-sitter-cpp.so.0
>     -rwxr-xr-x 1 root root 3175712 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so.0.14
> 
> Why do we have this extra complication with ~/.emacs.d/tree-sitter as a
> location for libraries?  In GNU/Linux distributions (well, in Gentoo at
> any rate), newly downloaded libraries are put into /usr/lib64, so that
> anybody naively updating a grammar library is going to have it masked out
> by the old one still in ~/.emacs.d/tree-sitter.  Maybe I don't understand
> the library mechanism as well as I ought to.

We support grammar libraries in ~/.emacs.d/tree-sitter so that users
could easily install versions of grammar libraries different from what
their sysadmins decide to install.

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Alan Mackenzie]

Hello, Eli.

On Mon, Aug 26, 2024 at 20:51:00 +0300, Eli Zaretskii wrote:
> > Date: Mon, 26 Aug 2024 17:25:34 +0000
> > Cc: Eli Zaretskii <eliz@gnu.org>, 64830@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > Ah, that's it!  I didn't know the grammar library in use was the one in
> > ~/.emacs/tree-sitter.  There, I've got 

> >     -rwxr-xr-x 1 acm acm 2319888 Jan 19  2023 libtree-sitter-cpp.so

> > , which is very old.  In my /usr/lib64, I've got

> >     lrwxrwxrwx 1 root root      26 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so.0 -> libtree-sitter-cpp.so.0.14
> >     lrwxrwxrwx 1 root root      23 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so -> libtree-sitter-cpp.so.0
> >     -rwxr-xr-x 1 root root 3175712 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so.0.14

> > Why do we have this extra complication with ~/.emacs.d/tree-sitter as a
> > location for libraries?  In GNU/Linux distributions (well, in Gentoo at
> > any rate), newly downloaded libraries are put into /usr/lib64, so that
> > anybody naively updating a grammar library is going to have it masked out
> > by the old one still in ~/.emacs.d/tree-sitter.  Maybe I don't understand
> > the library mechanism as well as I ought to.

> We support grammar libraries in ~/.emacs.d/tree-sitter so that users
> could easily install versions of grammar libraries different from what
> their sysadmins decide to install.

Ah, OK.  Seems like a good reason.

How about outputting a message looking something like this:

    Loading /home/acm/.emacs.d/tree-sitter/libtree-sitter.so ....Done

..  This wouldn't get in the way, yet might well prevent some of the
problems I've been having over the last few days.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Stefan Kangas]

Alan Mackenzie <acm@muc.de> writes:

> How about outputting a message looking something like this:
>
>     Loading /home/acm/.emacs.d/tree-sitter/libtree-sitter.so ....Done
>
> ..  This wouldn't get in the way, yet might well prevent some of the
> problems I've been having over the last few days.

Makes sense to me, FWIW.  Perhaps we should have a knob to
enable/disable such messages though.

I'm also not sure what should be the default.  Perhaps we should
experiment with it and take a final decision when we have some
experience with using it.

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Yuan Fu]

> On Aug 26, 2024, at 3:25 PM, Stefan Kangas <stefankangas@gmail.com> wrote:
> 
> Alan Mackenzie <acm@muc.de> writes:
> 
>> How about outputting a message looking something like this:
>> 
>>    Loading /home/acm/.emacs.d/tree-sitter/libtree-sitter.so ....Done
>> 
>> ..  This wouldn't get in the way, yet might well prevent some of the
>> problems I've been having over the last few days.
> 
> Makes sense to me, FWIW.  Perhaps we should have a knob to
> enable/disable such messages though.
> 
> I'm also not sure what should be the default.  Perhaps we should
> experiment with it and take a final decision when we have some
> experience with using it.

Yeah, that’ll be a very useful feature. To determining which grammar my Emacs instance loaded, I had to delete the object file and see if Emacs complains it can’t find any grammar. I think it’s fine to print this to message buffer by default, it’ll only happen once and we already do such things for loading a elisp file. 

I searched a bit and it seems ldaddr can find the location of a loaded shared library. Doesn’t seem hard to implement.

I also want a lisp function that can return the location of a loaded grammar file, would that be an overkill if we have the aforementioned printing?

Yuan

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Mon, 26 Aug 2024 18:58:13 -0700
> Cc: Alan Mackenzie <acm@muc.de>,
>  Eli Zaretskii <eliz@gnu.org>,
>  64830@debbugs.gnu.org
> 
> I also want a lisp function that can return the location of a loaded grammar file, would that be an overkill if we have the aforementioned printing?

Where/when would such a function be used?

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]



> On Aug 27, 2024, at 5:09 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Mon, 26 Aug 2024 18:58:13 -0700
>> Cc: Alan Mackenzie <acm@muc.de>,
>> Eli Zaretskii <eliz@gnu.org>,
>> 64830@debbugs.gnu.org
>> 
>> I also want a lisp function that can return the location of a loaded grammar file, would that be an overkill if we have the aforementioned printing?
> 
> Where/when would such a function be used?

For debugging. Eg, to verify that Emacs loaded the grammar file I think it loaded. Something like

(treesit-grammar-location 'c) ; => “/opt/local/lib/libtree-sitter-c.dylib”.

Yuan


[-- Attachment #2: show-location.patch --]
[-- Type: application/octet-stream, Size: 6726 bytes --]

From c64dc3e287e3db009589fb27d07d658baca9fe66 Mon Sep 17 00:00:00 2001
From: Yuan Fu <casouri@gmail.com>
Date: Tue, 27 Aug 2024 22:31:42 -0700
Subject: [PATCH] Add Ftreesit_grammar_location

* src/treesit.c (treesit_loaded_lang): New struct.
(treesit_load_language): Return a struct instead of just the language
object.  The struct contains both the language object and the path to
the shared library.
(Ftreesit_language_available_p, Ftreesit_language_abi_version)
(treesit_ensure_query_compiled, Ftreesit_parser_create): Update
call of treesit_load_language.
(Ftreesit_grammar_location): New function.
---
 src/treesit.c | 68 +++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 55 insertions(+), 13 deletions(-)

diff --git a/src/treesit.c b/src/treesit.c
index 5aedca44489..56cf8a1075a 100644
--- a/src/treesit.c
+++ b/src/treesit.c
@@ -19,6 +19,7 @@ Copyright (C) 2021-2024 Free Software Foundation, Inc.
 You should have received a copy of the GNU General Public License
 along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.  */
 
+#include <dlfcn.h>
 #include <config.h>
 #include "lisp.h"
 #include "buffer.h"
@@ -491,6 +492,14 @@ treesit_initialize (void)
 \f
 /*** Loading language library  */
 
+struct treesit_loaded_lang
+{
+  /* The language object.  */
+  TSLanguage *lang;
+  /* The path of the shared library.  */
+  const char *path;
+};
+
 /* Translate a symbol treesit-<lang> to a C name treesit_<lang>.  */
 static void
 treesit_symbol_to_c_name (char *symbol_name)
@@ -575,7 +584,7 @@ treesit_load_language_push_for_each_suffix (Lisp_Object lib_base_name,
 
    If error occurs, return NULL and fill SIGNAL_SYMBOL and SIGNAL_DATA
    with values suitable for xsignal.  */
-static TSLanguage *
+static struct treesit_loaded_lang
 treesit_load_language (Lisp_Object language_symbol,
 		       Lisp_Object *signal_symbol, Lisp_Object *signal_data)
 {
@@ -626,6 +635,7 @@ treesit_load_language (Lisp_Object language_symbol,
   dynlib_handle_ptr handle;
   const char *error;
   Lisp_Object error_list = Qnil;
+  struct treesit_loaded_lang loaded_lang = { NULL, NULL };
 
   tail = path_candidates;
   error = NULL;
@@ -650,7 +660,7 @@ treesit_load_language (Lisp_Object language_symbol,
          mismatch.  */
       *signal_symbol = Qtreesit_load_language_error;
       *signal_data = Fcons (Qnot_found, Fnreverse (error_list));
-      return NULL;
+      return loaded_lang;
     }
 
   /* Load TSLanguage.  */
@@ -672,7 +682,7 @@ treesit_load_language (Lisp_Object language_symbol,
     {
       *signal_symbol = Qtreesit_load_language_error;
       *signal_data = list2 (Qsymbol_error, build_string (error));
-      return NULL;
+      return loaded_lang;
     }
   TSLanguage *lang = (*langfn) ();
 
@@ -685,9 +695,15 @@ treesit_load_language (Lisp_Object language_symbol,
       *signal_symbol = Qtreesit_load_language_error;
       *signal_data = list2 (Qversion_mismatch,
 			    make_fixnum (ts_language_version (lang)));
-      return NULL;
+      return loaded_lang;
     }
-  return lang;
+
+  Dl_info info;
+  if (dladdr(langfn, &info))
+    loaded_lang.path = info.dli_fname;
+
+  loaded_lang.lang = lang;
+  return loaded_lang;
 }
 
 DEFUN ("treesit-language-available-p", Ftreesit_language_available_p,
@@ -704,7 +720,9 @@ DEFUN ("treesit-language-available-p", Ftreesit_language_available_p,
   treesit_initialize ();
   Lisp_Object signal_symbol = Qnil;
   Lisp_Object signal_data = Qnil;
-  if (treesit_load_language (language, &signal_symbol, &signal_data) == NULL)
+  struct treesit_loaded_lang loaded_lang
+    = treesit_load_language (language, &signal_symbol, &signal_data);
+  if (loaded_lang.lang == NULL)
     {
       if (NILP (detail))
 	return Qnil;
@@ -750,9 +768,9 @@ DEFUN ("treesit-language-abi-version", Ftreesit_language_abi_version,
     {
       Lisp_Object signal_symbol = Qnil;
       Lisp_Object signal_data = Qnil;
-      TSLanguage *ts_language = treesit_load_language (language,
-						       &signal_symbol,
-						       &signal_data);
+      struct treesit_loaded_lang lang
+	= treesit_load_language (language, &signal_symbol, &signal_data);
+      TSLanguage *ts_language = lang.lang;
       if (ts_language == NULL)
 	return Qnil;
       uint32_t version =  ts_language_version (ts_language);
@@ -760,6 +778,27 @@ DEFUN ("treesit-language-abi-version", Ftreesit_language_abi_version,
     }
 }
 
+DEFUN ("treesit-grammar-location", Ftreesit_grammar_location,
+       Streesit_grammar_location,
+       1, 1, 0,
+       doc: /* Return the path to the grammar file for LANGUAGE.
+
+If LANGUAGE isn't loaded yet, load it first.  If the langauge can't be
+loaded or the path couldn't be found, return nil.  */)
+  (Lisp_Object language)
+{
+  CHECK_SYMBOL (language);
+
+  Lisp_Object signal_symbol = Qnil;
+  Lisp_Object signal_data = Qnil;
+  struct treesit_loaded_lang lang
+    = treesit_load_language (language, &signal_symbol, &signal_data);
+
+  if (!lang.lang || !lang.path) return Qnil;
+
+  return build_string (lang.path);
+}
+
 \f
 /*** Parsing functions  */
 
@@ -1305,8 +1344,9 @@ treesit_ensure_query_compiled (Lisp_Object query, Lisp_Object *signal_symbol,
   Lisp_Object language = XTS_COMPILED_QUERY (query)->language;
   /* This is the main reason why we compile query lazily: to avoid
      loading languages early.  */
-  TSLanguage *treesit_lang = treesit_load_language (language, signal_symbol,
-						    signal_data);
+  struct treesit_loaded_lang lang
+    = treesit_load_language (language, signal_symbol, signal_data);
+  TSLanguage *treesit_lang = lang.lang;
   if (treesit_lang == NULL)
     return NULL;
 
@@ -1477,8 +1517,9 @@ DEFUN ("treesit-parser-create",
   Lisp_Object signal_symbol = Qnil;
   Lisp_Object signal_data = Qnil;
   TSParser *parser = ts_parser_new ();
-  TSLanguage *lang = treesit_load_language (language, &signal_symbol,
-					    &signal_data);
+  struct treesit_loaded_lang loaded_lang
+    = treesit_load_language (language, &signal_symbol, &signal_data);
+  TSLanguage *lang = loaded_lang.lang;
   if (lang == NULL)
     xsignal (signal_symbol, signal_data);
   /* We check language version when loading a language, so this should
@@ -4275,6 +4316,7 @@ cons (REGEXP . FN), which is a combination of a regexp and a predicate
   defsubr (&Streesit_language_available_p);
   defsubr (&Streesit_library_abi_version);
   defsubr (&Streesit_language_abi_version);
+  defsubr (&Streesit_grammar_location);
 
   defsubr (&Streesit_parser_p);
   defsubr (&Streesit_node_p);
-- 
2.39.5 (Apple Git-151)

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Tue, 27 Aug 2024 22:36:46 -0700
> Cc: Stefan Kangas <stefankangas@gmail.com>,
>  Alan Mackenzie <acm@muc.de>,
>  64830@debbugs.gnu.org
> 
> +struct treesit_loaded_lang
> +{
> +  /* The language object.  */
> +  TSLanguage *lang;
> +  /* The path of the shared library.  */
> +  const char *path;

Gnu Coding Standard frowns on using "path" for anything except
PATH-style directory lists.  Please use "filename" or "fname" or
something like that instead.

> +  Dl_info info;
> +  if (dladdr(langfn, &info))
> +    loaded_lang.path = info.dli_fname;

dladdr is non-portable.  We have dynlib_addr instead, and since
treesit.c correctly uses dynlib.c functions already, using dynlib_addr
is only natural.

> +  if (!lang.lang || !lang.path) return Qnil;
> +
> +  return build_string (lang.path);
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I think we should do

   return
     DECODE_FILE (make_unibyte_string (lang.filename, strlen (lang.filename));

instead, because dynlib_addr (and dladdr as well) return encoded file
names.

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 1297 bytes --]



> On Aug 28, 2024, at 5:33 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Tue, 27 Aug 2024 22:36:46 -0700
>> Cc: Stefan Kangas <stefankangas@gmail.com>,
>> Alan Mackenzie <acm@muc.de>,
>> 64830@debbugs.gnu.org
>> 
>> +struct treesit_loaded_lang
>> +{
>> +  /* The language object.  */
>> +  TSLanguage *lang;
>> +  /* The path of the shared library.  */
>> +  const char *path;
> 
> Gnu Coding Standard frowns on using "path" for anything except
> PATH-style directory lists.  Please use "filename" or "fname" or
> something like that instead.
> 
>> +  Dl_info info;
>> +  if (dladdr(langfn, &info))
>> +    loaded_lang.path = info.dli_fname;
> 
> dladdr is non-portable.  We have dynlib_addr instead, and since
> treesit.c correctly uses dynlib.c functions already, using dynlib_addr
> is only natural.
> 
>> +  if (!lang.lang || !lang.path) return Qnil;
>> +
>> +  return build_string (lang.path);
>     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> I think we should do
> 
>   return
>     DECODE_FILE (make_unibyte_string (lang.filename, strlen (lang.filename));
> 
> instead, because dynlib_addr (and dladdr as well) return encoded file
> names.

Ah, thanks for the review! TIL. Here’s the revised patch.

Yuan


[-- Attachment #2: grammar-location.patch --]
[-- Type: application/octet-stream, Size: 6681 bytes --]

From 97374b55f2a786bf24db63bf206ccf217927de4f Mon Sep 17 00:00:00 2001
From: Yuan Fu <casouri@gmail.com>
Date: Tue, 27 Aug 2024 22:31:42 -0700
Subject: [PATCH] Add Ftreesit_grammar_location

* src/treesit.c (treesit_loaded_lang): New struct.
(treesit_load_language): Return a struct instead of just the language
object.  The struct contains both the language object and the path to
the shared library.
(Ftreesit_language_available_p, Ftreesit_language_abi_version)
(treesit_ensure_query_compiled, Ftreesit_parser_create): Update
call of treesit_load_language.
(Ftreesit_grammar_location): New function.
---
 src/treesit.c | 68 +++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 55 insertions(+), 13 deletions(-)

diff --git a/src/treesit.c b/src/treesit.c
index 5aedca44489..09f4aeb0b99 100644
--- a/src/treesit.c
+++ b/src/treesit.c
@@ -22,6 +22,7 @@ Copyright (C) 2021-2024 Free Software Foundation, Inc.
 #include <config.h>
 #include "lisp.h"
 #include "buffer.h"
+#include "coding.h"
 
 #include "treesit.h"
 
@@ -491,6 +492,14 @@ treesit_initialize (void)
 \f
 /*** Loading language library  */
 
+struct treesit_loaded_lang
+{
+  /* The language object.  */
+  TSLanguage *lang;
+  /* The path to the shared library.  */
+  const char *filename;
+};
+
 /* Translate a symbol treesit-<lang> to a C name treesit_<lang>.  */
 static void
 treesit_symbol_to_c_name (char *symbol_name)
@@ -575,7 +584,7 @@ treesit_load_language_push_for_each_suffix (Lisp_Object lib_base_name,
 
    If error occurs, return NULL and fill SIGNAL_SYMBOL and SIGNAL_DATA
    with values suitable for xsignal.  */
-static TSLanguage *
+static struct treesit_loaded_lang
 treesit_load_language (Lisp_Object language_symbol,
 		       Lisp_Object *signal_symbol, Lisp_Object *signal_data)
 {
@@ -626,6 +635,7 @@ treesit_load_language (Lisp_Object language_symbol,
   dynlib_handle_ptr handle;
   const char *error;
   Lisp_Object error_list = Qnil;
+  struct treesit_loaded_lang loaded_lang = { NULL, NULL };
 
   tail = path_candidates;
   error = NULL;
@@ -650,7 +660,7 @@ treesit_load_language (Lisp_Object language_symbol,
          mismatch.  */
       *signal_symbol = Qtreesit_load_language_error;
       *signal_data = Fcons (Qnot_found, Fnreverse (error_list));
-      return NULL;
+      return loaded_lang;
     }
 
   /* Load TSLanguage.  */
@@ -672,7 +682,7 @@ treesit_load_language (Lisp_Object language_symbol,
     {
       *signal_symbol = Qtreesit_load_language_error;
       *signal_data = list2 (Qsymbol_error, build_string (error));
-      return NULL;
+      return loaded_lang;
     }
   TSLanguage *lang = (*langfn) ();
 
@@ -685,9 +695,14 @@ treesit_load_language (Lisp_Object language_symbol,
       *signal_symbol = Qtreesit_load_language_error;
       *signal_data = list2 (Qversion_mismatch,
 			    make_fixnum (ts_language_version (lang)));
-      return NULL;
+      return loaded_lang;
     }
-  return lang;
+
+  const char *sym;
+  dynlib_addr((void (*) (void)) langfn, &loaded_lang.filename, &sym);
+
+  loaded_lang.lang = lang;
+  return loaded_lang;
 }
 
 DEFUN ("treesit-language-available-p", Ftreesit_language_available_p,
@@ -704,7 +719,9 @@ DEFUN ("treesit-language-available-p", Ftreesit_language_available_p,
   treesit_initialize ();
   Lisp_Object signal_symbol = Qnil;
   Lisp_Object signal_data = Qnil;
-  if (treesit_load_language (language, &signal_symbol, &signal_data) == NULL)
+  struct treesit_loaded_lang loaded_lang
+    = treesit_load_language (language, &signal_symbol, &signal_data);
+  if (loaded_lang.lang == NULL)
     {
       if (NILP (detail))
 	return Qnil;
@@ -750,9 +767,9 @@ DEFUN ("treesit-language-abi-version", Ftreesit_language_abi_version,
     {
       Lisp_Object signal_symbol = Qnil;
       Lisp_Object signal_data = Qnil;
-      TSLanguage *ts_language = treesit_load_language (language,
-						       &signal_symbol,
-						       &signal_data);
+      struct treesit_loaded_lang lang
+	= treesit_load_language (language, &signal_symbol, &signal_data);
+      TSLanguage *ts_language = lang.lang;
       if (ts_language == NULL)
 	return Qnil;
       uint32_t version =  ts_language_version (ts_language);
@@ -760,6 +777,28 @@ DEFUN ("treesit-language-abi-version", Ftreesit_language_abi_version,
     }
 }
 
+DEFUN ("treesit-grammar-location", Ftreesit_grammar_location,
+       Streesit_grammar_location,
+       1, 1, 0,
+       doc: /* Return the path to the grammar file for LANGUAGE.
+
+If LANGUAGE isn't loaded yet, load it first.  If the langauge can't be
+loaded or the path couldn't be found, return nil.  */)
+  (Lisp_Object language)
+{
+  CHECK_SYMBOL (language);
+
+  Lisp_Object signal_symbol = Qnil;
+  Lisp_Object signal_data = Qnil;
+  struct treesit_loaded_lang lang
+    = treesit_load_language (language, &signal_symbol, &signal_data);
+
+  if (!lang.lang || !lang.filename) return Qnil;
+
+  return DECODE_FILE (make_unibyte_string (lang.filename,
+					   strlen (lang.filename)));
+}
+
 \f
 /*** Parsing functions  */
 
@@ -1305,8 +1344,9 @@ treesit_ensure_query_compiled (Lisp_Object query, Lisp_Object *signal_symbol,
   Lisp_Object language = XTS_COMPILED_QUERY (query)->language;
   /* This is the main reason why we compile query lazily: to avoid
      loading languages early.  */
-  TSLanguage *treesit_lang = treesit_load_language (language, signal_symbol,
-						    signal_data);
+  struct treesit_loaded_lang lang
+    = treesit_load_language (language, signal_symbol, signal_data);
+  TSLanguage *treesit_lang = lang.lang;
   if (treesit_lang == NULL)
     return NULL;
 
@@ -1477,8 +1517,9 @@ DEFUN ("treesit-parser-create",
   Lisp_Object signal_symbol = Qnil;
   Lisp_Object signal_data = Qnil;
   TSParser *parser = ts_parser_new ();
-  TSLanguage *lang = treesit_load_language (language, &signal_symbol,
-					    &signal_data);
+  struct treesit_loaded_lang loaded_lang
+    = treesit_load_language (language, &signal_symbol, &signal_data);
+  TSLanguage *lang = loaded_lang.lang;
   if (lang == NULL)
     xsignal (signal_symbol, signal_data);
   /* We check language version when loading a language, so this should
@@ -4275,6 +4316,7 @@ cons (REGEXP . FN), which is a combination of a regexp and a predicate
   defsubr (&Streesit_language_available_p);
   defsubr (&Streesit_library_abi_version);
   defsubr (&Streesit_language_abi_version);
+  defsubr (&Streesit_grammar_location);
 
   defsubr (&Streesit_parser_p);
   defsubr (&Streesit_node_p);
-- 
2.39.5 (Apple Git-151)

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Wed, 28 Aug 2024 21:54:26 -0700
> Cc: Stefan Kangas <stefankangas@gmail.com>,
>  Alan Mackenzie <acm@muc.de>,
>  64830@debbugs.gnu.org
> 
> Ah, thanks for the review! TIL. Here’s the revised patch.

A few more nits:

> +  const char *sym;
> +  dynlib_addr((void (*) (void)) langfn, &loaded_lang.filename, &sym);
               ^^
We leave one space between the function name and the opening paren.
Also, do you really need the type-cast for langfn?  Do you get
compiler warnings if you remove the cast?

> +DEFUN ("treesit-grammar-location", Ftreesit_grammar_location,
> +       Streesit_grammar_location,
> +       1, 1, 0,
> +       doc: /* Return the path to the grammar file for LANGUAGE.
                             ^^^^
"Path" again.  I suggest "absolute file name" instead.

> +If LANGUAGE isn't loaded yet, load it first.  If the langauge can't be
> +loaded or the path couldn't be found, return nil.  */)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I suggest "or the file name couldn't be determined".

Thanks.

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 1917 bytes --]

Finally able to address this again.

> On Aug 28, 2024, at 11:01 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Wed, 28 Aug 2024 21:54:26 -0700
>> Cc: Stefan Kangas <stefankangas@gmail.com>,
>> Alan Mackenzie <acm@muc.de>,
>> 64830@debbugs.gnu.org
>> 
>> Ah, thanks for the review! TIL. Here’s the revised patch.
> 
> A few more nits:
> 
>> +  const char *sym;
>> +  dynlib_addr((void (*) (void)) langfn, &loaded_lang.filename, &sym);
>               ^^
> We leave one space between the function name and the opening paren.
> Also, do you really need the type-cast for langfn?  Do you get
> compiler warnings if you remove the cast?

Oops. And yeah, I actually get an error:

treesit.c:702:16: error: incompatible function pointer types passing 'TSLanguage *(*)(void)' (aka 'struct TSLanguage *(*)(void)') to parameter of type 'void (*)(void)' [-Wincompatible-function-pointer-types]
  702 |   dynlib_addr (langfn, &loaded_lang.filename, &sym);
      |                ^~~~~~
./dynlib.h:39:26: note: passing argument to parameter 'ptr' here
   39 | void dynlib_addr (void (*ptr) (void), const char **file, const char **sym);
      |                          ^

The error seems reasonable to me, I just don’t know other way to suppress it.

>> +DEFUN ("treesit-grammar-location", Ftreesit_grammar_location,
>> +       Streesit_grammar_location,
>> +       1, 1, 0,
>> +       doc: /* Return the path to the grammar file for LANGUAGE.
>                             ^^^^
> "Path" again.  I suggest "absolute file name" instead.
> 
>> +If LANGUAGE isn't loaded yet, load it first.  If the langauge can't be
>> +loaded or the path couldn't be found, return nil.  */)
>          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> I suggest "or the file name couldn't be determined”.

Done. Please take a look at the latest patch, thanks!

Yuan


[-- Attachment #2: grammar-location.patch --]
[-- Type: application/octet-stream, Size: 6704 bytes --]

From 5158271bf6ef6d6c28d88fe21acd85169e10e93b Mon Sep 17 00:00:00 2001
From: Yuan Fu <casouri@gmail.com>
Date: Tue, 27 Aug 2024 22:31:42 -0700
Subject: [PATCH] Add Ftreesit_grammar_location

* src/treesit.c (treesit_loaded_lang): New struct.
(treesit_load_language): Return a struct instead of just the language
object.  The struct contains both the language object and the path to
the shared library.
(Ftreesit_language_available_p, Ftreesit_language_abi_version)
(treesit_ensure_query_compiled, Ftreesit_parser_create): Update
call of treesit_load_language.
(Ftreesit_grammar_location): New function.
---
 src/treesit.c | 68 +++++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 55 insertions(+), 13 deletions(-)

diff --git a/src/treesit.c b/src/treesit.c
index 5aedca44489..f2adf92d8c6 100644
--- a/src/treesit.c
+++ b/src/treesit.c
@@ -22,6 +22,7 @@ Copyright (C) 2021-2024 Free Software Foundation, Inc.
 #include <config.h>
 #include "lisp.h"
 #include "buffer.h"
+#include "coding.h"
 
 #include "treesit.h"
 
@@ -491,6 +492,14 @@ treesit_initialize (void)
 \f
 /*** Loading language library  */
 
+struct treesit_loaded_lang
+{
+  /* The language object.  */
+  TSLanguage *lang;
+  /* The path to the shared library.  */
+  const char *filename;
+};
+
 /* Translate a symbol treesit-<lang> to a C name treesit_<lang>.  */
 static void
 treesit_symbol_to_c_name (char *symbol_name)
@@ -575,7 +584,7 @@ treesit_load_language_push_for_each_suffix (Lisp_Object lib_base_name,
 
    If error occurs, return NULL and fill SIGNAL_SYMBOL and SIGNAL_DATA
    with values suitable for xsignal.  */
-static TSLanguage *
+static struct treesit_loaded_lang
 treesit_load_language (Lisp_Object language_symbol,
 		       Lisp_Object *signal_symbol, Lisp_Object *signal_data)
 {
@@ -626,6 +635,7 @@ treesit_load_language (Lisp_Object language_symbol,
   dynlib_handle_ptr handle;
   const char *error;
   Lisp_Object error_list = Qnil;
+  struct treesit_loaded_lang loaded_lang = { NULL, NULL };
 
   tail = path_candidates;
   error = NULL;
@@ -650,7 +660,7 @@ treesit_load_language (Lisp_Object language_symbol,
          mismatch.  */
       *signal_symbol = Qtreesit_load_language_error;
       *signal_data = Fcons (Qnot_found, Fnreverse (error_list));
-      return NULL;
+      return loaded_lang;
     }
 
   /* Load TSLanguage.  */
@@ -672,7 +682,7 @@ treesit_load_language (Lisp_Object language_symbol,
     {
       *signal_symbol = Qtreesit_load_language_error;
       *signal_data = list2 (Qsymbol_error, build_string (error));
-      return NULL;
+      return loaded_lang;
     }
   TSLanguage *lang = (*langfn) ();
 
@@ -685,9 +695,14 @@ treesit_load_language (Lisp_Object language_symbol,
       *signal_symbol = Qtreesit_load_language_error;
       *signal_data = list2 (Qversion_mismatch,
 			    make_fixnum (ts_language_version (lang)));
-      return NULL;
+      return loaded_lang;
     }
-  return lang;
+
+  const char *sym;
+  dynlib_addr ((void (*) void) langfn, &loaded_lang.filename, &sym);
+
+  loaded_lang.lang = lang;
+  return loaded_lang;
 }
 
 DEFUN ("treesit-language-available-p", Ftreesit_language_available_p,
@@ -704,7 +719,9 @@ DEFUN ("treesit-language-available-p", Ftreesit_language_available_p,
   treesit_initialize ();
   Lisp_Object signal_symbol = Qnil;
   Lisp_Object signal_data = Qnil;
-  if (treesit_load_language (language, &signal_symbol, &signal_data) == NULL)
+  struct treesit_loaded_lang loaded_lang
+    = treesit_load_language (language, &signal_symbol, &signal_data);
+  if (loaded_lang.lang == NULL)
     {
       if (NILP (detail))
 	return Qnil;
@@ -750,9 +767,9 @@ DEFUN ("treesit-language-abi-version", Ftreesit_language_abi_version,
     {
       Lisp_Object signal_symbol = Qnil;
       Lisp_Object signal_data = Qnil;
-      TSLanguage *ts_language = treesit_load_language (language,
-						       &signal_symbol,
-						       &signal_data);
+      struct treesit_loaded_lang lang
+	= treesit_load_language (language, &signal_symbol, &signal_data);
+      TSLanguage *ts_language = lang.lang;
       if (ts_language == NULL)
 	return Qnil;
       uint32_t version =  ts_language_version (ts_language);
@@ -760,6 +777,28 @@ DEFUN ("treesit-language-abi-version", Ftreesit_language_abi_version,
     }
 }
 
+DEFUN ("treesit-grammar-location", Ftreesit_grammar_location,
+       Streesit_grammar_location,
+       1, 1, 0,
+       doc: /* Return the absolute file name of the grammar file for LANGUAGE.
+
+If LANGUAGE isn't loaded yet, load it first.  If the langauge can't be
+loaded or the file name couldn't be determined, return nil.  */)
+  (Lisp_Object language)
+{
+  CHECK_SYMBOL (language);
+
+  Lisp_Object signal_symbol = Qnil;
+  Lisp_Object signal_data = Qnil;
+  struct treesit_loaded_lang lang
+    = treesit_load_language (language, &signal_symbol, &signal_data);
+
+  if (!lang.lang || !lang.filename) return Qnil;
+
+  return DECODE_FILE (make_unibyte_string (lang.filename,
+					   strlen (lang.filename)));
+}
+
 \f
 /*** Parsing functions  */
 
@@ -1305,8 +1344,9 @@ treesit_ensure_query_compiled (Lisp_Object query, Lisp_Object *signal_symbol,
   Lisp_Object language = XTS_COMPILED_QUERY (query)->language;
   /* This is the main reason why we compile query lazily: to avoid
      loading languages early.  */
-  TSLanguage *treesit_lang = treesit_load_language (language, signal_symbol,
-						    signal_data);
+  struct treesit_loaded_lang lang
+    = treesit_load_language (language, signal_symbol, signal_data);
+  TSLanguage *treesit_lang = lang.lang;
   if (treesit_lang == NULL)
     return NULL;
 
@@ -1477,8 +1517,9 @@ DEFUN ("treesit-parser-create",
   Lisp_Object signal_symbol = Qnil;
   Lisp_Object signal_data = Qnil;
   TSParser *parser = ts_parser_new ();
-  TSLanguage *lang = treesit_load_language (language, &signal_symbol,
-					    &signal_data);
+  struct treesit_loaded_lang loaded_lang
+    = treesit_load_language (language, &signal_symbol, &signal_data);
+  TSLanguage *lang = loaded_lang.lang;
   if (lang == NULL)
     xsignal (signal_symbol, signal_data);
   /* We check language version when loading a language, so this should
@@ -4275,6 +4316,7 @@ cons (REGEXP . FN), which is a combination of a regexp and a predicate
   defsubr (&Streesit_language_available_p);
   defsubr (&Streesit_library_abi_version);
   defsubr (&Streesit_language_abi_version);
+  defsubr (&Streesit_grammar_location);
 
   defsubr (&Streesit_parser_p);
   defsubr (&Streesit_node_p);
-- 
2.39.5 (Apple Git-151)


[-- Attachment #3: Type: text/plain, Size: 2 bytes --]

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Tue, 10 Sep 2024 22:09:04 -0700
> Cc: stefankangas@gmail.com,
>  acm@muc.de,
>  64830@debbugs.gnu.org
> 
> Done. Please take a look at the latest patch, thanks!

LGTM, but...

> +struct treesit_loaded_lang
> +{
> +  /* The language object.  */
> +  TSLanguage *lang;
> +  /* The path to the shared library.  */
> +  const char *filename;
> +};
> +

..."path" again.

> +DEFUN ("treesit-grammar-location", Ftreesit_grammar_location,
> +       Streesit_grammar_location,
> +       1, 1, 0,
> +       doc: /* Return the absolute file name of the grammar file for LANGUAGE.
> +
> +If LANGUAGE isn't loaded yet, load it first.  If the langauge can't be
                                                        ^^^^^^^^
A typo.

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Yuan Fu]

> On Sep 11, 2024, at 5:09 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Tue, 10 Sep 2024 22:09:04 -0700
>> Cc: stefankangas@gmail.com,
>> acm@muc.de,
>> 64830@debbugs.gnu.org
>> 
>> Done. Please take a look at the latest patch, thanks!
> 
> LGTM, but...
> 
>> +struct treesit_loaded_lang
>> +{
>> +  /* The language object.  */
>> +  TSLanguage *lang;
>> +  /* The path to the shared library.  */
>> +  const char *filename;
>> +};
>> +
> 
> ..."path" again.
> 
>> +DEFUN ("treesit-grammar-location", Ftreesit_grammar_location,
>> +       Streesit_grammar_location,
>> +       1, 1, 0,
>> +       doc: /* Return the absolute file name of the grammar file for LANGUAGE.
>> +
>> +If LANGUAGE isn't loaded yet, load it first.  If the langauge can't be
>                                                        ^^^^^^^^
> A typo.


Oops (^^;) That’s embarrassing. Ok, I fixed everything and pushed to master.

Also, since the original problem is fixed, closing this report.

Yuan

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> Date: Mon, 26 Aug 2024 19:50:43 +0000
> Cc: casouri@gmail.com, 64830@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > We support grammar libraries in ~/.emacs.d/tree-sitter so that users
> > could easily install versions of grammar libraries different from what
> > their sysadmins decide to install.
> 
> Ah, OK.  Seems like a good reason.
> 
> How about outputting a message looking something like this:
> 
>     Loading /home/acm/.emacs.d/tree-sitter/libtree-sitter.so ....Done
> 
> ..  This wouldn't get in the way, yet might well prevent some of the
> problems I've been having over the last few days.

As a diagnostic tool, by default off, maybe.  Annoying everyone with
this message, when it will almost always be redundant, doesn't sound
like TRT to me.

And note that in the usual case, when the grammar library is loaded
from a system-wide directory of shared libraries, Emacs doesn't even
know the absolute file name of the library, just its base name.

bug#64830: 30.0.50 C++ treesitter mode no coloration

[Eli Zaretskii]

> Date: Mon, 26 Aug 2024 17:25:34 +0000
> Cc: Eli Zaretskii <eliz@gnu.org>, 64830@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> Might it be worthwhile adding some functionality to Emacs whereby a user
> could check which versions of grammar libraries she's got?  Or is there
> something like that already?

Many (most?) grammar libraries don't have versions and don't make
releases, so this would be tricky to implement.