bug#64830: 30.0.50 C++ treesitter mode no coloration

[Alan Mackenzie <acm@muc.de>]

Hello, Yuan.

On Sun, Aug 25, 2024 at 15:40:31 -0700, Yuan Fu wrote:


> > On Aug 24, 2024, at 7:19 PM, Alan Mackenzie <acm@muc.de> wrote:

> > Hello, Yuan.

> > On Sat, Aug 24, 2024 at 13:43:25 -0700, Yuan Fu wrote:


> >>> On Aug 24, 2024, at 12:38 PM, Alan Mackenzie <acm@muc.de> wrote:
> >>> On Sat, Aug 24, 2024 at 11:35:36 -0700, Yuan Fu wrote:
> >>>>> On Aug 19, 2024, at 8:46 PM, Yuan Fu <casouri@gmail.com> wrote:
> >>>>>> On Aug 16, 2024, at 11:27 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> >>>>>>> Date: Fri, 16 Aug 2024 18:06:31 +0000
> >>>>>>> Cc: 64830@debbugs.gnu.org, casouri@gmail.com, acm@muc.de
> >>>>>>> From: Alan Mackenzie <acm@muc.de>

> >>>>>>>> Maybe your Emacs 30 build is old?

> >>>>>>> No.  I updated it on Wednesday, the most recent commit I have being:

> >>>>>>> commit 9bedb957bebdca99b1bb96f58ea790e20ed48dee (HEAD -> emacs-30,
> >>>>>>> origin/emacs-30)
> >>>>>>> Author: Eli Zaretskii <eliz@gnu.org>
> >>>>>>> Date:   Wed Aug 14 11:35:48 2024 +0300

> >>>>>>> Improve documentation of time-parsing functions

> >>>>>>> ..  I will update it right now and retry ....

> >>>>>>> ..... DONE.  It makes no difference.  I don't understand either
> >>>>>>> why I see this bug and you don't.

> >>>>>> Maybe try updating the C++ grammar library?

> >>>>>> Yuan, any ideas?

> >>>>> Nothing obviously wrong from a glance. I’m very busy recently but
> >>>>> I’ll have some time this week to look into this. Sorry for the
> >>>>> delay :-)

> >>>>> Yuan

> >>>> Upon closer inspection, I think this is caused by a recent change in
> >>>> c-ts-mode font-lock rules, in
> >>>> 014aab9847a0d3d898cb8cbc7224143f2d741abb.

> >>>> Alan, could you do this: don’t upgrade your c++ grammar and try this
> >>>> patch, if I was right it should fix your problem. Thanks!

> >>> I updated my emacs-30 branch, checked that that commit was included,
> >>> and rebuilt it.  The problem still exists on my copy of Emacs 30.
> >>> :-(

> >> No no I mean apply the attached patch and see if it fixes the problem.
> >> The commit hash I mentioned is the source of the bug, not the fix ;-)

> > Sorry about the misunderstanding.

> > I've now applied your patch, the one whose first hunk's header is:

> > @@ -537,6 +537,16 @@ c-ts-mode--top-level-label-matcher

> > , but it unfortunately doesn't solve the bug.  On templates-21.cc, one of
> > the test files from CC Mode, on doing M-x c++-ts-mode, there is no
> > fontification at all.  In *Messages* we have

> > Error during redisplay: (jit-lock-function 1) signaled
> > (treesit-query-error "Node type error at" 677 "[\"_Atomic\" \"break\"
> > \"case\" \"const\" \"continue\" \"default\" \"do\" \"else\" \"enum\"
> > \"extern\" \"for\" \"goto\" \"if\" \"inline\" \"register\" \"restrict\"
> > \"return\" \"sizeof\" \"static\" \"struct\" \"switch\" \"typedef\"
> > \"union\" \"volatile\" \"while\" \"and\" \"and_eq\" \"bitand\" \"bitor\"
> > \"catch\" \"class\" \"co_await\" \"co_return\" \"co_yield\" \"compl\"
> > \"concept\" \"consteval\" \"constexpr\" \"constinit\" \"decltype\"
> > \"delete\" \"explicit\" \"final\" \"friend\" \"mutable\" \"namespace\"
> > \"new\" \"noexcept\" \"not\" \"not_eq\" \"operator\" \"or\" \"or_eq\"
> > \"override\" \"private\" \"protected\" \"public\" \"requires\"
> > \"template\" \"throw\" \"try\" \"typename\" \"using\" \"xor\" \"xor_eq\"]
> > @font-lock-keyword-face (auto) @font-lock-keyword-face (this)
> > @font-lock-keyword-face (virtual) @font-lock-keyword-face" "Debug the
> > query with `treesit-query-validate'")

> > That "Node type error at" 677 isn't a buffer position - the buffer is
> > only 325 characters long.

> > Is there anything else I could do to help, here?

> Yes, could you try evaluating this three forms, and tell me their result?

Done.  But see below, first!

> (treesit-query-validate 'cpp '("virtual" @cap))

"QUERY is valid"

> (treesit-query-validate 'cpp '((virtual) @cap))

In *Tree-Sitter check query*, I got:
    Node type error at: 2
    (virtual) @cap
In the reponse area, I got:
    t
..

> (treesit-query-validate 'cpp '((auto) @font-lock-keyword-face
>                                (this) @font-lock-keyword-face
>                                (virtual) @font-lock-keyword-face))

In *Tree-Sitter check query*, I got:
    Node type error at: 64
    (auto) @font-lock-keyword-face (this) @font-lock-keyword-face (virtual) @font-lock-keyword-face
, and again in the response area:
    t
..

> To give more context, in the error you see, position 677 is a position
> in the query string, which points to the (virtual) part. That leads me
> to believe your grammar doesn’t recognize the (virtual) query. So in
> the patch I provided earlier, I added a function that tests whether the
> grammar recognizes (virtual), and if not, it uses the “virtual” query
> instead. The difference between (virtual) and “virtual” in the query is
> that (virtual) is a named node, and “virtual” is a keyword.

> I’m still not sure why you have this problem though, because I built
> libtree-sitter-cpp v0.22.2 and v0.22.0 and tested them locally, and
> they recognize (virtual) just fine. Anyway, evaluating those three
> forms will tell us what’s going on.

I downloaded my libtree-sitter-cpp.so.0.14 from my GNU/Linux
distribution, Gentoo.  It regards itself as dev-libs/tree-sitter-cpp
version 0.22.2.

Might it be worthwhile adding some functionality to Emacs whereby a user
could check which versions of grammar libraries she's got?  Or is there
something like that already?

> And to your question for building tree-sitter grammar, you can use the
> script at https://github.com/casouri/tree-sitter-module

> You can either run ./batch to build all the languages, or run ./build
> cpp to only build for cpp. Then just copy the grammar file in dist to
> ~/.emacs.d/tree-sitter.

Ah, that's it!  I didn't know the grammar library in use was the one in
~/.emacs/tree-sitter.  There, I've got 

    -rwxr-xr-x 1 acm acm 2319888 Jan 19  2023 libtree-sitter-cpp.so

, which is very old.  In my /usr/lib64, I've got

    lrwxrwxrwx 1 root root      26 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so.0 -> libtree-sitter-cpp.so.0.14
    lrwxrwxrwx 1 root root      23 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so -> libtree-sitter-cpp.so.0
    -rwxr-xr-x 1 root root 3175712 Aug  4 18:13 /usr/lib64/libtree-sitter-cpp.so.0.14

Why do we have this extra complication with ~/.emacs.d/tree-sitter as a
location for libraries?  In GNU/Linux distributions (well, in Gentoo at
any rate), newly downloaded libraries are put into /usr/lib64, so that
anybody naively updating a grammar library is going to have it masked out
by the old one still in ~/.emacs.d/tree-sitter.  Maybe I don't understand
the library mechanism as well as I ought to.

> Yuan

-- 
Alan Mackenzie (Nuremberg, Germany).