bug#75456: 31.0.50; Cannot find the definition of the predicate in `treesit-thing-settings'" sexp-list)

bug#75456: 31.0.50; Cannot find the definition of the predicate in `treesit-thing-settings'" sexp-list)

[Vincenzo Pupillo]

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]

Ciao, 
this problem occurs with js-ts-mode when jsdoc is enabled. Occurs when you 
place the cursor in a comment highlighted by jsdoc.
This is the error message:

"treesit-show-paren-data--categorize: peculiar error: ("Cannot find the 
definition of the predicate in `treesit-thing-settings'" sexp-list)"

Could this be due to commit 493bb2eaf1610ec950e4a2f7229d6e1f939064b9 ?

GNU Emacs 31.0.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version 3.24.43, cairo 
version 1.18.2) of 2025-01-09

Thanks.

Vincenzo

[-- Attachment #2: js_sexp-list_issue.png --]
[-- Type: image/png, Size: 19327 bytes --]

bug#75456: 31.0.50; Cannot find the definition of the predicate in `treesit-thing-settings'" sexp-list)

[Juri Linkov]

> Ciao, 
> this problem occurs with js-ts-mode when jsdoc is enabled. Occurs when you 
> place the cursor in a comment highlighted by jsdoc.
> This is the error message:
>
> "treesit-show-paren-data--categorize: peculiar error: ("Cannot find the 
> definition of the predicate in `treesit-thing-settings'" sexp-list)"

Recently I fixed bug#75198 and tested it on your mhtml-ts-mode
where it works nicely, and (treesit-language-at (point))
returns e.g. 'css' on embedded css.

However, now I discovered that (treesit-language-at (point))
returns 'javascript' on embedded jsdoc parts of a js file.

I wonder is this intended?

Since 'treesit-node-match-p' uses node's parser, I could use
(treesit-parser-language (treesit-node-parser (treesit-node-at (point))))
that returns 'jsdoc'.

But first I'd like to know why treesit-language-at doesn't return the same?

bug#75456: 31.0.50; Cannot find the definition of the predicate in `treesit-thing-settings'" sexp-list)

[Vincenzo Pupillo]

Ciao Juri,

In data giovedì 9 gennaio 2025 19:10:28 Ora standard dell’Europa centrale, 
Juri Linkov ha scritto:
> > Ciao,
> > this problem occurs with js-ts-mode when jsdoc is enabled. Occurs when you
> > place the cursor in a comment highlighted by jsdoc.
> > This is the error message:
> > 
> > "treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
> > definition of the predicate in `treesit-thing-settings'" sexp-list)"
> 
> Recently I fixed bug#75198 and tested it on your mhtml-ts-mode
> where it works nicely, and (treesit-language-at (point))
> returns e.g. 'css' on embedded css.
> 
> However, now I discovered that (treesit-language-at (point))
> returns 'javascript' on embedded jsdoc parts of a js file.
> 
> I wonder is this intended?
> 
> Since 'treesit-node-match-p' uses node's parser, I could use
> (treesit-parser-language (treesit-node-parser (treesit-node-at (point))))
> that returns 'jsdoc'.
> 
> But first I'd like to know why treesit-language-at doesn't return the same?

Local parsers, like jsdoc or phpdoc, are "ignored" by several functions used 
by treesit. But I don't know in detail how treesit works.
I tried to return local parser languages ​​in the past, but both indentation 
and font-locking got messed up.
I just tried and the result is the same (maybe I did something wrong), and 
then treesit-parser-list also returns jsdoc as if it were a global parser.

@Yuan can be of more help.

Thanks.

Vincenzo

bug#75456: 31.0.50; Cannot find the definition of the predicate in `treesit-thing-settings'" sexp-list)

[Yuan Fu]

> On Jan 9, 2025, at 2:42 PM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> 
> Ciao Juri,
> 
> In data giovedì 9 gennaio 2025 19:10:28 Ora standard dell’Europa centrale, 
> Juri Linkov ha scritto:
>>> Ciao,
>>> this problem occurs with js-ts-mode when jsdoc is enabled. Occurs when you
>>> place the cursor in a comment highlighted by jsdoc.
>>> This is the error message:
>>> 
>>> "treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
>>> definition of the predicate in `treesit-thing-settings'" sexp-list)"
>> 
>> Recently I fixed bug#75198 and tested it on your mhtml-ts-mode
>> where it works nicely, and (treesit-language-at (point))
>> returns e.g. 'css' on embedded css.
>> 
>> However, now I discovered that (treesit-language-at (point))
>> returns 'javascript' on embedded jsdoc parts of a js file.
>> 
>> I wonder is this intended?
>> 
>> Since 'treesit-node-match-p' uses node's parser, I could use
>> (treesit-parser-language (treesit-node-parser (treesit-node-at (point))))
>> that returns 'jsdoc'.
>> 
>> But first I'd like to know why treesit-language-at doesn't return the same?
> 
> Local parsers, like jsdoc or phpdoc, are "ignored" by several functions used 
> by treesit. But I don't know in detail how treesit works.
> I tried to return local parser languages ​​in the past, but both indentation 
> and font-locking got messed up.
> I just tried and the result is the same (maybe I did something wrong), and 
> then treesit-parser-list also returns jsdoc as if it were a global parser.
> 
> @Yuan can be of more help.
> 
> Thanks.
> 
> Vincenzo

If local parsers somehow doesn’t work, it’s a bug. What are the functions that ignore local parsers? And how exactly does indentation and font-lock mess up?

For indentation, there’s a special condition: if the largest node at point is a root node for the local parser, we don’t use it for matching indent rules, because it’s impossible to figure out how to indent it; instead, Emacs uses the host parser’s node at that point. (The logic is in treesit--indent-largest-node-at).

What do you mean by  treesit-parser-list also returns jsdoc as if it were a global parser? Does it return the jsdoc parser even you pass nil to the TAG parameter? If so, I think it’s because some treesit function accidentally created it when looking for a parser for jsdoc. I’ll try fix that.

Yuan

bug#75456: 31.0.50; Cannot find the definition of the predicate in `treesit-thing-settings'" sexp-list)

[Vincenzo Pupillo]

[-- Attachment #1: Type: text/plain, Size: 3466 bytes --]

Ciao Yuan

In data venerdì 10 gennaio 2025 04:23:42 Ora standard dell’Europa centrale, 
Yuan Fu ha scritto:
> > On Jan 9, 2025, at 2:42 PM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> > 
> > Ciao Juri,
> > 
> > In data giovedì 9 gennaio 2025 19:10:28 Ora standard dell’Europa centrale,
> > 
> > Juri Linkov ha scritto:
> >>> Ciao,
> >>> this problem occurs with js-ts-mode when jsdoc is enabled. Occurs when
> >>> you
> >>> place the cursor in a comment highlighted by jsdoc.
> >>> This is the error message:
> >>> 
> >>> "treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
> >>> definition of the predicate in `treesit-thing-settings'" sexp-list)"
> >> 
> >> Recently I fixed bug#75198 and tested it on your mhtml-ts-mode
> >> where it works nicely, and (treesit-language-at (point))
> >> returns e.g. 'css' on embedded css.
> >> 
> >> However, now I discovered that (treesit-language-at (point))
> >> returns 'javascript' on embedded jsdoc parts of a js file.
> >> 
> >> I wonder is this intended?
> >> 
> >> Since 'treesit-node-match-p' uses node's parser, I could use
> >> (treesit-parser-language (treesit-node-parser (treesit-node-at (point))))
> >> that returns 'jsdoc'.
> >> 
> >> But first I'd like to know why treesit-language-at doesn't return the
> >> same?
> > 
> > Local parsers, like jsdoc or phpdoc, are "ignored" by several functions
> > used by treesit. But I don't know in detail how treesit works.
> > I tried to return local parser languages ​​in the past, but both
> > indentation and font-locking got messed up.
> > I just tried and the result is the same (maybe I did something wrong), and
> > then treesit-parser-list also returns jsdoc as if it were a global parser.
> > 
> > @Yuan can be of more help.
> > 
> > Thanks.
> > 
> > Vincenzo
> 
> If local parsers somehow doesn’t work, it’s a bug. What are the functions
> that ignore local parsers? And how exactly does indentation and font-lock
> mess up?
> 
> For indentation, there’s a special condition: if the largest node at point
> is a root node for the local parser, we don’t use it for matching indent
> rules, because it’s impossible to figure out how to indent it; instead,
> Emacs uses the host parser’s node at that point. (The logic is in
> treesit--indent-largest-node-at).
> 
> What do you mean by  treesit-parser-list also returns jsdoc as if it were a
> global parser? Does it return the jsdoc parser even you pass nil to the TAG
> parameter? If so, I think it’s because some treesit function accidentally
> created it when looking for a parser for jsdoc. I’ll try fix that.
> 
> Yuan
As I wrote I might have made some mistakes (I tried with the new version of 
mhtml-ts-mode which is more complicated than js-ts-mode). So I preferred to 
write a patch for js-ts-mode. I could not replicate the indentation problems 
but only the font-locking ones.
If you try with the attached javascript file (setting treesit-font-lock-level 
to 4) you will see the problem and if you invoke (treesit-parser-list) you 
will see that jsdoc also appears in the list.


Vincenzo

p.s. Gmail doesn't like my .js file so I wrote it below:
/**
 * Foo
 * @param {int} a blah blah
 */
var foo = function(a) {
    console.log(a); // blah blah
};


/**
 * Bar
 * @param {int} a blah blah
 */
var bar = function(a) {
    console.log(a); // blah blah
};


[-- Attachment #2: 0001-language-at-point-function-that-handles-local-parser.patch --]
[-- Type: text/x-patch, Size: 1718 bytes --]

From 5981244ffa33496a6f1bc912acf914836d3559df Mon Sep 17 00:00:00 2001
From: Vincenzo Pupillo <v.pupillo@gmail.com>
Date: Fri, 10 Jan 2025 14:32:37 +0100
Subject: [PATCH] language-at-point function that handles local parser.

Just for testing.

* lisp/progmodes/js.el
(js--treesit-language-at-point): New function.
(js-ts-mode): Use the new function.
---
 lisp/progmodes/js.el | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/lisp/progmodes/js.el b/lisp/progmodes/js.el
index 101b882c718..3d172b2d2da 100644
--- a/lisp/progmodes/js.el
+++ b/lisp/progmodes/js.el
@@ -3718,6 +3718,23 @@ js--treesit-valid-imenu-entry
     ("lexical_declaration" (treesit-node-top-level node))
     (_ t)))
 
+(defun js--treesit-language-at-point (point)
+  "Return the language at POINT."
+  (let* ((node (treesit-node-at point 'javascript))
+         (node-type (treesit-node-type node))
+         (node-start (treesit-node-start node))
+         (node-end (treesit-node-end node)))
+    (if (not (treesit-ready-p 'jsdoc t))
+        'javascript
+      (if (equal node-type "comment")
+          (save-excursion
+            (message "node start = %s , end = %s" node-start node-end)
+            (goto-char node-start)
+            (if (search-forward "/**" node-end t)
+                'jsdoc
+              'javascript))
+        'javascript))))
+
 ;;; Main Function
 
 ;;;###autoload
@@ -3927,6 +3944,7 @@ js-ts-mode
 
     ;; Tree-sitter setup.
     (setq-local treesit-primary-parser (treesit-parser-create 'javascript))
+    (setq-local treesit-language-at-point-function #'js--treesit-language-at-point)
 
     ;; Indent.
     (setq-local treesit-simple-indent-rules js--treesit-indent-rules)
-- 
2.47.1

bug#75456: 31.0.50; Cannot find the definition of the predicate in `treesit-thing-settings'" sexp-list)

[Yuan Fu]

> On Jan 10, 2025, at 5:50 AM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
> 
> Ciao Yuan
> 
> In data venerdì 10 gennaio 2025 04:23:42 Ora standard dell’Europa centrale, 
> Yuan Fu ha scritto:
>>> On Jan 9, 2025, at 2:42 PM, Vincenzo Pupillo <v.pupillo@gmail.com> wrote:
>>> 
>>> Ciao Juri,
>>> 
>>> In data giovedì 9 gennaio 2025 19:10:28 Ora standard dell’Europa centrale,
>>> 
>>> Juri Linkov ha scritto:
>>>>> Ciao,
>>>>> this problem occurs with js-ts-mode when jsdoc is enabled. Occurs when
>>>>> you
>>>>> place the cursor in a comment highlighted by jsdoc.
>>>>> This is the error message:
>>>>> 
>>>>> "treesit-show-paren-data--categorize: peculiar error: ("Cannot find the
>>>>> definition of the predicate in `treesit-thing-settings'" sexp-list)"
>>>> 
>>>> Recently I fixed bug#75198 and tested it on your mhtml-ts-mode
>>>> where it works nicely, and (treesit-language-at (point))
>>>> returns e.g. 'css' on embedded css.
>>>> 
>>>> However, now I discovered that (treesit-language-at (point))
>>>> returns 'javascript' on embedded jsdoc parts of a js file.
>>>> 
>>>> I wonder is this intended?
>>>> 
>>>> Since 'treesit-node-match-p' uses node's parser, I could use
>>>> (treesit-parser-language (treesit-node-parser (treesit-node-at (point))))
>>>> that returns 'jsdoc'.
>>>> 
>>>> But first I'd like to know why treesit-language-at doesn't return the
>>>> same?
>>> 
>>> Local parsers, like jsdoc or phpdoc, are "ignored" by several functions
>>> used by treesit. But I don't know in detail how treesit works.
>>> I tried to return local parser languages ​​in the past, but both
>>> indentation and font-locking got messed up.
>>> I just tried and the result is the same (maybe I did something wrong), and
>>> then treesit-parser-list also returns jsdoc as if it were a global parser.
>>> 
>>> @Yuan can be of more help.
>>> 
>>> Thanks.
>>> 
>>> Vincenzo
>> 
>> If local parsers somehow doesn’t work, it’s a bug. What are the functions
>> that ignore local parsers? And how exactly does indentation and font-lock
>> mess up?
>> 
>> For indentation, there’s a special condition: if the largest node at point
>> is a root node for the local parser, we don’t use it for matching indent
>> rules, because it’s impossible to figure out how to indent it; instead,
>> Emacs uses the host parser’s node at that point. (The logic is in
>> treesit--indent-largest-node-at).
>> 
>> What do you mean by  treesit-parser-list also returns jsdoc as if it were a
>> global parser? Does it return the jsdoc parser even you pass nil to the TAG
>> parameter? If so, I think it’s because some treesit function accidentally
>> created it when looking for a parser for jsdoc. I’ll try fix that.
>> 
>> Yuan
> As I wrote I might have made some mistakes (I tried with the new version of 
> mhtml-ts-mode which is more complicated than js-ts-mode). So I preferred to 
> write a patch for js-ts-mode. I could not replicate the indentation problems 
> but only the font-locking ones.
> If you try with the attached javascript file (setting treesit-font-lock-level 
> to 4) you will see the problem and if you invoke (treesit-parser-list) you 
> will see that jsdoc also appears in the list.
> 
> 
> Vincenzo
> 
> p.s. Gmail doesn't like my .js file so I wrote it below:
> /**
> * Foo
> * @param {int} a blah blah
> */
> var foo = function(a) {
>    console.log(a); // blah blah
> };
> 
> 
> /**
> * Bar
> * @param {int} a blah blah
> */
> var bar = function(a) {
>    console.log(a); // blah blah
> };
> 
> <0001-language-at-point-function-that-handles-local-parser.patch>

I just pushed a fix, now you shouldn’t see jsdoc parsers anymore. Can you see if this fixes the problem you had?

Yuan

bug#75456: 31.0.50; Cannot find the definition of the predicate in `treesit-thing-settings'" sexp-list)

[Vincenzo Pupillo]

Ciao Yuan,

In data sabato 18 gennaio 2025 07:27:32 Ora standard dell’Europa centrale, 
Yuan Fu ha scritto:
> ....
> 
> I just pushed a fix, now you shouldn’t see jsdoc parsers anymore. Can you
> see if this fixes the problem you had?
> 
I only tested js.el with 0001-language-at-point-function-that-handles-local-
parser.patch
I tested only js.el with the 0001-language-at-point-function-that-handles-
local-parser.patch because it was less complex. The font-locking is ok but the 
indentation of jsdoc type comments breaks because there are no specific rules 
(for jsdoc) and also this message appears:

Error running timer 'show-paren-function': (treesit-no-parser jsdoc)

The comment:
;; 2. In most cases, a function shouldn't implicitly create a parser.
;; All parsers should be created explicitly by user.

Does this imply that we need to create local parsers as well?


> Yuan