Tree sitter question / problem

Tree sitter question / problem

[Perry Smith]

[-- Attachment #1.1: Type: text/plain, Size: 2870 bytes --]

As a result of not knowing what I’m doing, this took me a half a day to figure this out.  And I fear it is going to be hard for me to explain things.  Feel free to ask me questions.

I’m working on ruby-ts-mode and it is patterned mostly after c-ts-mode because I want the versatility.

I have this bit of code and it is the content of the comment that is the topic of the email:

;; doc/keywords.rdoc in the Ruby git repository considers these to be
;; reserved keywords.  If these keywords are added to the list, it
;; causes the font-lock to stop working.
;;
;; "__ENCODING__" "__FILE__" "__LINE__" "false" "self" "super" "true"
;;
;; "nil" (which does not exhibit this issue) is also considered a
;; keyword but I removed it and added it as a constant.
;;
(defun ruby-ts-mode--keywords (language)
  "Ruby keywords for tree-sitter font-locking.
Currently LANGUAGE is ignored but shoule be set to `ruby'."
  (let ((common-keywords
         '("BEGIN" "END" "alias" "and" "begin" "break" "case" "class"
           "def" "defined?" "do" "else" "elsif" "end" "ensure" "for"
           "if" "in" "module" "next" "not" "or" "redo" "rescue"
           "retry" "return" "then" "undef" "unless" "until" "when"
           "while" "yield")))
    common-keywords))

I also have this in my ruby-ts-mode--font-lock-settings:

   :language language
   :feature 'keyword
   `([,@(ruby-ts-mode--keywords language)] @font-lock-keyword-face)

(I understand this is just two snippets of a more complex set up but I’m hoping that c-ts-mode can be reviewed for more context if needed.)

The Ruby parser I got from git@github.com:casouri/tree-sitter-module.git and followed the instructions.

As the comment states, if I add “false” into the list of keywords, then **none** of the keywords get highlighted.  I can understand that “false” doesn’t get highlighted for some reason but having the entire query stop working seems like it might be a bug or at least a weakness that should be addressed.

If I turn on treesit-inspect-mode and put the cursor on top of false in a Ruby file, it says only (false) instead of something like (while “while”) that other keyword nodes that do not exhibit this problem show and I’m jumping to the conclusion that that is part of the problem.

Perhaps the parser should report (false “false”) but at the same time, it seems rather weak that the whole query stops working.

Thank you,
Perry
p.s. while I know of no other language that is similar to Ruby, I’m adding in the versatility for the mode to be readily adapted to other languages and also has the ability, like c-mode, to easily have different styles.  I was never 100% satisfied with existing modes and always wanted to tweak them.  I think with the tree sitter concepts, this is going to be much easier and more frequent.


[-- Attachment #1.2: Type: text/html, Size: 5260 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Tree sitter question / problem

[Theodor Thornhill]

Hi there, Perry

Perry Smith <pedz@easesoftware.com> writes:

> As a result of not knowing what I’m doing, this took me a half a day to figure this out.  And I fear it is going to be hard for me to explain things.  Feel free to ask me questions.
>
> I’m working on ruby-ts-mode and it is patterned mostly after c-ts-mode because I want the versatility.
>
> I have this bit of code and it is the content of the comment that is the topic of the email:
>
> ;; doc/keywords.rdoc in the Ruby git repository considers these to be
> ;; reserved keywords.  If these keywords are added to the list, it
> ;; causes the font-lock to stop working.
> ;;
> ;; "__ENCODING__" "__FILE__" "__LINE__" "false" "self" "super" "true"
> ;;
> ;; "nil" (which does not exhibit this issue) is also considered a
> ;; keyword but I removed it and added it as a constant.
> ;;
> (defun ruby-ts-mode--keywords (language)
>   "Ruby keywords for tree-sitter font-locking.
> Currently LANGUAGE is ignored but shoule be set to `ruby'."
>   (let ((common-keywords
>          '("BEGIN" "END" "alias" "and" "begin" "break" "case" "class"
>            "def" "defined?" "do" "else" "elsif" "end" "ensure" "for"
>            "if" "in" "module" "next" "not" "or" "redo" "rescue"
>            "retry" "return" "then" "undef" "unless" "until" "when"
>            "while" "yield")))
>     common-keywords))
>

If you try (treesit-query-validate 'ruby (ruby-ts-mode--keywords 'ruby))
you should get a validation error and some help.  My guess is that
'false' and 'true' are nodes in their own right, such as (true) (false).

...

Yeah, you need to use the nodes separately [0].

> I also have this in my ruby-ts-mode--font-lock-settings:
>
>    :language language
>    :feature 'keyword
>    `([,@(ruby-ts-mode--keywords language)] @font-lock-keyword-face)
>
> (I understand this is just two snippets of a more complex set up but I’m hoping that c-ts-mode can be reviewed for more context if needed.)
>
> The Ruby parser I got from git@github.com:casouri/tree-sitter-module.git and followed the instructions.
>
> As the comment states, if I add “false” into the list of keywords,
> then **none** of the keywords get highlighted.  I can understand that
> “false” doesn’t get highlighted for some reason but having the entire
> query stop working seems like it might be a bug or at least a weakness
> that should be addressed.

That may be, but this is for major mode authors, so it is likely we can
live with these difficulties, IMO.

>
> If I turn on treesit-inspect-mode and put the cursor on top of false
> in a Ruby file, it says only (false) instead of something like (while
> “while”) that other keyword nodes that do not exhibit this problem
> show and I’m jumping to the conclusion that that is part of the
> problem.

This is because it is a node, and not a keyword.


[0]: https://github.com/tree-sitter/tree-sitter-ruby/blob/master/grammar.js#L533-L541