Validating tree-sitter major modes and grammar

Validating tree-sitter major modes and grammar

[Yuan Fu]

Continuing from the tree-sitter maternity thread, I cooked up some script to go over each builtin tree-sitter mode, clone the grammars it uses, and check whether the font-lock queries are compatible with the latest version of the grammar. If everything works fine, the script adds some comment in the source file listing the version that was checked.

For example, I ran the script for c-ts-mode and c++-ts-mode, and this is the comment inserted to c-ts-mode.el:

;;; Tree-sitter language versions
;;
;; c-ts-mode is known to work with the following languages and version:
;; - tree-sitter-c: v0.20.8-61-g3efee11
;;
;; c++-ts-mode is known to work with the following languages and version:
;; - tree-sitter-cpp: v0.20.5-49-gf41b4f6
;;
;; We try our best to make builtin modes work with latest grammar
;; versions, so a more recent grammar version has a good chance to work.
;; Send us a bug report if it doesn't.

The version is taken from “git describe”. If the grammar isn’t versioned, it’ll just be the hash of HEAD. In order for “git describe” to work, I had to do full clones, and some tree-sitter grammar’s repo is very slow to clone. I wonder if there’s anything we can do to speed it up? 

Any thoughts/comments? I pushed the code to master so others can play with it.

BTW, the script revealed that cmake-ts-mode isn’t compatible with the latest grammar. I’ll try fix it later.

Yuan

Re: Validating tree-sitter major modes and grammar

[Eli Zaretskii]

> From: Yuan Fu <casouri@gmail.com>
> Date: Fri, 20 Dec 2024 01:09:10 -0800
> Cc: Peter Oliver <p.d.oliver@mavit.org.uk>,
>  Björn Bidar <bjorn.bidar@thaodan.de>,
>  Stefan Kangas <stefankangas@gmail.com>
> 
> Continuing from the tree-sitter maternity thread, I cooked up some script to go over each builtin tree-sitter mode, clone the grammars it uses, and check whether the font-lock queries are compatible with the latest version of the grammar. If everything works fine, the script adds some comment in the source file listing the version that was checked.
> 
> For example, I ran the script for c-ts-mode and c++-ts-mode, and this is the comment inserted to c-ts-mode.el:

I'm not sure what exactly you are suggesting, but commands and Lisp
programs that are for the Emacs maintainers (as opposed to users)
should be in the admin/ directory, not in the lisp/ directory.

If you think this should be used by Emacs users, please describe how
would a user use such a command, because I don't think I understand
that.

Re: Validating tree-sitter major modes and grammar

[Yuan Fu]

> On Dec 20, 2024, at 4:24 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Fri, 20 Dec 2024 01:09:10 -0800
>> Cc: Peter Oliver <p.d.oliver@mavit.org.uk>,
>> Björn Bidar <bjorn.bidar@thaodan.de>,
>> Stefan Kangas <stefankangas@gmail.com>
>> 
>> Continuing from the tree-sitter maternity thread, I cooked up some script to go over each builtin tree-sitter mode, clone the grammars it uses, and check whether the font-lock queries are compatible with the latest version of the grammar. If everything works fine, the script adds some comment in the source file listing the version that was checked.
>> 
>> For example, I ran the script for c-ts-mode and c++-ts-mode, and this is the comment inserted to c-ts-mode.el:
> 
> I'm not sure what exactly you are suggesting, but commands and Lisp
> programs that are for the Emacs maintainers (as opposed to users)
> should be in the admin/ directory, not in the lisp/ directory.
> 
> If you think this should be used by Emacs users, please describe how
> would a user use such a command, because I don't think I understand
> that.

It’s for maintainers, I moved the script to admin/treesit-admin.el.

Yuan

Re: Validating tree-sitter major modes and grammar

[Björn Bidar]

Yuan Fu <casouri@gmail.com> writes:

> Continuing from the tree-sitter maternity thread, I cooked up some
> script to go over each builtin tree-sitter mode, clone the grammars it
> uses, and check whether the font-lock queries are compatible with the
> latest version of the grammar. If everything works fine, the script
> adds some comment in the source file listing the version that was
> checked.

Oh that sounds very good. This could be a good workaround for the sort
of fast and lose nature of tree-sitter.