[PATCH] Testing strategy for Tree-sitter modes

[PATCH] Testing strategy for Tree-sitter modes

[Daniel Martín]

[-- Attachment #1: Type: text/plain, Size: 917 bytes --]


I think the development of Tree-sitter modes would benefit from having a
regression test suite that ensures that bug fixes for specific cases
don't regress in the future.

As a start, I've attached to this mail some ERT-based tests for the
indentation feature of c-ts-mode.  I've used the convenient feature of
"erts" files to create the tests, which I think make them readable and
maintainable.

The tests are skipped if the Emacs build does not have Tree-sitter
enabled, or if the corresponding grammar is not installed.  Perhaps it
would be a good idea to install these Tree-sitter dependencies in the
container images used for EMBA (https://emba.gnu.org/emacs), so that
we're able to run Tree-sitter tests in continuous integration, and even
play with different versions of Tree-sitter or the Tree-sitter grammars
to detect incompatibilities.  I'll leave this decision to the people in
charge of EMBA.

Thanks.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-c-ts-mode-tests.patch --]
[-- Type: text/x-patch, Size: 2810 bytes --]

From c557b9aa3d733d9b2d4ab2b13b5b0c644d7675f5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Daniel=20Mart=C3=ADn?= <mardani29@yahoo.es>
Date: Sun, 8 Jan 2023 14:04:24 +0100
Subject: [PATCH] Add c-ts-mode tests

* test/lisp/progmodes/c-ts-mode-resources/indent.erts: New .erts file
to test indentation of typical C constructs and prevent regression of
bug fixes.
* test/lisp/progmodes/c-ts-mode-tests.el: New file with c-ts-mode
tests.
---
 .../progmodes/c-ts-mode-resources/indent.erts | 44 +++++++++++++++++++
 test/lisp/progmodes/c-ts-mode-tests.el        | 31 +++++++++++++
 2 files changed, 75 insertions(+)
 create mode 100644 test/lisp/progmodes/c-ts-mode-resources/indent.erts
 create mode 100644 test/lisp/progmodes/c-ts-mode-tests.el

diff --git a/test/lisp/progmodes/c-ts-mode-resources/indent.erts b/test/lisp/progmodes/c-ts-mode-resources/indent.erts
new file mode 100644
index 00000000000..5defcbd3c83
--- /dev/null
+++ b/test/lisp/progmodes/c-ts-mode-resources/indent.erts
@@ -0,0 +1,44 @@
+Code:
+  (lambda ()
+    (c-ts-mode)
+    (indent-region (point-min) (point-max)))
+
+Name: Basic
+
+=-=
+int
+main (void)
+{
+  return 0;
+}
+=-=-=
+
+Name: Hanging Braces (GNU Style)
+
+=-=
+int
+main (void)
+{
+  if (true)
+    {
+    }
+}
+=-=-=
+
+Name: Multiline Parameter List (bug#60398)
+
+=-=
+int f2(int x,
+       int y) {
+  return x + y;
+};
+=-=-=
+
+Name: Multiline Block Comments (bug#60270)
+
+=-=
+/**
+ * @some_func:
+ * @arg1:
+ */
+=-=-=
diff --git a/test/lisp/progmodes/c-ts-mode-tests.el b/test/lisp/progmodes/c-ts-mode-tests.el
new file mode 100644
index 00000000000..8606faf9913
--- /dev/null
+++ b/test/lisp/progmodes/c-ts-mode-tests.el
@@ -0,0 +1,31 @@
+;;; c-ts-mode-tests.el --- Tests for Tree-sitter-based C mode         -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2023 Free Software Foundation, Inc.
+
+;; This file is part of GNU Emacs.
+
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Code:
+
+(require 'ert)
+(require 'ert-x)
+(require 'treesit)
+
+(ert-deftest c-ts-mode-test-indentation ()
+  (skip-unless (treesit-ready-p 'c))
+  (ert-test-erts-file (ert-resource-file "indent.erts")))
+
+(provide 'c-ts-mode-tests)
+;;; c-ts-mode-tests.el ends here
-- 
2.34.1

Re: [PATCH] Testing strategy for Tree-sitter modes

[Eli Zaretskii]

> From: Daniel Martín <mardani29@yahoo.es>
> Cc: casouri@gmail.com
> Date: Sun, 08 Jan 2023 15:23:57 +0100
> 
> I think the development of Tree-sitter modes would benefit from having a
> regression test suite that ensures that bug fixes for specific cases
> don't regress in the future.

Thanks, adding tests is always welcome.

Re: [PATCH] Testing strategy for Tree-sitter modes

[Michael Albinus]

Daniel Martín <mardani29@yahoo.es> writes:

Hi Daniel,

> The tests are skipped if the Emacs build does not have Tree-sitter
> enabled, or if the corresponding grammar is not installed.  Perhaps it
> would be a good idea to install these Tree-sitter dependencies in the
> container images used for EMBA (https://emba.gnu.org/emacs), so that
> we're able to run Tree-sitter tests in continuous integration, and even
> play with different versions of Tree-sitter or the Tree-sitter grammars
> to detect incompatibilities.  I'll leave this decision to the people in
> charge of EMBA.

Good idea.

See the source tree, file test/infra/Dockerfile.emba. If you could
provide an extension for this, adding the Tree-sitter dependencies, we
could arrange to bring it to EMBA.

(Personally, I don't know what to do wrt Tree-sitter)

> Thanks.

Best regards, Michael.

Re: [PATCH] Testing strategy for Tree-sitter modes

[Jostein Kjønigsen]

On 08.01.2023 15:23, Daniel Martín wrote:
> I think the development of Tree-sitter modes would benefit from having a
> regression test suite that ensures that bug fixes for specific cases
> don't regress in the future.
>
> As a start, I've attached to this mail some ERT-based tests for the
> indentation feature of c-ts-mode.  I've used the convenient feature of
> "erts" files to create the tests, which I think make them readable and
> maintainable.
>
> The tests are skipped if the Emacs build does not have Tree-sitter
> enabled, or if the corresponding grammar is not installed.  Perhaps it
> would be a good idea to install these Tree-sitter dependencies in the
> container images used for EMBA (https://emba.gnu.org/emacs), so that
> we're able to run Tree-sitter tests in continuous integration, and even
> play with different versions of Tree-sitter or the Tree-sitter grammars
> to detect incompatibilities.  I'll leave this decision to the people in
> charge of EMBA.
>
> Thanks.
>
I fully support the addition of the possibility to have a test-suite run 
on CI.

When we ported csharp-mode and typescript-mode from ELPA to new ts-based 
major modes in Emacs-core, I always felt like this was the one thing I 
really wished we'd have here as well.

If this can be added easily, it would make it much easier (and safer!) 
to make changes to tree-sitter grammars in major-modes with complex 
grammer, and where it's not always obvious what rule covers what 
use-case(s).

--
Jostein