bug#37476: [PATCH] Document ert test selectors in batch mode

bug#37476: [PATCH] Document ert test selectors in batch mode

[Stefan Kangas]

[-- Attachment #1: Type: text/plain, Size: 223 bytes --]

I think it would be good to document how to use test selectors on the
ert manual page that talks about running tests in batch mode.  I came
up with the following suggested patch.

Any comments?

Best regards,
Stefan Kangas

[-- Attachment #2: 0001-Document-ert-test-selectors-in-batch-mode.patch --]
[-- Type: text/x-patch, Size: 1521 bytes --]

From a6101b99da1b046cd7c1cf3bb6e019dc0e842c20 Mon Sep 17 00:00:00 2001
From: Stefan Kangas <stefankangas@gmail.com>
Date: Sat, 21 Sep 2019 14:58:08 +0200
Subject: [PATCH] Document ert test selectors in batch mode

* doc/misc/ert.texi (Running Tests in Batch Mode): Document test
selectors.
---
 doc/misc/ert.texi | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/doc/misc/ert.texi b/doc/misc/ert.texi
index d2d86555e3..e853faf323 100644
--- a/doc/misc/ert.texi
+++ b/doc/misc/ert.texi
@@ -337,10 +337,20 @@ Running Tests in Batch Mode
 
 In quiet mode ERT prints only unexpected results and summary.
 
-If ERT is not part of your Emacs distribution, you may need to use
+You can specify selectors to only run a subset of your tests
+(@pxref{Test Selectors}).  For example, the following would run all
+tests where the name of the test matches the string ``to-match''.
+
+@example
+emacs -batch -l ert -l my-tests.el \
+      -eval '(ert-run-tests-batch-and-exit "to-match")'
+@end example
+
+If you are using a version of Emacs older than 24.1, ERT is not part
+of your Emacs distribution.  In this case, you may need to use
 @code{-L /path/to/ert/} so that Emacs can find it.  You may need
-additional @code{-L} flags to ensure that @code{my-tests.el} and all the
-files that it requires are on your @code{load-path}.
+additional @code{-L} flags to ensure that @code{my-tests.el} and all
+the files that it requires are on your @code{load-path}.
 
 
 @node Test Selectors
-- 
2.20.1

bug#37476: [PATCH] Document ert test selectors in batch mode

[Lars Ingebrigtsen]

Stefan Kangas <stefan@marxist.se> writes:

> I think it would be good to document how to use test selectors on the
> ert manual page that talks about running tests in batch mode.  I came
> up with the following suggested patch.
>
> Any comments?

[...]

> -If ERT is not part of your Emacs distribution, you may need to use
> +You can specify selectors to only run a subset of your tests
> +(@pxref{Test Selectors}).  For example, the following would run all
> +tests where the name of the test matches the string ``to-match''.
> +
> +@example
> +emacs -batch -l ert -l my-tests.el \
> +      -eval '(ert-run-tests-batch-and-exit "to-match")'
> +@end example

I guess that would be nice to have documented, but it's not a use case
I've ever had.  Tests are usually grouped in separate files, so you'd
usually have (as in Emacs) "make foo-tests" or whatever in your
Makefiles.

> +If you are using a version of Emacs older than 24.1, ERT is not part
> +of your Emacs distribution.  In this case, you may need to use

The Emacs manuals usually document what's currently in Emacs, so it's a
bit odd to be talking about Emacs 24.1 here, isn't it?  So perhaps that
paragraph should just be removed?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37476: [PATCH] Document ert test selectors in batch mode

[Stefan Kangas]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> > -If ERT is not part of your Emacs distribution, you may need to use
> > +You can specify selectors to only run a subset of your tests
> > +(@pxref{Test Selectors}).  For example, the following would run all
> > +tests where the name of the test matches the string ``to-match''.
> > +
> > +@example
> > +emacs -batch -l ert -l my-tests.el \
> > +      -eval '(ert-run-tests-batch-and-exit "to-match")'
> > +@end example
>
> I guess that would be nice to have documented, but it's not a use case
> I've ever had.  Tests are usually grouped in separate files, so you'd
> usually have (as in Emacs) "make foo-tests" or whatever in your
> Makefiles.

I find that it's sometimes easier to run just one unit test from a
file, since there is less output and therefore less scrolling.  I've
had to search the web to find out how to do this before.

I'm not sure I understand the second sentence.  Package developers
might have a Makefile similar to the one Emacs core has, which allows
you to easily specify test selectors, but they might also be missing
one.

> > +If you are using a version of Emacs older than 24.1, ERT is not part
> > +of your Emacs distribution.  In this case, you may need to use
>
> The Emacs manuals usually document what's currently in Emacs, so it's a
> bit odd to be talking about Emacs 24.1 here, isn't it?  So perhaps that
> paragraph should just be removed?

I completely agree, and was simply trying to be conservative.  If
that's what we want, I'll be happy to remove it.

Thanks for reviewing the patch.

Best regards,
Stefan Kangas

bug#37476: [PATCH] Document ert test selectors in batch mode

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Sun, 22 Sep 2019 15:38:54 +0200
> Cc: 37476@debbugs.gnu.org
> 
> > > +If you are using a version of Emacs older than 24.1, ERT is not part
> > > +of your Emacs distribution.  In this case, you may need to use
> >
> > The Emacs manuals usually document what's currently in Emacs, so it's a
> > bit odd to be talking about Emacs 24.1 here, isn't it?  So perhaps that
> > paragraph should just be removed?
> 
> I completely agree, and was simply trying to be conservative.  If
> that's what we want, I'll be happy to remove it.

It should be removed, yes.

bug#37476: [PATCH] Document ert test selectors in batch mode

[Lars Ingebrigtsen]

Stefan Kangas <stefan@marxist.se> writes:

> I find that it's sometimes easier to run just one unit test from a
> file, since there is less output and therefore less scrolling.  I've
> had to search the web to find out how to do this before.

Makes sense.  I usually just run unit test inside the running Emacs when
I want to test one specific thing.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37476: [PATCH] Document ert test selectors in batch mode

[Stefan Kangas]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> > I find that it's sometimes easier to run just one unit test from a
> > file, since there is less output and therefore less scrolling.  I've
> > had to search the web to find out how to do this before.
>
> Makes sense.  I usually just run unit test inside the running Emacs when
> I want to test one specific thing.

No more comments in a week, so I've now pushed the attached as commit
cc7f2fb0cc.

(I removed the part about Emacs 24.1.)

Best regards,
Stefan Kangas

[-- Attachment #2: 0001-Document-ert-test-selectors-in-batch-mode-Bug-37476.patch --]
[-- Type: text/x-patch, Size: 1264 bytes --]

From cc7f2fb0cc1c077d0648e48d0d9efb713c375abe Mon Sep 17 00:00:00 2001
From: Stefan Kangas <stefankangas@gmail.com>
Date: Sat, 21 Sep 2019 14:58:08 +0200
Subject: [PATCH] Document ert test selectors in batch mode (Bug#37476)

* doc/misc/ert.texi (Running Tests in Batch Mode): Document test
selectors.
---
 doc/misc/ert.texi | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/doc/misc/ert.texi b/doc/misc/ert.texi
index 2dbba095d6..889216cbef 100644
--- a/doc/misc/ert.texi
+++ b/doc/misc/ert.texi
@@ -337,10 +337,14 @@ Running Tests in Batch Mode
 
 In quiet mode ERT prints only unexpected results and summary.
 
-If ERT is not part of your Emacs distribution, you may need to use
-@code{-L /path/to/ert/} so that Emacs can find it.  You may need
-additional @code{-L} flags to ensure that @code{my-tests.el} and all the
-files that it requires are on your @code{load-path}.
+You can specify selectors to only run a subset of your tests
+(@pxref{Test Selectors}).  For example, the following would run all
+tests where the name of the test matches the string ``to-match''.
+
+@example
+emacs -batch -l ert -l my-tests.el \
+      -eval '(ert-run-tests-batch-and-exit "to-match")'
+@end example
 
 
 @node Test Selectors
-- 
2.20.1

bug#37476: [PATCH] Document ert test selectors in batch mode

[Michael Albinus]

Stefan Kangas <stefan@marxist.se> writes:

Hi Stefan,

sorry to enter this thread such late; somehow I've missed it.

> diff --git a/doc/misc/ert.texi b/doc/misc/ert.texi
> index 2dbba095d6..889216cbef 100644
> --- a/doc/misc/ert.texi
> +++ b/doc/misc/ert.texi
> @@ -337,10 +337,14 @@ Running Tests in Batch Mode
>
>  In quiet mode ERT prints only unexpected results and summary.
>
> -If ERT is not part of your Emacs distribution, you may need to use
> -@code{-L /path/to/ert/} so that Emacs can find it.  You may need
> -additional @code{-L} flags to ensure that @code{my-tests.el} and all the
> -files that it requires are on your @code{load-path}.
> +You can specify selectors to only run a subset of your tests
> +(@pxref{Test Selectors}).  For example, the following would run all
> +tests where the name of the test matches the string ``to-match''.
> +
> +@example
> +emacs -batch -l ert -l my-tests.el \
> +      -eval '(ert-run-tests-batch-and-exit "to-match")'
> +@end example

SELECTOR, if it is a string, is a regular expression. We shall say
so, "... matches the regular expression ...".

> Best regards,
> Stefan Kangas

Best regards, Michael.

bug#37476: [PATCH] Document ert test selectors in batch mode

[Stefan Kangas]

Hi Michael,

Michael Albinus <michael.albinus@gmx.de> writes:
> SELECTOR, if it is a string, is a regular expression. We shall say
> so, "... matches the regular expression ...".

Thanks for pointing that out.  I've now changed that on master.

Best regards,
Stefan Kangas