[PATCH] test/README: have matching test script file names

[PATCH] test/README: have matching test script file names

[Tomi Ollila]

A while ago test script names were changed to format
Tddd-basename.sh. Update README to reflect that.

While at it, included some small requirements updates.
---

'A while ago' here means like 25 months (change was done Jan 2014).
I started doing this change in March 2014 but dropped after got
too exhaustive thoughts of how to continue. Now I did minor change
to make TEST_EMACS=my-emacs TEST_EMACSCLIENT=my-emacsclient make test
lines fit on 80 columns...

 test/README | 34 ++++++++++++++++++++++++++--------
 1 file changed, 26 insertions(+), 8 deletions(-)

diff --git a/test/README b/test/README
index e54e36b..bd9ab54 100644
--- a/test/README
+++ b/test/README
@@ -8,10 +8,17 @@ enhance.
 
 Prerequisites
 -------------
+The test system itself requires:
+
+  - bash(1) version 4.0 or newer
+
+Without bash 4.0+ the tests just refuse to run.
+
 Some tests require external dependencies to run. Without them, they
 will be skipped, or (rarely) marked failed. Please install these, so
 that you know if you break anything.
 
+  - GNU tar(1)
   - dtach(1)
   - emacs(1)
   - emacsclient(1)
@@ -19,14 +26,21 @@ that you know if you break anything.
   - gpg(1)
   - python(1)
 
+If your system lacks these tools or have older, non-upgreable versions
+of these, please (possibly compile and) install these to some other
+path, for example /usr/local/bin or /opt/gnu/bin. Then prepend the
+chosen directory to your PATH before running the tests.
+
+e.g. env PATH=/opt/gnu/bin:$PATH make test
+
 Running Tests
 -------------
 The easiest way to run tests is to say "make test", (or simply run the
 notmuch-test script). Either command will run all available tests.
 
 Alternately, you can run a specific subset of tests by simply invoking
-one of the executable scripts in this directory, (such as ./search,
-./reply, etc). Note that you will probably want "make test-binaries"
+one of the executable scripts in this directory, (such as ./T*-search.sh,
+./T*-reply.sh, etc). Note that you will probably want "make test-binaries"
 before running individual tests.
 
 The following command-line options are available when running tests:
@@ -80,9 +94,9 @@ can be specified as follows:
 You can choose an emacs binary (and corresponding emacsclient) to run
 the tests in one of the following ways.
 
-	TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient make test
-	TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient ./emacs
-	make test TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient
+	TEST_EMACS=my-emacs TEST_EMACSCLIENT=my-emacsclient make test
+	TEST_EMACS=my-emacs TEST_EMACSCLIENT=my-emacsclient ./T*-emacs.sh
+	make test TEST_EMACS=my-emacs TEST_EMACSCLIENT=my-emacsclient
 
 Some tests may require a c compiler. You can choose the name and flags similarly
 to with emacs, e.g.
@@ -126,9 +140,13 @@ skipped by the user, as failures.
 
 Writing Tests
 -------------
-The test script is written as a shell script.  It should start with
-the standard "#!/usr/bin/env bash" with copyright notices, and an
-assignment to variable 'test_description', like this:
+The test script is written as a shell script. It is to be named as
+Tddd-testname.sh where 'ddd' is three digits and 'testname' the "bare"
+name of your test. Tests will be run in order the 'ddd' part determines.
+
+The test script should start with the standard "#!/usr/bin/env bash"
+with copyright notices, and an assignment to variable 'test_description',
+like this:
 
 	#!/usr/bin/env bash
 	#
-- 
1.9.3

Re: [PATCH] test/README: have matching test script file names

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1469 bytes --]

On Fri, Feb 12, 2016 at 09:17:00PM +0200, Tomi Ollila wrote:
> +The test system itself requires:
> +
> +  - bash(1) version 4.0 or newer
> +
> +Without bash 4.0+ the tests just refuse to run.

This was surprising to me (obviously I haven't looked at the test
library closely ;), because Git tries to maintain compatiblity with
POSIX [1] and Git is the source of notmuch's test framework [2].  But
it looks like notmuch's fork of the test framework was Bash-specific
from 0d67c52f (notmuch-test: Use named-parameters for generate_message
function, 2010-02-03).  That commit happened before the Git framework,
but we still have the code it added:

    # This is our (bash-specific) magic for doing named parameters                                                                                                            
    local -A template="($@)"

Carl explicitly motivated the change in his commit message.  +1 to
documenting the requirement explicitly in the README, although users
should have been getting nice error messages since 20018a0c (test:
Emit a friendly error message if run with bash < 4.0, 2010-10-28).

Cheers,
Trevor

[1]: https://git.kernel.org/cgit/git/git.git/tree/Documentation/CodingGuidelines?h=v2.7.1
[2]: 0083854 (Copy test framework from Git, 2010-06-10)

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH] test/README: have matching test script file names

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

> A while ago test script names were changed to format
> Tddd-basename.sh. Update README to reflect that.
>
> While at it, included some small requirements updates.

pushed these documentation changes.

d