[PATCH 1/2] tests: make timeout configurable with NOTMUCH_TEST_TIMEOUT (default: 2m)

[PATCH 1/2] tests: make timeout configurable with NOTMUCH_TEST_TIMEOUT (default: 2m)

[Daniel Kahn Gillmor]

The current 2 minute timeout is reasonable, but to exercise the test
suite or induce timeout failures, we might want to make it shorter.
This makes it configurable so you can run (for example):

    make check NOTMUCH_TEST_TIMEOUT=10s

We stick with the default of 2m.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
---
 test/notmuch-test | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/test/notmuch-test b/test/notmuch-test
index bd3e080a..dbca39ca 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -32,8 +32,9 @@ rm -rf $NOTMUCH_BUILDDIR/test/test-results
 
 # Test for timeout utility
 if command -v timeout >/dev/null; then
-    TEST_TIMEOUT_CMD="timeout 2m"
-    echo "INFO: using 2 minute timeout for tests"
+    TEST_TIMEOUT=${NOTMUCH_TEST_TIMEOUT:-2m}
+    TEST_TIMEOUT_CMD="timeout $TEST_TIMEOUT"
+    echo "INFO: using $TEST_TIMEOUT timeout for tests"
 else
     TEST_TIMEOUT_CMD=""
 fi
-- 
2.20.1

[PATCH 2/2] tests: fail and report when a parallel build fails (or times out)

[Daniel Kahn Gillmor]

When a parallel build fails (or when it times out, if timeout is
present), the test suite should not blithely succeed.  Catch these
failures and at least report them.

Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
---
 test/notmuch-test | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/test/notmuch-test b/test/notmuch-test
index dbca39ca..bbc2dc31 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -49,6 +49,11 @@ if test -z "$NOTMUCH_TEST_SERIALIZE" && command -v parallel >/dev/null ; then
         echo "INFO: running tests with moreutils parallel"
         $TEST_TIMEOUT_CMD parallel -- $TESTS
     fi
+    RES=$?
+    if [[ $RES != 0 ]]; then
+        echo "parallel test suite returned error code $RES"
+        exit $RES
+    fi
 else
     for test in $TESTS; do
         $TEST_TIMEOUT_CMD $test "$@" &
-- 
2.20.1

Re: [PATCH 1/2] tests: make timeout configurable with NOTMUCH_TEST_TIMEOUT (default: 2m)

[Tomi Ollila]

On Mon, May 20 2019, Daniel Kahn Gillmor wrote:

> The current 2 minute timeout is reasonable, but to exercise the test
> suite or induce timeout failures, we might want to make it shorter.
> This makes it configurable so you can run (for example):
>
>     make check NOTMUCH_TEST_TIMEOUT=10s
>
> We stick with the default of 2m.
>
> Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>


Series LGTM.

Tomi


> ---
>  test/notmuch-test | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/test/notmuch-test b/test/notmuch-test
> index bd3e080a..dbca39ca 100755
> --- a/test/notmuch-test
> +++ b/test/notmuch-test
> @@ -32,8 +32,9 @@ rm -rf $NOTMUCH_BUILDDIR/test/test-results
>  
>  # Test for timeout utility
>  if command -v timeout >/dev/null; then
> -    TEST_TIMEOUT_CMD="timeout 2m"
> -    echo "INFO: using 2 minute timeout for tests"
> +    TEST_TIMEOUT=${NOTMUCH_TEST_TIMEOUT:-2m}
> +    TEST_TIMEOUT_CMD="timeout $TEST_TIMEOUT"
> +    echo "INFO: using $TEST_TIMEOUT timeout for tests"
>  else
>      TEST_TIMEOUT_CMD=""
>  fi
> -- 
> 2.20.1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/2] tests: make timeout configurable with NOTMUCH_TEST_TIMEOUT (default: 2m)

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

> On Mon, May 20 2019, Daniel Kahn Gillmor wrote:
>
>> The current 2 minute timeout is reasonable, but to exercise the test
>> suite or induce timeout failures, we might want to make it shorter.
>> This makes it configurable so you can run (for example):
>>
>>     make check NOTMUCH_TEST_TIMEOUT=10s
>>
>> We stick with the default of 2m.
>>
>> Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
>
>
> Series LGTM.
>
> Tomi

series pushed.

d