Re: [PATCH v2] test: report summary even when aborting

[Tomi Ollila <tomi.ollila@iki.fi>]

On Sun, May 26 2019, Daniel Kahn Gillmor wrote:

> In certain cases of test suite failure, the summary report was not
> being printed.  In particular, any failure on the parallel test suite,
> and any aborted test in the serialized test suite would end up hiding
> the summary.
>
> It's better to always show the summary where we can (while preserving
> the return code).  If we do abort due to this high-level failure,
> though, we should also announce to the user that we're doing so as
> close to the end of the process as possible, to make it easier to find
> the problem.
>
> Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

LGTM.

Tomi


> ---
>  test/notmuch-test | 14 +++++++++++---
>  1 file changed, 11 insertions(+), 3 deletions(-)
>
> diff --git a/test/notmuch-test b/test/notmuch-test
> index 50ed8721..b58fd3b3 100755
> --- a/test/notmuch-test
> +++ b/test/notmuch-test
> @@ -45,6 +45,8 @@ else
>  fi
>  
>  trap 'e=$?; kill $!; exit $e' HUP INT TERM
> +
> +META_FAILURE=
>  # Run the tests
>  if test -z "$NOTMUCH_TEST_SERIALIZE" && command -v parallel >/dev/null ; then
>      test -t 1 && export COLORS_WITHOUT_TTY=t || :
> @@ -57,8 +59,7 @@ if test -z "$NOTMUCH_TEST_SERIALIZE" && command -v parallel >/dev/null ; then
>      fi
>      RES=$?
>      if [[ $RES != 0 ]]; then
> -        echo "parallel test suite returned error code $RES"
> -        exit $RES
> +        META_FAILURE="parallel test suite returned error code $RES"
>      fi
>  else
>      for test in $TESTS; do
> @@ -69,7 +70,8 @@ else
>          RES=$?
>          testname=$(basename $test .sh)
>          if [[ $RES != 0 && ! -e "$NOTMUCH_BUILDDIR/test/test-results/$testname" ]]; then
> -            exit $RES
> +            META_FAILURE="Aborting on $testname (returned $RES)"
> +            break
>          fi
>      done
>  fi
> @@ -79,6 +81,12 @@ trap - HUP INT TERM
>  echo
>  $NOTMUCH_SRCDIR/test/aggregate-results.sh $NOTMUCH_BUILDDIR/test/test-results/*
>  ev=$?
> +if [ -n "$META_FAILURE" ]; then
> +    printf 'ERROR: %s\n' "$META_FAILURE"
> +    if [ $ev = 0 ]; then
> +        ev=$RES
> +    fi
> +fi
>  
>  # Clean up
>  rm -rf $NOTMUCH_BUILDDIR/test/test-results
> -- 
> 2.20.1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch