From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 7C30B6DE12DE for ; Sun, 26 May 2019 14:35:09 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: 0.427 X-Spam-Level: X-Spam-Status: No, score=0.427 tagged_above=-999 required=5 tests=[AWL=-0.225, SPF_NEUTRAL=0.652] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dAVCOso9Pp3Z for ; Sun, 26 May 2019 14:35:08 -0700 (PDT) Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34]) by arlo.cworth.org (Postfix) with ESMTP id EA83B6DE0B36 for ; Sun, 26 May 2019 14:35:06 -0700 (PDT) Received: from guru.guru-group.fi (localhost [IPv6:::1]) by guru.guru-group.fi (Postfix) with ESMTP id 8FDE6100069; Mon, 27 May 2019 00:35:01 +0300 (EEST) From: Tomi Ollila To: Daniel Kahn Gillmor , Notmuch Mail Subject: Re: [PATCH v2] test: report summary even when aborting In-Reply-To: <20190526150313.2786-1-dkg@fifthhorseman.net> References: <20190526150313.2786-1-dkg@fifthhorseman.net> User-Agent: Notmuch/0.28.3+84~g41389bb (https://notmuchmail.org) Emacs/25.2.1 (x86_64-unknown-linux-gnu) X-Face: HhBM'cA~ MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 May 2019 21:35:09 -0000 On Sun, May 26 2019, Daniel Kahn Gillmor wrote: > In certain cases of test suite failure, the summary report was not > being printed. In particular, any failure on the parallel test suite, > and any aborted test in the serialized test suite would end up hiding > the summary. > > It's better to always show the summary where we can (while preserving > the return code). If we do abort due to this high-level failure, > though, we should also announce to the user that we're doing so as > close to the end of the process as possible, to make it easier to find > the problem. > > Signed-off-by: Daniel Kahn Gillmor LGTM. Tomi > --- > test/notmuch-test | 14 +++++++++++--- > 1 file changed, 11 insertions(+), 3 deletions(-) > > diff --git a/test/notmuch-test b/test/notmuch-test > index 50ed8721..b58fd3b3 100755 > --- a/test/notmuch-test > +++ b/test/notmuch-test > @@ -45,6 +45,8 @@ else > fi > > trap 'e=$?; kill $!; exit $e' HUP INT TERM > + > +META_FAILURE= > # Run the tests > if test -z "$NOTMUCH_TEST_SERIALIZE" && command -v parallel >/dev/null ; then > test -t 1 && export COLORS_WITHOUT_TTY=t || : > @@ -57,8 +59,7 @@ if test -z "$NOTMUCH_TEST_SERIALIZE" && command -v parallel >/dev/null ; then > fi > RES=$? > if [[ $RES != 0 ]]; then > - echo "parallel test suite returned error code $RES" > - exit $RES > + META_FAILURE="parallel test suite returned error code $RES" > fi > else > for test in $TESTS; do > @@ -69,7 +70,8 @@ else > RES=$? > testname=$(basename $test .sh) > if [[ $RES != 0 && ! -e "$NOTMUCH_BUILDDIR/test/test-results/$testname" ]]; then > - exit $RES > + META_FAILURE="Aborting on $testname (returned $RES)" > + break > fi > done > fi > @@ -79,6 +81,12 @@ trap - HUP INT TERM > echo > $NOTMUCH_SRCDIR/test/aggregate-results.sh $NOTMUCH_BUILDDIR/test/test-results/* > ev=$? > +if [ -n "$META_FAILURE" ]; then > + printf 'ERROR: %s\n' "$META_FAILURE" > + if [ $ev = 0 ]; then > + ev=$RES > + fi > +fi > > # Clean up > rm -rf $NOTMUCH_BUILDDIR/test/test-results > -- > 2.20.1 > > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > https://notmuchmail.org/mailman/listinfo/notmuch