From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 41C426DE0ED0 for ; Sat, 25 May 2019 12:42:08 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: 0.428 X-Spam-Level: X-Spam-Status: No, score=0.428 tagged_above=-999 required=5 tests=[AWL=-0.224, SPF_NEUTRAL=0.652] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MZaWSEXzwIt9 for ; Sat, 25 May 2019 12:42:06 -0700 (PDT) Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34]) by arlo.cworth.org (Postfix) with ESMTP id A1A3E6DE0ECA for ; Sat, 25 May 2019 12:42:05 -0700 (PDT) Received: from guru.guru-group.fi (localhost [IPv6:::1]) by guru.guru-group.fi (Postfix) with ESMTP id 957DC1000D0; Sat, 25 May 2019 22:41:58 +0300 (EEST) From: Tomi Ollila To: Daniel Kahn Gillmor , Notmuch Mail Subject: Re: [PATCH] test: report summary even when aborting In-Reply-To: <20190525173050.8707-1-dkg@fifthhorseman.net> References: <20190525173050.8707-1-dkg@fifthhorseman.net> User-Agent: Notmuch/0.28.3+84~g41389bb (https://notmuchmail.org) Emacs/25.2.1 (x86_64-unknown-linux-gnu) X-Face: HhBM'cA~ MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 May 2019 19:42:08 -0000 On Sat, May 25 2019, Daniel Kahn Gillmor wrote: > In certain cases of test suite failure, the summary report was not > being printed. In particular, any failure on the parallel test suite, > and any aborted test in the serialized test suite would end up hiding > the summary. > > It's better to always show the summary where we can (while preserving > the return code). This is an improvement, but it looks like it is possible aggregate-results.sh prints everything good (like all 3 tests passed) since it may be that there are only good logs in test-results/. That might, in some cases, be confusing (exit value of notmuch-test is not always visible). One option could be giving $RES to aggregate-results.sh and if it is not zero then that could print different text -- just that it requires a bit more coding... Tomi > > Signed-off-by: Daniel Kahn Gillmor > --- > test/notmuch-test | 11 ++++++++--- > 1 file changed, 8 insertions(+), 3 deletions(-) > > diff --git a/test/notmuch-test b/test/notmuch-test > index 50ed8721..d835e152 100755 > --- a/test/notmuch-test > +++ b/test/notmuch-test > @@ -45,6 +45,8 @@ else > fi > > trap 'e=$?; kill $!; exit $e' HUP INT TERM > + > +RES=0 > # Run the tests > if test -z "$NOTMUCH_TEST_SERIALIZE" && command -v parallel >/dev/null ; then > test -t 1 && export COLORS_WITHOUT_TTY=t || : > @@ -58,7 +60,6 @@ if test -z "$NOTMUCH_TEST_SERIALIZE" && command -v parallel >/dev/null ; then > RES=$? > if [[ $RES != 0 ]]; then > echo "parallel test suite returned error code $RES" > - exit $RES > fi > else > for test in $TESTS; do > @@ -69,7 +70,7 @@ else > RES=$? > testname=$(basename $test .sh) > if [[ $RES != 0 && ! -e "$NOTMUCH_BUILDDIR/test/test-results/$testname" ]]; then > - exit $RES > + echo "Aborting on $testname (returned $RES)" > fi > done > fi > @@ -78,7 +79,11 @@ trap - HUP INT TERM > # Report results > echo > $NOTMUCH_SRCDIR/test/aggregate-results.sh $NOTMUCH_BUILDDIR/test/test-results/* > -ev=$? > +if [ "$RES" = 0 ]; then > + ev=$? > +else > + ev=$RES > +fi > > # Clean up > rm -rf $NOTMUCH_BUILDDIR/test/test-results > -- > 2.20.1 > > _______________________________________________ > notmuch mailing list > notmuch@notmuchmail.org > https://notmuchmail.org/mailman/listinfo/notmuch