From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id DBD466DE10A1 for ; Sun, 26 May 2019 08:06:23 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.16 X-Spam-Level: X-Spam-Status: No, score=-0.16 tagged_above=-999 required=5 tests=[AWL=0.041, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pEF_0iHttNjs for ; Sun, 26 May 2019 08:06:23 -0700 (PDT) Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118]) by arlo.cworth.org (Postfix) with ESMTPS id 22C2F6DE0EEA for ; Sun, 26 May 2019 08:06:23 -0700 (PDT) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019; t=1558883180; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=HXxYFaj3E8rmU/ieD4NyBandRhi90ruRVt1wB7xPb/c=; b=1/FRUCAa/NLYI0M9Nn69e7zGpcOwEtBQKY8bYgZAIgcZ0+u7xyMUwif0 UX8dqD1+7GU9qf4luhda/kmAxN3AAQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fifthhorseman.net; i=@fifthhorseman.net; q=dns/txt; s=2019rsa; t=1558883180; h=from : to : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : from; bh=HXxYFaj3E8rmU/ieD4NyBandRhi90ruRVt1wB7xPb/c=; b=nU01AtVAb/Ky2dIlW9nN8YAqvD4a5PKnE6O3ULW3OrHnvkmLDhq0QO22 Ysl+NOyZPFz89pwWRlg1PcZZDaa3D43pfEvo0zHfhQzCuN1mI3N6tNImNr cbqX7nMMJVGwGwRYQ56CuvzU0tqoXaCdDq0nXWjzeO6jVfpJx70pNm6awo MtVP57FSlNJBeEQOhDPj6LfFCdBqc7ztHytO0MOL8IcNDYKwshDSzhRn2B ImMP/1ZzUwhsNpdvSe3xofaE3ru+rZVIkRkQnNRH0pFQFlYpz4PNKJofsf bkVruoivtvOUvgyJw3ZtHNalIFBIX1Dz+FjpCRQ1vd1R/d3ZR/Zmqg== Received: from fifthhorseman.net (cpe-74-71-53-242.nyc.res.rr.com [74.71.53.242]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by che.mayfirst.org (Postfix) with ESMTPSA id 66700F99D for ; Sun, 26 May 2019 11:06:19 -0400 (EDT) Received: by fifthhorseman.net (Postfix, from userid 1000) id ACBDB201E0; Sun, 26 May 2019 11:03:13 -0400 (EDT) From: Daniel Kahn Gillmor To: Notmuch Mail Subject: [PATCH v2] test: report summary even when aborting Date: Sun, 26 May 2019 11:03:13 -0400 Message-Id: <20190526150313.2786-1-dkg@fifthhorseman.net> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 May 2019 15:06:24 -0000 In certain cases of test suite failure, the summary report was not being printed. In particular, any failure on the parallel test suite, and any aborted test in the serialized test suite would end up hiding the summary. It's better to always show the summary where we can (while preserving the return code). If we do abort due to this high-level failure, though, we should also announce to the user that we're doing so as close to the end of the process as possible, to make it easier to find the problem. Signed-off-by: Daniel Kahn Gillmor --- test/notmuch-test | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/test/notmuch-test b/test/notmuch-test index 50ed8721..b58fd3b3 100755 --- a/test/notmuch-test +++ b/test/notmuch-test @@ -45,6 +45,8 @@ else fi trap 'e=$?; kill $!; exit $e' HUP INT TERM + +META_FAILURE= # Run the tests if test -z "$NOTMUCH_TEST_SERIALIZE" && command -v parallel >/dev/null ; then test -t 1 && export COLORS_WITHOUT_TTY=t || : @@ -57,8 +59,7 @@ if test -z "$NOTMUCH_TEST_SERIALIZE" && command -v parallel >/dev/null ; then fi RES=$? if [[ $RES != 0 ]]; then - echo "parallel test suite returned error code $RES" - exit $RES + META_FAILURE="parallel test suite returned error code $RES" fi else for test in $TESTS; do @@ -69,7 +70,8 @@ else RES=$? testname=$(basename $test .sh) if [[ $RES != 0 && ! -e "$NOTMUCH_BUILDDIR/test/test-results/$testname" ]]; then - exit $RES + META_FAILURE="Aborting on $testname (returned $RES)" + break fi done fi @@ -79,6 +81,12 @@ trap - HUP INT TERM echo $NOTMUCH_SRCDIR/test/aggregate-results.sh $NOTMUCH_BUILDDIR/test/test-results/* ev=$? +if [ -n "$META_FAILURE" ]; then + printf 'ERROR: %s\n' "$META_FAILURE" + if [ $ev = 0 ]; then + ev=$RES + fi +fi # Clean up rm -rf $NOTMUCH_BUILDDIR/test/test-results -- 2.20.1