From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 51757431FBC for ; Sun, 8 Sep 2013 08:53:45 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HabxIQnpw6mA for ; Sun, 8 Sep 2013 08:53:38 -0700 (PDT) Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34]) by olra.theworths.org (Postfix) with ESMTP id AC749431FAE for ; Sun, 8 Sep 2013 08:53:38 -0700 (PDT) Received: by guru.guru-group.fi (Postfix, from userid 501) id 85213100093; Sun, 8 Sep 2013 18:53:32 +0300 (EEST) From: Tomi Ollila To: notmuch@notmuchmail.org Subject: [PATCH v2] test: exit with nonzero value when not all tests completed successfully Date: Sun, 8 Sep 2013 18:53:30 +0300 Message-Id: <1378655610-21497-1-git-send-email-tomi.ollila@iki.fi> X-Mailer: git-send-email 1.8.0 Cc: tomi.ollila@iki.fi X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Sep 2013 15:53:45 -0000 If any of the tests in our test system is not passing the execution of the test suite completes with nonzero exit value. It is better to rely on the exit value of the test system instead of some arbitrary strings in test output (or use both). --- This is v2 of id:1378479935-26312-1-git-send-email-tomi.ollila@iki.fi thanks to Mark's observation in id:87d2olg9bz.fsf@qmul.ac.uk which widened my thought region :D Tomi test/aggregate-results.sh | 9 +++++++++ test/notmuch-test | 3 +++ 2 files changed, 12 insertions(+) diff --git a/test/aggregate-results.sh b/test/aggregate-results.sh index 732d6ca..b016edb 100755 --- a/test/aggregate-results.sh +++ b/test/aggregate-results.sh @@ -1,5 +1,7 @@ #!/usr/bin/env bash +set -eu + fixed=0 success=0 failed=0 @@ -79,3 +81,10 @@ if [ "$skipped" != "0" ]; then tests=$(pluralize "test" $skipped) echo "$skipped $tests skipped." fi + +if [ $success -gt 0 -a $fixed -eq 0 -a $failed -eq 0 -a $skipped -eq 0 ] +then + exit 0 +else + exit 1 +fi diff --git a/test/notmuch-test b/test/notmuch-test index 6db7979..aa28bb0 100755 --- a/test/notmuch-test +++ b/test/notmuch-test @@ -97,6 +97,9 @@ trap - HUP INT TERM # Report results ./aggregate-results.sh test-results/* +ev=$? # Clean up rm -rf test-results corpus.mail + +exit $ev -- 1.8.0