unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
From: Dmitry Kurochkin <dmitry.kurochkin@gmail.com>
To: notmuch@notmuchmail.org
Subject: [PATCH v3 1/7] test: add support for external executable dependencies
Date: Sun, 27 Nov 2011 22:36:13 +0400	[thread overview]
Message-ID: <1322418979-19180-2-git-send-email-dmitry.kurochkin@gmail.com> (raw)
In-Reply-To: <1322418979-19180-1-git-send-email-dmitry.kurochkin@gmail.com>

There is existing support for general prerequisites in the test suite.
But it is not very convenient to use: every test case has to keep
track for it's dependencies and they have to be explicitly listed.

The patch aims to add better support for a particular type of external
dependencies: external executables.  The main idea is to replace
missing external binaries with shell functions that have the same
name.  These functions always fail and keep track of missing
dependencies for a subtest.  The result reporting functions later can
check that an external binaries are missing and correctly report SKIP
result instead of FAIL.  The primary benefit is that the test cases do
not need to declare their dependencies or be changed in any way.
---
 test/test-lib.sh |   49 +++++++++++++++++++++++++++++++++++++++++--------
 1 files changed, 41 insertions(+), 8 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 076f929..0996a74 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -548,6 +548,19 @@ test_have_prereq () {
 	esac
 }
 
+# declare prerequisite for the given external binary
+test_declare_external_prereq () {
+	binary="$1"
+	test "$#" = 2 && name=$2 || name="$binary(1)"
+
+	hash $binary 2>/dev/null || eval "
+$binary () {
+	echo -n \"\$test_subtest_missing_external_prereqs_\" | grep -e \" $name \" ||
+	test_subtest_missing_external_prereqs_=\"$test_subtest_missing_external_prereqs_ $name\"
+	false
+}"
+}
+
 # You are not expected to call test_ok_ and test_failure_ directly, use
 # the text_expect_* functions instead.
 
@@ -624,18 +637,31 @@ test_skip () {
 	fi
 	case "$to_skip" in
 	t)
-		test_reset_state_
-		say_color skip >&3 "skipping test: $@"
-		say_color skip "%-6s" "SKIP"
-		echo " $1"
-		: true
+		test_report_skip_ "$@"
 		;;
 	*)
-		false
+		test_check_missing_external_prereqs_ "$@"
 		;;
 	esac
 }
 
+test_check_missing_external_prereqs_ () {
+	if test -n "$test_subtest_missing_external_prereqs_"; then
+		say_color skip >&3 "missing prerequisites:"
+		echo "$test_subtest_missing_external_prereqs_" >&3
+		test_report_skip_ "$@"
+	else
+		false
+	fi
+}
+
+test_report_skip_ () {
+	test_reset_state_
+	say_color skip >&3 "skipping test: $@"
+	say_color skip "%-6s" "SKIP"
+	echo " $1"
+}
+
 test_subtest_known_broken () {
 	test_subtest_known_broken_=t
 }
@@ -648,7 +674,10 @@ test_expect_success () {
 	if ! test_skip "$@"
 	then
 		test_run_ "$2"
-		if [ "$?" = 0 -a "$eval_ret" = 0 ]
+		run_ret="$?"
+		# test_run_ may update missing external prerequisites
+		test_check_missing_external_prereqs_ "$@" ||
+		if [ "$run_ret" = 0 -a "$eval_ret" = 0 ]
 		then
 			test_ok_ "$1"
 		else
@@ -665,7 +694,10 @@ test_expect_code () {
 	if ! test_skip "$@"
 	then
 		test_run_ "$3"
-		if [ "$?" = 0 -a "$eval_ret" = "$1" ]
+		run_ret="$?"
+		# test_run_ may update missing external prerequisites,
+		test_check_missing_external_prereqs_ "$@" ||
+		if [ "$run_ret" = 0 -a "$eval_ret" = "$1" ]
 		then
 			test_ok_ "$2"
 		else
@@ -870,6 +902,7 @@ test_emacs () {
 
 test_reset_state_ () {
 	test_subtest_known_broken_=
+	test_subtest_missing_external_prereqs_=
 }
 
 
-- 
1.7.7.3

  reply	other threads:[~2011-11-27 18:36 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-11-17 13:05 [PATCH v2 00/10] test: (hopefully) better test prerequisites Dmitry Kurochkin
2011-11-17 13:05 ` [PATCH v2 01/10] test: move subtest variables reset into a dedicated function Dmitry Kurochkin
2011-11-17 13:05 ` [PATCH v2 02/10] test: set EMACS_SERVER variable only after dtach(1) was successfully started Dmitry Kurochkin
2011-11-17 13:05 ` [PATCH v2 03/10] test: add test state reset to test_expect_* functions that did not have it Dmitry Kurochkin
2011-11-27 16:19   ` David Bremner
2011-11-27 18:36     ` [PATCH v3 00/7] test: (hopefully) better test prerequisites Dmitry Kurochkin
2011-11-27 18:36       ` Dmitry Kurochkin [this message]
2011-11-27 18:36       ` [PATCH v3 2/7] test: fix "skipping test" verbose output Dmitry Kurochkin
2011-11-27 18:36       ` [PATCH v3 3/7] test: skip all subtests if external dependencies are missing during init Dmitry Kurochkin
2011-11-27 18:36       ` [PATCH v3 4/7] test: declare external dependencies for the tests Dmitry Kurochkin
2011-11-27 18:36       ` [PATCH v3 5/7] test: add function to explicitly check for external dependencies Dmitry Kurochkin
2011-11-27 18:36       ` [PATCH v3 6/7] test: check if emacs is available in the beginning of test_emacs Dmitry Kurochkin
2011-11-27 18:36       ` [PATCH v3 7/7] test: fix "Stashing in notmuch-search" test when emacs is not available Dmitry Kurochkin
2011-11-28  7:31       ` [PATCH v3 00/7] test: (hopefully) better test prerequisites David Bremner
2012-01-12 17:14       ` Pieter Praet
2012-01-12 17:16         ` [PATCH] test: don't bail out of `run_emacs' too early when missing prereqs Pieter Praet
2012-01-12 17:34           ` Dmitry Kurochkin
2012-01-14  9:07             ` Pieter Praet
2012-01-14  9:09               ` Pieter Praet
2012-01-15 13:50                 ` Dmitry Kurochkin
2012-01-16  2:44                 ` David Bremner
2012-01-15 13:56               ` Dmitry Kurochkin
2012-01-16 10:40                 ` Pieter Praet
2011-11-17 13:05 ` [PATCH v2 04/10] test: add support for external executable dependencies Dmitry Kurochkin
2011-11-17 13:05 ` [PATCH v2 05/10] test: fix "skipping test" verbose output Dmitry Kurochkin
2011-11-17 13:05 ` [PATCH v2 06/10] test: skip all subtests if external dependencies are missing during init Dmitry Kurochkin
2011-11-17 13:06 ` [PATCH v2 07/10] test: declare external dependencies for the tests Dmitry Kurochkin
2011-11-17 13:06 ` [PATCH v2 08/10] test: add function to explicitly check for external dependencies Dmitry Kurochkin
2011-11-17 13:06 ` [PATCH v2 09/10] test: check if emacs is available in the beginning of test_emacs Dmitry Kurochkin
2011-11-17 13:06 ` [PATCH v2 10/10] test: fix "Stashing in notmuch-search" test when emacs is not available Dmitry Kurochkin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://notmuchmail.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1322418979-19180-2-git-send-email-dmitry.kurochkin@gmail.com \
    --to=dmitry.kurochkin@gmail.com \
    --cc=notmuch@notmuchmail.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).