[RFC WIP PATCH] test: use GNU make instead of parallel(1) to run tests in parallel

[RFC WIP PATCH] test: use GNU make instead of parallel(1) to run tests in parallel

[Tomi Ollila]

quick try -- more to follow if there is interest :D

moreutils parallel and gnu parallel are fine tools, but if tests
can be run in parallel with gnu make (which is required anyway)
those are not needed to be installed -- not in linux nor in any
other system where such tools may be harder to install...

current version:
make test                           - use nproc(1) to choose how many in pa
NOTMUCH_TEST_PARALLEL= make test    - don't run tests in parallel
NOTMUCH_TEST_PARALLEL=1 make test   - like above, but uses make in between
NOTMUCH_TEST_PARALLEL=99 make test  - run 99 (all, as we have 90+) in paral

(the latests did execute OK in 10 year old laptop with 4 gig of memory)

todo:
 - how to choose gnu make on non-linux systems (gmake, $MAKE ...?)
 - whether to use -O (output-sync -- and how new make needed for that)
 - check decisions made in this...
 - unverbose test run (from make execution instead of @env... in makefile)
 - whether to have user-env-defined NOTMUCH_TEST_PARALLEL or something else
---
 test/notmuch-test | 23 ++++++++++++-----------
 test/parallel.mk  | 13 +++++++++++++
 2 files changed, 25 insertions(+), 11 deletions(-)
 create mode 100644 test/parallel.mk

diff --git a/test/notmuch-test b/test/notmuch-test
index 14747bdb..f8ab676a 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -46,21 +46,22 @@ else
     TEST_TIMEOUT_CMD=""
 fi
 
+# if NOTMUCH_TEST_PARALLEL is unset, set it to the number of cores
+# if NOTMUCH_TEST_PARALLEL is set but empty, don't run tests in parallel
+# user can set NOTMUCH_TEST_PARALLEL to a numeric value...
+: ${NOTMUCH_TEST_PARALLEL=`exec nproc`} || : # nproc may not be available in all systems...
+
 META_FAILURE=
 RES=0
 # Run the tests
-if test -z "${NOTMUCH_TEST_SERIALIZE-}" && command -v parallel >/dev/null ; then
-    test -t 1 && export COLORS_WITHOUT_TTY=t || :
-    if parallel --minversion 0 >/dev/null 2>&1 ; then
-        echo "INFO: running tests with GNU parallel"
-        printf '%s\n' $TESTS | $TEST_TIMEOUT_CMD parallel || RES=$?
-    else
-        echo "INFO: running tests with moreutils parallel"
-        $TEST_TIMEOUT_CMD parallel -- $TESTS || RES=$?
-    fi
-    if [ $RES != 0 ]; then
+if test "$NOTMUCH_TEST_PARALLEL"; then
+    test -t 1 && export COLORS_WITHOUT_TTY=t
+    echo "INFO: running tests parallel, $NOTMUCH_TEST_PARALLEL tests..."
+    make -k -f $NOTMUCH_SRCDIR/test/parallel.mk -j $NOTMUCH_TEST_PARALLEL \
+            TIMEOUT_CMD="$TEST_TIMEOUT_CMD" $TESTS || {
+        RES=$?
         META_FAILURE="parallel test suite returned error code $RES"
-    fi
+    }
 else
     trap 'e=$?; trap - 0; kill ${!-}; exit $e' 0 HUP INT TERM
     for test in $TESTS; do
diff --git a/test/parallel.mk b/test/parallel.mk
new file mode 100644
index 00000000..4002f499
--- /dev/null
+++ b/test/parallel.mk
@@ -0,0 +1,13 @@
+#
+# named as parallel.mk to show what is the purpose, as
+# the contents of this makefile looks more generic
+#
+
+TIMEOUT_CMD := # could not figure out good generic name for this
+
+.PHONY: $(MAKECMDGOALS)
+$(MAKECMDGOALS):
+	env -u MAKEFLAGS $(TIMEOUT_CMD) $@
+
+.SUFFIXES:
+MAKEFLAGS += --no-builtin-rules --warn-undefined-variables
-- 
2.31.1

Re: [RFC WIP PATCH] test: use GNU make instead of parallel(1) to run tests in parallel

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

> quick try -- more to follow if there is interest :D
>
> moreutils parallel and gnu parallel are fine tools, but if tests
> can be run in parallel with gnu make (which is required anyway)
> those are not needed to be installed -- not in linux nor in any
> other system where such tools may be harder to install...

At least for me, installing moreutils is not a burden. So someone else
will have to drive this if they are motivated.

d