From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 0NTZJop9G2EzCgAAgWs5BA (envelope-from ) for ; Tue, 17 Aug 2021 11:12:42 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 8EFbIop9G2F6dQAAbx9fmQ (envelope-from ) for ; Tue, 17 Aug 2021 09:12:42 +0000 Received: from mail.notmuchmail.org (nmbug.tethera.net [144.217.243.247]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 96EF325D66 for ; Tue, 17 Aug 2021 11:12:41 +0200 (CEST) Received: from nmbug.tethera.net (localhost [127.0.0.1]) by mail.notmuchmail.org (Postfix) with ESMTP id D673F1FFB4; Tue, 17 Aug 2021 05:12:34 -0400 (EDT) Received: from lahtoruutu.iki.fi (unknown [IPv6:2a0b:5c81:1c1::37]) by mail.notmuchmail.org (Postfix) with ESMTPS id DF87E1FE70 for ; Tue, 17 Aug 2021 05:12:31 -0400 (EDT) Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: too) by lahtoruutu.iki.fi (Postfix) with ESMTPSA id 1F4B01B00273; Tue, 17 Aug 2021 12:12:20 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iki.fi; s=lahtoruutu; t=1629191540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=3fqkaVO+84l0tOmvqxYhECI3iuar0p3XUa/HSI11bO0=; b=PsfBcYhm4VS8204tirx7QqznXtS9s++USdwgIpTzcpXxH/SmV/FbPoqfVFE/9EItkwsa8p iMsQ21rBoYqFGK+E2uWBt+cBCvANiOWqN1ToXL9975q1mMpXWZwFvF/CngoIxJecZgkQof Q59LUqAhjhGMV26n0++ZIivQF4JGtmdqVceyjykzKH4Wc+GKmHkW9wcuvas9AasPV+eV2/ t4Q2lNknIE+ORKtudOx9L02W0ApFgY2xUNmUgGzkFy1vQaSzG4oQeR2W93pg7pEGs1XZsC EnFb5BKeHUXJYdlQWHaPJihZaoCyr/HyQAAAIjBesdQgU0+9cP6+iYrz0YJaJQ== From: Tomi Ollila To: notmuch@notmuchmail.org Subject: [RFC WIP PATCH] test: use GNU make instead of parallel(1) to run tests in parallel Date: Tue, 17 Aug 2021 12:12:16 +0300 Message-Id: <20210817091216.26044-1-tomi.ollila@iki.fi> X-Mailer: git-send-email 2.30.0 MIME-Version: 1.0 ARC-Authentication-Results: i=1; ORIGINATING; auth=pass smtp.auth=too smtp.mailfrom=tomi.ollila@iki.fi ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=iki.fi; s=lahtoruutu; t=1629191540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=3fqkaVO+84l0tOmvqxYhECI3iuar0p3XUa/HSI11bO0=; b=LvaU3GbqsYDekJmh14bqw1myn8f/Syqv6Q3qsNfHMroPZ00xl6zGlIsunat4e6B4gJ1cSs 3OpqguZ57ZJjx/Xhl6mlAjvJH73zU9ichFHU4PTbw81gIy3k67Pv0E6PL1fg/BCUvJ8lT/ lF4e1OMxJyK3DaNwSAGZaKly3SrKoIuX7OW0Q0cO4qoK6Aq8HXmDTZgzR2QQYDzNuSxDN4 pArRE6H0mIWjujJJ+bUhCKgLd6vTTQmBLVYhhjoSZcrwa07oYoyEEK6xzb6o9GEbXMKZ4q inFD3NFh3x/DyRZ0O+0y5UWnqocbVXmQruTdu6bom7IUmQJwUi/y9iliPVbr/w== ARC-Seal: i=1; s=lahtoruutu; d=iki.fi; t=1629191540; a=rsa-sha256; cv=none; b=Zfjj4TORBk3KMLzIiSxydincQWHKcSuTrhnGwCS2tvAkbRWK9xg+Y5vFzgxg8P1cusRWHV YV7Sqk4Xnr5f7eKIlnjMiDjy/X94Q3ToMrv8fSCIhext3SfaPN5N2vXndDUHcX0ck0u/PO G1RDl9J2z/E4HGktxnzQiktB3Pw3lHhSXagn/8IfsD+SAqpZ8UC681U4XCmAmBZX1nzeyk JA5mz5SZCzZCZqf5zVHFoXfWnDSS07nBijeRRx3Nx4mGi1YARxVvjZZeDBCeizPKpVTmd1 saZIwvmnrOe/lV4w0gIzrsGqP9V4hQSEuO++XfLoc3LboW0sF7nMOcBpSi9HeA== Message-ID-Hash: 3DI6JPLRHXPM54CQEMHQRDYQVC5M3Q4E X-Message-ID-Hash: 3DI6JPLRHXPM54CQEMHQRDYQVC5M3Q4E X-MailFrom: tomi.ollila@iki.fi X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-notmuch.notmuchmail.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; suspicious-header CC: tomi.ollila@iki.fi X-Mailman-Version: 3.2.1 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1629191562; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=f2BxGvMFheLk8BPaPh7cI2EXzZGz2M3J7KFD5dV80Tw=; b=I9CJEDWY8NAsM75BoSGv1qRzV/NHE3Kczvtr7GfyGdSehivBloqtV79DN5wQgGm6L3F8mP /fwhxjeq/eGZmUew7I0tnaXGc7k8ZKNlmTabUHybvO9Uuf/Zg+5QAcFIPsYobDjKg7LEGX EWw13Fid62q2T6gauqh4L4UKTzD+bsQ1qvO3qUV8fLneN377kJE8ag2fY8DWZ5+0Xagkof 0Hn/+XTR/Wr71bt6qelXV2BM9j37R8p6gol7T7Fcwz1TIZaXXCQe+J9ZFSOdMoghSjRLGn VrImPG5eeGqhx4ZGQHmN/9fawvUHbnQnhjhzAMuv8K+ofBOc8Zon/OgpAHooJQ== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1629191562; a=rsa-sha256; cv=fail; b=Gn5A0TyIcil0T9mNUluwYOw9nTBQ+y0LpQ1MX2UNMYt/ad/p40tg300Ww/kds+JClIwLDN rSt5hngLkUeLxxUjYgHTaLBZI7U4tPTNlxFPQ+Q9QeTE9+50yMA7F0OCiSkFtMxam4tKZT jLSMybFKKjPqNAjhV074qEDZWRrx9O7LGOjhyZ73v1PueAAa2Pt4rjth6qa1QLTFpfpBNx M5vkz7B+TAowVp3JsjUfVSx0QslA6xQ+aMtXKElHMRZ8ilO58ZEJeDGbN+yWmGKQ73m98h A8iSfemtTEt68eaCM34m+Env3Jh3SMApKKmWdyobgRJh1YcYYk4XyhRlBLbMsQ== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=iki.fi header.s=lahtoruutu header.b=PsfBcYhm; arc=reject ("signature check failed: fail, {[1] = sig:iki.fi:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Spam-Score: 3.47 Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=iki.fi header.s=lahtoruutu header.b=PsfBcYhm; arc=reject ("signature check failed: fail, {[1] = sig:iki.fi:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 144.217.243.247 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Migadu-Queue-Id: 96EF325D66 X-Spam-Score: 3.47 X-Migadu-Scanner: scn1.migadu.com X-TUID: yu9RudL5wz2A quick try -- more to follow if there is interest :D moreutils parallel and gnu parallel are fine tools, but if tests can be run in parallel with gnu make (which is required anyway) those are not needed to be installed -- not in linux nor in any other system where such tools may be harder to install... current version: make test - use nproc(1) to choose how many in pa NOTMUCH_TEST_PARALLEL= make test - don't run tests in parallel NOTMUCH_TEST_PARALLEL=1 make test - like above, but uses make in between NOTMUCH_TEST_PARALLEL=99 make test - run 99 (all, as we have 90+) in paral (the latests did execute OK in 10 year old laptop with 4 gig of memory) todo: - how to choose gnu make on non-linux systems (gmake, $MAKE ...?) - whether to use -O (output-sync -- and how new make needed for that) - check decisions made in this... - unverbose test run (from make execution instead of @env... in makefile) - whether to have user-env-defined NOTMUCH_TEST_PARALLEL or something else --- test/notmuch-test | 23 ++++++++++++----------- test/parallel.mk | 13 +++++++++++++ 2 files changed, 25 insertions(+), 11 deletions(-) create mode 100644 test/parallel.mk diff --git a/test/notmuch-test b/test/notmuch-test index 14747bdb..f8ab676a 100755 --- a/test/notmuch-test +++ b/test/notmuch-test @@ -46,21 +46,22 @@ else TEST_TIMEOUT_CMD="" fi +# if NOTMUCH_TEST_PARALLEL is unset, set it to the number of cores +# if NOTMUCH_TEST_PARALLEL is set but empty, don't run tests in parallel +# user can set NOTMUCH_TEST_PARALLEL to a numeric value... +: ${NOTMUCH_TEST_PARALLEL=`exec nproc`} || : # nproc may not be available in all systems... + META_FAILURE= RES=0 # Run the tests -if test -z "${NOTMUCH_TEST_SERIALIZE-}" && command -v parallel >/dev/null ; then - test -t 1 && export COLORS_WITHOUT_TTY=t || : - if parallel --minversion 0 >/dev/null 2>&1 ; then - echo "INFO: running tests with GNU parallel" - printf '%s\n' $TESTS | $TEST_TIMEOUT_CMD parallel || RES=$? - else - echo "INFO: running tests with moreutils parallel" - $TEST_TIMEOUT_CMD parallel -- $TESTS || RES=$? - fi - if [ $RES != 0 ]; then +if test "$NOTMUCH_TEST_PARALLEL"; then + test -t 1 && export COLORS_WITHOUT_TTY=t + echo "INFO: running tests parallel, $NOTMUCH_TEST_PARALLEL tests..." + make -k -f $NOTMUCH_SRCDIR/test/parallel.mk -j $NOTMUCH_TEST_PARALLEL \ + TIMEOUT_CMD="$TEST_TIMEOUT_CMD" $TESTS || { + RES=$? META_FAILURE="parallel test suite returned error code $RES" - fi + } else trap 'e=$?; trap - 0; kill ${!-}; exit $e' 0 HUP INT TERM for test in $TESTS; do diff --git a/test/parallel.mk b/test/parallel.mk new file mode 100644 index 00000000..4002f499 --- /dev/null +++ b/test/parallel.mk @@ -0,0 +1,13 @@ +# +# named as parallel.mk to show what is the purpose, as +# the contents of this makefile looks more generic +# + +TIMEOUT_CMD := # could not figure out good generic name for this + +.PHONY: $(MAKECMDGOALS) +$(MAKECMDGOALS): + env -u MAKEFLAGS $(TIMEOUT_CMD) $@ + +.SUFFIXES: +MAKEFLAGS += --no-builtin-rules --warn-undefined-variables -- 2.31.1