From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
To: 45948@debbugs.gnu.org
Subject: [bug#45948] [PATCH 4/5] build: test-driver.scm: Provide the ability to filter on test case names.
Date: Mon, 18 Jan 2021 01:25:00 -0500 [thread overview]
Message-ID: <20210118062501.27022-5-maxim.cournoyer@gmail.com> (raw)
In-Reply-To: <20210118062501.27022-1-maxim.cournoyer@gmail.com>
* build-aux/test-driver.scm (show-help): Add help text for the new --select
and --exclude options.
(%options): Add the new select and exclude options.
(test-runner-gnu): Pass them to the test runner. Update doc.
(test-match-name*, test-match-name*/negated, %test-match-all): New variables.
(main): Compute the test specifier based on the values of the new options and
apply it to the current test runner when running the test file.
* doc/guix.texi (Running the Test Suite): Document the new options.
---
build-aux/test-driver.scm | 66 ++++++++++++++++++++++++++++++++-------
doc/guix.texi | 12 +++++++
2 files changed, 66 insertions(+), 12 deletions(-)
diff --git a/build-aux/test-driver.scm b/build-aux/test-driver.scm
index 767a9fb25d..3ad6b0c93f 100644
--- a/build-aux/test-driver.scm
+++ b/build-aux/test-driver.scm
@@ -27,6 +27,8 @@
(use-modules (ice-9 getopt-long)
(ice-9 pretty-print)
+ (ice-9 regex)
+ (srfi srfi-1)
(srfi srfi-26)
(srfi srfi-64))
@@ -34,14 +36,19 @@
(display "Usage:
test-driver --test-name=NAME --log-file=PATH --trs-file=PATH
[--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--select=REGEXP] [--exclude=REGEXP]
[--enable-hard-errors={yes|no}] [--brief={yes|no}}] [--]
TEST-SCRIPT [TEST-SCRIPT-ARGUMENTS]
-The '--test-name', '--log-file' and '--trs-file' options are mandatory.\n"))
+The '--test-name', '--log-file' and '--trs-file' options are mandatory.
+The '--select' and '--exclude' options allow selecting or excluding individual
+test cases via a regexp, respectively.\n"))
(define %options
'((test-name (value #t) (required? #t))
(log-file (value #t))
(trs-file (value #t))
+ (select (value #t))
+ (exclude (value #t))
(color-tests (value #t))
(expect-failure (value #t)) ;XXX: not implemented yet
(enable-hard-errors (value #t)) ;not implemented in SRFI-64
@@ -76,14 +83,22 @@ The '--test-name', '--log-file' and '--trs-file' options are mandatory.\n"))
"^[[m") ;no color
result)))
+\f
+;;;
+;;; SRFI 64 custom test runner.
+;;;
+
(define* (test-runner-gnu test-name #:key color? brief?
(out-port (current-output-port))
- (trs-port (%make-void-port "w")))
+ (trs-port (%make-void-port "w"))
+ select exclude)
"Return an custom SRFI-64 test runner. TEST-NAME is a string specifying the
file name of the current the test. COLOR? specifies whether to use colors,
and BRIEF?, well, you know. OUT-PORT and TRS-PORT must be output ports.
OUT-PORT defaults to the current output port, while TRS-PORT defaults to a
-void port, which means no TRS output is logged."
+void port, which means no TRS output is logged. SELECT and EXCLUDE may take a
+regular expression to select or exclude individual test cases based on their
+names."
(define (test-on-test-begin-gnu runner)
;; Procedure called at the start of an individual test case, before the
@@ -148,6 +163,24 @@ void port, which means no TRS output is logged."
(test-runner-on-bad-end-name! runner test-on-bad-end-name-simple)
runner))
+\f
+;;;
+;;; SRFI 64 test specifiers.
+;;;
+(define (test-match-name* regexp)
+ "Return a test specifier that matches a test name against REGEXP."
+ (lambda (runner)
+ (string-match regexp (test-runner-test-name runner))))
+
+(define (test-match-name*/negated regexp)
+ "Return a negated test specifier version of test-match-name*."
+ (lambda (runner)
+ (not (string-match regexp (test-runner-test-name runner)))))
+
+;;; XXX: test-match-all is a syntax, which isn't convenient to use with a list
+;;; of test specifiers computed at run time.
+(define %test-match-all (@@ (srfi srfi-64) %test-match-all))
+
\f
;;;
;;; Entry point.
@@ -158,15 +191,22 @@ void port, which means no TRS output is logged."
(option (cut option-ref opts <> <>)))
(cond
((option 'help #f) (show-help))
- ((option 'version #f) (format #t "test-driver.scm ~A" script-version))
+ ((option 'version #f) (format #t "test-driver.scm ~A" 'script-version))
(else
- (let ((log (and=> (option 'log-file #f) (cut open-file <> "w0")))
- (trs (and=> (option 'trs-file #f) (cut open-file <> "wl")))
- (out (duplicate-port (current-output-port) "wl"))
- (test-name (option 'test-name #f))
- (color-tests (if (assoc 'color-tests opts)
- (option->boolean opts 'color-tests)
- #t)))
+ (let* ((log (and=> (option 'log-file #f) (cut open-file <> "w0")))
+ (trs (and=> (option 'trs-file #f) (cut open-file <> "wl")))
+ (out (duplicate-port (current-output-port) "wl"))
+ (test-name (option 'test-name #f))
+ (select (option 'select #f))
+ (exclude (option 'exclude #f))
+ (test-specifiers (filter-map
+ identity
+ (list (and=> select test-match-name*)
+ (and=> exclude test-match-name*/negated))))
+ (test-specifier (apply %test-match-all test-specifiers))
+ (color-tests (if (assoc 'color-tests opts)
+ (option->boolean opts 'color-tests)
+ #t)))
(when log
(redirect-port log (current-output-port))
(redirect-port log (current-warning-port))
@@ -176,7 +216,9 @@ void port, which means no TRS output is logged."
#:color? color-tests
#:brief? (option->boolean opts 'brief)
#:out-port out #:trs-port trs)
- (load-from-path test-name))
+ (test-apply test-specifier
+ (lambda _
+ (load-from-path test-name))))
(and=> log close-port)
(and=> trs close-port)
(close-port out))))
diff --git a/doc/guix.texi b/doc/guix.texi
index 194bb3a314..d59d2806cb 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -915,6 +915,18 @@ the @code{SCM_LOG_DRIVER_FLAGS} makefile variable as in this example:
make check TESTS="tests/base64.scm" SCM_LOG_DRIVER_FLAGS="--brief=no"
@end example
+The underlying SRFI 64 custom Automake test driver used for the 'check'
+test suite (located at @file{build-aux/test-driver.scm}) also allows
+selecting which test cases to run at a finer level, via its
+@option{--select} and @option{--exclude} options. Here's an example, to
+run all the test cases from the @file{tests/packages.scm} test file
+whose names start with ``transaction-upgrade-entry'':
+
+@example
+export SCM_LOG_DRIVER_FLAGS="--select=^transaction-upgrade-entry"
+make check TESTS="tests/packages.scm"
+@end example
+
Upon failure, please email @email{bug-guix@@gnu.org} and attach the
@file{test-suite.log} file. Please specify the Guix version being used
as well as version numbers of the dependencies (@pxref{Requirements}) in
--
2.29.2
next prev parent reply other threads:[~2021-01-18 6:27 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-18 6:18 [bug#45948] [PATCH 0/5] Improvements to the Automake SRFI 64 test driver Maxim Cournoyer
2021-01-18 6:24 ` [bug#45948] [PATCH 1/5] build: test-driver.scm Make output redirection optional Maxim Cournoyer
2021-01-18 6:24 ` [bug#45948] [PATCH 2/5] build: test-driver.scm: Define the --test-name option as required Maxim Cournoyer
2021-01-18 14:37 ` [bug#45948] [PATCH 0/5] Improvements to the Automake SRFI 64 test driver Maxim Cournoyer
2021-01-18 6:24 ` [bug#45948] [PATCH 3/5] build: test-driver.scm: Enable colored test results by default Maxim Cournoyer
2021-01-30 21:39 ` [bug#45948] [PATCH 0/5] Improvements to the Automake SRFI 64 test driver Ludovic Courtès
2021-02-01 2:47 ` Maxim Cournoyer
2021-01-18 6:24 ` [bug#45948] [PATCH 4/5] build: test-driver.scm: Add test cases filtering options Maxim Cournoyer
2021-01-18 14:38 ` [bug#45948] [PATCH 0/5] Improvements to the Automake SRFI 64 test driver Maxim Cournoyer
2021-01-18 6:25 ` Maxim Cournoyer [this message]
2021-01-30 21:34 ` Ludovic Courtès
2021-02-01 3:44 ` bug#45948: " Maxim Cournoyer
2021-01-18 6:25 ` [bug#45948] [PATCH 5/5] build: test-driver.scm: Add a new '--errors-only' option Maxim Cournoyer
2021-01-30 21:32 ` [bug#45948] [PATCH 0/5] Improvements to the Automake SRFI 64 test driver Ludovic Courtès
2021-02-01 22:18 ` Ludovic Courtès
2021-02-02 5:47 ` Maxim Cournoyer
2021-02-02 8:36 ` zimoun
2021-02-02 12:52 ` Maxim Cournoyer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210118062501.27022-5-maxim.cournoyer@gmail.com \
--to=maxim.cournoyer@gmail.com \
--cc=45948@debbugs.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).