* [PATCH] build: Add a scheme custom test driver using SRFI-64.
@ 2015-11-01 2:59 Mathieu Lirzin
2015-11-01 17:13 ` Ludovic Courtès
0 siblings, 1 reply; 8+ messages in thread
From: Mathieu Lirzin @ 2015-11-01 2:59 UTC (permalink / raw)
To: guix-devel
[-- Attachment #1: Type: text/plain, Size: 1951 bytes --]
Hello Guix,
While reading the Automake manual I have discovered that it provides an
API for custom test drivers, so I've tried to implement something for
the scheme API for test suites (SRFI-64) which Guix is using. The
result provides support for multiple test cases in a single file, and
allow ‘make check’ display the results individually:
PASS: tests/base32.scm - bytevector->base32-string
PASS: tests/base32.scm - base32-string->bytevector
PASS: tests/base32.scm - nix-base32-string->bytevector
PASS: tests/base32.scm - &invalid-base32-character
SKIP: tests/base32.scm - sha256 & bytevector->nix-base32-string
PASS: tests/base64.scm - empty string
It is now possible to define individual XFAIL test cases:
(test-expect-fail "absurd")
(test-assert "absurd" (= 0 1))
And the ‘.log’ files now use somekind of recutils format:
test-name: url-fetch, file
location: /home/mthl/src/gnu/guix/tests/builders.scm:78
source:
+ (test-assert
+ "url-fetch, file"
+ (let* ((file (search-path %load-path "guix.scm"))
+ (hash (call-with-input-file file port-sha256))
+ (out (url-fetch* %store file 'sha256 hash)))
+ (and (file-exists? out) (valid-path? %store out))))
value: #t
result: PASS
Finally the ‘.log’ files are no longer fragmented between
“${top_srcdir}” and “tests/”. If something went wrong (FAIL, SKIP,
XPASS) then the entire log file is copied in ‘test-suite.log’ so no need
to ask for an additional file when someone is reporting some ‘make
check’ error.
One caveat is that ‘tests/cpio.scm‘ is now failing.
Since this script is not intented to be exclusively used by Guix, I have
used a generic copyright notice. I guess Guix is the best place to
challenge and improve it, but IMO it will be better hosted elsewhere
like in Gnulib. Opinions?
Comments are welcome!
--
Mathieu Lirzin
[-- Attachment #2: 0001-build-Add-a-scheme-custom-test-driver-using-SRFI-64.patch --]
[-- Type: text/x-diff, Size: 22176 bytes --]
From 977e0586a6689df204fd9b522f8508ec318180c0 Mon Sep 17 00:00:00 2001
From: Mathieu Lirzin <mthl@openmailbox.org>
Date: Mon, 26 Oct 2015 23:47:24 +0100
Subject: [PATCH] build: Add a scheme custom test driver using SRFI-64.
This provides support for multiple scheme test cases in a unique file and
fixes the fragmentation of '.log' files.
* build-aux/test-driver.scm: New file.
* Makefile.am (SCM_LOG_DRIVER): New variable.
(SCM_LOG_COMPILER, AM_SCM_LOG_FLAGS): Delete variables.
(AM_TESTS_ENVIRONMENT): Set GUILE_AUTO_COMPILE to 0.
* tests/base32.scm, tests/build-utils.scm, tests/builders.scm,
tests/challenge.scm, tests/containers.scm, tests/cpan.scm,
tests/cpio.scm, tests/cran.scm, tests/derivations.scm, tests/elpa.scm,
tests/file-systems.scm, tests/gem.scm, tests/gexp.scm,
tests/graph.scm, tests/gremlin.scm, tests/hackage.scm, tests/hash.scm,
tests/lint.scm, tests/monads.scm, tests/nar.scm, tests/packages.scm,
tests/pk-crypto.scm, tests/pki.scm, tests/profiles.scm,
tests/publish.scm, tests/pypi.scm, tests/records.scm,
tests/scripts.scm, tests/services.scm, tests/sets.scm, tests/size.scm,
tests/snix.scm, tests/store.scm, tests/substitute.scm,
tests/syscalls.scm, tests/ui.scm, tests/union.scm, tests/utils.scm:
Don't exit at the end of each test.
---
Makefile.am | 6 +-
build-aux/test-driver.scm | 157 ++++++++++++++++++++++++++++++++++++++++++++++
tests/base32.scm | 3 -
tests/build-utils.scm | 3 -
tests/builders.scm | 3 -
tests/challenge.scm | 3 -
tests/containers.scm | 3 -
tests/cpan.scm | 3 -
tests/cpio.scm | 3 -
tests/cran.scm | 3 -
tests/derivations.scm | 3 -
tests/elpa.scm | 3 -
tests/file-systems.scm | 3 -
tests/gem.scm | 3 -
tests/gexp.scm | 3 -
tests/graph.scm | 3 -
tests/gremlin.scm | 3 -
tests/hackage.scm | 3 -
tests/hash.scm | 3 -
tests/lint.scm | 3 -
tests/monads.scm | 3 -
tests/nar.scm | 3 -
tests/packages.scm | 3 -
tests/pk-crypto.scm | 3 -
tests/pki.scm | 3 -
tests/profiles.scm | 3 -
tests/publish.scm | 3 -
tests/pypi.scm | 3 -
tests/records.scm | 3 -
tests/scripts.scm | 3 -
tests/services.scm | 3 -
tests/sets.scm | 3 -
tests/size.scm | 3 -
tests/snix.scm | 3 -
tests/store.scm | 3 -
tests/substitute.scm | 3 -
tests/syscalls.scm | 3 -
tests/ui.scm | 3 -
tests/union.scm | 3 -
tests/utils.scm | 3 -
40 files changed, 160 insertions(+), 117 deletions(-)
create mode 100644 build-aux/test-driver.scm
diff --git a/Makefile.am b/Makefile.am
index 4f90b1d..2db6b25 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -268,10 +268,10 @@ TESTS = $(SCM_TESTS) $(SH_TESTS)
TEST_EXTENSIONS = .scm .sh
-AM_TESTS_ENVIRONMENT = abs_top_srcdir="$(abs_top_srcdir)"
+AM_TESTS_ENVIRONMENT = abs_top_srcdir="$(abs_top_srcdir)" GUILE_AUTO_COMPILE=0
-SCM_LOG_COMPILER = $(top_builddir)/test-env $(GUILE)
-AM_SCM_LOG_FLAGS = --no-auto-compile -L "$(top_srcdir)"
+SCM_LOG_DRIVER = $(top_builddir)/test-env $(GUILE) --no-auto-compile \
+ -e main $(top_srcdir)/build-aux/test-driver.scm 2>/dev/null
SH_LOG_COMPILER = $(top_builddir)/test-env $(SHELL)
AM_SH_LOG_FLAGS = -x -e
diff --git a/build-aux/test-driver.scm b/build-aux/test-driver.scm
new file mode 100644
index 0000000..52f7cfb
--- /dev/null
+++ b/build-aux/test-driver.scm
@@ -0,0 +1,157 @@
+;;;; test-driver.scm --- SRFI-64 custom driver for GNU Automake test suites.
+
+(define script-version "2015-11-01.01") ;UTC
+
+;;; Copyright © 2015 Mathieu Lirzin <mthl@gnu.org>
+;;;
+;;; This program is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; This program is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+;;;; Commentary:
+;;;
+;;; This script provides a custom test driver for GNU Automake using the
+;;; SRFI-64 Scheme API for test suites.
+;;;
+;;;; Code:
+
+(use-modules (ice-9 getopt-long)
+ (ice-9 pretty-print)
+ (srfi srfi-64))
+
+(define (show-help)
+ (display "Usage:
+ test-driver --test-name=NAME --log-file=PATH --trs-file=PATH
+ [--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--enable-hard-errors={yes|no}] [--]
+ TEST-SCRIPT [TEST-SCRIPT-ARGUMENTS]
+The '--test-name', '--log-file' and '--trs-file' options are mandatory.\n"))
+
+(define %options
+ '((test-name (value #t))
+ (log-file (value #t))
+ (trs-file (value #t))
+ (color-tests (value #t))
+ (expect-failure (value #t)) ;XXX: not implemented yet
+ (enable-hard-errors (value #t)) ;XXX: not implemented in SRFI-64
+ (help (single-char #\h) (value #f))
+ (version (single-char #\V) (value #f))))
+
+(define* (test-display field value #:optional (port (current-output-port)))
+ "Display \"FIELD: VALUE\\n\" on PORT."
+ (format port "~A: ~A\n" field value))
+
+(define* (result->string symbol #:key colorize)
+ "Return SYMBOL as an upper case string. Use colors when COLORIZE is #t."
+ (let ((result (string-upcase (symbol->string symbol))))
+ (if colorize
+ (string-append (case symbol
+ ((pass) "^[[0;32m") ;green
+ ((xfail) "^[[1;32m") ;light green
+ ((skip) "^[[1;34m") ;blue
+ ((fail xpass) "^[[0;31m") ;red
+ ((error) "^[[0;35m")) ;magenta
+ result
+ "^[[m") ;no color
+ result)))
+
+(define* (test-runner-gnu test-name #:key color-tests out-port trs-port)
+ "Return an custom SRFI-64 test runner. The current output port is supposed
+to be redirected to a '.log' file."
+
+ (define (test-on-test-begin-gnu runner)
+ "Procedure called at the start of an individual test-case, before the test
+expression (and expected value) are evaluated."
+ (let ((results (test-result-alist runner)))
+ (test-display "test-name" (assq-ref results 'test-name))
+ (test-display "location"
+ (string-append
+ (assq-ref results 'source-file) ":"
+ (number->string (assq-ref results 'source-line))))
+ (display "source:\n")
+ (pretty-print (assq-ref results 'source-form) #:per-line-prefix "+ ")))
+
+ (define (test-on-test-end-gnu runner)
+ "Procedure called at the end of an individual testcase, when the result of
+the test is available."
+ (let ((results (test-result-alist runner)))
+ ;; Display the result of the test on the console (with colors).
+ (test-display
+ (result->string (test-result-kind runner) #:colorize color-tests)
+ (string-append test-name " - " (test-runner-test-name runner))
+ out-port)
+ (when (assq 'actual-value results)
+ (test-display "value" (assq-ref results 'actual-value)))
+ (test-display "result" (result->string (assq-ref results 'result-kind)))
+ (newline)
+ (test-display ":test-result"
+ (string-append (result->string (test-result-kind runner))
+ " " (test-runner-test-name runner))
+ trs-port)))
+
+ (define (test-on-group-end-gnu runner)
+ "Procedure meant to be called by a `test-end', including at the end of a
+test-group."
+ (let ((fail (or (positive? (test-runner-fail-count runner))
+ (positive? (test-runner-xpass-count runner))))
+ (skip (or (positive? (test-runner-skip-count runner))
+ (positive? (test-runner-xfail-count runner)))))
+ ;; XXX: Since multiple tests share one '.trs' file, we need to choose
+ ;; what to report in the "global" results which would require more
+ ;; refinements.
+ (test-display ":global-test-result" (if fail "FAIL" "PASS") trs-port)
+ (test-display ":recheck" (if fail "yes" "no") trs-port)
+ (test-display ":copy-in-global-log" (if (or fail skip) "yes" "no") trs-port)
+ #f))
+
+ (let ((runner (test-runner-null)))
+ (test-runner-on-test-begin! runner test-on-test-begin-gnu)
+ (test-runner-on-test-end! runner test-on-test-end-gnu)
+ (test-runner-on-group-end! runner test-on-group-end-gnu)
+ (test-runner-on-bad-end-name! runner test-on-bad-end-name-simple)
+ runner))
+
+\f
+;;;
+;;; Entry point.
+;;;
+
+(define (main . args)
+ (let ((opts (getopt-long (command-line) %options)))
+ (when (option-ref opts 'help #f)
+ (show-help)
+ (exit 0))
+ (when (option-ref opts 'version #f)
+ (format #t "test-driver.scm ~A" script-version)
+ (exit 0))
+ (let ((trs (open-output-file (option-ref opts 'trs-file ""))))
+ (test-runner-current
+ (test-runner-gnu (option-ref opts 'test-name #f)
+ #:out-port (current-output-port)
+ #:trs-port trs
+ #:color-tests (option-ref opts 'color-tests #f)))
+ (with-output-to-file (option-ref opts 'log-file "")
+ (lambda ()
+ (set-current-error-port (current-output-port))
+ (load-from-path (option-ref opts 'test-name #f))))
+ (close-port trs)
+ (exit 0))))
+
+;;; Local Variables:
+;;; eval: (add-hook 'write-file-functions 'time-stamp)
+;;; time-stamp-start: "(define script-version \""
+;;; time-stamp-format: "%:y-%02m-%02d.%02H"
+;;; time-stamp-time-zone: "UTC"
+;;; time-stamp-end: "\") ;UTC"
+;;; End:
+
+;;;; test-driver.scm ends here.
diff --git a/tests/base32.scm b/tests/base32.scm
index dcd926f..194f8da 100644
--- a/tests/base32.scm
+++ b/tests/base32.scm
@@ -101,6 +101,3 @@
l))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/build-utils.scm b/tests/build-utils.scm
index b0a4d15..cc96738 100644
--- a/tests/build-utils.scm
+++ b/tests/build-utils.scm
@@ -141,6 +141,3 @@
(equal? str "hello world\n")))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/builders.scm b/tests/builders.scm
index a7c3e42..bb9e0fa 100644
--- a/tests/builders.scm
+++ b/tests/builders.scm
@@ -113,6 +113,3 @@
(file-exists? (string-append out "/bin/hello")))))
(test-end "builders")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/challenge.scm b/tests/challenge.scm
index e53cacd..9505042 100644
--- a/tests/challenge.scm
+++ b/tests/challenge.scm
@@ -106,9 +106,6 @@
(test-end)
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'with-derivation-narinfo* 'scheme-indent-function 2)
;;; End:
diff --git a/tests/containers.scm b/tests/containers.scm
index 0ba8149..ccbcfa5 100644
--- a/tests/containers.scm
+++ b/tests/containers.scm
@@ -141,6 +141,3 @@
(zero? result)))))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/cpan.scm b/tests/cpan.scm
index 2f95135..b92a78c 100644
--- a/tests/cpan.scm
+++ b/tests/cpan.scm
@@ -94,6 +94,3 @@
(pk 'fail x #f)))))
(test-end "cpan")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/cpio.scm b/tests/cpio.scm
index cf65f98..516de06 100644
--- a/tests/cpio.scm
+++ b/tests/cpio.scm
@@ -79,6 +79,3 @@
(stat:size (stat file))))))))))))))
(test-end "cpio")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/cran.scm b/tests/cran.scm
index ba5699a..135f308 100644
--- a/tests/cran.scm
+++ b/tests/cran.scm
@@ -172,6 +172,3 @@
(pk 'fail x #f))))))
(test-end "cran")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/derivations.scm b/tests/derivations.scm
index 9fc96c7..2e10b70 100644
--- a/tests/derivations.scm
+++ b/tests/derivations.scm
@@ -946,6 +946,3 @@
(call-with-input-file out get-string-all))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/elpa.scm b/tests/elpa.scm
index e8adde3..46c6ac2 100644
--- a/tests/elpa.scm
+++ b/tests/elpa.scm
@@ -104,6 +104,3 @@ information about package NAME. (Function 'elpa-package-info'.)"
(eval-test-with-elpa "auctex"))
(test-end "elpa")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/file-systems.scm b/tests/file-systems.scm
index c36509b..aed27e8 100644
--- a/tests/file-systems.scm
+++ b/tests/file-systems.scm
@@ -51,6 +51,3 @@
(equal? form '(uuid "foobar"))))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/gem.scm b/tests/gem.scm
index 9efbda3..43a6c53 100644
--- a/tests/gem.scm
+++ b/tests/gem.scm
@@ -77,6 +77,3 @@
(pk 'fail x #f)))))
(test-end "gem")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/gexp.scm b/tests/gexp.scm
index 4860a8e..ca7ad9a 100644
--- a/tests/gexp.scm
+++ b/tests/gexp.scm
@@ -754,9 +754,6 @@
(test-end "gexp")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;; Local Variables:
;; eval: (put 'test-assertm 'scheme-indent-function 1)
;; End:
diff --git a/tests/graph.scm b/tests/graph.scm
index f454b06..4de1702 100644
--- a/tests/graph.scm
+++ b/tests/graph.scm
@@ -188,6 +188,3 @@ edges."
(equal? edges `((,out ,txt)))))))))))
(test-end "graph")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/gremlin.scm b/tests/gremlin.scm
index dc9f78c..2885554 100644
--- a/tests/gremlin.scm
+++ b/tests/gremlin.scm
@@ -64,6 +64,3 @@
"ORIGIN/foo")))
(test-end "gremlin")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/hackage.scm b/tests/hackage.scm
index 229bee3..969266b 100644
--- a/tests/hackage.scm
+++ b/tests/hackage.scm
@@ -137,6 +137,3 @@ library
(x (pk 'fail x #f))))
(test-end "hackage")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/hash.scm b/tests/hash.scm
index 9bcd694..8039549 100644
--- a/tests/hash.scm
+++ b/tests/hash.scm
@@ -126,6 +126,3 @@ In Guile <= 2.0.9, CBIPs were always fully buffered, so the
(list hello hash world))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/lint.scm b/tests/lint.scm
index 3f14956..db29915 100644
--- a/tests/lint.scm
+++ b/tests/lint.scm
@@ -550,9 +550,6 @@ requests."
(test-end "lint")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;; Local Variables:
;; eval: (put 'with-http-server 'scheme-indent-function 1)
;; eval: (put 'with-warnings 'scheme-indent-function 0)
diff --git a/tests/monads.scm b/tests/monads.scm
index 62a07a2..60d41ae 100644
--- a/tests/monads.scm
+++ b/tests/monads.scm
@@ -254,6 +254,3 @@
'())))
(test-end "monads")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/nar.scm b/tests/nar.scm
index e24a638..9796980 100644
--- a/tests/nar.scm
+++ b/tests/nar.scm
@@ -320,9 +320,6 @@
(test-end "nar")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'with-file-tree 'scheme-indent-function 2)
;;; End:
diff --git a/tests/packages.scm b/tests/packages.scm
index b28ae0b..b037a4f 100644
--- a/tests/packages.scm
+++ b/tests/packages.scm
@@ -727,9 +727,6 @@
(test-end "packages")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'dummy-package 'scheme-indent-function 1)
;;; End:
diff --git a/tests/pk-crypto.scm b/tests/pk-crypto.scm
index f5008f3..5024a15 100644
--- a/tests/pk-crypto.scm
+++ b/tests/pk-crypto.scm
@@ -287,6 +287,3 @@
(canonical-sexp->sexp (sexp->canonical-sexp sexp))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/pki.scm b/tests/pki.scm
index 51f2119..876ad98 100644
--- a/tests/pki.scm
+++ b/tests/pki.scm
@@ -119,6 +119,3 @@
(corrupt-signature 'c))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/profiles.scm b/tests/profiles.scm
index cc9a822..360ce27 100644
--- a/tests/profiles.scm
+++ b/tests/profiles.scm
@@ -307,9 +307,6 @@
(test-end "profiles")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'dummy-package 'scheme-indent-function 1)
;;; End:
diff --git a/tests/publish.scm b/tests/publish.scm
index 4d72fdc..9a0bed5 100644
--- a/tests/publish.scm
+++ b/tests/publish.scm
@@ -109,6 +109,3 @@ References: ~a~%"
(call-with-input-file temp read-string))))
(test-end "publish")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/pypi.scm b/tests/pypi.scm
index c772474..4f7beb1 100644
--- a/tests/pypi.scm
+++ b/tests/pypi.scm
@@ -106,6 +106,3 @@ baz > 13.37")
(pk 'fail x #f)))))
(test-end "pypi")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/records.scm b/tests/records.scm
index 800ed03..c6f85d4 100644
--- a/tests/records.scm
+++ b/tests/records.scm
@@ -305,6 +305,3 @@ Description: 1st line,
'("a")))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/scripts.scm b/tests/scripts.scm
index 3bf41ae..3901710 100644
--- a/tests/scripts.scm
+++ b/tests/scripts.scm
@@ -64,9 +64,6 @@
(test-end "scripts")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'with-environment-variable 'scheme-indent-function 2)
;;; End:
diff --git a/tests/services.scm b/tests/services.scm
index 7d2e31b..804d6c9 100644
--- a/tests/services.scm
+++ b/tests/services.scm
@@ -115,6 +115,3 @@
(null? (e s3)))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/sets.scm b/tests/sets.scm
index 0a89591..cdb7efe 100644
--- a/tests/sets.scm
+++ b/tests/sets.scm
@@ -47,6 +47,3 @@
(set-contains? s3 b))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/size.scm b/tests/size.scm
index a110604..fcd5902 100644
--- a/tests/size.scm
+++ b/tests/size.scm
@@ -88,9 +88,6 @@
(test-end "size")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'match* 'scheme-indent-function 1)
;;; End:
diff --git a/tests/snix.scm b/tests/snix.scm
index a66b2c7..4c31e33 100644
--- a/tests/snix.scm
+++ b/tests/snix.scm
@@ -71,6 +71,3 @@
(pk 'fail x #f))))
(test-end "snix")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/store.scm b/tests/store.scm
index 96b6478..1f3123b 100644
--- a/tests/store.scm
+++ b/tests/store.scm
@@ -706,6 +706,3 @@
(call-with-output-string (cut write-file item <>))))))))
(test-end "store")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/substitute.scm b/tests/substitute.scm
index 8569812..2e963d1 100644
--- a/tests/substitute.scm
+++ b/tests/substitute.scm
@@ -275,9 +275,6 @@ a file for NARINFO."
(test-end "substitute")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'with-narinfo 'scheme-indent-function 1)
;;; eval: (put 'test-quit 'scheme-indent-function 2)
diff --git a/tests/syscalls.scm b/tests/syscalls.scm
index 86783b9..06870a7 100644
--- a/tests/syscalls.scm
+++ b/tests/syscalls.scm
@@ -236,6 +236,3 @@
(lo (interface-address lo)))))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/ui.scm b/tests/ui.scm
index bd4c907..133982a 100644
--- a/tests/ui.scm
+++ b/tests/ui.scm
@@ -250,6 +250,3 @@ Second line" 24))
(show-manifest-transaction store m t)))))))))
(test-end "ui")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/union.scm b/tests/union.scm
index 22ba67c..cccf397 100644
--- a/tests/union.scm
+++ b/tests/union.scm
@@ -125,6 +125,3 @@
(eq? 'directory (stat:type (lstat "bin"))))))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/utils.scm b/tests/utils.scm
index b65d6d2..1e763a2 100644
--- a/tests/utils.scm
+++ b/tests/utils.scm
@@ -321,6 +321,3 @@
(test-end)
(false-if-exception (delete-file temp-file))
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
--
2.6.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH] build: Add a scheme custom test driver using SRFI-64.
2015-11-01 2:59 [PATCH] build: Add a scheme custom test driver using SRFI-64 Mathieu Lirzin
@ 2015-11-01 17:13 ` Ludovic Courtès
2015-11-01 19:48 ` Mathieu Lirzin
0 siblings, 1 reply; 8+ messages in thread
From: Ludovic Courtès @ 2015-11-01 17:13 UTC (permalink / raw)
To: Mathieu Lirzin; +Cc: guix-devel
Mathieu Lirzin <mthl@gnu.org> skribis:
> While reading the Automake manual I have discovered that it provides an
> API for custom test drivers, so I've tried to implement something for
> the scheme API for test suites (SRFI-64) which Guix is using. The
> result provides support for multiple test cases in a single file, and
> allow ‘make check’ display the results individually:
>
> PASS: tests/base32.scm - bytevector->base32-string
> PASS: tests/base32.scm - base32-string->bytevector
> PASS: tests/base32.scm - nix-base32-string->bytevector
> PASS: tests/base32.scm - &invalid-base32-character
> SKIP: tests/base32.scm - sha256 & bytevector->nix-base32-string
> PASS: tests/base64.scm - empty string
Awesome! Are the “inner” tests displayed by default? Or is there some
environment variable to control that?
I’d prefer the default to display only file names, as is currently the
case (it’s more concise.)
> And the ‘.log’ files now use somekind of recutils format:
Cool!
> Finally the ‘.log’ files are no longer fragmented between
> “${top_srcdir}” and “tests/”. If something went wrong (FAIL, SKIP,
> XPASS) then the entire log file is copied in ‘test-suite.log’ so no need
> to ask for an additional file when someone is reporting some ‘make
> check’ error.
Nice. “Running the Test Suite” needs to be updated accordingly.
> One caveat is that ‘tests/cpio.scm‘ is now failing.
Why is that? Does it relate to this change?
> Since this script is not intented to be exclusively used by Guix, I have
> used a generic copyright notice. I guess Guix is the best place to
> challenge and improve it, but IMO it will be better hosted elsewhere
> like in Gnulib. Opinions?
I think we could start using it and testing it for a while in Guix, and
eventually submit it for inclusion in Gnulib once we are more confident.
> From 977e0586a6689df204fd9b522f8508ec318180c0 Mon Sep 17 00:00:00 2001
> From: Mathieu Lirzin <mthl@openmailbox.org>
> Date: Mon, 26 Oct 2015 23:47:24 +0100
> Subject: [PATCH] build: Add a scheme custom test driver using SRFI-64.
>
> This provides support for multiple scheme test cases in a unique file and
> fixes the fragmentation of '.log' files.
>
> * build-aux/test-driver.scm: New file.
> * Makefile.am (SCM_LOG_DRIVER): New variable.
> (SCM_LOG_COMPILER, AM_SCM_LOG_FLAGS): Delete variables.
> (AM_TESTS_ENVIRONMENT): Set GUILE_AUTO_COMPILE to 0.
> * tests/base32.scm, tests/build-utils.scm, tests/builders.scm,
> tests/challenge.scm, tests/containers.scm, tests/cpan.scm,
> tests/cpio.scm, tests/cran.scm, tests/derivations.scm, tests/elpa.scm,
> tests/file-systems.scm, tests/gem.scm, tests/gexp.scm,
> tests/graph.scm, tests/gremlin.scm, tests/hackage.scm, tests/hash.scm,
> tests/lint.scm, tests/monads.scm, tests/nar.scm, tests/packages.scm,
> tests/pk-crypto.scm, tests/pki.scm, tests/profiles.scm,
> tests/publish.scm, tests/pypi.scm, tests/records.scm,
> tests/scripts.scm, tests/services.scm, tests/sets.scm, tests/size.scm,
> tests/snix.scm, tests/store.scm, tests/substitute.scm,
> tests/syscalls.scm, tests/ui.scm, tests/union.scm, tests/utils.scm:
> Don't exit at the end of each test.
AIUI each tests/foo.scm file is still run with its own Guile instance,
in ./test-env, right? (This is important. It also means tests can run
in parallel.)
If this is OK, it LGTM.
However, I’m unsure if we should push it now, or after the release. On
one hand, I’d rather avoid potentially disrupting changes like this
now. On the other hand, since it makes it easier (and different) to
report test failures, it’s nice.
How confident are you? :-)
What do others think?
Thanks!
Ludo’.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] build: Add a scheme custom test driver using SRFI-64.
2015-11-01 17:13 ` Ludovic Courtès
@ 2015-11-01 19:48 ` Mathieu Lirzin
2015-11-01 23:23 ` Mathieu Lirzin
2015-11-02 8:59 ` Ludovic Courtès
0 siblings, 2 replies; 8+ messages in thread
From: Mathieu Lirzin @ 2015-11-01 19:48 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel
Hi,
ludo@gnu.org (Ludovic Courtès) writes:
> Mathieu Lirzin <mthl@gnu.org> skribis:
>
>> While reading the Automake manual I have discovered that it provides an
>> API for custom test drivers, so I've tried to implement something for
>> the scheme API for test suites (SRFI-64) which Guix is using. The
>> result provides support for multiple test cases in a single file, and
>> allow ‘make check’ display the results individually:
>>
>> PASS: tests/base32.scm - bytevector->base32-string
>> PASS: tests/base32.scm - base32-string->bytevector
>> PASS: tests/base32.scm - nix-base32-string->bytevector
>> PASS: tests/base32.scm - &invalid-base32-character
>> SKIP: tests/base32.scm - sha256 & bytevector->nix-base32-string
>> PASS: tests/base64.scm - empty string
>
> Awesome! Are the “inner” tests displayed by default? Or is there some
> environment variable to control that?
Yes, each test case is displayed by default.
> I’d prefer the default to display only file names, as is currently the
> case (it’s more concise.)
IMO This would be reasonable to have an option for making the output more
concise since there are 563 test cases in Guix. this is not part of the
test driver API specification, but it does not seem too hard to add an
additional option to the test driver script like what the TAP driver is
doing:
https://www.gnu.org/software/automake/manual/automake.html#Use-TAP-with-the-Automake-test-harness
I think ‘--brief’ could be a good name for this option. Default
verbosity would specified by adding “AM_SCM_LOG_DRIVER= --brief=yes” in
Makefile.am, then the user can overide that with:
make check -j4 TESTS=tests/rocket-science.scm SCM_LOG_DRIVER="--brief=no"
>> Finally the ‘.log’ files are no longer fragmented between
>> “${top_srcdir}” and “tests/”. If something went wrong (FAIL, SKIP,
>> XPASS) then the entire log file is copied in ‘test-suite.log’ so no need
>> to ask for an additional file when someone is reporting some ‘make
>> check’ error.
>
> Nice. “Running the Test Suite” needs to be updated accordingly.
Thanks for reminding me this. ;)
>> One caveat is that ‘tests/cpio.scm‘ is now failing.
>
> Why is that? Does it relate to this change?
I didn't try to debug the problem but it has appeared when I started
messing with redirection of the output/error ports. Here is the failure
output.
--8<---------------cut here---------------start------------->8---
test-name: bit-identical to GNU cpio's output
location: /home/mthl/src/gnu/guix/tests/cpio.scm:49
source:
+ (test-assert
+ "bit-identical to GNU cpio's output"
+ (call-with-temporary-output-file
+ (lambda (link _)
+ (delete-file link)
+ (symlink "chbouib" link)
+ (let ((files (cons* "/"
+ (canonicalize-path
+ (dirname (search-path %load-path "guix.scm")))
+ link
+ (map (compose
+ canonicalize-path
+ (cut search-path %load-path <>))
+ '("guix.scm"
+ "guix/build/syscalls.scm"
+ "guix/packages.scm")))))
+ (call-with-temporary-output-file
+ (lambda (ref-file _)
+ (let ((pipe (open-pipe*
+ OPEN_WRITE
+ %cpio-program
+ "-o"
+ "-O"
+ ref-file
+ "-H"
+ "newc"
+ "--null")))
+ (for-each
+ (lambda (file) (format pipe "~a\x00" file))
+ files)
+ (and (zero? (close-pipe pipe))
+ (call-with-temporary-output-file
+ (lambda (file port)
+ (write-cpio-archive files port)
+ (close-port port)
+ (or (file=? ref-file file)
+ (throw 'cpio-archives-differ
+ files
+ ref-file
+ file
+ (stat:size (stat ref-file))
+ (stat:size (stat file))))))))))))))
value: #f
result: FAIL
--8<---------------cut here---------------end--------------->8---
>> Since this script is not intented to be exclusively used by Guix, I have
>> used a generic copyright notice. I guess Guix is the best place to
>> challenge and improve it, but IMO it will be better hosted elsewhere
>> like in Gnulib. Opinions?
>
> I think we could start using it and testing it for a while in Guix, and
> eventually submit it for inclusion in Gnulib once we are more confident.
So you recommend to add ”this file is part of GNU Guix” and use “GNU
Guix” instead of “this program” for now?
>> From 977e0586a6689df204fd9b522f8508ec318180c0 Mon Sep 17 00:00:00 2001
>> From: Mathieu Lirzin <mthl@openmailbox.org>
>> Date: Mon, 26 Oct 2015 23:47:24 +0100
>> Subject: [PATCH] build: Add a scheme custom test driver using SRFI-64.
>>
>> This provides support for multiple scheme test cases in a unique file and
>> fixes the fragmentation of '.log' files.
>>
>> * build-aux/test-driver.scm: New file.
>> * Makefile.am (SCM_LOG_DRIVER): New variable.
>> (SCM_LOG_COMPILER, AM_SCM_LOG_FLAGS): Delete variables.
>> (AM_TESTS_ENVIRONMENT): Set GUILE_AUTO_COMPILE to 0.
>> * tests/base32.scm, tests/build-utils.scm, tests/builders.scm,
>> tests/challenge.scm, tests/containers.scm, tests/cpan.scm,
>> tests/cpio.scm, tests/cran.scm, tests/derivations.scm, tests/elpa.scm,
>> tests/file-systems.scm, tests/gem.scm, tests/gexp.scm,
>> tests/graph.scm, tests/gremlin.scm, tests/hackage.scm, tests/hash.scm,
>> tests/lint.scm, tests/monads.scm, tests/nar.scm, tests/packages.scm,
>> tests/pk-crypto.scm, tests/pki.scm, tests/profiles.scm,
>> tests/publish.scm, tests/pypi.scm, tests/records.scm,
>> tests/scripts.scm, tests/services.scm, tests/sets.scm, tests/size.scm,
>> tests/snix.scm, tests/store.scm, tests/substitute.scm,
>> tests/syscalls.scm, tests/ui.scm, tests/union.scm, tests/utils.scm:
>> Don't exit at the end of each test.
>
> AIUI each tests/foo.scm file is still run with its own Guile instance,
> in ./test-env, right? (This is important. It also means tests can run
> in parallel.)
>
> If this is OK, it LGTM.
I think it is OK :)
> However, I’m unsure if we should push it now, or after the release. On
> one hand, I’d rather avoid potentially disrupting changes like this
> now. On the other hand, since it makes it easier (and different) to
> report test failures, it’s nice.
>
> How confident are you? :-)
IMHO we should wait after the next release in order to make this test
driver more bullet proof.
Thanks for your comments!
--
Mathieu Lirzin
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] build: Add a scheme custom test driver using SRFI-64.
2015-11-01 19:48 ` Mathieu Lirzin
@ 2015-11-01 23:23 ` Mathieu Lirzin
2015-11-02 8:48 ` Ludovic Courtès
2015-11-02 8:59 ` Ludovic Courtès
1 sibling, 1 reply; 8+ messages in thread
From: Mathieu Lirzin @ 2015-11-01 23:23 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel
[-- Attachment #1: Type: text/plain, Size: 1646 bytes --]
Mathieu Lirzin <mthl@gnu.org> writes:
> ludo@gnu.org (Ludovic Courtès) writes:
>> Awesome! Are the “inner” tests displayed by default? Or is there some
>> environment variable to control that?
>
> Yes, each test case is displayed by default.
>
>> I’d prefer the default to display only file names, as is currently the
>> case (it’s more concise.)
>
> IMO This would be reasonable to have an option for making the output more
> concise since there are 563 test cases in Guix. this is not part of the
> test driver API specification, but it does not seem too hard to add an
> additional option to the test driver script like what the TAP driver is
> doing:
>
> https://www.gnu.org/software/automake/manual/automake.html#Use-TAP-with-the-Automake-test-harness
>
> I think ‘--brief’ could be a good name for this option. Default
> verbosity would specified by adding “AM_SCM_LOG_DRIVER= --brief=yes” in
^^^
AM_SCM_LOG_DRIVER_FLAGS
> Makefile.am, then the user can overide that with:
>
> make check -j4 TESTS=tests/rocket-science.scm SCM_LOG_DRIVER="--brief=no"
^^^
SCM_LOG_DRIVER_FLAGS
>
[...]
>> Nice. “Running the Test Suite” needs to be updated accordingly.
>
> Thanks for reminding me this. ;)
Here is an updated patch with the implementation of what is described
above. As a bonus There is a bug fix for the ‘--color-tests’ option and
a few docstring/comment changes.
--
Mathieu Lirzin
[-- Attachment #2: 0001-build-Add-a-scheme-custom-test-driver-using-SRFI-64.patch --]
[-- Type: text/x-diff, Size: 24071 bytes --]
From 8a1f52e08c8d33a33066271f0f39d6072baf9854 Mon Sep 17 00:00:00 2001
From: Mathieu Lirzin <mthl@gnu.org>
Date: Mon, 26 Oct 2015 23:47:24 +0100
Subject: [PATCH] build: Add a scheme custom test driver using SRFI-64.
This provides support for multiple scheme test cases in a unique file and
fixes the fragmentation of '.log' files.
* build-aux/test-driver.scm: New file.
* Makefile.am (SCM_LOG_DRIVER, AM_SCM_LOG_DRIVER_FLAGS): New variables.
(SCM_LOG_COMPILER, AM_SCM_LOG_FLAGS): Delete variables.
(AM_TESTS_ENVIRONMENT): Set GUILE_AUTO_COMPILE to 0.
* doc/guix.texi (Running the Test Suite): Reporting a bug does not
require to provide a specific '.log' file anymore.
* tests/base32.scm, tests/build-utils.scm, tests/builders.scm,
tests/challenge.scm, tests/containers.scm, tests/cpan.scm,
tests/cpio.scm, tests/cran.scm, tests/derivations.scm, tests/elpa.scm,
tests/file-systems.scm, tests/gem.scm, tests/gexp.scm,
tests/graph.scm, tests/gremlin.scm, tests/hackage.scm, tests/hash.scm,
tests/lint.scm, tests/monads.scm, tests/nar.scm, tests/packages.scm,
tests/pk-crypto.scm, tests/pki.scm, tests/profiles.scm,
tests/publish.scm, tests/pypi.scm, tests/records.scm,
tests/scripts.scm, tests/services.scm, tests/sets.scm, tests/size.scm,
tests/snix.scm, tests/store.scm, tests/substitute.scm,
tests/syscalls.scm, tests/ui.scm, tests/union.scm, tests/utils.scm:
Don't exit at the end of each test.
---
Makefile.am | 7 +-
build-aux/test-driver.scm | 174 ++++++++++++++++++++++++++++++++++++++++++++++
doc/guix.texi | 8 +--
tests/base32.scm | 3 -
tests/build-utils.scm | 3 -
tests/builders.scm | 3 -
tests/challenge.scm | 3 -
tests/containers.scm | 3 -
tests/cpan.scm | 3 -
tests/cpio.scm | 3 -
tests/cran.scm | 3 -
tests/derivations.scm | 3 -
tests/elpa.scm | 3 -
tests/file-systems.scm | 3 -
tests/gem.scm | 3 -
tests/gexp.scm | 3 -
tests/graph.scm | 3 -
tests/gremlin.scm | 3 -
tests/hackage.scm | 3 -
tests/hash.scm | 3 -
tests/lint.scm | 3 -
tests/monads.scm | 3 -
tests/nar.scm | 3 -
tests/packages.scm | 3 -
tests/pk-crypto.scm | 3 -
tests/pki.scm | 3 -
tests/profiles.scm | 3 -
tests/publish.scm | 3 -
tests/pypi.scm | 3 -
tests/records.scm | 3 -
tests/scripts.scm | 3 -
tests/services.scm | 3 -
tests/sets.scm | 3 -
tests/size.scm | 3 -
tests/snix.scm | 3 -
tests/store.scm | 3 -
tests/substitute.scm | 3 -
tests/syscalls.scm | 3 -
tests/ui.scm | 3 -
tests/union.scm | 3 -
tests/utils.scm | 3 -
41 files changed, 181 insertions(+), 122 deletions(-)
create mode 100644 build-aux/test-driver.scm
diff --git a/Makefile.am b/Makefile.am
index b2ee324..ce4cd53 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -268,10 +268,11 @@ TESTS = $(SCM_TESTS) $(SH_TESTS)
TEST_EXTENSIONS = .scm .sh
-AM_TESTS_ENVIRONMENT = abs_top_srcdir="$(abs_top_srcdir)"
+AM_TESTS_ENVIRONMENT = abs_top_srcdir="$(abs_top_srcdir)" GUILE_AUTO_COMPILE=0
-SCM_LOG_COMPILER = $(top_builddir)/test-env $(GUILE)
-AM_SCM_LOG_FLAGS = --no-auto-compile -L "$(top_srcdir)"
+SCM_LOG_DRIVER = $(top_builddir)/test-env $(GUILE) --no-auto-compile \
+ -e main $(top_srcdir)/build-aux/test-driver.scm 2>/dev/null
+AM_SCM_LOG_DRIVER_FLAGS = --brief=yes
SH_LOG_COMPILER = $(top_builddir)/test-env $(SHELL)
AM_SH_LOG_FLAGS = -x -e
diff --git a/build-aux/test-driver.scm b/build-aux/test-driver.scm
new file mode 100644
index 0000000..c982d86
--- /dev/null
+++ b/build-aux/test-driver.scm
@@ -0,0 +1,174 @@
+;;;; test-driver.scm --- SRFI-64 custom driver for GNU Automake test suites.
+
+(define script-version "2015-11-01.22") ;UTC
+
+;;; Copyright © 2015 Mathieu Lirzin <mthl@gnu.org>
+;;;
+;;; This program is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; This program is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+;;;; Commentary:
+;;;
+;;; This script provides a custom test driver for GNU Automake using the
+;;; SRFI-64 Scheme API for test suites.
+;;;
+;;;; Code:
+
+(use-modules (ice-9 getopt-long)
+ (ice-9 pretty-print)
+ (srfi srfi-26)
+ (srfi srfi-64))
+
+(define (show-help)
+ (display "Usage:
+ test-driver --test-name=NAME --log-file=PATH --trs-file=PATH
+ [--expect-failure={yes|no}] [--color-tests={yes|no}]
+ [--enable-hard-errors={yes|no}] [--brief={yes|no}}] [--]
+ TEST-SCRIPT [TEST-SCRIPT-ARGUMENTS]
+The '--test-name', '--log-file' and '--trs-file' options are mandatory.\n"))
+
+(define %options
+ '((test-name (value #t))
+ (log-file (value #t))
+ (trs-file (value #t))
+ (color-tests (value #t))
+ (expect-failure (value #t)) ;XXX: not implemented yet
+ (enable-hard-errors (value #t)) ;XXX: not implemented in SRFI-64
+ (brief (value #t))
+ (help (single-char #\h) (value #f))
+ (version (single-char #\V) (value #f))))
+
+(define (option->boolean options key)
+ "Return #t if the value associated with KEY in OPTIONS is \"yes\"."
+ (and=> (option-ref options key #f) (cut string=? <> "yes")))
+
+(define* (test-display field value #:optional (port (current-output-port)))
+ "Display \"FIELD: VALUE\\n\" on PORT."
+ (format port "~A: ~A\n" field value))
+
+(define* (result->string symbol #:key colorize)
+ "Return SYMBOL as an upper case string. Use colors when COLORIZE is #t."
+ (let ((result (string-upcase (symbol->string symbol))))
+ (if colorize
+ (string-append (case symbol
+ ((pass) "^[[0;32m") ;green
+ ((xfail) "^[[1;32m") ;light green
+ ((skip) "^[[1;34m") ;blue
+ ((fail xpass) "^[[0;31m") ;red
+ ((error) "^[[0;35m")) ;magenta
+ result
+ "^[[m") ;no color
+ result)))
+
+(define* (test-runner-gnu test-name #:key color-tests brief out-port trs-port)
+ "Return an custom SRFI-64 test runner. TEST-NAME is a string specifying the
+file name of the current the test. COLOR-TESTS and BRIEF are booleans.
+OUT-PORT and TRS-PORT must be output ports. The current output port is
+supposed to be redirected to a '.log' file."
+
+ (define (test-on-test-begin-gnu runner)
+ "Procedure called at the start of an individual test-case, before the test
+expression (and expected value) are evaluated."
+ (let ((results (test-result-alist runner)))
+ (test-display "test-name" (assq-ref results 'test-name))
+ (test-display "location"
+ (string-append
+ (assq-ref results 'source-file) ":"
+ (number->string (assq-ref results 'source-line))))
+ (display "source:\n")
+ (pretty-print (assq-ref results 'source-form) #:per-line-prefix "+ ")))
+
+ (define (test-on-test-end-gnu runner)
+ "Procedure called at the end of an individual testcase, when the result of
+the test is available."
+ (let ((results (test-result-alist runner)))
+ (unless brief
+ ;; Display the result of each test case on the console.
+ (test-display
+ (result->string (test-result-kind runner) #:colorize color-tests)
+ (string-append test-name " - " (test-runner-test-name runner))
+ out-port))
+ (when (assq 'actual-value results)
+ (test-display "value" (assq-ref results 'actual-value)))
+ (test-display "result" (result->string (assq-ref results 'result-kind)))
+ (newline)
+ (test-display ":test-result"
+ (string-append (result->string (test-result-kind runner))
+ " " (test-runner-test-name runner))
+ trs-port)))
+
+ (define (test-on-group-end-gnu runner)
+ "Procedure called by a `test-end', including at the end of a test-group."
+ (let ((fail (or (positive? (test-runner-fail-count runner))
+ (positive? (test-runner-xpass-count runner))))
+ (skip (or (positive? (test-runner-skip-count runner))
+ (positive? (test-runner-xfail-count runner)))))
+ ;; XXX: Since multiple tests share one '.trs' file, we need to choose
+ ;; what to report in the "global" results which would require more
+ ;; refinements.
+ (test-display ":global-test-result" (if fail "FAIL" "PASS") trs-port)
+ (test-display ":recheck" (if fail "yes" "no") trs-port)
+ (test-display ":copy-in-global-log"
+ (if (or fail skip) "yes" "no")
+ trs-port)
+ (when brief
+ ;; Display the global test group result on the console.
+ (test-display (result->string (if fail 'fail 'pass)
+ #:colorize color-tests)
+ test-name
+ out-port))
+ #f))
+
+ (let ((runner (test-runner-null)))
+ (test-runner-on-test-begin! runner test-on-test-begin-gnu)
+ (test-runner-on-test-end! runner test-on-test-end-gnu)
+ (test-runner-on-group-end! runner test-on-group-end-gnu)
+ (test-runner-on-bad-end-name! runner test-on-bad-end-name-simple)
+ runner))
+
+\f
+;;;
+;;; Entry point.
+;;;
+
+(define (main . args)
+ (let ((opts (getopt-long (command-line) %options)))
+ (when (option-ref opts 'help #f)
+ (show-help)
+ (exit 0))
+ (when (option-ref opts 'version #f)
+ (format #t "test-driver.scm ~A" script-version)
+ (exit 0))
+ (let ((trs (open-output-file (option-ref opts 'trs-file ""))))
+ (test-runner-current
+ (test-runner-gnu (option-ref opts 'test-name #f)
+ #:color-tests (option->boolean opts 'color-tests)
+ #:brief (option->boolean opts 'brief)
+ #:out-port (current-output-port)
+ #:trs-port trs))
+ (with-output-to-file (option-ref opts 'log-file "")
+ (lambda ()
+ (set-current-error-port (current-output-port))
+ (load-from-path (option-ref opts 'test-name #f))))
+ (close-port trs)
+ (exit 0))))
+
+;;; Local Variables:
+;;; eval: (add-hook 'write-file-functions 'time-stamp)
+;;; time-stamp-start: "(define script-version \""
+;;; time-stamp-format: "%:y-%02m-%02d.%02H"
+;;; time-stamp-time-zone: "UTC"
+;;; time-stamp-end: "\") ;UTC"
+;;; End:
+
+;;;; test-driver.scm ends here.
diff --git a/doc/guix.texi b/doc/guix.texi
index e8b79ec..b916a84 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -479,11 +479,9 @@ that is created for test purposes will already have various things in
cache.
Upon failure, please email @email{bug-guix@@gnu.org} and attach the
-@file{test-suite.log} file. When @file{tests/@var{something}.scm}
-fails, please also attach the @file{@var{something}.log} file available
-in the top-level build directory. Please specify the Guix version being
-used as well as version numbers of the dependencies
-(@pxref{Requirements}) in your message.
+@file{test-suite.log} file. Please specify the Guix version being used
+as well as version numbers of the dependencies (@pxref{Requirements}) in
+your message.
@node Setting Up the Daemon
@section Setting Up the Daemon
diff --git a/tests/base32.scm b/tests/base32.scm
index dcd926f..194f8da 100644
--- a/tests/base32.scm
+++ b/tests/base32.scm
@@ -101,6 +101,3 @@
l))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/build-utils.scm b/tests/build-utils.scm
index b0a4d15..cc96738 100644
--- a/tests/build-utils.scm
+++ b/tests/build-utils.scm
@@ -141,6 +141,3 @@
(equal? str "hello world\n")))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/builders.scm b/tests/builders.scm
index a7c3e42..bb9e0fa 100644
--- a/tests/builders.scm
+++ b/tests/builders.scm
@@ -113,6 +113,3 @@
(file-exists? (string-append out "/bin/hello")))))
(test-end "builders")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/challenge.scm b/tests/challenge.scm
index e53cacd..9505042 100644
--- a/tests/challenge.scm
+++ b/tests/challenge.scm
@@ -106,9 +106,6 @@
(test-end)
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'with-derivation-narinfo* 'scheme-indent-function 2)
;;; End:
diff --git a/tests/containers.scm b/tests/containers.scm
index 0ba8149..ccbcfa5 100644
--- a/tests/containers.scm
+++ b/tests/containers.scm
@@ -141,6 +141,3 @@
(zero? result)))))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/cpan.scm b/tests/cpan.scm
index 2f95135..b92a78c 100644
--- a/tests/cpan.scm
+++ b/tests/cpan.scm
@@ -94,6 +94,3 @@
(pk 'fail x #f)))))
(test-end "cpan")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/cpio.scm b/tests/cpio.scm
index cf65f98..516de06 100644
--- a/tests/cpio.scm
+++ b/tests/cpio.scm
@@ -79,6 +79,3 @@
(stat:size (stat file))))))))))))))
(test-end "cpio")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/cran.scm b/tests/cran.scm
index ba5699a..135f308 100644
--- a/tests/cran.scm
+++ b/tests/cran.scm
@@ -172,6 +172,3 @@
(pk 'fail x #f))))))
(test-end "cran")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/derivations.scm b/tests/derivations.scm
index 9fc96c7..2e10b70 100644
--- a/tests/derivations.scm
+++ b/tests/derivations.scm
@@ -946,6 +946,3 @@
(call-with-input-file out get-string-all))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/elpa.scm b/tests/elpa.scm
index e8adde3..46c6ac2 100644
--- a/tests/elpa.scm
+++ b/tests/elpa.scm
@@ -104,6 +104,3 @@ information about package NAME. (Function 'elpa-package-info'.)"
(eval-test-with-elpa "auctex"))
(test-end "elpa")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/file-systems.scm b/tests/file-systems.scm
index c36509b..aed27e8 100644
--- a/tests/file-systems.scm
+++ b/tests/file-systems.scm
@@ -51,6 +51,3 @@
(equal? form '(uuid "foobar"))))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/gem.scm b/tests/gem.scm
index 9efbda3..43a6c53 100644
--- a/tests/gem.scm
+++ b/tests/gem.scm
@@ -77,6 +77,3 @@
(pk 'fail x #f)))))
(test-end "gem")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/gexp.scm b/tests/gexp.scm
index 4860a8e..ca7ad9a 100644
--- a/tests/gexp.scm
+++ b/tests/gexp.scm
@@ -754,9 +754,6 @@
(test-end "gexp")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;; Local Variables:
;; eval: (put 'test-assertm 'scheme-indent-function 1)
;; End:
diff --git a/tests/graph.scm b/tests/graph.scm
index f454b06..4de1702 100644
--- a/tests/graph.scm
+++ b/tests/graph.scm
@@ -188,6 +188,3 @@ edges."
(equal? edges `((,out ,txt)))))))))))
(test-end "graph")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/gremlin.scm b/tests/gremlin.scm
index dc9f78c..2885554 100644
--- a/tests/gremlin.scm
+++ b/tests/gremlin.scm
@@ -64,6 +64,3 @@
"ORIGIN/foo")))
(test-end "gremlin")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/hackage.scm b/tests/hackage.scm
index 229bee3..969266b 100644
--- a/tests/hackage.scm
+++ b/tests/hackage.scm
@@ -137,6 +137,3 @@ library
(x (pk 'fail x #f))))
(test-end "hackage")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/hash.scm b/tests/hash.scm
index 9bcd694..8039549 100644
--- a/tests/hash.scm
+++ b/tests/hash.scm
@@ -126,6 +126,3 @@ In Guile <= 2.0.9, CBIPs were always fully buffered, so the
(list hello hash world))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/lint.scm b/tests/lint.scm
index 3f14956..db29915 100644
--- a/tests/lint.scm
+++ b/tests/lint.scm
@@ -550,9 +550,6 @@ requests."
(test-end "lint")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;; Local Variables:
;; eval: (put 'with-http-server 'scheme-indent-function 1)
;; eval: (put 'with-warnings 'scheme-indent-function 0)
diff --git a/tests/monads.scm b/tests/monads.scm
index 62a07a2..60d41ae 100644
--- a/tests/monads.scm
+++ b/tests/monads.scm
@@ -254,6 +254,3 @@
'())))
(test-end "monads")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/nar.scm b/tests/nar.scm
index e24a638..9796980 100644
--- a/tests/nar.scm
+++ b/tests/nar.scm
@@ -320,9 +320,6 @@
(test-end "nar")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'with-file-tree 'scheme-indent-function 2)
;;; End:
diff --git a/tests/packages.scm b/tests/packages.scm
index b28ae0b..b037a4f 100644
--- a/tests/packages.scm
+++ b/tests/packages.scm
@@ -727,9 +727,6 @@
(test-end "packages")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'dummy-package 'scheme-indent-function 1)
;;; End:
diff --git a/tests/pk-crypto.scm b/tests/pk-crypto.scm
index f5008f3..5024a15 100644
--- a/tests/pk-crypto.scm
+++ b/tests/pk-crypto.scm
@@ -287,6 +287,3 @@
(canonical-sexp->sexp (sexp->canonical-sexp sexp))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/pki.scm b/tests/pki.scm
index 51f2119..876ad98 100644
--- a/tests/pki.scm
+++ b/tests/pki.scm
@@ -119,6 +119,3 @@
(corrupt-signature 'c))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/profiles.scm b/tests/profiles.scm
index cc9a822..360ce27 100644
--- a/tests/profiles.scm
+++ b/tests/profiles.scm
@@ -307,9 +307,6 @@
(test-end "profiles")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'dummy-package 'scheme-indent-function 1)
;;; End:
diff --git a/tests/publish.scm b/tests/publish.scm
index 4d72fdc..9a0bed5 100644
--- a/tests/publish.scm
+++ b/tests/publish.scm
@@ -109,6 +109,3 @@ References: ~a~%"
(call-with-input-file temp read-string))))
(test-end "publish")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/pypi.scm b/tests/pypi.scm
index c772474..4f7beb1 100644
--- a/tests/pypi.scm
+++ b/tests/pypi.scm
@@ -106,6 +106,3 @@ baz > 13.37")
(pk 'fail x #f)))))
(test-end "pypi")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/records.scm b/tests/records.scm
index 800ed03..c6f85d4 100644
--- a/tests/records.scm
+++ b/tests/records.scm
@@ -305,6 +305,3 @@ Description: 1st line,
'("a")))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/scripts.scm b/tests/scripts.scm
index 3bf41ae..3901710 100644
--- a/tests/scripts.scm
+++ b/tests/scripts.scm
@@ -64,9 +64,6 @@
(test-end "scripts")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'with-environment-variable 'scheme-indent-function 2)
;;; End:
diff --git a/tests/services.scm b/tests/services.scm
index 7d2e31b..804d6c9 100644
--- a/tests/services.scm
+++ b/tests/services.scm
@@ -115,6 +115,3 @@
(null? (e s3)))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/sets.scm b/tests/sets.scm
index 0a89591..cdb7efe 100644
--- a/tests/sets.scm
+++ b/tests/sets.scm
@@ -47,6 +47,3 @@
(set-contains? s3 b))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/size.scm b/tests/size.scm
index a110604..fcd5902 100644
--- a/tests/size.scm
+++ b/tests/size.scm
@@ -88,9 +88,6 @@
(test-end "size")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'match* 'scheme-indent-function 1)
;;; End:
diff --git a/tests/snix.scm b/tests/snix.scm
index a66b2c7..4c31e33 100644
--- a/tests/snix.scm
+++ b/tests/snix.scm
@@ -71,6 +71,3 @@
(pk 'fail x #f))))
(test-end "snix")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/store.scm b/tests/store.scm
index 60d1085..9906390 100644
--- a/tests/store.scm
+++ b/tests/store.scm
@@ -773,6 +773,3 @@
(call-with-output-string (cut write-file item <>))))))))
(test-end "store")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/substitute.scm b/tests/substitute.scm
index 9d907e7..69b272f 100644
--- a/tests/substitute.scm
+++ b/tests/substitute.scm
@@ -275,9 +275,6 @@ a file for NARINFO."
(test-end "substitute")
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
-
;;; Local Variables:
;;; eval: (put 'with-narinfo 'scheme-indent-function 1)
;;; eval: (put 'test-quit 'scheme-indent-function 2)
diff --git a/tests/syscalls.scm b/tests/syscalls.scm
index 86783b9..06870a7 100644
--- a/tests/syscalls.scm
+++ b/tests/syscalls.scm
@@ -236,6 +236,3 @@
(lo (interface-address lo)))))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/ui.scm b/tests/ui.scm
index bd4c907..133982a 100644
--- a/tests/ui.scm
+++ b/tests/ui.scm
@@ -250,6 +250,3 @@ Second line" 24))
(show-manifest-transaction store m t)))))))))
(test-end "ui")
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/union.scm b/tests/union.scm
index 22ba67c..cccf397 100644
--- a/tests/union.scm
+++ b/tests/union.scm
@@ -125,6 +125,3 @@
(eq? 'directory (stat:type (lstat "bin"))))))))
(test-end)
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
diff --git a/tests/utils.scm b/tests/utils.scm
index b65d6d2..1e763a2 100644
--- a/tests/utils.scm
+++ b/tests/utils.scm
@@ -321,6 +321,3 @@
(test-end)
(false-if-exception (delete-file temp-file))
-
-\f
-(exit (= (test-runner-fail-count (test-runner-current)) 0))
--
2.6.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH] build: Add a scheme custom test driver using SRFI-64.
2015-11-01 23:23 ` Mathieu Lirzin
@ 2015-11-02 8:48 ` Ludovic Courtès
0 siblings, 0 replies; 8+ messages in thread
From: Ludovic Courtès @ 2015-11-02 8:48 UTC (permalink / raw)
To: Mathieu Lirzin; +Cc: guix-devel
Mathieu Lirzin <mthl@gnu.org> skribis:
> Mathieu Lirzin <mthl@gnu.org> writes:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>>> Awesome! Are the “inner” tests displayed by default? Or is there some
>>> environment variable to control that?
>>
>> Yes, each test case is displayed by default.
>>
>>> I’d prefer the default to display only file names, as is currently the
>>> case (it’s more concise.)
>>
>> IMO This would be reasonable to have an option for making the output more
>> concise since there are 563 test cases in Guix. this is not part of the
>> test driver API specification, but it does not seem too hard to add an
>> additional option to the test driver script like what the TAP driver is
>> doing:
>>
>> https://www.gnu.org/software/automake/manual/automake.html#Use-TAP-with-the-Automake-test-harness
>>
>> I think ‘--brief’ could be a good name for this option. Default
>> verbosity would specified by adding “AM_SCM_LOG_DRIVER= --brief=yes” in
> ^^^
> AM_SCM_LOG_DRIVER_FLAGS
Great.
> From 8a1f52e08c8d33a33066271f0f39d6072baf9854 Mon Sep 17 00:00:00 2001
> From: Mathieu Lirzin <mthl@gnu.org>
> Date: Mon, 26 Oct 2015 23:47:24 +0100
> Subject: [PATCH] build: Add a scheme custom test driver using SRFI-64.
>
> This provides support for multiple scheme test cases in a unique file and
> fixes the fragmentation of '.log' files.
>
> * build-aux/test-driver.scm: New file.
> * Makefile.am (SCM_LOG_DRIVER, AM_SCM_LOG_DRIVER_FLAGS): New variables.
> (SCM_LOG_COMPILER, AM_SCM_LOG_FLAGS): Delete variables.
> (AM_TESTS_ENVIRONMENT): Set GUILE_AUTO_COMPILE to 0.
> * doc/guix.texi (Running the Test Suite): Reporting a bug does not
> require to provide a specific '.log' file anymore.
> * tests/base32.scm, tests/build-utils.scm, tests/builders.scm,
> tests/challenge.scm, tests/containers.scm, tests/cpan.scm,
> tests/cpio.scm, tests/cran.scm, tests/derivations.scm, tests/elpa.scm,
> tests/file-systems.scm, tests/gem.scm, tests/gexp.scm,
> tests/graph.scm, tests/gremlin.scm, tests/hackage.scm, tests/hash.scm,
> tests/lint.scm, tests/monads.scm, tests/nar.scm, tests/packages.scm,
> tests/pk-crypto.scm, tests/pki.scm, tests/profiles.scm,
> tests/publish.scm, tests/pypi.scm, tests/records.scm,
> tests/scripts.scm, tests/services.scm, tests/sets.scm, tests/size.scm,
> tests/snix.scm, tests/store.scm, tests/substitute.scm,
> tests/syscalls.scm, tests/ui.scm, tests/union.scm, tests/utils.scm:
> Don't exit at the end of each test.
Looks nice.
If you want you can push it to a wip- branch, and we’ll apply it after
the release. Or you can keep it locally for later.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] build: Add a scheme custom test driver using SRFI-64.
2015-11-01 19:48 ` Mathieu Lirzin
2015-11-01 23:23 ` Mathieu Lirzin
@ 2015-11-02 8:59 ` Ludovic Courtès
2015-11-05 21:44 ` Mathieu Lirzin
1 sibling, 1 reply; 8+ messages in thread
From: Ludovic Courtès @ 2015-11-02 8:59 UTC (permalink / raw)
To: Mathieu Lirzin; +Cc: guix-devel
Mathieu Lirzin <mthl@gnu.org> skribis:
> ludo@gnu.org (Ludovic Courtès) writes:
[...]
>>> One caveat is that ‘tests/cpio.scm‘ is now failing.
>>
>> Why is that? Does it relate to this change?
>
> I didn't try to debug the problem but it has appeared when I started
> messing with redirection of the output/error ports. Here is the failure
> output.
>
> test-name: bit-identical to GNU cpio's output
> location: /home/mthl/src/gnu/guix/tests/cpio.scm:49
Is GNU cpio available in $PATH? Which version is that? Could you run
this test in ‘master’ to see if the problem shows up?
>>> Since this script is not intented to be exclusively used by Guix, I have
>>> used a generic copyright notice. I guess Guix is the best place to
>>> challenge and improve it, but IMO it will be better hosted elsewhere
>>> like in Gnulib. Opinions?
>>
>> I think we could start using it and testing it for a while in Guix, and
>> eventually submit it for inclusion in Gnulib once we are more confident.
>
> So you recommend to add ”this file is part of GNU Guix” and use “GNU
> Guix” instead of “this program” for now?
I think you can leave “this program” so nothing will need to be changed
when you move it elsewhere.
>> However, I’m unsure if we should push it now, or after the release. On
>> one hand, I’d rather avoid potentially disrupting changes like this
>> now. On the other hand, since it makes it easier (and different) to
>> report test failures, it’s nice.
>>
>> How confident are you? :-)
>
> IMHO we should wait after the next release in order to make this test
> driver more bullet proof.
Sounds reasonable.
Thanks!
Ludo’.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] build: Add a scheme custom test driver using SRFI-64.
2015-11-02 8:59 ` Ludovic Courtès
@ 2015-11-05 21:44 ` Mathieu Lirzin
2015-11-06 21:20 ` Ludovic Courtès
0 siblings, 1 reply; 8+ messages in thread
From: Mathieu Lirzin @ 2015-11-05 21:44 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel
Hi,
As suggested, I've created a “wip-check” for this patch.
ludo@gnu.org (Ludovic Courtès) writes:
> Mathieu Lirzin <mthl@gnu.org> skribis:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>>>> One caveat is that ‘tests/cpio.scm‘ is now failing.
>>>
>>> Why is that? Does it relate to this change?
>>
>> I didn't try to debug the problem but it has appeared when I started
>> messing with redirection of the output/error ports. Here is the failure
>> output.
>>
>> test-name: bit-identical to GNU cpio's output
>> location: /home/mthl/src/gnu/guix/tests/cpio.scm:49
>
> Is GNU cpio available in $PATH? Which version is that? Could you run
> this test in ‘master’ to see if the problem shows up?
Yeah the test pass in master.
$ which cpio
/bin/cpio
$cpio --version
cpio (GNU cpio) 2.11
Copyright (C) 2010 Free Software Foundation, Inc.
[...]
Thanks for looking into it,
--
Mathieu Lirzin
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] build: Add a scheme custom test driver using SRFI-64.
2015-11-05 21:44 ` Mathieu Lirzin
@ 2015-11-06 21:20 ` Ludovic Courtès
0 siblings, 0 replies; 8+ messages in thread
From: Ludovic Courtès @ 2015-11-06 21:20 UTC (permalink / raw)
To: Mathieu Lirzin; +Cc: guix-devel
Mathieu Lirzin <mthl@gnu.org> skribis:
> As suggested, I've created a “wip-check” for this patch.
Cool.
[...]
>> Is GNU cpio available in $PATH? Which version is that? Could you run
>> this test in ‘master’ to see if the problem shows up?
>
> Yeah the test pass in master.
So I suppose I can leave the burden on you to find out what’s going on?
:-)
Ludo’.
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2015-11-06 21:20 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-11-01 2:59 [PATCH] build: Add a scheme custom test driver using SRFI-64 Mathieu Lirzin
2015-11-01 17:13 ` Ludovic Courtès
2015-11-01 19:48 ` Mathieu Lirzin
2015-11-01 23:23 ` Mathieu Lirzin
2015-11-02 8:48 ` Ludovic Courtès
2015-11-02 8:59 ` Ludovic Courtès
2015-11-05 21:44 ` Mathieu Lirzin
2015-11-06 21:20 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).