Re: [PATCH 0/5] Automake custom test driver using SRFI-64.

[ludo@gnu.org (Ludovic Courtès)]

[-- Attachment #1: Type: text/plain, Size: 1114 bytes --]

Mathieu Lirzin <mthl@gnu.org> skribis:

> Mathieu Lirzin (5):
>   tests: Silence %cpio-program.
>   build: Add a Guile custom test driver using SRFI-64.
>   tests: Silence guix-daemon.
>   tests: Silence %have-nix-hash?.
>   tests: Silence tar.

On closer inspection, this is awesome.  :-)

I withdraw my comment about losing some of the “noise” since all we’re
losing is pointless info, and we still have the “useful noise” (like
‘pk’ output) in those nicely structured log files.  So I’m happy!

‘make recheck’ still works; it still reruns the whole file (even if only
one test within the file failed), which is surprising but OK and
probably unavoidable given how SRFI-64 works.

The only things I would change is the attached patch: the change to
‘test-env’ removes the “Terminated” messages that Eric mentioned, the
rest improves a docstring in ‘test-driver.scm’ and uses the common
convention for Boolean variables.

If you can integrate something along these lines, then please merge!

I hereby dub you Automake Wizard.

Thank you!

Ludo’.


[-- Attachment #2: Type: text/x-patch, Size: 4562 bytes --]

diff --git a/build-aux/test-driver.scm b/build-aux/test-driver.scm
index 4439b21..467741f 100644
--- a/build-aux/test-driver.scm
+++ b/build-aux/test-driver.scm
@@ -1,6 +1,6 @@
 ;;;; test-driver.scm - SRFI-64 custom driver for GNU Automake test suites
 
-(define script-version "2016-03-28.13") ;UTC
+(define script-version "2016-04-01.21") ;UTC
 
 ;;; Copyright © 2015, 2016 Mathieu Lirzin <mthl@gnu.org>
 ;;;
@@ -30,8 +30,7 @@
              (srfi srfi-64))
 
 (define (show-help)
-  (display "Usage:
-  test-driver --test-name=NAME --log-file=PATH --trs-file=PATH
+  (display "Usage: test-driver --test-name=NAME --log-file=PATH --trs-file=PATH
               [--expect-failure={yes|no}] [--color-tests={yes|no}]
               [--enable-hard-errors={yes|no}] [--brief={yes|no}}] [--]
               TEST-SCRIPT [TEST-SCRIPT-ARGUMENTS]
@@ -75,11 +74,11 @@ The '--test-name', '--log-file' and '--trs-file' options are mandatory.\n"))
                        "[m")          ;no color
         result)))
 
-(define* (test-runner-gnu test-name #:key color-tests brief out-port trs-port)
+(define* (test-runner-gnu test-name #:key color? brief? out-port trs-port)
   "Return an custom SRFI-64 test runner.  TEST-NAME is a string specifying the
-file name of the current the test.  COLOR-TESTS and BRIEF are booleans.
-OUT-PORT and TRS-PORT must be output ports.  The current output port is
-supposed to be redirected to a '.log' file."
+file name of the current the test.  COLOR? specifies whether to use colors,
+and BRIEF?, well, you know.  OUT-PORT and TRS-PORT must be output ports.  The
+current output port is supposed to be redirected to a '.log' file."
 
   (define (test-on-test-begin-gnu runner)
     ;; Procedure called at the start of an individual test-case, before the
@@ -97,10 +96,10 @@ supposed to be redirected to a '.log' file."
     (let* ((results (test-result-alist runner))
            (result? (cut assq <> results))
            (result  (cut assq-ref results <>)))
-      (unless brief
+      (unless brief?
         ;; Display the result of each test case on the console.
         (test-display
-         (result->string (test-result-kind runner) #:colorize color-tests)
+         (result->string (test-result-kind runner) #:colorize color?)
          (string-append test-name " - " (test-runner-test-name runner))
          out-port))
       (when (result? 'expected-value)
@@ -134,10 +133,10 @@ supposed to be redirected to a '.log' file."
       (test-display ":copy-in-global-log"
                     (if (or fail skip) "yes" "no")
                     trs-port)
-      (when brief
+      (when brief?
         ;; Display the global test group result on the console.
         (test-display (result->string (if fail 'fail (if skip 'skip 'pass))
-                                      #:colorize color-tests)
+                                      #:colorize color?)
                       test-name
                       out-port))
       #f))
@@ -167,8 +166,8 @@ supposed to be redirected to a '.log' file."
         (setvbuf out _IOLBF)
         (test-with-runner
             (test-runner-gnu (option 'test-name #f)
-                             #:color-tests (option->boolean opts 'color-tests)
-                             #:brief (option->boolean opts 'brief)
+                             #:color? (option->boolean opts 'color-tests)
+                             #:brief? (option->boolean opts 'brief)
                              #:out-port out #:trs-port trs)
           (parameterize ((current-output-port log)
                          (current-error-port log)
diff --git a/test-env.in b/test-env.in
index 040175a..2c71d58 100644
--- a/test-env.in
+++ b/test-env.in
@@ -32,6 +32,10 @@ unset CDPATH
 
 if [ -x "@abs_top_builddir@/guix-daemon" ]
 then
+    # Silence the daemon's output, which is often useless, as well as that of
+    # Bash (such as "Terminated" messages when 'guix-daemon' is killed.)
+    exec 2> /dev/null
+
     NIX_STORE_DIR="@GUIX_TEST_ROOT@/store"
 
     # Do that because store.scm calls `canonicalize-path' on it.
@@ -97,7 +101,7 @@ then
     # unavailable, for instance if we're not running as root.
     "@abs_top_builddir@/pre-inst-env"				\
 	"@abs_top_builddir@/guix-daemon" --disable-chroot	\
-	--substitute-urls="$GUIX_BINARY_SUBSTITUTE_URL" 2>/dev/null &
+	--substitute-urls="$GUIX_BINARY_SUBSTITUTE_URL" &
 
     daemon_pid=$!
     trap "kill $daemon_pid ; rm -rf $NIX_STATE_DIR" EXIT