Re: [PATCH 0/5] Automake custom test driver using SRFI-64.

[Mathieu Lirzin <mthl@gnu.org>]

Eric Bavier <ericbavier@openmailbox.org> writes:

> In addition to the test status, I get messages like this when running
> `make check`:
>
> -----
> ;;; note: source file /home/bavier/projects/guix/srfi/srfi-64.scm
> ;;;       newer than compiled /gnu/store/4r7iv7lz7vbcvzdjilqpiflzkh8nff3b-profile/lib/guile/2.0/ccache/srfi/srfi-64.go
> PASS: tests/elpa.scm
> ./test-env: line 1: 19815 Terminated              "/home/bavier/projects/guix/build/master/pre-inst-env" "/home/bavier/projects/guix/build/master/guix-daemon" --disable-chroot --substitute-urls="$GUIX_BINARY_SUBSTITUTE_URL" 2> /dev/null
> -----
>
> It seems that we don't compile any of the srfi/srfi-* source?

If you have a recent Guile version (>= 2.0.9) SRFI-64 is already
included and should be used without falling back to Guix embedded copy.
I have only tested with Guile 2.0.11 (from Debian) so I guess this
problem is appearing when using Guile from Guix.

Maybe 'Makefile.am' should include something like:

--8<---------------cut here---------------start------------->8---
if !HAVE_SRFI_64
GOBJECTS += srfi/srfi-64.go srfi/srfi-64.upstream.go
endif
--8<---------------cut here---------------end--------------->8---

with some m4 magic to define AM_CONDITIONAL([HAVE_SRFI_64], ... ) when
srfi-64 is found in Guile.  :)

> I don't recall seeing the "Terminated" messages before (anyone else
> see this?), but this patch to test-env.in seems to help:
>
> --- a/test-env.in
> +++ b/test-env.in
> @@ -100,7 +100,8 @@ then
>         --substitute-urls="$GUIX_BINARY_SUBSTITUTE_URL" 2>/dev/null &
>
>      daemon_pid=$!
> -    trap "kill $daemon_pid ; rm -rf $NIX_STATE_DIR" EXIT
> +    trap "kill $daemon_pid ; wait $daemon_pid 2>/dev/null ;
> +          rm -rf $NIX_STATE_DIR" EXIT
>  fi

I don't have the "terminated" message on my system.  Does it appear for
all the tests?  If not, can you identify what tests are creating those
messages? (by launching ‘make check TESTS="tests/suspect.scm"’).

> The result output seems more informative.  The counts include
> individual tests within each tests/*.scm, correct?

Yes, every individual test case is included in the summary.

> ============================================================================
> Testsuite summary for GNU Guix 0.9.1
> ============================================================================
> # TOTAL: 677
> # PASS:  671
> # SKIP:  6
> # XFAIL: 0
> # FAIL:  0
> # XPASS: 0
> # ERROR: 0
> ============================================================================
>
> I notice in the test-suite.log that logs are included from tests that
> are skipped.  This seems appropriate, but it appears to also include
> PASS output from other tests in that suite (e.g. all of tests/gexp.log
> is included since two of the 49 tests are skipp). Is it possible to
> include output from only the skipped tests?  I'm not familiar for the
> trs spec.

It is not possible unfortunately.  The '.trs' files have only a
':copy-in-global-log:' field which defines if the corresponding '.log'
file is copied verbatim in test-suite.log or not.

> Thanks for this work.  I had considered reworking the test suites with
> Autoconf's AutoTest for better unity, but I think what you've done here
> is quite a bit more elegant with less disruption.

I should learn how to use AutoTest.  Thanks for your review and your
kind words.

-- 
Mathieu Lirzin