bug#45948: [PATCH 0/5] Improvements to the Automake SRFI 64 test driver.

[Maxim Cournoyer <maxim.cournoyer@gmail.com>]

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

[...]

> I never felt the need for this since most individual files run quickly
> enough (and those that don’t should be optimized…), but it can be
> useful.

What triggered it for me was trying to iterate using tests added to the
tests/packages.scm test module:

--8<---------------cut here---------------start------------->8---
$ time make check TESTS=tests/packages.scm VERBOSE=1
SCM_LOG_DRIVER_FLAGS="--brief=no"

time make check TESTS=tests/packages.scm VERBOSE=1 SCM_LOG_DRIVER_FLAGS="--brief=no"
[...]
PASS: tests/packages.scm - package-patched-vulnerabilities
PASS: tests/packages.scm - fold-packages
PASS: tests/packages.scm - fold-packages, hidden package
[... time passes ...]
PASS: tests/packages.scm - fold-available-packages with/without cache
[...]
PASS: tests/packages.scm - find-package-locations with cache
PASS: tests/packages.scm - specification->location
============================================================================
Testsuite summary for GNU Guix UNKNOWN
============================================================================
# TOTAL: 100
# PASS:  100
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
[...]
real    0m46.172s
user    1m4.885s
sys     0m0.376s
--8<---------------cut here---------------end--------------->8---

That's on the fastest machine I have access to (on my vintage desktop,
it took nearly 4 minutes).  The slowest test seems to be
'fold-available-packages with/without cache'.

Now with the new select, one can do:

--8<---------------cut here---------------start------------->8---
$ time make check TESTS=tests/packages.scm VERBOSE=1 SCM_LOG_DRIVER_FLAGS="--brief=no --select='bag->derivation' --errors-only=yes"
[...]
PASS: tests/packages.scm - bag->derivation
PASS: tests/packages.scm - bag->derivation, cross-compilation
============================================================================
Testsuite summary for GNU Guix UNKNOWN
============================================================================
# TOTAL: 100
# PASS:  2
# SKIP:  98
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
[...]
real    0m1.569s
user    0m2.382s
sys     0m0.154s
--8<---------------cut here---------------end--------------->8---

1.6 s; better than 46 s!

We can also check the time the suspected slow test took:

$ time make check TESTS=tests/packages.scm SCM_LOG_DRIVER_FLAGS="--select='fold-available-packages with/without cache'"
[...]
PASS: tests/packages.scm - fold-available-packages with/without cache
============================================================================
Testsuite summary for GNU Guix UNKNOWN
============================================================================
# TOTAL: 100
# PASS:  1
# SKIP:  99
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
[...]
real    0m36.627s
user    0m45.731s
sys     0m0.264s

So yes, this is the most expensive test of tests/packages.scm.

>> +;;; XXX: test-match-all is a syntax, which isn't convenient to use with a list
>> +;;; of test specifiers computed at run time.
>> +(define %test-match-all (@@ (srfi srfi-64) %test-match-all))
>
> Since this is an internal procedure in Guile that could vanish anytime,
> I recommend copying it here (it’s just 9 lines).

Done!

>> +The underlying SRFI 64 custom Automake test driver used for the 'check'
>> +test suite (located at @file{build-aux/test-driver.scm}) also allows
>
> Maybe shorten to “The underlying test driver (located at
> @file{build-aux/test-driver.scm}) also allows”.

I see value in explicitly stating what it is, as it took me some effort
to be able to answer that question when I started looking at it (the
test driver).

I've now pushed this series to master; thank you for the review!

Maxim