Failing fast in SRFI-64 test groups

Failing fast in SRFI-64 test groups

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 349 bytes --]

Hey,

If each test in a group is effectively dependent on all the tests
before it, then it would be useful to abort the group if any tests
fail. I had a read through the docs, and a quick look at the code, but
didn't spot a way of doing this (short of implementing a custom test
runner). Is if possible to do this easily?

Thanks,

Chris

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: Failing fast in SRFI-64 test groups

[Taylan Ulrich Bayırlı/Kammer]

Christopher Baines <mail@cbaines.net> writes:

> Hey,
>
> If each test in a group is effectively dependent on all the tests
> before it, then it would be useful to abort the group if any tests
> fail. I had a read through the docs, and a quick look at the code, but
> didn't spot a way of doing this (short of implementing a custom test
> runner). Is if possible to do this easily?
>
> Thanks,
>
> Chris

You could define a test runner with a custom on-test-end procedure that
throws an exception if the result was a failure, and wrap all your
test-group expressions in a corresponding exception handler, possibly
with a macro to cut some boilerplate.

    (define test-group-abort 'test-group-abort)
    
    (define my-test-runner (test-runner-simple))
    (test-runner-on-test-end!
      my-test-runner
      (lambda (runner)
        (test-on-test-end-simple runner)  ;call default implementation
        (when (not (test-passed? runner))
          (throw test-group-abort))))

    (define-syntax test-group+catch
      (syntax-rules ()
        ((_ <name> <expr> ...)
         (catch test-group-abort
           (lambda ()
             (test-group <name> <expr> ...))
           (lambda (key)
             #f)))))

    ;; usage example:
    (test-runner-current my-test-runner)
    (test-begin "foo")
    (test-group+catch "bar"
      (test-assert "failure" #f)
      (test-assert "success" #t))
    (test-end "foo")
    ;passed: 0
    ;failed: 1

If using Emacs, you can run the following elisp code to make Emacs use
correct indentation for 'test-group+catch':

    (put 'test-group+catch 'scheme-indent-function 1)

Taylan