srfi-64 tests passing when they should not

[Zelphir Kaltstahl <zelphirkaltstahl@posteo.de>]

Hello Guile users!

I noticed something strange about SRFI-64 unit tests, when using test-equal with
an expected result of false (#f):

~~~~
(import
  ;; unit tests
  (srfi srfi-64)
  ;; module to test
  (list-utils))


(test-begin "list-utils-test")

(test-group
 "list-prefixes-test"

 (test-equal "gives all list prefixes -- 00"
   '(("static") ("static" "img"))
   (list-prefixes '("static" "img" "logo.png")))

 (test-equal "gives false for one element lists -- 00"
   #f
   (list-prefixes '("static")))

 (test-assert "gives false for zero element lists -- 00"
   (equal? #f (list-prefixes '()))))

(test-end "list-utils-test")
~~~~

This code is for a still empty module (not even a module defined in that
(list-utils) file. So I expect all tests to fail, due to the error of unbound
variables.

However, the result is surprisingly 2 passing tests, despite errors happening
during the test:

~~~~
%%%% Starting test list-utils-test
Group begin: list-utils-test
Group begin: list-prefixes-test
Test begin:
  test-name: "gives all list prefixes -- 00"
  source-file: "test/test-list-utils.scm"
  source-line: 13
  source-form: (test-equal "gives all list prefixes -- 00" (quote (("static") ("static" "img"))) (list-prefixes (quote ("static" "img" "logo.png"))))
Test end:
  result-kind: fail
  actual-value: #f
  actual-error: (unbound-variable #f "Unbound variable: ~S" (list-prefixes) #f)
  expected-value: (("static") ("static" "img"))
Test begin:
  test-name: "gives false for one element lists -- 00"
  source-file: "test/test-list-utils.scm"
  source-line: 17
  source-form: (test-equal "gives false for one element lists -- 00" #f (list-prefixes (quote ("static"))))
Test end:
  result-kind: pass
  actual-value: #f
  actual-error: (unbound-variable #f "Unbound variable: ~S" (list-prefixes) #f)
  expected-value: #f
Test begin:
  test-name: "gives false for zero element lists -- 00"
  source-file: "test/test-list-utils.scm"
  source-line: 21
  source-form: (test-equal "gives false for zero element lists -- 00" #f (list-prefixes (quote ())))
Test end:
  result-kind: pass
  actual-value: #f
  actual-error: (unbound-variable #f "Unbound variable: ~S" (list-prefixes) #f)
  expected-value: #f
Group end: list-prefixes-test
Group end: list-utils-test
# of expected passes      2
# of unexpected failures  1
~~~~

The first 2 tests are surprisingly passing. This is also the reason, why I used
test-assert and manually wrote the (equal? ...) in the last test, to see,
whether it makes any difference. Indeed it does.

Why does die implementation behave like this?

Best regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl