* bug#72384: srfi-64: test-end should not clear fail list
@ 2024-07-30 19:52 Tomas Volf
2024-10-01 23:38 ` Taylan Kammer
0 siblings, 1 reply; 2+ messages in thread
From: Tomas Volf @ 2024-07-30 19:52 UTC (permalink / raw)
To: 72384
Hello,
I think I found a bug in (srfi srfi-64) module shipped with GNU Guile.
Reading the specification for test-expect-fail I do not see a mandate to clear
expect-fail list on test-end. test-skip does have such provision, but it is
lacking in the test-expect-fail. Therefore I think current behavior is wrong:
(use-modules (srfi srfi-64))
(test-begin "x")
(test-begin "group1")
(test-expect-fail "test-a")
(test-assert "test-a" #t)
(pk (test-result-kind))
(test-end "group1")
(test-assert "test-a" #t)
(pk (test-result-kind))
(test-end)
Leading to:
;;; (xpass)
;;; (pass)
Have a nice day
Tomas Volf
^ permalink raw reply [flat|nested] 2+ messages in thread
* bug#72384: srfi-64: test-end should not clear fail list
2024-07-30 19:52 bug#72384: srfi-64: test-end should not clear fail list Tomas Volf
@ 2024-10-01 23:38 ` Taylan Kammer
0 siblings, 0 replies; 2+ messages in thread
From: Taylan Kammer @ 2024-10-01 23:38 UTC (permalink / raw)
To: Tomas Volf, 72384
[-- Attachment #1: Type: text/plain, Size: 2040 bytes --]
On 30.07.2024 21:52, Tomas Volf wrote:
> Hello,
>
> I think I found a bug in (srfi srfi-64) module shipped with GNU Guile.
>
> Reading the specification for test-expect-fail I do not see a mandate to clear
> expect-fail list on test-end. test-skip does have such provision, but it is
> lacking in the test-expect-fail. Therefore I think current behavior is wrong:
>
> (use-modules (srfi srfi-64))
>
> (test-begin "x")
>
> (test-begin "group1")
> (test-expect-fail "test-a")
> (test-assert "test-a" #t)
> (pk (test-result-kind))
> (test-end "group1")
> (test-assert "test-a" #t)
> (pk (test-result-kind))
>
> (test-end)
>
> Leading to:
>
> ;;; (xpass)
>
> ;;; (pass)
>
> Have a nice day
> Tomas Volf
>
>
I'm inclined to see this as an error/omission in the spec itself. It makes sense for test-end to clear the expected-fail list, just like it clears the skip list. If it didn't, it might cause one to accidentally mark tests as "expected failure" that weren't meant to be marked as such. Consider the following:
(test-group "group1"
(test-expect-fail "test-a")
(test-assert "test-a" #t))
(test-group "group2"
(test-assert "test-a" #t))
Since test-group is equivalent to a pair of test-begin/test-end calls, this code would be very "deceptive" if the implicit test-end didn't clear the expected-fail list. After all, the two groups look completely disjoint, and one wouldn't expect any state from the former to implicitly bleed into the latter. Also, if I'm not mistaken, there's not even a way to clear the expected-fail list explicitly.
I actually have some "real-world" code that uses repetitive names in a test suite within different groups, so this isn't just a theoretical issue either:
https://codeberg.org/taylan/scheme-bytestructures/src/branch/master/run-tests.body.scm
Notice how often the names "ref" and "set" are used.
So, I think the behavior of the reference implementation is correct/desirable here. My implementation of SRFI-64 does the same.
- Taylan
[-- Attachment #2: Type: text/html, Size: 2625 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2024-10-01 23:38 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-07-30 19:52 bug#72384: srfi-64: test-end should not clear fail list Tomas Volf
2024-10-01 23:38 ` Taylan Kammer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).