* bug#72369: srfi-64: test-end fails to signal an error with null runner
@ 2024-07-30 19:51 Tomas Volf
2024-09-30 16:22 ` Taylan Kammer
0 siblings, 1 reply; 4+ messages in thread
From: Tomas Volf @ 2024-07-30 19:51 UTC (permalink / raw)
To: 72369
Hello,
I think I found a bug in (srfi srfi-64) module shipped with GNU Guile.
The specification says the following about the test-end:
> An error is reported if the suite-name does not match the current test group
> name.
Thus the following should signal an error:
(use-modules (srfi srfi-64))
(let ((r (test-runner-null)))
(test-runner-current r)
(test-begin "x")
(test-end "y"))
However it does not.
Have a nice day
Tomas Volf
^ permalink raw reply [flat|nested] 4+ messages in thread
* bug#72369: srfi-64: test-end fails to signal an error with null runner 2024-07-30 19:51 bug#72369: srfi-64: test-end fails to signal an error with null runner Tomas Volf @ 2024-09-30 16:22 ` Taylan Kammer 2024-10-02 19:57 ` Tomas Volf 0 siblings, 1 reply; 4+ messages in thread From: Taylan Kammer @ 2024-09-30 16:22 UTC (permalink / raw) To: Tomas Volf, 72369 On 30.07.2024 21:51, Tomas Volf wrote: > Hello, > > I think I found a bug in (srfi srfi-64) module shipped with GNU Guile. > > The specification says the following about the test-end: > >> An error is reported if the suite-name does not match the current test group >> name. > Thus the following should signal an error: > > (use-modules (srfi srfi-64)) > (let ((r (test-runner-null))) > (test-runner-current r) > (test-begin "x") > (test-end "y")) > > However it does not. > > Have a nice day > Tomas Volf This would be easy to change, but the on-bad-end-name handler would be kind of useless if test-end was hardcoded to always raise an error. I think the intended meaning of the spec is that the default/simple test runner reports an error in this case (by implementing the on-bad-end-name handler) but not test-end itself. One could argue that "reporting" an error is not the same thing as signaling/raising one. We could make test-end always print something to stderr, but not actually raise an error, so it technically fulfills the spec's promise that it "reports" an error, but the usefulness of this is unclear to me. Opinions welcome. - Taylan ^ permalink raw reply [flat|nested] 4+ messages in thread
* bug#72369: srfi-64: test-end fails to signal an error with null runner 2024-09-30 16:22 ` Taylan Kammer @ 2024-10-02 19:57 ` Tomas Volf 2024-10-02 21:16 ` Taylan Kammer 0 siblings, 1 reply; 4+ messages in thread From: Tomas Volf @ 2024-10-02 19:57 UTC (permalink / raw) To: Taylan Kammer; +Cc: 72369 Taylan Kammer <taylan.kammer@gmail.com> writes: Hi, > On 30.07.2024 21:51, Tomas Volf wrote: >> Hello, >> >> I think I found a bug in (srfi srfi-64) module shipped with GNU Guile. >> >> The specification says the following about the test-end: >> >>> An error is reported if the suite-name does not match the current test group >>> name. >> Thus the following should signal an error: >> >> (use-modules (srfi srfi-64)) >> (let ((r (test-runner-null))) >> (test-runner-current r) >> (test-begin "x") >> (test-end "y")) >> >> However it does not. >> >> Have a nice day >> Tomas Volf > > This would be easy to change, but the on-bad-end-name handler would be kind of > useless if test-end was hardcoded to always raise an error. I think the intended > meaning of the spec is that the default/simple test runner reports an error in > this case (by implementing the on-bad-end-name handler) but not test-end itself. > > One could argue that "reporting" an error is not the same thing as > signaling/raising one. We could make test-end always print something to stderr, > but not actually raise an error, so it technically fulfills the spec's promise > that it "reports" an error, but the usefulness of this is unclear to me. > > Opinions welcome. I think I reacted to these concerns in response to #72365, but for completeness pasting the same here: I agree that raising an error is good behavior. However I do not think that on-bad-end-name-function is a place where to do it. In my opinion the name mismatch is a hard error, in my implementation subclass of &programming-error[4]. If I am writing new test runner, the specification does not mention that raising the error is *my* responsibility, just that test-end will signal an error. To rephrase that: test-end is mandated to signal error, but custom test runner has no provision requiring it to do it in on-bad-end-name-function. Hence I believe test-end needs to be the one to signal the error. However! That does not make on-bad-end-name-function useless. The specification does not mandate *how* the error signaled by test-end should look like, hence there is no *portable* way to detect it. Custom runner, if it needs to report name mismatch specially, can just produce specific log line in the callback (or even signal its own exception first before test-end does). Let me know what you think (either here or in #72365 ^_^ ). Tomas -- There are only two hard things in Computer Science: cache invalidation, naming things and off-by-one errors. ^ permalink raw reply [flat|nested] 4+ messages in thread
* bug#72369: srfi-64: test-end fails to signal an error with null runner 2024-10-02 19:57 ` Tomas Volf @ 2024-10-02 21:16 ` Taylan Kammer 0 siblings, 0 replies; 4+ messages in thread From: Taylan Kammer @ 2024-10-02 21:16 UTC (permalink / raw) To: Tomas Volf; +Cc: 72369 [-- Attachment #1: Type: text/plain, Size: 165 bytes --] On 02.10.2024 21:57, Tomas Volf wrote: > Let me know what you think (either here or in #72365 ^_^ ). > > Tomas Sure thing! Responded in the other thread. - Taylan [-- Attachment #2: Type: text/html, Size: 650 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2024-10-02 21:16 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-07-30 19:51 bug#72369: srfi-64: test-end fails to signal an error with null runner Tomas Volf 2024-09-30 16:22 ` Taylan Kammer 2024-10-02 19:57 ` Tomas Volf 2024-10-02 21:16 ` Taylan Kammer
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).