* Replacing Guile test-suite with SRFI-64? @ 2023-09-23 12:49 Bruno Victal 2023-09-23 15:13 ` Mike Gran 2023-10-31 13:20 ` Maxim Cournoyer 0 siblings, 2 replies; 9+ messages in thread From: Bruno Victal @ 2023-09-23 12:49 UTC (permalink / raw) To: guile-devel Hi, Would a rewrite of the Guile tests using SRFI-64 be a welcome idea? IMO the advantages of doing so are: * Less “specifics” that need to be learned solely for working with Guile. It's easier for people already using SRFI-64 to “bring in” their experience. * Standardized/portable. Straightforward to integrate external modules or SRFI sample implementations into Guile. * Pre-existing documentation. Opinions? -- Thanks, Bruno. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Replacing Guile test-suite with SRFI-64? 2023-09-23 12:49 Replacing Guile test-suite with SRFI-64? Bruno Victal @ 2023-09-23 15:13 ` Mike Gran 2023-09-24 22:27 ` Maxime Devos 2023-09-26 12:25 ` Bruno Victal 2023-10-31 13:20 ` Maxim Cournoyer 1 sibling, 2 replies; 9+ messages in thread From: Mike Gran @ 2023-09-23 15:13 UTC (permalink / raw) To: guile-devel@gnu.org, Bruno Victal > Hi, > Would a rewrite of the Guile tests using SRFI-64 be a welcome idea? > IMO the advantages of doing so are: ... >Opinions? IMHO, I think Guile should not depend too heavily on Guile for testing itself. It is bad bootstrapping practice. And personally, I find that errors in srfi-64 tests are more difficult to interpret that errors in the guile test suite framework. The logging is worse. - Mike ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Replacing Guile test-suite with SRFI-64? 2023-09-23 15:13 ` Mike Gran @ 2023-09-24 22:27 ` Maxime Devos 2023-09-24 23:32 ` Mike Gran 2023-09-26 12:25 ` Bruno Victal 1 sibling, 1 reply; 9+ messages in thread From: Maxime Devos @ 2023-09-24 22:27 UTC (permalink / raw) To: Mike Gran, guile-devel@gnu.org, Bruno Victal [-- Attachment #1.1.1: Type: text/plain, Size: 1482 bytes --] Op 23-09-2023 om 17:13 schreef Mike Gran: >> Hi, > >> Would a rewrite of the Guile tests using SRFI-64 be a welcome idea? >> IMO the advantages of doing so are: > > ... > >> Opinions? > > IMHO, I think Guile should not depend too heavily on Guile for testing itself. > It is bad bootstrapping practice. Whether you use Guile's custom test utilities or SRFI-64, in both cases you are equally depending on Guile to test itself. A benefit of SRFI-64 is that the SRFI-64 implementation has tests whereas (IIRC) Guile's custom thing doesn't. Another benefit is that the SRFI-64 implementation is also tested in other Scheme implementations, which partially mitigates the ‘tester testing itself’ issue (it turns out that it has bugs, see patch by Taylan Kammer (*)). Also, there are not bootstrapping issues, because Guile is built before the tests are run (‘make check’ implies ‘make’). > And personally, I find that errors in srfi-64 tests are more difficult to interpret > that errors in the guile test suite framework. The logging is worse. There is a not-yet-applied patch (*) that changes the SRFI implementation to be better, but I don't know if it improves error messages and logging. If there is anything concrete you consider worse about error reporting in SRFI-64, perhaps the test runner could simply be tweaked to fix that. (*) https://lists.nongnu.org/archive/html/guile-devel/2021-05/msg00007.html [-- Attachment #1.1.2: OpenPGP public key --] [-- Type: application/pgp-keys, Size: 929 bytes --] [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 236 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Replacing Guile test-suite with SRFI-64? 2023-09-24 22:27 ` Maxime Devos @ 2023-09-24 23:32 ` Mike Gran 2023-09-25 9:48 ` Maxime Devos 0 siblings, 1 reply; 9+ messages in thread From: Mike Gran @ 2023-09-24 23:32 UTC (permalink / raw) To: guile-devel@gnu.org, Bruno Victal, Maxime Devos > A benefit of SRFI-64 is that the SRFI-64 implementation has tests > whereas (IIRC) Guile's custom thing doesn't. OK two separate issues. 1. Mike G's gripes about SRFI-64. 2. Updating over 65,000 lines of test code I don't want to be put in the position of having to make a critique of SRFI-64, and, once my critique is dissected, make it appear that that is justification for updating the test suite. I wouldn't be the decision maker, anyway. It is 2 MB of test code that has served Guile well so far. I think the argument for srfi-64 is not self-evident. That srfi-64 is familiar to some and has a test suite does not prove that the current test suite is difficult for newcomers to understand and is buggy. Respectfully, -Mike ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Replacing Guile test-suite with SRFI-64? 2023-09-24 23:32 ` Mike Gran @ 2023-09-25 9:48 ` Maxime Devos 2023-09-26 8:17 ` Taylan Kammer 0 siblings, 1 reply; 9+ messages in thread From: Maxime Devos @ 2023-09-25 9:48 UTC (permalink / raw) To: Mike Gran, guile-devel@gnu.org, Bruno Victal [-- Attachment #1.1.1: Type: text/plain, Size: 2708 bytes --] Op 25-09-2023 om 01:32 schreef Mike Gran: >> A benefit of SRFI-64 is that the SRFI-64 implementation has tests >> whereas (IIRC) Guile's custom thing doesn't. > > OK two separate issues. > 1. Mike G's gripes about SRFI-64. > 2. Updating over 65,000 lines of test code These aren't separate issues -- if SRFI-64 is worse than what Guile has currently and SRFI-64 can't be improved to make it better, then that's a good reason > I don't want to be put in the position of having to make a critique of > SRFI-64, and, once my critique is dissected, make it > appear that that is justification for updating the test suite. > > I wouldn't be the decision maker, anyway. This last sentence is irrelevant. > It is 2 MB of test code that has served Guile well so far. Nobody claimed that these 2MB hasn't served Guile well. Instead, the proposal is about serving Guile better. That things might be going well currently doesn't mean we can't do better. Also, I dispute that it has served Guile well. It is difficult to select which tests to run (it is easier to delete all the other tests than to find/remember the documentation stating how to do this), whereas with typical SRFI-64 test runners you can just set TESTS=some-scheme-file.scm. Also test-suite/README is rather obsolete. A more general problem is that Guile's test library is undocumented. > I think the argument for srfi-64 is not self-evident. Nobody claimed it is self-evident -- instead, Bruno and I gave some explanation on how SRFI-64 can be better and didn't say ‘SRFI-64 better because duh SRFI-64, self-evident, no?’. (And conversely, you pointed out some problems with the current SRFI-64.) > That > srfi-64 is familiar to some and has a test suite does not prove > that the current test suite is difficult for newcomers to understand > and is buggy. This message is weird -- you already made a (short) critique about SRFI-64, and nobody claimed ‘has test suite -> current test suite is difficult to understand / buggy’. To be clear, this critique is the following message. > IMHO, I think Guile should not depend too heavily on Guile for testing itself. > It is bad bootstrapping practice. > > And personally, I find that errors in srfi-64 tests are more difficult to interpret > that errors in the guile test suite framework. The logging is worse. And if it turns out to be justification for updating the test suite, then how would that be a bad thing? After all, it would be justified. (Also, which test suite are you talking about -- the test suite of SRFI-64, or Guile's test suite in its entirety?) Best regards, Maxime Devos. [-- Attachment #1.1.2: OpenPGP public key --] [-- Type: application/pgp-keys, Size: 929 bytes --] [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 236 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Replacing Guile test-suite with SRFI-64? 2023-09-25 9:48 ` Maxime Devos @ 2023-09-26 8:17 ` Taylan Kammer 2023-09-27 16:12 ` Taylan Kammer 0 siblings, 1 reply; 9+ messages in thread From: Taylan Kammer @ 2023-09-26 8:17 UTC (permalink / raw) To: guile-devel Re. issues with the standard SRFI 64 implementation, let me point out that I've written an alternative implementation with cleaner code and behavior of the default test runner; it's part of this (currently dormant) project: https://github.com/TaylanUB/scheme-srfis (I've not yet had time to migrate away from GitHub. Sorry.) If I remember correctly, it also runs slightly faster. To use with Guile, just add the root dir of the repo to the load path and import (srfi srfi-64) as usual; it should override the default one if the Guile load path is set up correctly. To make sure you're not overriding other SRFIs offered by Guile, remove all but the following files: srfi/64/* srfi/64.sld srfi/srfi-64.sld Or copy just those into a different directory and add that to your Guile load path. You could also rename 'srfi-64.sld' so as not to override the default SRFI 64, and I think it should still work under the changed name so long as srfi/64/* aren't renamed. -- Taylan ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Replacing Guile test-suite with SRFI-64? 2023-09-26 8:17 ` Taylan Kammer @ 2023-09-27 16:12 ` Taylan Kammer 0 siblings, 0 replies; 9+ messages in thread From: Taylan Kammer @ 2023-09-27 16:12 UTC (permalink / raw) To: guile-devel On 26.09.2023 10:17, Taylan Kammer wrote: > > https://github.com/TaylanUB/scheme-srfis > > (I've not yet had time to migrate away from GitHub. Sorry.) > Fixed: https://codeberg.org/taylan/scheme-srfis -- Taylan ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Replacing Guile test-suite with SRFI-64? 2023-09-23 15:13 ` Mike Gran 2023-09-24 22:27 ` Maxime Devos @ 2023-09-26 12:25 ` Bruno Victal 1 sibling, 0 replies; 9+ messages in thread From: Bruno Victal @ 2023-09-26 12:25 UTC (permalink / raw) To: Mike Gran; +Cc: guile-devel@gnu.org Hi Mike, On 2023-09-23 16:13, Mike Gran wrote: > And personally, I find that errors in srfi-64 tests are more difficult to interpret > that errors in the guile test suite framework. The logging is worse. If I'm not mistaken the logging behavior can be customized by writing your own SRFI-64 test runner. (as an example Guix does this in [1]) [1]: <https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/build/marionette.scm?id=b2887da40c6cb9d541a9df67aa726694f64d8943#n453> -- Furthermore, I consider that nonfree software must be eradicated. Cheers, Bruno. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Replacing Guile test-suite with SRFI-64? 2023-09-23 12:49 Replacing Guile test-suite with SRFI-64? Bruno Victal 2023-09-23 15:13 ` Mike Gran @ 2023-10-31 13:20 ` Maxim Cournoyer 1 sibling, 0 replies; 9+ messages in thread From: Maxim Cournoyer @ 2023-10-31 13:20 UTC (permalink / raw) To: Bruno Victal; +Cc: guile-devel, spk121, maximedevos, taylan.kammer Hi, Bruno Victal <mirai@makinata.eu> writes: > Hi, > > Would a rewrite of the Guile tests using SRFI-64 be a welcome idea? > IMO the advantages of doing so are: > > * Less “specifics” that need to be learned solely for working with > Guile. It's easier for people already using SRFI-64 to “bring in” their > experience. > > * Standardized/portable. Straightforward to integrate external modules > or SRFI sample implementations into Guile. > > * Pre-existing documentation. I do not have extensive experience with the current test runner used by Guile itself, but it sounds like a good idea to standardize to use SRFI-64, as more people already know that library, and many SRFIs published, e.g. <https://github.com/scheme-requests-for-implementation/srfi-209/tree/master>, come with a SRFI-64 test suite, so it'd be less boilerplate to integrate them. Also, I like the idea of dogfooding *the* testing library of Scheme/Guile. If it fails short of the current custom testing framework used by Guile, we should improve it. This effort will help testing any Guile/Scheme authored project out there, so is worth it in my opinion. -- Thanks, Maxim ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2023-10-31 13:20 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-09-23 12:49 Replacing Guile test-suite with SRFI-64? Bruno Victal 2023-09-23 15:13 ` Mike Gran 2023-09-24 22:27 ` Maxime Devos 2023-09-24 23:32 ` Mike Gran 2023-09-25 9:48 ` Maxime Devos 2023-09-26 8:17 ` Taylan Kammer 2023-09-27 16:12 ` Taylan Kammer 2023-09-26 12:25 ` Bruno Victal 2023-10-31 13:20 ` Maxim Cournoyer
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).