From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Tomas Volf <~@wolfsden.cz> Newsgroups: gmane.lisp.guile.bugs Subject: bug#72365: srfi-64: test-on-bad-end-name-simple is not allowed to raise an exception Date: Thu, 03 Oct 2024 00:17:47 +0200 Message-ID: <87bk0240ac.fsf@wolfsden.cz> References: <2a1fc413-ccd8-470a-9088-6d06386f32ec@gmail.com> <87ploi4rkw.fsf@wolfsden.cz> <8320ebef-80f6-46ee-b01e-620282926f9c@gmail.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33445"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: 72365@debbugs.gnu.org To: Taylan Kammer Original-X-From: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Thu Oct 03 00:18:32 2024 Return-path: Envelope-to: guile-bugs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sw7fy-0008Ws-BZ for guile-bugs@m.gmane-mx.org; Thu, 03 Oct 2024 00:18:30 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sw7fY-000497-47; Wed, 02 Oct 2024 18:18:04 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sw7fV-00048b-Pw for bug-guile@gnu.org; Wed, 02 Oct 2024 18:18:01 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sw7fV-00070h-H4 for bug-guile@gnu.org; Wed, 02 Oct 2024 18:18:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:References:In-Reply-To:From:To:Subject; bh=bssnvYwRwNZ3yRH9/5Q73x66Yzt/Xw5bAh0gzLH75uY=; b=XP0JtWTX3fE6JSzRPqEM1b98jDZ0ULf5nEwMszBwr1To5PrGuS6tyCggri3yHfxjWd597w/WEu+Zb5LaEXAwD+YT8K03tmjzUTc5N5i/G7HBl8cH1nSDucndgCUNWGuXCj+WWaKnUHe//mQ7EI6B05GsCCVWKSBkKWL8Y7Hceu46bae9Aqj+GuA0W9NuX+i/txcTBtKWKWR+cDZha3b7ZRenscL1+lz99gMftlS1I5mKzRMgK/vf15zlMarkE3hPA6BiNH139IpisGQ5agEY32iGN7y6ZTM8xHucU12BqA0wx6zf5gtXKSrZ29IYLa/1W+R4d8wS5MFfioObRUnBHw==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sw7fW-00053H-E6 for bug-guile@gnu.org; Wed, 02 Oct 2024 18:18:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Tomas Volf <~@wolfsden.cz> Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Wed, 02 Oct 2024 22:18:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 72365 X-GNU-PR-Package: guile Original-Received: via spool by 72365-submit@debbugs.gnu.org id=B72365.172790747419399 (code B ref 72365); Wed, 02 Oct 2024 22:18:02 +0000 Original-Received: (at 72365) by debbugs.gnu.org; 2 Oct 2024 22:17:54 +0000 Original-Received: from localhost ([127.0.0.1]:59464 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sw7fN-00052n-MH for submit@debbugs.gnu.org; Wed, 02 Oct 2024 18:17:54 -0400 Original-Received: from wolfsden.cz ([37.205.8.62]:47844) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <~@wolfsden.cz>) id 1sw7fL-00052a-91 for 72365@debbugs.gnu.org; Wed, 02 Oct 2024 18:17:52 -0400 Original-Received: by wolfsden.cz (Postfix, from userid 104) id C666431214B; Wed, 2 Oct 2024 22:17:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=wolfsden.cz; s=mail; t=1727907468; bh=ZPz0ZEPOU+V10yNV57eslz8fQPp7tnpNgnnGchTBGZs=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=Nlf57oHOkubkCh5kmavcaXAa08XW/8n6oCtvcYWSFKmcOfKsFm6tYu2ryc34nQGSV uakKCQ1AzTr7kyfcUmomr6whhWKeoUkqB0d31DRWy9svHhrySZuSIWxT1+gA43YoEv 5Lo7v3iLYKq5S4JoTyCgt4h2A7T12441F7Ui6fsXAHkQbzpbTt9moeGs0XnlpYKgw7 4t+0rcGPp3RTrNQl3gAOusy3es+XbaLVgPVdCdvPsLJZOomQi8RZl1PHmXAR1iUjhB nfImUlAZr+zhUn+9HxH9MevpMgVtc9ouhAUvmR0T2M7nZaMlHWK97Qfod7jQ9uO9+7 t5ODxWu4CIiJ28vN6maCPicYJ+AMSR2DAo7y/Z78M4l4RfyQGYNTHuhhWH6RguXSwD 6uRD1DW83pRb6HCfJocR2qJ8eny2GWUaIlO5+/DFoLCsP2aPZRR8LJF7FUwdELJuq6 J0COBG+tczGuEVF7EOh/nxOjFWSFM4UvUGtpu9EAGs+1xWB9xSvJLxFgz4TW9PiNl+ xuvA8QDSgXbSL7tqO70hgLg7x4ewFNwo8hgqd5UpAIcWjnwT24K/i/F2HumpMGs9DM Tn9PC2mB9fI7DTF5CQCWbb0FMqjNbqKio7ySkq3yaTf1naLe/IGqZDahol+q6rtggA fq7vW1RrDNMFa1dc73a/vas8= Original-Received: from localhost (unknown [128.0.188.242]) by wolfsden.cz (Postfix) with ESMTPSA id F2CA3313FB3; Wed, 2 Oct 2024 22:17:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=wolfsden.cz; s=mail; t=1727907468; bh=ZPz0ZEPOU+V10yNV57eslz8fQPp7tnpNgnnGchTBGZs=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=Nlf57oHOkubkCh5kmavcaXAa08XW/8n6oCtvcYWSFKmcOfKsFm6tYu2ryc34nQGSV uakKCQ1AzTr7kyfcUmomr6whhWKeoUkqB0d31DRWy9svHhrySZuSIWxT1+gA43YoEv 5Lo7v3iLYKq5S4JoTyCgt4h2A7T12441F7Ui6fsXAHkQbzpbTt9moeGs0XnlpYKgw7 4t+0rcGPp3RTrNQl3gAOusy3es+XbaLVgPVdCdvPsLJZOomQi8RZl1PHmXAR1iUjhB nfImUlAZr+zhUn+9HxH9MevpMgVtc9ouhAUvmR0T2M7nZaMlHWK97Qfod7jQ9uO9+7 t5ODxWu4CIiJ28vN6maCPicYJ+AMSR2DAo7y/Z78M4l4RfyQGYNTHuhhWH6RguXSwD 6uRD1DW83pRb6HCfJocR2qJ8eny2GWUaIlO5+/DFoLCsP2aPZRR8LJF7FUwdELJuq6 J0COBG+tczGuEVF7EOh/nxOjFWSFM4UvUGtpu9EAGs+1xWB9xSvJLxFgz4TW9PiNl+ xuvA8QDSgXbSL7tqO70hgLg7x4ewFNwo8hgqd5UpAIcWjnwT24K/i/F2HumpMGs9DM Tn9PC2mB9fI7DTF5CQCWbb0FMqjNbqKio7ySkq3yaTf1naLe/IGqZDahol+q6rtggA fq7vW1RrDNMFa1dc73a/vas8= In-Reply-To: <8320ebef-80f6-46ee-b01e-620282926f9c@gmail.com> (Taylan Kammer's message of "Wed, 2 Oct 2024 23:07:01 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.lisp.guile.bugs:11025 Archived-At: Taylan Kammer writes: > Do I understand correctly that this is an additional test suite for > testing SRFI-64 itself? Like the "meta test suite" shipped with > SRFI-64? Yes, exactly. Vast majority of the tests are just derived from the specification, with few non-portable written just for my implementation, but those can be turned off. > > Is there a brief description somewhere on how to run it with Guile? > Would be really neat if I can use it to further test my > implementation. It is not hard, but at the same time the test suite is not really stand-alone, so the instructions will be bit hackish. 1. Download https://files.wolfsden.cz/releases/guile-wolfsden/guile-wolfsden-0.0.1.tar.gz 2. Unpack it. 3. Open `build-aux/srfi64test-driver.scm'. 4. On line 35 replace 'wolfsden with 'guile. 5. Open `Makefile.am'. 6. Delete lines 17-19 (assignment to `TESTS' variable). 7. Open `tests/srfi-64/local.mk'. 8. On line 4 change `TESTS +=' to `TESTS ='. 9. Build the project and run the tests: $ autoreconf -fvi $ ./configure --disable-doc-snarf $ make $ make check Due to step 4. only portable (== written based on specification) tests will run, and they will run against (srfi srfi-64). Since you library is available as that module, just make sure it is on the load path. When I follow the steps, I get: # TOTAL: 340 # PASS: 265 # SKIP: 36 # XFAIL: 0 # FAIL: 39 # XPASS: 0 # ERROR: 0 You should get less FAILs I guess (since you have fixed many problems already, and some you did not even have). I am sure you will dispute some of those tests. ^_^ >> You can find my version here[0]. If you do not use Guix, building from >> tarball[1] might be easier. Contrary to your version, mine is available >> as (wolfsden srfi srfi-64). >> >> 0: https://git.wolfsden.cz/guile-wolfsden/ >> 1: https://wolfsden.cz/project/guile-wolfsden.html > > Your implementation seems written specifically with Guile in mind, > which is a big plus I guess. Yes, I decided to write my version in as-readable manner as possible (well, at least I hope the code is readable), at the cost of portability. Since I have seen what portability did to (srfi srfi-64). > If the quality of the implementations is the same or higher, in terms of > observable behavior, then it should be preferred for Guile, I think. If I find > the time, I'll see if I can use your implementation to run some of my test > suites, like the bytestructures test suite, and report if I notice any > issues. Oh, that would be much appreciated. I did test my version against Guix's test suite (and it revealed 4 bugs in Guix's tests) and none in my library, so I hope results for your project would be similar. >>> In one case, the reference implementation clearly violates the specification: >>> The simple test runner uses the `aux` field which the spec claims it doesn't >>> use. (My implementation fixes this.) However, in this case it's not that >>> clear-cut. >>> >>> In this case, I think raising an error is good default behavior, since the >>> mismatched end name indicates a problem with the test suite itself rather than >>> the code being tested. If it poses a problem to the user, one can override that >>> callback with the `test-runner-on-bad-end-name!` setter. >>> >>> What do you think? >> I agree that raising an error is good behavior. However I do not think >> that on-bad-end-name-function is a place where to do it. In my opinion >> the name mismatch is a hard error, in my implementation subclass of >> &programming-error[4]. If I am writing new test runner, the >> specification does not mention that raising the error is *my* >> responsibility, just that test-end will signal an error. >> >> To rephrase that: test-end is mandated to signal error, but custom test >> runner has no provision requiring it to do it in >> on-bad-end-name-function. Hence I believe test-end needs to be the one >> to signal the error. > > Makes sense I guess. I've generally tried to imitate the reference > implementation's behavior as closely as possible in such matters, worrying that > there might be code out there that relies on its various quirks, but maybe I'm > being too paranoid. I tried to not use reference implementation that much, and instead relied on the specification. It was slow and painful process. > I don't have a strong opinion either way. The number of people, who want to > write a test runner that does something special on bad-end-name (something other > than raise an error), is probably very small. I definitely agree on this one. > > - Making `test-end` itself raise an error would probably be most convenient, so test runner authors don't have to take care of it. > > - But if `test-end` doesn't do it, it's not a big deal either IMO, because all > they would need to do is to call `(test-runner-on-bad-end-name! my-runner > test-on-bad-end-name-simple)` to make their custom runner raise an error as > well. (And, if they want to do something before, they can use a procedure that > ends with the call `(test-on-bad-end-name-simple ...)`.) > > The latter is my preference, because enabling the behavior via a single line of > code is easy, whereas disabling it would be difficult / impossible if `test-end` > were to be hardcoded to raise an error. But if a SRFI-64 implementation made its > `test-end` always raise an error, it probably wouldn't anyone in practice, so I > wouldn't see it as a real problem. I still think test-end itself raising is what specification mandates (whether it *should* mandate it is a different question :) ), however I agree, I also am skeptical anyone's code actually cares either way. Tomas -- There are only two hard things in Computer Science: cache invalidation, naming things and off-by-one errors.