From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Taylan Kammer Newsgroups: gmane.lisp.guile.bugs Subject: bug#72371: srfi-64: test marked for skip and as expected failure has wrong result-kind in on-test-begin-function Date: Tue, 1 Oct 2024 23:45:49 +0200 Message-ID: <47bec123-76f8-4117-93ba-aed16b8c659c@gmail.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="14427"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mozilla Thunderbird To: Tomas Volf <~@wolfsden.cz>, 72371@debbugs.gnu.org Original-X-From: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Tue Oct 01 23:47:31 2024 Return-path: Envelope-to: guile-bugs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1svkiP-0003at-Tz for guile-bugs@m.gmane-mx.org; Tue, 01 Oct 2024 23:47:30 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1svki9-0002nn-VF; Tue, 01 Oct 2024 17:47:14 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1svki4-0002nT-Dz for bug-guile@gnu.org; Tue, 01 Oct 2024 17:47:08 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1svkhy-0008Ay-EO for bug-guile@gnu.org; Tue, 01 Oct 2024 17:47:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=In-Reply-To:From:References:MIME-Version:Date:To:Subject; bh=Wc/J4mfkdC3m5lGDsLnP0fbX+DWA18MgbGPHzEQl1Lc=; b=iUpXZHMr5z4iKhjiB1QYE7RLM7U6KJ5NLty3Oq6F+y7n2nILwKwaj+onTQAT6ZoE6etpPHAexw9nRmOlCBRLsdX04HXdxRFRuMjXdsPEUzYzM3vcBeThHLXIzuZLtYiXV0ma5KLq8ul2BA00cK37cSGOPWFNbyvgTwlASL0QLtxJK1mRD77rJ0jDYoSJnUzeIFcFe8xNLjeRdSEr/XDg7RFZ784duGxAYSk9X0Foue8y5m/bnWRDeFOKN4J1lAGCXLKtDQGrkyjOVgGlCnTTANq7u68gIDVnsu6abON6HBSQJUWlEVy+lAC6gDwNCYfTH/ahVyaA8vUa6+f5x6dc2Q==; Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1svkhx-0004Tb-Un for bug-guile@gnu.org; Tue, 01 Oct 2024 17:47:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Taylan Kammer Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Tue, 01 Oct 2024 21:47:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 72371 X-GNU-PR-Package: guile Original-Received: via spool by 72371-submit@debbugs.gnu.org id=B72371.172781921917200 (code B ref 72371); Tue, 01 Oct 2024 21:47:01 +0000 Original-Received: (at 72371) by debbugs.gnu.org; 1 Oct 2024 21:46:59 +0000 Original-Received: from localhost ([127.0.0.1]:53878 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1svkhv-0004TM-4c for submit@debbugs.gnu.org; Tue, 01 Oct 2024 17:46:59 -0400 Original-Received: from mail-ed1-f49.google.com ([209.85.208.49]:40007) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1svkht-0004TC-H0 for 72371@debbugs.gnu.org; Tue, 01 Oct 2024 17:46:58 -0400 Original-Received: by mail-ed1-f49.google.com with SMTP id 4fb4d7f45d1cf-5c88367684bso798368a12.0 for <72371@debbugs.gnu.org>; Tue, 01 Oct 2024 14:46:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727819152; x=1728423952; darn=debbugs.gnu.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id:from :to:cc:subject:date:message-id:reply-to; bh=Wc/J4mfkdC3m5lGDsLnP0fbX+DWA18MgbGPHzEQl1Lc=; b=RiIDBOdwYut6XoU+/HpdJW3l8BteGqe+ACqqBOMeL7tlmNGs6p5rVx4ZsrRwW8VnQe KbQ1mLd/m16YmooP89QT20wRQVQVYIUR6gL/h7nH1aErBLsF15D6bno3kOcuOC6v9NCf LfokzBYrexiw9sf6I2yLGdAzTlAfasHDqwDZVG0qbZRpnCqx0p1q2iAb1zuvkQL2D/xE +va9M8y1BROoXtta2hR5bAyUngpMDWhYF9BaiwrVFuL/RUZFDAMo1fxecXXNvxhLZfhF 3acl65bhjbD+RlnNjXAISIR1EiVLeYEee50I58SqsWwD1OFbP5yMip+0Xxhq1WKC7gvl GYfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727819152; x=1728423952; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Wc/J4mfkdC3m5lGDsLnP0fbX+DWA18MgbGPHzEQl1Lc=; b=YLSryfqrroqT8PL4DTO0A+phMp6/SfzepPQyqO2GY9yzPr2XATylexn+z6xhd5NsNO wh3WeXVMo1ZFiJ6OtWgPCTsWlJCbKUpbx8tWArf1iU9d8aShvo5aENa27Xy1fnb5sU+w 3xABWk5dG/XJ5Xc2/a1IW4Rpp1peBz/oPlD+TmeErsGKV9CimtMjUDNhDm0+mSGSmRkQ d8qQFfLVHGXHdG1qxtGFzcxq7wZk2xCRC3kHwg3RrrDP3GfIDKRV8KmCuXs5bpHVX0Dn LcdNvx+ttJJPVt0qLvEm/gVX7QESgc73G2DsDW5fXRamA5fVogOpWbdlMwJB8QgrMzk7 JQmg== X-Forwarded-Encrypted: i=1; AJvYcCUt28Qu4KgJSEw/9zpGhpD7zwItzC2mccoBSUU58AdVcaBuuzpZf48D4q9jBeqD/1pXW0G3kA==@debbugs.gnu.org X-Gm-Message-State: AOJu0YyShaYEtZAsVvis5NOoD82QNYbV+XX2pbwyH2LSGyrSJgfIAZ0b +M15h2z3diMBLInaL9W06m03MFyNJMHqLY28zKx1G12vyepik2vWUJuWJIgb X-Google-Smtp-Source: AGHT+IG4GQmfCLop5UiFTcTshv63IGSLR1MTC4t8cS7bOH16oejkB5XkU0Ddb7fCzV9/zYhushtPug== X-Received: by 2002:a05:6402:35c6:b0:5c2:6850:7b2 with SMTP id 4fb4d7f45d1cf-5c8b1b7a6cdmr216153a12.6.1727819151671; Tue, 01 Oct 2024 14:45:51 -0700 (PDT) Original-Received: from ?IPV6:2003:106:8f04:c300:95ac:529d:6db3:196b? (p200301068f04c30095ac529d6db3196b.dip0.t-ipconnect.de. [2003:106:8f04:c300:95ac:529d:6db3:196b]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88248b628sm6665103a12.66.2024.10.01.14.45.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 01 Oct 2024 14:45:50 -0700 (PDT) Content-Language: en-US In-Reply-To: X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.lisp.guile.bugs:11002 Archived-At: On 30.07.2024 21:51, Tomas Volf wrote: > Hello, > > I think I found a bug in (srfi srfi-64) module shipped with GNU Guile. > > The specification says the following regarding the test-result-kind: > >> If we've started on a new test, but don't have a result yet, then the result >> kind is 'xfail if the test is expected to fail, 'skip if the test is supposed >> to be skipped, or #f otherwise. > Thus I believe that following should print `xfail': > > (use-modules (srfi srfi-64)) > (test-begin "x") > > (test-runner-on-test-begin! (test-runner-current) > (λ (runner) > (pk (test-result-kind)))) > > (test-skip 1) > (test-expect-fail 1) > (test-assert #t) > > (test-end) > > However it does not: > > ;;; (skip) > > Have a nice day > Tomas Volf > I think this is a case where the spec didn't actually consider what should happen if skip and expect-fail are combined. Otherwise, I would expect to see a more explicit description of what should happen in such cases. In other words, I think the English description of what's supposed to happen, that you've quoted, is *not* intended to be read like procedural pseudo-code: "If expected to fail, return 'xfail; if supposed to be skipped, return 'skip." The reference implementation does it the exact other way around, in a rather straightforward manner (two consecutive clasuses of a cond expression), so I don't think it's a bug. Intuitively, I also think it makes the most sense to treat skipping as a higher priority. While an xfail test is still executed, a skipped test is not executed at all, which is a more significant change in the test suite's behavior and should be honored IMO. If I've marked a test to be skipped, it could be because executing it currently leads to a crash or an infinite loop, so it would be important to skip it even if it's marked as xfail. So, I think the observed behavior is probably best, and intended. Opinions welcome. - Taylan