[bug#57050] [PATCH v4 08/14] gnu: racket: Update to 8.6.

["Philip McGrath" <philip@philipmcgrath.com>]

On Sat, Aug 27, 2022, at 3:21 PM, Liliana Marie Prikler wrote:
> Am Samstag, dem 27.08.2022 um 14:55 -0400 schrieb Philip McGrath:
>> +       ((#:tests? _ #t)
>> +        ;; FIXME: There have been some flaky test failures. Some
>> have been
>> +        ;; fixed upstream post-release but have proven non-trivial
>> to
>> +        ;; backport; at least one issue remains. Re-enable tests
>> once
>> +        ;; https://github.com/racket/racket/issues/4359 is fixed.
>> +        #f)
> Rather than discarding all tests, just discard the flaky ones.

I've tried to explain this before, but, as far as I've been able to determine, the problem doesn't seem to be that a particular test or group of tests is flaky: the mysterious problems all seem to be related to setting up the test environment. There have been problems, for example, with files not being found, or the complex pile of patches used to construct variants on the expected error output not working properly. At least the unix, windows, ieee, date, oop, primvars, and misc test groups have been affected. Since it's not been easy to reproduce the problem (I've thought it was fixed several times), I have no reason to believe it wouldn't affect other test groups if we skipped those. An unfortunate consequence of the way the tests are organized, skipping particular test groups means that expected error output is never checked and requires manually choosing the configurations (combinations of compiler/interpreter settings) you want to test.

I do not see a viable path to fixing the tests for this release, and I don't know a way of running a subset of tests that would be both reliable and meaningful.

Other distributions are also skipping these tests.

In fact, Guix is currently skipping an unknown (probably large) portion of these tests because of the issue with "/bin/sh", which has lead to things silently not running.

I hope we will get this fixed upstream for the 8.7 release. In fact, I've not yet encountered the problem while building from Git HEAD, but it's not clear what change might be relevant. (Indeed, the problem might still be there, and I might just not have hit it yet.)

-Philip