Re: T460: new sporadic failures with emacs 29

[Michael J Gruber <michaeljgruber+grubix+git@gmail.com>]

Am Do., 31. Aug. 2023 um 17:17 Uhr schrieb David Bremner <bremner@unb.ca>:
>
> Michael J Gruber <michaeljgruber+grubix+git@gmail.com> writes:
>
> >
> > I still get those issues. OTOH, skipping T460.14 did not show any
> > adverse side effects. So I'll do that for emacs29.
> > I might be nice to mark some tests ignored rather than skipped so that
> > we notice when they do not fail sporadically any more. That is, *if*
> > we look at the output of a passing test suite ...
> >
>
> It is possible to selectively mark tests as broken, but it requires
> patching the test suite, and it sets a failing exit code if those tests
> start passing.

Yes, that's why I wrote "ignore". Something like NOTMUCH_IGNORE_TESTS
which runs the test, outputs the diff on fail, but "succeeds" without
counting towards pass/fail, and reports the number of ignored
pass/fail separately - basically "known_broken" without the
"known/expectation".

I just don't know whether it's worth it. Other folks disable a whole
test suite when they want to get a package update going ...

Michael