Re: Flickering tests

[Andrea Corallo <acorallo@gnu.org>]

john muhl <jm@pub.pink> writes:

> Andrea Corallo <acorallo@gnu.org> writes:
>
>> Andrea Corallo <acorallo@gnu.org> writes:
>>
>>> Hello,
>>>
>>> lately I see some flickering tests in the testsuite.
>>>
>>> I see these tests failing irregularly on emacs-30 master and igc:
>>>
>>> dbus-test06-register-property-emits-signal
>>> dbus-test05-register-signal-with-nils
>>> dbus-test06-property-types
>>> dbus-test03-peer-interface
>>> secrets-test03-items
>>
>> Forgot to add to the list these two:
>>
>> threads-condvar-wait
>
> I haven’t seen any of the above failing.
>
>> track-changes-tests--random
>
> This one has always been flaky on my test machines.
>
> I also see intermittent failures for:
>
>   src/process-tests
>   lisp/eshell/esh-cmd-tests
>   lisp/eshell/esh-proc-tests
>   lisp/eshell/esh-var-tests
>   lisp/gnus/mml-sec-tests
>   lisp/net/network-stream-tests
>   lisp/progmodes/eglot-tests
>
> I assumed these were due to the machines the being underpowered

Mmmh, ideally tests should not be sensitive to the computational power
of the machine.

> but can dig up the logs if someone wants to have a look at them.

I think we should put an effort in reaching a stable a clean testsuite,
should we have separate bugs for each test where we do the
investigation/fix?