Re: Flickering tests

[Michael Albinus <Michael.Albinus@gmx.de>]

Andrea Corallo <acorallo@gnu.org> writes:

Hi Andrea,

> Mmhh, I've the impression this happen on my system only when it's under
> heavy load.  I've 64 cores and if I "make check -j64" I've say 50%
> probability to hit these, but if I run only:
>
> ./src/emacs -batch -l ert -l ./test/lisp/net/dbus-tests.el -f ert-run-tests-batch-and-exit
>
> they all pass.  Perhaps some timeout need to be adjusted?

It's not the a I believe. D-Bus tests are sending events to the D-Buson
OS level, and receiving events which will be interpreted. They are not
written such a way that all tests can run in parallel.

If you invoke "make check -j64", it could happen that some D-Bus tests
run in parallel, and returning events do arrive in another Emacs
instance but the one which has send the evnt. This might be even more
problematic, because recently (on Sunday, commit
4e22ef870c4b650f29c4441ac51b6a2ac506ea57) a test which registers for
wildcard events has been added. This test resides in my local Emacs git
for 3 months w/o problems; I've simply waited until the FSF papers of
the author, Robert Church, were on file.

What could we do? Could we exclude dbus-tests.el from running as
simultaneous job?

Or could we simply skip D-Bus tests, if we detect that the tests run via
simultaneous make jobs? How to check, on Lisp level?

We always have the option to mark this test as :unstable, so we could
stabilize the Emacs 30 release this way. But it would be a pity.

Btw, could you check whether the D-Bus tests flickery goes away if we
tag dbus-test05-register-signal-with-nils as :unstable?

> Andrea

Best regards, Michael.