Re: Flickering tests

[Michael Albinus <Michael.Albinus@gmx.de>]

[-- Attachment #1: Type: text/plain, Size: 1306 bytes --]

Andrea Corallo <acorallo@gnu.org> writes:

Hi Andrea,

> Mmmh not sure is possible but also desirable, we want to be able to run
> as many tests as possible even with -jN.

Last days, I've digged further into this. I still couldn't provoke the
error (my machine has only 8 processing units). However, ...

>> Btw, could you check whether the D-Bus tests flickery goes away if we
>> tag dbus-test05-register-signal-with-nils as :unstable?
>
> From the bunch of test I did seems this is solving, I'll mark it as
> :unstable for now till a better solution is found.

... I've found suspicious code in this test. Usually, all tests work
deterministic. A method call or a signal is raised, and the returned
message from D-Bus is analyzed. But in this test, a signal handler with
a wildcard filter is registered. That means, *any* signal from D-Bus is
catched, also signals which aren't relevant for this test. The more
pressure you give on the system ("make -j96"), the more likely it is.

So I have modified the signal handler to handle only signals we're
interested in. See the appended patch. Could you, pls, check whether it
helps? I guess we need it only in the master branch, because the problem
appears only in dbus-test.el. dbusbind.c and dbus.el are OK.

> Thanks
>
>   Andrea

Best regards, Michael.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 2851 bytes --]

diff --git a/test/lisp/net/dbus-tests.el b/test/lisp/net/dbus-tests.el
index b3337132fb6..0e0996b373e 100644
--- a/test/lisp/net/dbus-tests.el
+++ b/test/lisp/net/dbus-tests.el
@@ -25,7 +25,7 @@
 (require 'ert-x)
 (require 'dbus)

-(defvar dbus-debug nil)
+(defvar dbus-debug)
 (declare-function dbus-get-unique-name "dbusbind.c" (bus))

 (defconst dbus--test-enabled-session-bus
@@ -732,12 +732,28 @@ dbus-test04-call-method-timeout
     ;; Cleanup.
     (dbus-unregister-service :session dbus--test-service)))

+(defvar dbus--test-event-expected nil
+  "The expected event in `dbus--test-signal-handler'.")
+
 (defvar dbus--test-signal-received nil
   "Received signal value in `dbus--test-signal-handler'.")

 (defun dbus--test-signal-handler (&rest args)
   "Signal handler for `dbus-test*-signal' and `dbus-test08-register-monitor'."
-  (setq dbus--test-signal-received args))
+  (ignore-error dbus-error
+    (let ((last-input-event last-input-event))
+      (when (or (null dbus--test-event-expected)
+                (and (equal (dbus-event-bus-name last-input-event)
+                            (dbus-event-bus-name dbus--test-event-expected))
+                     (equal (dbus-event-message-type last-input-event)
+                            (dbus-event-message-type dbus--test-event-expected))
+                     (equal (dbus-event-service-name last-input-event)
+                            (dbus-event-service-name dbus--test-event-expected))
+                     (equal (dbus-event-path-name last-input-event)
+                            (dbus-event-path-name dbus--test-event-expected))
+                     (equal (dbus-event-member-name last-input-event)
+                            (dbus-event-member-name dbus--test-event-expected))))
+        (setq dbus--test-signal-received args)))))

 (defun dbus--test-timeout-handler (&rest _ignore)
   "Timeout handler, reporting a failed test."
@@ -804,6 +820,14 @@ dbus-test05-register-signal-with-nils
             (handler #'dbus--test-signal-handler)
             registered)

+        ;; Filter received signals in signal handler.
+        (setq dbus--test-event-expected
+              `(dbus-event :session ,dbus-message-type-signal
+                0 ;; Serial number doesn't matter.
+	        ,(dbus-get-unique-name :session)
+                nil ;; Destination doesn't matter.
+	        ,dbus--test-path ,dbus--test-interface "Member" ,handler))
+
         ;; Register signal handler.
         (should
          (equal
@@ -842,6 +866,7 @@ dbus-test05-register-signal-with-nils
         (should-not (dbus-unregister-object registered)))

     ;; Cleanup.
+    (setq dbus--test-event-expected nil)
     (dbus-unregister-service :session dbus--test-service)))

 (ert-deftest dbus-test06-register-property ()