From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrea Corallo Newsgroups: gmane.emacs.devel Subject: Re: Flickering tests Date: Wed, 03 Jul 2024 10:52:37 -0400 Message-ID: References: <87ed8bhnrb.fsf@gmx.de> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="8954"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: emacs-devel@gnu.org To: Michael Albinus Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Jul 03 16:53:52 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sP1Mm-00029F-CV for ged-emacs-devel@m.gmane-mx.org; Wed, 03 Jul 2024 16:53:52 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sP1M1-0000ee-5W; Wed, 03 Jul 2024 10:53:05 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sP1Lv-0000Pc-Tc for emacs-devel@gnu.org; Wed, 03 Jul 2024 10:53:00 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sP1Lv-0003tE-06; Wed, 03 Jul 2024 10:52:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=C1xx4jHkp+TCk38mnapqplM6vULgAnjgSk6LSIALuOg=; b=cfDx2TjuqB/rKnorxPgF JgMUHqpV1a1Opj1nSUZqU1gfRCGGw29Um5Hqh9XvjQ6Gql1CbA7S9s9T+yL5xfSywQGqPsuEW5Eje BJc2/QKc/oqlekYj0N6ZKsdGzi25KWXoKUn/01KRR2rsTsPYEAnhOquYXwH/ES4E22r7j1eaE4tnQ au6U42pXCT1NUfyvXxxVQEkFUdvNeFRn360L5plRFoOlQahegU7yfyUI7/hHeaCVoRCb0JJO7+kFq 3NezVtTJESSnmhAzYGaX4/foFW13abC5RLuIGkp1aoMDVhyO4wWMNXX4SfR3bPZMLg+4Iig1panwC GDx9VSafVhFZ9A==; Original-Received: from acorallo by fencepost.gnu.org with local (Exim 4.90_1) (envelope-from ) id 1sP1LZ-0005Gq-IA; Wed, 03 Jul 2024 10:52:56 -0400 In-Reply-To: (Andrea Corallo's message of "Tue, 02 Jul 2024 11:25:41 -0400") X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:321257 Archived-At: Andrea Corallo writes: > Michael Albinus writes: > >> Andrea Corallo writes: >> >>> Hello, >> >> Hi Andrea, >> >>> lately I see some flickering tests in the testsuite. >>> >>> I see these tests failing irregularly on emacs-30 master and igc: >>> >>> dbus-test06-register-property-emits-signal >>> dbus-test05-register-signal-with-nils >>> dbus-test06-property-types >>> dbus-test03-peer-interface >>> secrets-test03-items >>> >>> Am I the only one? >> >> I don't see it here. >> >> Could you pls show the error messages of the failed tests? >> >>> Andrea >> >> Best regards, Michael. > > Hi Michael, > > this what I saw on a recent master for > dbus-test05-register-signal-with-nils and > dbus-test05-register-signal-with-nils: > > Test dbus-test05-register-signal-with-nils backtrace: > signal(ert-test-failed (((should (equal dbus--test-signal-received ' > ert-fail(((should (equal dbus--test-signal-received '("foo"))) :form > #f(compiled-function () #)() > #f(compiled-function () #)() > handler-bind-1(#f(compiled-function () # ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test > ert-run-test(#s(ert-test :name dbus-test05-register-signal-with-nils > ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m > ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co > ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable))) > ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un > eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) ( > command-line-1(("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-l > command-line() > normal-top-level() > Test dbus-test05-register-signal-with-nils condition: > (ert-test-failed > ((should (equal dbus--test-signal-received '("foo"))) :form > (equal > ((":1.171874" "/org/gtk/vfs/mount/1" "foo.tar.gz" > "archive:host=file%253A%252F%252F%252Fhome%252Fandcor03%252Femacs-ci%252Femacs-ci-repo1%252Ftest%252Flisp%252Fnet%252Ftramp-archive-resources%252Ffoo.tar.gz" > "" > ". GThemedIcon drive-removable-media drive-removable drive drive-removable-media-symbolic drive-removable-symbolic drive-symbolic" > ". GThemedIcon drive-removable-media-symbolic drive-removable-symbolic drive-symbolic drive-removable-media drive-removable drive" > "" t ... ...)) > ("foo")) > :value nil :explanation > (list-elt 0 > (different-types > (":1.171874" "/org/gtk/vfs/mount/1" "foo.tar.gz" > "archive:host=file%253A%252F%252F%252Fhome%252Fandcor03%252Femacs-ci%252Femacs-ci-repo1%252Ftest%252Flisp%252Fnet%252Ftramp-archive-resources%252Ffoo.tar.gz" > "" > ". GThemedIcon drive-removable-media drive-removable drive drive-removable-media-symbolic drive-removable-symbolic drive-symbolic" > ". GThemedIcon drive-removable-media-symbolic drive-removable-symbolic drive-symbolic drive-removable-media drive-removable drive" > "" t ... ...) > "foo")))) > FAILED 12/17 dbus-test05-register-signal-with-nils (0.112019 sec) at lisp/net/dbus-tests.el:795 > > Test dbus-test06-register-property-emits-signal backtrace: > signal(ert-test-failed (((should (equal dbus--test-signal-received ` > ert-fail(((should (equal dbus--test-signal-received `(,dbus--test-in > #f(compiled-function () #)() > #f(compiled-function () #)() > handler-bind-1(#f(compiled-function () # ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test > ert-run-test(#s(ert-test :name dbus-test06-register-property-emits-s > ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m > ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co > ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable))) > ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un > eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) ( > command-line-1(("-L" ":." "-l" "ert" "--eval" "(setq treesit-extra-l > command-line() > normal-top-level() > Test dbus-test06-register-property-emits-signal condition: > (ert-test-failed > ((should (equal dbus--test-signal-received `...)) :form > (equal ("org.gnu.Emacs.TestDBus") > ("org.gnu.Emacs.TestDBus.Interface" (...) nil)) > :value nil :explanation > (proper-lists-of-different-length 1 3 ("org.gnu.Emacs.TestDBus") > ("org.gnu.Emacs.TestDBus.Interface" > (...) nil) > first-mismatch-at 0))) > FAILED 15/17 dbus-test06-register-property-emits-signal (0.008444 sec) at lisp/net/dbus-tests.el:1163 > > Andrea Mmhh, I've the impression this happen on my system only when it's under heavy load. I've 64 cores and if I "make check -j64" I've say 50% probability to hit these, but if I run only: ./src/emacs -batch -l ert -l ./test/lisp/net/dbus-tests.el -f ert-run-tests-batch-and-exit they all pass. Perhaps some timeout need to be adjusted? Andrea