From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrea Corallo Newsgroups: gmane.emacs.devel Subject: Re: Flickering tests Date: Wed, 03 Jul 2024 03:25:08 -0400 Message-ID: References: <87v81nl00w.fsf@pub.pink> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="5960"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: emacs-devel@gnu.org, Michael Albinus To: john muhl Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Jul 03 09:25:52 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sOuNE-0001F3-DH for ged-emacs-devel@m.gmane-mx.org; Wed, 03 Jul 2024 09:25:52 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sOuMq-0008IV-9E; Wed, 03 Jul 2024 03:25:28 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sOuMl-0008B6-2U for emacs-devel@gnu.org; Wed, 03 Jul 2024 03:25:25 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sOuMi-0002m1-Ra; Wed, 03 Jul 2024 03:25:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=snnNSg4rsSCxEwv4rVXSVnDur1nRAGXgCh4q2SDOadE=; b=JZ4tuIjiB1WqgG9d1p+D GEVysLXt4K0MPCq7BFDR4RFfv7K2bF2llLZojSdu/gtkzxFKG+S1toBbVM017i94nCTmNHF1yyrpD d6P8S96QnRnqtS+Xh46tmEUTQ0pq/ATNz1F+cNlSTWYcaiYqOWjAdLWz9n+yvhi8wHjJ4WJzKbM8A yyDFI9lBOyFjNbfi5iwVObDpwAtCXt2sXK/c2cjF1TbKq3SGENgrbkHc7eSU5cicnMckehLJPNJRc /615+Hrgn6/BC8MHqDyTPItRBkjbr5+KOQkylt5762kxUHNJZe8yT3aAVd+LjvYvpFoGIdy4nlp9P AFlVliDlioQ+3A==; Original-Received: from acorallo by fencepost.gnu.org with local (Exim 4.90_1) (envelope-from ) id 1sOuMW-00008M-55; Wed, 03 Jul 2024 03:25:13 -0400 In-Reply-To: <87v81nl00w.fsf@pub.pink> (john muhl's message of "Tue, 02 Jul 2024 20:47:11 -0500") X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:321235 Archived-At: john muhl writes: > Andrea Corallo writes: > >> Andrea Corallo writes: >> >>> Hello, >>> >>> lately I see some flickering tests in the testsuite. >>> >>> I see these tests failing irregularly on emacs-30 master and igc: >>> >>> dbus-test06-register-property-emits-signal >>> dbus-test05-register-signal-with-nils >>> dbus-test06-property-types >>> dbus-test03-peer-interface >>> secrets-test03-items >> >> Forgot to add to the list these two: >> >> threads-condvar-wait > > I haven=E2=80=99t seen any of the above failing. > >> track-changes-tests--random > > This one has always been flaky on my test machines. > > I also see intermittent failures for: > > src/process-tests > lisp/eshell/esh-cmd-tests > lisp/eshell/esh-proc-tests > lisp/eshell/esh-var-tests > lisp/gnus/mml-sec-tests > lisp/net/network-stream-tests > lisp/progmodes/eglot-tests > > I assumed these were due to the machines the being underpowered Mmmh, ideally tests should not be sensitive to the computational power of the machine. > but can dig up the logs if someone wants to have a look at them. I think we should put an effort in reaching a stable a clean testsuite, should we have separate bugs for each test where we do the investigation/fix?