From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?Q?G=C3=A1bor?= Boskovits Subject: bug#30526: java-testng: non-deterministic test failures Date: Mon, 26 Feb 2018 08:23:15 +0100 Message-ID: References: <87606tjd8n.fsf@elephly.net> <20180225121658.639b198b@lepiller.eu> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="001a113f6c0e2dc16a0566186271" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:59317) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqD8o-0003Zm-QT for bug-guix@gnu.org; Mon, 26 Feb 2018 02:24:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqD8k-0002Yx-Q3 for bug-guix@gnu.org; Mon, 26 Feb 2018 02:24:06 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:53095) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eqD8k-0002Yp-LT for bug-guix@gnu.org; Mon, 26 Feb 2018 02:24:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1eqD8k-0000w8-FA for bug-guix@gnu.org; Mon, 26 Feb 2018 02:24:02 -0500 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <20180225121658.639b198b@lepiller.eu> List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Julien Lepiller Cc: 30526@debbugs.gnu.org --001a113f6c0e2dc16a0566186271 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable 2018-02-25 12:16 GMT+01:00 Julien Lepiller : > Le Mon, 19 Feb 2018 14:55:04 +0100, > Ricardo Wurmus a =C3=A9crit : > > > Sometimes I get this error when building java-testng 6.14.2: > > > > --8<---------------cut here---------------start------------->8--- > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > TestNG > > Total tests run: 1630, Failures: 1, Skips: 0 > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > Failures in :TestNG, :Regression2 > > test.testng387.TestNG387.testInvocationCounterIsCorrect > ForMethodWithDataProvider() > > StackTrace: > > java.lang.AssertionError: Arrays do not have the same size:3 !=3D 4 > > at > > test.testng387.TestNG387.testInvocationCounterIsCorrect > ForMethodWithDataProvider(Unknown > > Source) ... Removed 27 stack frames > > > > phase `check' failed after 297.3 seconds > > --8<---------------cut here---------------end--------------->8--- > > > > > > Hi, > > I've updated testng to the latest version (6.14.3) and it seems to > solve the issue. Can you confirm it? > > > Ok, I will check that today. It seems, that these are related to some race conditions in the parallel execution of tests. We also have verifyThatTestMethodsRunInParallelThreads failing sometimes. I will report back with results today. --001a113f6c0e2dc16a0566186271 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
2018= -02-25 12:16 GMT+01:00 Julien Lepiller <julien@lepiller.eu>= :
Le Mon, 19 Feb 2018 14:55:04 +0100,
Ricardo Wurmus <rekado@elephly.net= > a =C3=A9crit :

> Sometimes I get this error when building java-testng 6.14.2:
>
> --8<---------------cut here---------------start-------------&g= t;8---
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D
> TestNG
> Total tests run: 1630, Failures: 1, Skips: 0
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D
>
> Failures in <suite> :TestNG, <test> :Regression2
> test.testng387.TestNG387.testInvocationCounterIsCorrectForMe= thodWithDataProvider()
> StackTrace:
>=C2=A0 java.lang.AssertionError: Arrays do not have the same size:3 != =3D 4
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0at
> test.testng387.TestNG387.testInvocationCounterIsCorrectForMe= thodWithDataProvider(Unknown
> Source) ... Removed 27 stack frames
>
> phase `check' failed after 297.3 seconds
> --8<---------------cut here---------------end---------------&g= t;8---
>
>

Hi,

I've updated testng to the latest version (6.14.3) and it seems to
solve the issue. Can you confirm it?


=C2=A0
Ok, I will check that today.=C2=A0
It seems, that these are related to some race conditions in the p= arallel execution of tests. We also have verifyThatTestMethodsRunInParallel= Threads failing sometimes. I will report back with results today.
--001a113f6c0e2dc16a0566186271--