* bug#30526: java-testng: non-deterministic test failures
@ 2018-02-19 13:55 Ricardo Wurmus
2018-02-21 9:07 ` Gábor Boskovits
2018-02-25 11:16 ` Julien Lepiller
0 siblings, 2 replies; 6+ messages in thread
From: Ricardo Wurmus @ 2018-02-19 13:55 UTC (permalink / raw)
To: 30526
Sometimes I get this error when building java-testng 6.14.2:
--8<---------------cut here---------------start------------->8---
===============================================
TestNG
Total tests run: 1630, Failures: 1, Skips: 0
===============================================
Failures in <suite> :TestNG, <test> :Regression2
test.testng387.TestNG387.testInvocationCounterIsCorrectForMethodWithDataProvider()
StackTrace:
java.lang.AssertionError: Arrays do not have the same size:3 != 4
at test.testng387.TestNG387.testInvocationCounterIsCorrectForMethodWithDataProvider(Unknown Source)
... Removed 27 stack frames
phase `check' failed after 297.3 seconds
--8<---------------cut here---------------end--------------->8---
--
Ricardo
GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
https://elephly.net
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#30526: java-testng: non-deterministic test failures
2018-02-19 13:55 bug#30526: java-testng: non-deterministic test failures Ricardo Wurmus
@ 2018-02-21 9:07 ` Gábor Boskovits
2018-02-21 9:20 ` Gábor Boskovits
2018-02-25 11:16 ` Julien Lepiller
1 sibling, 1 reply; 6+ messages in thread
From: Gábor Boskovits @ 2018-02-21 9:07 UTC (permalink / raw)
To: Ricardo Wurmus; +Cc: 30526
[-- Attachment #1: Type: text/plain, Size: 1145 bytes --]
2018-02-19 14:55 GMT+01:00 Ricardo Wurmus <rekado@elephly.net>:
> Sometimes I get this error when building java-testng 6.14.2:
>
> --8<---------------cut here---------------start------------->8---
> ===============================================
> TestNG
> Total tests run: 1630, Failures: 1, Skips: 0
> ===============================================
>
> Failures in <suite> :TestNG, <test> :Regression2
> test.testng387.TestNG387.testInvocationCounterIsCorrect
> ForMethodWithDataProvider()
> StackTrace:
> java.lang.AssertionError: Arrays do not have the same size:3 != 4
> at test.testng387.TestNG387.testInvocationCounterIsCorrect
> ForMethodWithDataProvider(Unknown Source)
> ... Removed 27 stack frames
>
> phase `check' failed after 297.3 seconds
> --8<---------------cut here---------------end--------------->8---
>
>
>
This actually seems to be known upstream, I consider reopening this bug:
https://github.com/cbeust/testng/issues/582.
It seems they just disabled the failing tests. I guess we should do the
same.
> --
> Ricardo
>
> GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
> https://elephly.net
>
>
>
>
>
[-- Attachment #2: Type: text/html, Size: 2050 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#30526: java-testng: non-deterministic test failures
2018-02-21 9:07 ` Gábor Boskovits
@ 2018-02-21 9:20 ` Gábor Boskovits
0 siblings, 0 replies; 6+ messages in thread
From: Gábor Boskovits @ 2018-02-21 9:20 UTC (permalink / raw)
To: Ricardo Wurmus; +Cc: 30526
[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]
2018-02-21 10:07 GMT+01:00 Gábor Boskovits <boskovits@gmail.com>:
> 2018-02-19 14:55 GMT+01:00 Ricardo Wurmus <rekado@elephly.net>:
>
>> Sometimes I get this error when building java-testng 6.14.2:
>>
>> --8<---------------cut here---------------start------------->8---
>> ===============================================
>> TestNG
>> Total tests run: 1630, Failures: 1, Skips: 0
>> ===============================================
>>
>> Failures in <suite> :TestNG, <test> :Regression2
>> test.testng387.TestNG387.testInvocationCounterIsCorrectForMe
>> thodWithDataProvider()
>> StackTrace:
>> java.lang.AssertionError: Arrays do not have the same size:3 != 4
>> at test.testng387.TestNG387.testInvocationCounterIsCorrectForMethodWithDataProvider(Unknown
>> Source)
>> ... Removed 27 stack frames
>>
>> phase `check' failed after 297.3 seconds
>> --8<---------------cut here---------------end--------------->8---
>>
>>
>>
> This actually seems to be known upstream, I consider reopening this bug:
> https://github.com/cbeust/testng/issues/582.
> It seems they just disabled the failing tests. I guess we should do the
> same.
>
I contacted upstream, and asked the status of this. I will report back when
they say something.
>
>
>> --
>> Ricardo
>>
>> GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
>> https://elephly.net
>>
>>
>>
>>
>>
>
[-- Attachment #2: Type: text/html, Size: 2879 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#30526: java-testng: non-deterministic test failures
2018-02-19 13:55 bug#30526: java-testng: non-deterministic test failures Ricardo Wurmus
2018-02-21 9:07 ` Gábor Boskovits
@ 2018-02-25 11:16 ` Julien Lepiller
2018-02-26 7:23 ` Gábor Boskovits
1 sibling, 1 reply; 6+ messages in thread
From: Julien Lepiller @ 2018-02-25 11:16 UTC (permalink / raw)
To: 30526
Le Mon, 19 Feb 2018 14:55:04 +0100,
Ricardo Wurmus <rekado@elephly.net> a écrit :
> Sometimes I get this error when building java-testng 6.14.2:
>
> --8<---------------cut here---------------start------------->8---
> ===============================================
> TestNG
> Total tests run: 1630, Failures: 1, Skips: 0
> ===============================================
>
> Failures in <suite> :TestNG, <test> :Regression2
> test.testng387.TestNG387.testInvocationCounterIsCorrectForMethodWithDataProvider()
> StackTrace:
> java.lang.AssertionError: Arrays do not have the same size:3 != 4
> at
> test.testng387.TestNG387.testInvocationCounterIsCorrectForMethodWithDataProvider(Unknown
> Source) ... Removed 27 stack frames
>
> phase `check' failed after 297.3 seconds
> --8<---------------cut here---------------end--------------->8---
>
>
Hi,
I've updated testng to the latest version (6.14.3) and it seems to
solve the issue. Can you confirm it?
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#30526: java-testng: non-deterministic test failures
2018-02-25 11:16 ` Julien Lepiller
@ 2018-02-26 7:23 ` Gábor Boskovits
2018-02-27 7:19 ` Gábor Boskovits
0 siblings, 1 reply; 6+ messages in thread
From: Gábor Boskovits @ 2018-02-26 7:23 UTC (permalink / raw)
To: Julien Lepiller; +Cc: 30526
[-- Attachment #1: Type: text/plain, Size: 1368 bytes --]
2018-02-25 12:16 GMT+01:00 Julien Lepiller <julien@lepiller.eu>:
> Le Mon, 19 Feb 2018 14:55:04 +0100,
> Ricardo Wurmus <rekado@elephly.net> a écrit :
>
> > Sometimes I get this error when building java-testng 6.14.2:
> >
> > --8<---------------cut here---------------start------------->8---
> > ===============================================
> > TestNG
> > Total tests run: 1630, Failures: 1, Skips: 0
> > ===============================================
> >
> > Failures in <suite> :TestNG, <test> :Regression2
> > test.testng387.TestNG387.testInvocationCounterIsCorrect
> ForMethodWithDataProvider()
> > StackTrace:
> > java.lang.AssertionError: Arrays do not have the same size:3 != 4
> > at
> > test.testng387.TestNG387.testInvocationCounterIsCorrect
> ForMethodWithDataProvider(Unknown
> > Source) ... Removed 27 stack frames
> >
> > phase `check' failed after 297.3 seconds
> > --8<---------------cut here---------------end--------------->8---
> >
> >
>
> Hi,
>
> I've updated testng to the latest version (6.14.3) and it seems to
> solve the issue. Can you confirm it?
>
>
>
Ok, I will check that today.
It seems, that these are related to some race conditions in the parallel
execution of tests. We also have verifyThatTestMethodsRunInParallelThreads
failing sometimes. I will report back with results today.
[-- Attachment #2: Type: text/html, Size: 1936 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* bug#30526: java-testng: non-deterministic test failures
2018-02-26 7:23 ` Gábor Boskovits
@ 2018-02-27 7:19 ` Gábor Boskovits
0 siblings, 0 replies; 6+ messages in thread
From: Gábor Boskovits @ 2018-02-27 7:19 UTC (permalink / raw)
To: Julien Lepiller; +Cc: 30526, 30526-done
[-- Attachment #1: Type: text/plain, Size: 1613 bytes --]
2018-02-26 8:23 GMT+01:00 Gábor Boskovits <boskovits@gmail.com>:
> 2018-02-25 12:16 GMT+01:00 Julien Lepiller <julien@lepiller.eu>:
>
>> Le Mon, 19 Feb 2018 14:55:04 +0100,
>> Ricardo Wurmus <rekado@elephly.net> a écrit :
>>
>> > Sometimes I get this error when building java-testng 6.14.2:
>> >
>> > --8<---------------cut here---------------start------------->8---
>> > ===============================================
>> > TestNG
>> > Total tests run: 1630, Failures: 1, Skips: 0
>> > ===============================================
>> >
>> > Failures in <suite> :TestNG, <test> :Regression2
>> > test.testng387.TestNG387.testInvocationCounterIsCorrectForMe
>> thodWithDataProvider()
>> > StackTrace:
>> > java.lang.AssertionError: Arrays do not have the same size:3 != 4
>> > at
>> > test.testng387.TestNG387.testInvocationCounterIsCorrectForMe
>> thodWithDataProvider(Unknown
>> > Source) ... Removed 27 stack frames
>> >
>> > phase `check' failed after 297.3 seconds
>> > --8<---------------cut here---------------end--------------->8---
>> >
>> >
>>
>> Hi,
>>
>> I've updated testng to the latest version (6.14.3) and it seems to
>> solve the issue. Can you confirm it?
>>
>>
>>
> Ok, I will check that today.
> It seems, that these are related to some race conditions in the parallel
> execution of tests. We also have verifyThatTestMethodsRunInParallelThreads
> failing sometimes. I will report back with results today.
>
guix build --rounds=30 run to completion with this update, referring to
discussion on the IRC I'm closing this bug as fixed.
[-- Attachment #2: Type: text/html, Size: 2529 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-02-27 7:20 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-02-19 13:55 bug#30526: java-testng: non-deterministic test failures Ricardo Wurmus
2018-02-21 9:07 ` Gábor Boskovits
2018-02-21 9:20 ` Gábor Boskovits
2018-02-25 11:16 ` Julien Lepiller
2018-02-26 7:23 ` Gábor Boskovits
2018-02-27 7:19 ` Gábor Boskovits
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).