unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
@ 2018-01-31  3:07 Mark H Weaver
  2018-02-03 19:53 ` Efraim Flashner
                   ` (3 more replies)
  0 siblings, 4 replies; 14+ messages in thread
From: Mark H Weaver @ 2018-01-31  3:07 UTC (permalink / raw)
  To: 30299

On core-updates, Hydra has been unable to successfully build 'shepherd'
on any system except x86_64-linux.  I can also report that on my
mips64el-linux GuixSD system, which is running something close to
'core-updates', I had to disable tests on shepherd in order to build it.
I don't know about aarch64-linux.

These are the tests that commonly fail:

  FAIL: tests/respawn.sh
  FAIL: tests/respawn-throttling.sh
  FAIL: tests/basic.sh

      Mark

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
  2018-01-31  3:07 bug#30299: [core-updates] shepherd fails tests on all systems except x86_64 Mark H Weaver
@ 2018-02-03 19:53 ` Efraim Flashner
  2018-02-04 22:48 ` Ludovic Courtès
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 14+ messages in thread
From: Efraim Flashner @ 2018-02-03 19:53 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: 30299

[-- Attachment #1: Type: text/plain, Size: 839 bytes --]

On Tue, Jan 30, 2018 at 10:07:35PM -0500, Mark H Weaver wrote:
> On core-updates, Hydra has been unable to successfully build 'shepherd'
> on any system except x86_64-linux.  I can also report that on my
> mips64el-linux GuixSD system, which is running something close to
> 'core-updates', I had to disable tests on shepherd in order to build it.
> I don't know about aarch64-linux.
> 
> These are the tests that commonly fail:
> 
>   FAIL: tests/respawn.sh
>   FAIL: tests/respawn-throttling.sh
>   FAIL: tests/basic.sh
> 
>       Mark

Shepherd built successfully on aarch64-linux, and again with '--check'.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
  2018-01-31  3:07 bug#30299: [core-updates] shepherd fails tests on all systems except x86_64 Mark H Weaver
  2018-02-03 19:53 ` Efraim Flashner
@ 2018-02-04 22:48 ` Ludovic Courtès
  2018-02-14  8:52   ` Mark H Weaver
  2018-02-14  9:16 ` Mark H Weaver
  2020-03-19  8:28 ` Brice Waegeneire
  3 siblings, 1 reply; 14+ messages in thread
From: Ludovic Courtès @ 2018-02-04 22:48 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: 30299

Hello,

Mark H Weaver <mhw@netris.org> skribis:

> On core-updates, Hydra has been unable to successfully build 'shepherd'
> on any system except x86_64-linux.  I can also report that on my
> mips64el-linux GuixSD system, which is running something close to
> 'core-updates', I had to disable tests on shepherd in order to build it.
> I don't know about aarch64-linux.
>
> These are the tests that commonly fail:
>
>   FAIL: tests/respawn.sh
>   FAIL: tests/respawn-throttling.sh
>   FAIL: tests/basic.sh

This is a non-deterministic failure.  I could reproduce the
tests/basic.sh one and it is fixed by this:

  https://git.savannah.gnu.org/cgit/shepherd.git/commit/?id=cc9564586729a5bb90dd5d2722b543fdde9ab821

I’ll roll a new Shepherd release soon and update the package.

The rest may be a duplicate of 23811, though I couldn’t reproduce it.

Thanks,
Ludo’.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
  2018-02-04 22:48 ` Ludovic Courtès
@ 2018-02-14  8:52   ` Mark H Weaver
  2018-02-14 13:00     ` Ludovic Courtès
  2018-02-15 19:21     ` Mark H Weaver
  0 siblings, 2 replies; 14+ messages in thread
From: Mark H Weaver @ 2018-02-14  8:52 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 30299

ludo@gnu.org (Ludovic Courtès) writes:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> On core-updates, Hydra has been unable to successfully build 'shepherd'
>> on any system except x86_64-linux.  I can also report that on my
>> mips64el-linux GuixSD system, which is running something close to
>> 'core-updates', I had to disable tests on shepherd in order to build it.
>> I don't know about aarch64-linux.
>>
>> These are the tests that commonly fail:
>>
>>   FAIL: tests/respawn.sh
>>   FAIL: tests/respawn-throttling.sh
>>   FAIL: tests/basic.sh
>
> This is a non-deterministic failure.  I could reproduce the
> tests/basic.sh one and it is fixed by this:
>
>   https://git.savannah.gnu.org/cgit/shepherd.git/commit/?id=cc9564586729a5bb90dd5d2722b543fdde9ab821

Thank you!

> I’ll roll a new Shepherd release soon and update the package.

For now, I added your patch to the shepherd package in core-updates, in
commit f2d2ee42f168909f27c0c3b6532ef16febfd3b86.

      Mark

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
  2018-01-31  3:07 bug#30299: [core-updates] shepherd fails tests on all systems except x86_64 Mark H Weaver
  2018-02-03 19:53 ` Efraim Flashner
  2018-02-04 22:48 ` Ludovic Courtès
@ 2018-02-14  9:16 ` Mark H Weaver
  2020-03-19  8:28 ` Brice Waegeneire
  3 siblings, 0 replies; 14+ messages in thread
From: Mark H Weaver @ 2018-02-14  9:16 UTC (permalink / raw)
  To: 30299

Mark H Weaver <mhw@netris.org> writes:

> On core-updates, Hydra has been unable to successfully build 'shepherd'
> on any system except x86_64-linux.  I can also report that on my
> mips64el-linux GuixSD system, which is running something close to
> 'core-updates', I had to disable tests on shepherd in order to build it.
> I don't know about aarch64-linux.
>
> These are the tests that commonly fail:
>
>   FAIL: tests/respawn.sh
>   FAIL: tests/respawn-throttling.sh
>   FAIL: tests/basic.sh

FYI, after 18 failed attempts, Hydra finally built shepherd on i686
successfully on the 19th try:

  https://hydra.gnu.org/build/2425454#tabs-buildsteps

Now that I've added Ludovic's patch, hopefully it will require fewer
attempts this time :)

       Mark

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
  2018-02-14  8:52   ` Mark H Weaver
@ 2018-02-14 13:00     ` Ludovic Courtès
  2018-02-15 19:21     ` Mark H Weaver
  1 sibling, 0 replies; 14+ messages in thread
From: Ludovic Courtès @ 2018-02-14 13:00 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: 30299

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> On core-updates, Hydra has been unable to successfully build 'shepherd'
>>> on any system except x86_64-linux.  I can also report that on my
>>> mips64el-linux GuixSD system, which is running something close to
>>> 'core-updates', I had to disable tests on shepherd in order to build it.
>>> I don't know about aarch64-linux.
>>>
>>> These are the tests that commonly fail:
>>>
>>>   FAIL: tests/respawn.sh
>>>   FAIL: tests/respawn-throttling.sh
>>>   FAIL: tests/basic.sh
>>
>> This is a non-deterministic failure.  I could reproduce the
>> tests/basic.sh one and it is fixed by this:
>>
>>   https://git.savannah.gnu.org/cgit/shepherd.git/commit/?id=cc9564586729a5bb90dd5d2722b543fdde9ab821
>
> Thank you!
>
>> I’ll roll a new Shepherd release soon and update the package.
>
> For now, I added your patch to the shepherd package in core-updates, in
> commit f2d2ee42f168909f27c0c3b6532ef16febfd3b86.

Thanks, and sorry for not acting earlier!  (I was waiting for a reply
from the Translation Project to make the new release, but that hasn’t
happened yet.)

Ludo’.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
  2018-02-14  8:52   ` Mark H Weaver
  2018-02-14 13:00     ` Ludovic Courtès
@ 2018-02-15 19:21     ` Mark H Weaver
  2018-02-15 21:26       ` Ludovic Courtès
  2018-02-17  0:04       ` Ludovic Courtès
  1 sibling, 2 replies; 14+ messages in thread
From: Mark H Weaver @ 2018-02-15 19:21 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 30299

Hi Ludovic,

Mark H Weaver <mhw@netris.org> writes:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> On core-updates, Hydra has been unable to successfully build 'shepherd'
>>> on any system except x86_64-linux.  I can also report that on my
>>> mips64el-linux GuixSD system, which is running something close to
>>> 'core-updates', I had to disable tests on shepherd in order to build it.
>>> I don't know about aarch64-linux.
>>>
>>> These are the tests that commonly fail:
>>>
>>>   FAIL: tests/respawn.sh
>>>   FAIL: tests/respawn-throttling.sh
>>>   FAIL: tests/basic.sh
>>
>> This is a non-deterministic failure.  I could reproduce the
>> tests/basic.sh one and it is fixed by this:
>>
>>   https://git.savannah.gnu.org/cgit/shepherd.git/commit/?id=cc9564586729a5bb90dd5d2722b543fdde9ab821
>
> For now, I added your patch to the shepherd package in core-updates, in
> commit f2d2ee42f168909f27c0c3b6532ef16febfd3b86.

With your patch, Hydra built shepherd successfully on i686-linux on the
first try, which is much better than the 19th try :)

However, on armhf-linux, three tests failed: respawn.sh,
respawn-throttling.sh, and pid-file.sh.

  https://hydra.gnu.org/build/2499835

We should probably arrange for test-suite.log to be printed when "make
check" fails.  We could add this functionality to selected packages like
shepherd and guix the next time we update them, and maybe consider
adding something generic to gnu-build-system's check phase in the next
core-updates cycle.  What do you think?

      Mark

--8<---------------cut here---------------start------------->8---
starting phase `check'
make  check-am
make[1]: Entering directory '/tmp/guix-build-shepherd-0.3.2.drv-0/shepherd-0.3.2'
make  check-TESTS
make[2]: Entering directory '/tmp/guix-build-shepherd-0.3.2.drv-0/shepherd-0.3.2'
make[3]: Entering directory '/tmp/guix-build-shepherd-0.3.2.drv-0/shepherd-0.3.2'
PASS: tests/misbehaved-client.sh
FAIL: tests/respawn.sh
PASS: tests/no-home.sh
FAIL: tests/pid-file.sh
PASS: tests/status-sexp.sh
PASS: tests/sigint.sh
PASS: tests/basic.sh
FAIL: tests/respawn-throttling.sh
============================================================================
Testsuite summary for GNU Shepherd 0.3.2
============================================================================
# TOTAL: 8
# PASS:  5
# SKIP:  0
# XFAIL: 0
# FAIL:  3
# XPASS: 0
# ERROR: 0
============================================================================
See ./test-suite.log
Please report to bug-guix@gnu.org
============================================================================
make[3]: *** [Makefile:1220: test-suite.log] Error 1
make[3]: Leaving directory '/tmp/guix-build-shepherd-0.3.2.drv-0/shepherd-0.3.2'
make[2]: *** [Makefile:1328: check-TESTS] Error 2
make[2]: Leaving directory '/tmp/guix-build-shepherd-0.3.2.drv-0/shepherd-0.3.2'
make[1]: *** [Makefile:1527: check-am] Error 2
make[1]: Leaving directory '/tmp/guix-build-shepherd-0.3.2.drv-0/shepherd-0.3.2'
make: *** [Makefile:1529: check] Error 2
phase `check' failed after 29.6 seconds
builder for `/gnu/store/sk0l3ll4x1ddn9zwxmfqjipr578hiqx1-shepherd-0.3.2.drv' failed with exit code 1
@ build-failed /gnu/store/sk0l3ll4x1ddn9zwxmfqjipr578hiqx1-shepherd-0.3.2.drv - 1 builder for `/gnu/store/sk0l3ll4x1ddn9zwxmfqjipr578hiqx1-shepherd-0.3.2.drv' failed with exit code 1
--8<---------------cut here---------------end--------------->8---

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
  2018-02-15 19:21     ` Mark H Weaver
@ 2018-02-15 21:26       ` Ludovic Courtès
  2018-02-17  0:04       ` Ludovic Courtès
  1 sibling, 0 replies; 14+ messages in thread
From: Ludovic Courtès @ 2018-02-15 21:26 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: 30299

Hi Mark,

Mark H Weaver <mhw@netris.org> skribis:

> Mark H Weaver <mhw@netris.org> writes:

[...]

>>> This is a non-deterministic failure.  I could reproduce the
>>> tests/basic.sh one and it is fixed by this:
>>>
>>>   https://git.savannah.gnu.org/cgit/shepherd.git/commit/?id=cc9564586729a5bb90dd5d2722b543fdde9ab821
>>
>> For now, I added your patch to the shepherd package in core-updates, in
>> commit f2d2ee42f168909f27c0c3b6532ef16febfd3b86.
>
> With your patch, Hydra built shepherd successfully on i686-linux on the
> first try, which is much better than the 19th try :)

We’re making progress.  :-)

> However, on armhf-linux, three tests failed: respawn.sh,
> respawn-throttling.sh, and pid-file.sh.
>
>   https://hydra.gnu.org/build/2499835

I’ll try to reproduce the failure here.

> We should probably arrange for test-suite.log to be printed when "make
> check" fails.  We could add this functionality to selected packages like
> shepherd and guix the next time we update them, and maybe consider
> adding something generic to gnu-build-system's check phase in the next
> core-updates cycle.  What do you think?

Definitely.  We discussed it before and I think it’s a good idea.  I
wanted to add it to ‘guix’ as a starting point but never got around to
doing it.

Thanks,
Ludo’.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
  2018-02-15 19:21     ` Mark H Weaver
  2018-02-15 21:26       ` Ludovic Courtès
@ 2018-02-17  0:04       ` Ludovic Courtès
  2018-02-24 23:00         ` Andreas Enge
  1 sibling, 1 reply; 14+ messages in thread
From: Ludovic Courtès @ 2018-02-17  0:04 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: 30299

Hello,

Mark H Weaver <mhw@netris.org> skribis:

> However, on armhf-linux, three tests failed: respawn.sh,
> respawn-throttling.sh, and pid-file.sh.
>
>   https://hydra.gnu.org/build/2499835

(Similar issue on aarch64:
<https://berlin.guixsd.org/build/2419870/log/raw>.  Though of course it
passed on the 2nd and 3rd attempts…)

I was able to reproduce a tests/respawn.sh failure on hardware (ARMv7).
The issue is that a service is not respawned, and the log shows:

--8<---------------cut here---------------start------------->8---
+ assert_killed_service_is_respawned t-service2-pid-695
++ cat t-service2-pid-695
+ old_pid=789
+ rm t-service2-pid-695
+ kill 789
+ wait_for_file t-service2-pid-695
+ i=0
+ test -f t-service2-pid-695
+ test 0 -lt 20
+ sleep 0.3
++ expr 0 + 1

[...]

2018-02-16 11:13:31 Service root has been started.
2018-02-16 11:13:32 Service test1 has been started.
2018-02-16 11:13:34 Service test2 has been started.
2018-02-16 11:13:35 Respawning test1.
2018-02-16 11:13:35 Service test1 has been started.
2018-02-16 11:13:36 Respawning test2.
2018-02-16 11:13:37 Service test2 has been started.
2018-02-16 11:13:37 Respawning test1.
2018-02-16 11:13:37 Service test1 has been started.
2018-02-16 11:13:38 Respawning test2.
2018-02-16 11:13:43 Service test2 could not be started.
--8<---------------cut here---------------end--------------->8---

So SIGCHLD was correctly delivered, but somehow restarting that service
didn’t work (its PID file didn’t show up again; the 5 seconds between
“Respawning” and “could not be started” correspond to the delay in
‘read-pid-file’ in (shepherd service)).  

These test failures seem to be more frequent when the machine is loaded.

Ludo’.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
  2018-02-17  0:04       ` Ludovic Courtès
@ 2018-02-24 23:00         ` Andreas Enge
  0 siblings, 0 replies; 14+ messages in thread
From: Andreas Enge @ 2018-02-24 23:00 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 30299

Hello,

I changed the severity to "serious", since this bug prevents installing
GuixSD on arm, or creating an installation image. Also, it is now present
on master instead of core-updates.

Andreas

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
  2018-01-31  3:07 bug#30299: [core-updates] shepherd fails tests on all systems except x86_64 Mark H Weaver
                   ` (2 preceding siblings ...)
  2018-02-14  9:16 ` Mark H Weaver
@ 2020-03-19  8:28 ` Brice Waegeneire
  2020-03-19  9:16   ` Marius Bakke
  3 siblings, 1 reply; 14+ messages in thread
From: Brice Waegeneire @ 2020-03-19  8:28 UTC (permalink / raw)
  To: 30299

Hello,

Does this bug is still relevant?
It was reported on core-updates 2 years ago with shepherd 0.3.2.
I can't see any CI failures[0] related to a test failing since cuirass 
was setup, the only time the build of shepherd failed[1] it wasn't due 
to a test.
I wasn't able to reproduce the failing test with some building rounds on 
armhf and i686.

> I changed the severity to "serious", since this bug prevents installing
> GuixSD on arm, or creating an installation image. Also, it is now 
> present
> on master instead of core-updates.

As far as I know this isn't the case anymore, so at least the priority 
should be lowered.

[0]: http://ci.guix.gnu.org/search?query=shepherd
[1]: http://ci.guix.gnu.org/build/1312246/details

Brice.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#30299: [core-updates] shepherd fails tests on all systems except x86_64
  2020-03-19  8:28 ` Brice Waegeneire
@ 2020-03-19  9:16   ` Marius Bakke
  2020-03-20  9:58     ` bug#23811: shepherd: tests/respawn-throttling.sh fails sporadically Brice Waegeneire
  0 siblings, 1 reply; 14+ messages in thread
From: Marius Bakke @ 2020-03-19  9:16 UTC (permalink / raw)
  To: Brice Waegeneire, 30299-done

[-- Attachment #1: Type: text/plain, Size: 551 bytes --]

Brice Waegeneire <brice@waegenei.re> writes:

> Hello,
>
> Does this bug is still relevant?
> It was reported on core-updates 2 years ago with shepherd 0.3.2.
> I can't see any CI failures[0] related to a test failing since cuirass 
> was setup, the only time the build of shepherd failed[1] it wasn't due 
> to a test.
> I wasn't able to reproduce the failing test with some building rounds on 
> armhf and i686.

Thank you for doing bug triage Brice.  I'm closing the issue as we
haven't had problems building the shepherd in a while.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#23811: shepherd: tests/respawn-throttling.sh fails sporadically
  2020-03-19  9:16   ` Marius Bakke
@ 2020-03-20  9:58     ` Brice Waegeneire
  2020-03-20 17:26       ` Marius Bakke
  0 siblings, 1 reply; 14+ messages in thread
From: Brice Waegeneire @ 2020-03-20  9:58 UTC (permalink / raw)
  To: 23811

Hello,

#23811 (merged with #28588) are about the same tests failing as
the recently closed #30299 and were reported 4 et 3 years.
I suggest to close then too.

Brice.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* bug#23811: shepherd: tests/respawn-throttling.sh fails sporadically
  2020-03-20  9:58     ` bug#23811: shepherd: tests/respawn-throttling.sh fails sporadically Brice Waegeneire
@ 2020-03-20 17:26       ` Marius Bakke
  0 siblings, 0 replies; 14+ messages in thread
From: Marius Bakke @ 2020-03-20 17:26 UTC (permalink / raw)
  To: Brice Waegeneire, 23811-done

[-- Attachment #1: Type: text/plain, Size: 237 bytes --]

Brice Waegeneire <brice@waegenei.re> writes:

> Hello,
>
> #23811 (merged with #28588) are about the same tests failing as
> the recently closed #30299 and were reported 4 et 3 years.
> I suggest to close then too.

Good catch, closing!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2020-03-20 17:27 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-31  3:07 bug#30299: [core-updates] shepherd fails tests on all systems except x86_64 Mark H Weaver
2018-02-03 19:53 ` Efraim Flashner
2018-02-04 22:48 ` Ludovic Courtès
2018-02-14  8:52   ` Mark H Weaver
2018-02-14 13:00     ` Ludovic Courtès
2018-02-15 19:21     ` Mark H Weaver
2018-02-15 21:26       ` Ludovic Courtès
2018-02-17  0:04       ` Ludovic Courtès
2018-02-24 23:00         ` Andreas Enge
2018-02-14  9:16 ` Mark H Weaver
2020-03-19  8:28 ` Brice Waegeneire
2020-03-19  9:16   ` Marius Bakke
2020-03-20  9:58     ` bug#23811: shepherd: tests/respawn-throttling.sh fails sporadically Brice Waegeneire
2020-03-20 17:26       ` Marius Bakke

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).