bug#33362: System tests stuck in "shepherd[1]: waiting for udevd..."

[Mark H Weaver <mhw@netris.org>]

Hi Simon,

zimoun <zimon.toutoune@gmail.com> writes:

> On Mon, 12 Nov 2018 at 20:09, Mark H Weaver <mhw@netris.org> wrote:
>
>> As I write this, there are two system test builds that have been stuck
>> for many hours, endlessly printing "shepherd[1]: waiting for udevd...":
>>
>>   https://hydra.gnu.org/build/3153725
>>   https://hydra.gnu.org/build/3154365
>>
>> They are both on i686-linux, and on the 'core-updates' branch, but with
>> two different build slaves (hydra.gnunet.org and guix.sjd.se).
>>
>> I will now abort these builds, to free up build slots for other jobs.
>
> I am doing bug triage and I hit this old one #33362 [1] from 2018.  It
> is about hydra which is down now, IIRC.

I doubt that this bug was about Hydra.  It guess that the bug was in our
system test derivations.  As far as I know, the only relevance of Hydra
to this bug is that Hydra was our CI system at that time, and therefore
it was Hydra that brought this bug to my attention.

> Does it still make sense to keep it open?  Or can we close it?

If the bug hasn't occurred recently, then I agree it's okay to close it.

It would be good to check our modern Cuirass-based ci.guix.gnu.org to
find out whether this failure mode is still occurring in our system
tests.

I see that Cuirass's web interface has improved quite significantly in
the last couple of years, and I'm very grateful to those who've worked
on it.  However, Cuirass still seems to be missing some important
functionality that Hydra had.  Most notably, unless I missed something,
it seems to lack the ability to compare the results of two evaluations
and show the *differences* between those results, i.e. to enumerate the
newly failing jobs, the newly succeeding jobs, and the newly aborted
jobs.

Without that functionality, it's not easy for us to notice when a job
starts to fail, unless a user files a bug report.  The total list of job
failures has always been too large to easily notice changes in that list
without assistance.

What do you think?

Thanks again for your work on this.

      Regards,
        Mark

-- 
Disinformation flourishes because many people care deeply about injustice
but very few check the facts.  Ask me about <https://stallmansupport.org>.