From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark H Weaver Subject: bug#35181: Hydra offloads often get stuck while exporting build requisites Date: Sun, 07 Apr 2019 12:41:38 -0400 Message-ID: <87mul17oo2.fsf@netris.org> Mime-Version: 1.0 Content-Type: text/plain Return-path: Received: from eggs.gnu.org ([209.51.188.92]:56940) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hDAtn-00017f-Uq for bug-guix@gnu.org; Sun, 07 Apr 2019 12:44:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hDAtm-0006GA-RD for bug-guix@gnu.org; Sun, 07 Apr 2019 12:44:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:35103) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hDAtm-0006G0-K1 for bug-guix@gnu.org; Sun, 07 Apr 2019 12:44:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hDAtm-00081k-8t for bug-guix@gnu.org; Sun, 07 Apr 2019 12:44:02 -0400 Sender: "Debbugs-submit" Resent-Message-ID: Received: from eggs.gnu.org ([209.51.188.92]:56861) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hDAt7-00013r-Ou for bug-guix@gnu.org; Sun, 07 Apr 2019 12:43:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hDAt6-00060g-Pj for bug-guix@gnu.org; Sun, 07 Apr 2019 12:43:21 -0400 Received: from world.peace.net ([64.112.178.59]:56576) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hDAt6-0005x5-JP for bug-guix@gnu.org; Sun, 07 Apr 2019 12:43:20 -0400 List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: 35181@debbugs.gnu.org It has become extremely frequent for builds offloaded by hydra.gnu.org to its x86 build slave hydra.gnunet.org to get stuck indefinitely while exporting prerequisites for the build to the build slave. As I write this, both of hydra.gnunet.org's build slots (one for x86_64-linux, and one for i686-linux) are stuck in this way. Here are the stuck builds: https://hydra.gnu.org/build/3432052 https://hydra.gnu.org/build/3432472 and here are the tails of their nix build logs: --8<---------------cut here---------------start------------->8--- performing build 3432052 these derivations will be built: /gnu/store/k27i3lkb38gr3mw0mridymhik3qsg6w7-font-fira-sans-4.202.drv process 14769 acquired build slot '/var/guix/offload/hydra.gnunet.org/1' load on machine 'hydra.gnunet.org' is 1.54 (normalized: 0.385) sending 1 store item to 'hydra.gnunet.org'... exporting path `/gnu/store/gzd2cisahj50nff16p8ji813p683p5r4-font-fira-sans-4.202-checkout' --8<---------------cut here---------------end--------------->8--- --8<---------------cut here---------------start------------->8--- performing build 3432472 these derivations will be built: /gnu/store/5ivay4l7bn0sqsi7k53j4qv3kndrby17-font-google-material-design-icons-3.0.1.drv process 8985 acquired build slot '/var/guix/offload/hydra.gnunet.org/0' load on machine 'hydra.gnunet.org' is 1.98 (normalized: 0.99) sending 1 store item to 'hydra.gnunet.org'... exporting path `/gnu/store/kaj5xgnz04l4mzgj05sc5v6j7cvpbrrd-font-google-material-design-icons-3.0.1-checkout' --8<---------------cut here---------------end--------------->8--- The first of these builds has been stuck for about 10.25 hours, and the second has been stuck for 11 hours. In the recent past, I've discovered them stuck for over 24 hours. Mark