From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ivan Petkov Subject: bug#35139: Rust builds systematically time out Date: Thu, 4 Apr 2019 08:47:42 -0700 Message-ID: <101FBDE5-97FA-4449-9076-DD24C56B8715@gmail.com> References: <878swqtabb.fsf@gnu.org> <87bm1mglus.fsf@gmx.com> Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Content-Type: multipart/alternative; boundary="Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D" Return-path: Received: from eggs.gnu.org ([209.51.188.92]:34613) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hC4bv-0004nA-Fl for bug-guix@gnu.org; Thu, 04 Apr 2019 11:49:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hC4bu-0002vw-9Z for bug-guix@gnu.org; Thu, 04 Apr 2019 11:49:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:59280) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hC4bu-0002vd-6D for bug-guix@gnu.org; Thu, 04 Apr 2019 11:49:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hC4bt-0003CX-Un for bug-guix@gnu.org; Thu, 04 Apr 2019 11:49:01 -0400 Sender: "Debbugs-submit" Resent-Message-ID: Received: from eggs.gnu.org ([209.51.188.92]:34295) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hC4az-0004id-Rp for bug-guix@gnu.org; Thu, 04 Apr 2019 11:48:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hC4ay-0001OT-N2 for bug-guix@gnu.org; Thu, 04 Apr 2019 11:48:05 -0400 In-Reply-To: <87bm1mglus.fsf@gmx.com> List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Pierre Langlois , Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 35139@debbugs.gnu.org --Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Apr 4, 2019, at 1:59 AM, Ludovic Court=C3=A8s wrote: >=20 > The build nodes may be slower than the front-end, but still, it seems > unlikely that it would take more than 6h there. (That could happen if > the test suite, which lasts 2.1h, were =E2=80=9Cembarrassingly = parallel=E2=80=9D, but > we=E2=80=99re running tests with =E2=80=98-j1=E2=80=99.) >=20 > To summarize, there are two problems: >=20 > 1. Rust takes too long to build. What can we do about it? Enable > parallel builds? Rust tests are designed to run in parallel, as long as you have enough RAM, file descriptors, etc. available on the machine for the amount of concurrency being used. The compiler test suite is largely just = compiling files, so the most important resource is probably available RAM/swap. > On Apr 4, 2019, at 2:28 AM, Pierre Langlois = wrote: >=20 > One thing I suggested in the past was to remove the check phase *only* > for rust packages used for bootstrapping. This way we still run the > tests for the final rust but not at every step in the chain. >=20 > Although, I wonder if we're more likely to miss a bug if we do this, = I'm > not sure. Although that definitely will speed the bootstrap chain, I=E2=80=99m = concerned that if a dependency package ever gets updated and breaks things we = wouldn=E2=80=99t know without running the test suite. Maybe if the bootstrapped versions don=E2=80=99t ever change skipping = the check phase will be safe, but I think we should try running parallel tests = first and see how far that gets us. =E2=80=94Ivan= --Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On Apr 4, 2019, at 1:59 AM, Ludovic Court=C3=A8= s <ludo@gnu.org> = wrote:

The build nodes may be slower = than the front-end, but still, it seems
unlikely that it = would take more than 6h there.  (That could happen if
the test suite, which lasts 2.1h, were =E2=80=9Cembarrassingly = parallel=E2=80=9D, but
we=E2=80=99re running tests with = =E2=80=98-j1=E2=80=99.)

To summarize, there = are two problems:

 1. Rust takes too = long to build.  What can we do about it?  Enable
    parallel builds?

Rust = tests are designed to run in parallel, as long as you have = enough
RAM, file descriptors, etc. available on the = machine for the amount of
concurrency being used. = The compiler test suite is largely just compiling
files, so the most important resource is probably available = RAM/swap.

On Apr 4, 2019, at 2:28 AM, Pierre Langlois = <pierre.langlois@gmx.com> wrote:
One thing I = suggested in the past was to remove the check phase *only*
for rust packages used for = bootstrapping. This way we still run the
tests for the final rust but not at every step in the = chain.

Although, I = wonder if we're more likely to miss a bug if we do this, I'm
not sure.

Although that definitely will speed the bootstrap chain, = I=E2=80=99m concerned that
if a dependency package = ever gets updated and breaks things we wouldn=E2=80=99t
know without running the test suite.

Maybe if the bootstrapped versions = don=E2=80=99t ever change skipping the check
phase = will be safe, but I think we should try running parallel tests = first
and see how far that gets us.

=E2=80=94Ivan
= --Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D--