From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id cFkuNeu5I2DJSgAA0tVLHw (envelope-from ) for ; Wed, 10 Feb 2021 10:48:11 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id yLToMOu5I2BpZQAAB5/wlQ (envelope-from ) for ; Wed, 10 Feb 2021 10:48:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1592D9402AB for ; Wed, 10 Feb 2021 10:48:10 +0000 (UTC) Received: from localhost ([::1]:60898 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l9n2X-0002wc-MR for larch@yhetil.org; Wed, 10 Feb 2021 05:48:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:45712) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l9n2Q-0002vN-5I for bug-guix@gnu.org; Wed, 10 Feb 2021 05:48:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:43991) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1l9n2P-0001zR-Ta for bug-guix@gnu.org; Wed, 10 Feb 2021 05:48:01 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1l9n2P-0002Cz-O7 for bug-guix@gnu.org; Wed, 10 Feb 2021 05:48:01 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#46402: Cuirass rebuilds the same packae multiple times Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 10 Feb 2021 10:48:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46402 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mathieu Othacehe Received: via spool by 46402-submit@debbugs.gnu.org id=B46402.16129540258419 (code B ref 46402); Wed, 10 Feb 2021 10:48:01 +0000 Received: (at 46402) by debbugs.gnu.org; 10 Feb 2021 10:47:05 +0000 Received: from localhost ([127.0.0.1]:55537 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9n1V-0002Bj-7T for submit@debbugs.gnu.org; Wed, 10 Feb 2021 05:47:05 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55896) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1l9n1S-0002BE-8Y for 46402@debbugs.gnu.org; Wed, 10 Feb 2021 05:47:04 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:41504) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l9n1L-0001Uy-RU; Wed, 10 Feb 2021 05:46:55 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=42098 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l9n1I-0008Lp-NN; Wed, 10 Feb 2021 05:46:53 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20210209141915.40114e57@tachikoma.lepiller.eu> <87lfbxs0w9.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 22 =?UTF-8?Q?Pluvi=C3=B4se?= an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 10 Feb 2021 11:46:51 +0100 In-Reply-To: <87lfbxs0w9.fsf@gnu.org> (Mathieu Othacehe's message of "Tue, 09 Feb 2021 16:42:46 +0100") Message-ID: <87zh0c8ajo.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 46402@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.86 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 1592D9402AB X-Spam-Score: -2.86 X-Migadu-Scanner: scn0.migadu.com X-TUID: 72KXPNi+IafM Hi, Mathieu Othacehe skribis: >> here is an example: https://ci.guix.gnu.org/build/287478/details and >> https://ci.guix.gnu.org/build/287476/details are being built at the >> same time, and the logs currently show they are both running the test >> phase of the php package. > > Thanks for the report. This problem has been briefly discussed > yesterday. It has been introduced by the new remote building mechanism > in Cuirass. Hydra solves it by breaking each build into buildsteps > corresponding to the derivation inputs. > > The buildsteps are then submitted to the workers in a logical order. I > proposed to introduce a similar mechanism in Cuirass but Ludo expressed > doubts. Ludo do you think this problem could be solved otherwise? I=E2=80=99m not sure exactly but I can share my feelings. :-) Seems to me that =E2=80=98BuildSteps=E2=80=99 is an orthogonal concern that= has little to do with Cuirass=E2=80=99 job and with its data model. In Hydra I saw th= at as a (necessary) kludge. I like the way the Coordinator does it, and AIUI it=E2=80=99s pretty much t= he same as what the daemon is doing: submit build requests in topological order, such that when a derivation build is submitted, its prerequisites are known to be built already. I suppose what makes it more difficult here is that we have this extra =E2=80=9Cjob=E2=80=9D abstraction on top of derivations; everything in Cuir= ass revolves around jobs, which leads to this impedance mismatch. If Cuirass would instead delegate derivation build requests to a Coordinator/daemon-like thing, it wouldn=E2=80=99t have to worry about those details. That would better separate concerns. This is quite a hand-wavy reply but I hope it=E2=80=99s useful! Thanks, Ludo=E2=80=99.