From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: [GSoC] Rewrite Hydra to be more integrated with Guix. Date: Wed, 23 Mar 2016 15:08:40 +0100 Message-ID: <87io0duw0n.fsf@gnu.org> References: <87bn6c944a.fsf@gnu.org> <20160322213124.GB2135@solar> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:50073) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aijSr-0007CV-9L for guix-devel@gnu.org; Wed, 23 Mar 2016 10:08:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aijSn-00048K-Ss for guix-devel@gnu.org; Wed, 23 Mar 2016 10:08:49 -0400 In-Reply-To: <20160322213124.GB2135@solar> (Andreas Enge's message of "Tue, 22 Mar 2016 22:31:24 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Andreas Enge Cc: guix-devel Andreas Enge skribis: > In the end, I think it would be interesting to not use the offload approa= ch, > where the offloading machine sends all inputs to the build machine, which > returns the result, so that more or less the complete set of binary packa= ges > flows along the tree of build machines to the root and the other way roun= d. > (As I recently understood during discussions with Ludovic, currently we h= ave > a tree of height one: hydra is the root and all build machines are the le= aves; > but the offloading mechanism could transparently work with trees of bigger > height, where hydra offloads to a root of a smaller build cluster, which > itself offloads to one of its subordinate build machines. And so on. The = same > thing would also work in the Debian setting, assuming that build requests > come from the leaves and are forwarded towards the root, while a work pac= kage > is handed down the other way.) Now it would be interesting if machines ju= st > got build jobs (or made requests) and looked for the necessary inputs ins= ide > a big magma/cloud/gnunet/ipfs/mirror network, compiled the package, and > dropped the result into the same magma. Already in our current set-up, it > would be useful if hydra need not send the build inputs, but build machin= es > could just fetch them from an arbitrary nginx mirror, where they are cach= ed. I agree. However, I think there are a lot of unknowns, which makes it unsuitable for a GSoC project. That=E2=80=99s why I was suggesting that Mathieu do not consider workload distribution (meaning we would keep using the existing offload mechanism) as part of this project. Ludo=E2=80=99.