From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id OCVtHO1mJGCKDwAA0tVLHw (envelope-from ) for ; Wed, 10 Feb 2021 23:06:21 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id ED0TGO1mJGB1XgAAbx9fmQ (envelope-from ) for ; Wed, 10 Feb 2021 23:06:21 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A813B940274 for ; Wed, 10 Feb 2021 23:06:20 +0000 (UTC) Received: from localhost ([::1]:33752 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l9yYt-00085B-Dy for larch@yhetil.org; Wed, 10 Feb 2021 18:06:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:60734) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l9xzh-0008ME-Ir for guix-devel@gnu.org; Wed, 10 Feb 2021 17:29:57 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:57609) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l9xzh-00029S-1K; Wed, 10 Feb 2021 17:29:57 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=46192 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l9xza-0007EE-1l; Wed, 10 Feb 2021 17:29:51 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Mathieu Othacehe Subject: Re: The Guix Build Coordinator in 2021 References: <878s7xx9tx.fsf@cbaines.net> <87sg642nqy.fsf@gnu.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 22 =?utf-8?Q?Pluvi=C3=B4se?= an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Wed, 10 Feb 2021 23:29:47 +0100 In-Reply-To: <87sg642nqy.fsf@gnu.org> (Mathieu Othacehe's message of "Wed, 10 Feb 2021 11:58:13 +0100") Message-ID: <87blcr1rqc.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.86 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: A813B940274 X-Spam-Score: -2.86 X-Migadu-Scanner: scn0.migadu.com X-TUID: giMuRHOULncG Hi! Mathieu Othacehe skribis: > We are now in a situation where our continuous integration system, > while performing better and better is getting out of hand. Here are the > different software I'm keeping track of: > > * Cuirass, deployed on ci.guix.gnu.org > * The Guix Build Coordinator, deployed on guix.cbaines.net > * The Guix Data Service, deployed on data.guix.gnu.org > * Patchwork, deployed on patchwork.cbaines.net I can see why one would think things are getting out of hands, or at least that we have prolific developers. :-) Still, these four things play different roles: the Coordinator =E2=80=9Cjus= t=E2=80=9D build things in a distributed fashion, the Data Service evaluates channels and stores metadata, while Patchwork does patchy things. Cuirass is in-between because it evaluates things, distributes builds, and stores metadata. It made sense to bundle functionalities like this, because that=E2=80=99s the subset we=E2=80=99re interested in (build stuff,= distribute builds, provide some metadata useful for QA). The Coordinator/Data Service split is a nice separation of concerns IMO. Each is good at doing its job; the combination isn=E2=80=99t quite comparab= le to Cuirass because it=E2=80=99s more generic, but similar information and acti= ons are available. > All those services have databases, using different DBMS on different > servers. Those databases are sometimes overlapping, in the same way as > some of the features of those software. Maybe it=E2=80=99s OK to have multiple databases? The daemon has one too. = :-) It can be seen as an implementation detail. (Plus, I=E2=80=99m told that microservices are hype.) > In particular I feel that what's implemented in the Guix Build > Coordinator can be seen as a subset of Cuirass functionalities. As you > know, I'm reluctant to the idea of connecting Cuirass to the Guix Build > Coordinator, because most of Cuirass PostgreSQL database content would > be duplicated in the SQLite database of the GBC. Does it really need to be duplicated though? Maybe Cuirass could keep links to Coordinator builds, just like it doesn=E2=80=99t duplicate the sto= re database. > On the other hand, maintaining those two software in separate ways seems > like a huge waste of time given the very limited number of people > contributing the maintenance of the CI system. > > Furthermore, some of the features we are implementing here, should be > part of the "guix-daemon" itself, which makes me think that we should > not place too much effort in their development. Yeah, but we need to go incremental IMO. The daemon rewrite is a bit of a jump, which is why it hasn=E2=80=99t gone anywhere yet. In the meantime,= I think it=E2=80=99s OK to daemon-like functionality elsewhere; perhaps that= =E2=80=99ll be reused eventually in the daemon. Ludo=E2=80=99.