From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id +AuPKw8jBGJLHAEAgWs5BA (envelope-from ) for ; Wed, 09 Feb 2022 21:24:47 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id GCLUKA8jBGLHhgAA9RJhRA (envelope-from ) for ; Wed, 09 Feb 2022 21:24:47 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3715D29C82 for ; Wed, 9 Feb 2022 21:24:47 +0100 (CET) Received: from localhost ([::1]:35844 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nHtW9-0006NS-65 for larch@yhetil.org; Wed, 09 Feb 2022 15:24:45 -0500 Received: from eggs.gnu.org ([209.51.188.92]:39912) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nHtUD-0006EO-3e for guix-devel@gnu.org; Wed, 09 Feb 2022 15:23:04 -0500 Received: from mira.cbaines.net ([212.71.252.8]:36306) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nHtTz-0002A0-Uy for guix-devel@gnu.org; Wed, 09 Feb 2022 15:22:34 -0500 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id A2CF927BBE9 for ; Wed, 9 Feb 2022 20:22:30 +0000 (GMT) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 4865e42b for ; Wed, 9 Feb 2022 20:22:29 +0000 (UTC) User-agent: mu4e 1.6.10; emacs 27.2 From: Christopher Baines To: guix-devel@gnu.org Subject: Start of 2022 update on bordeaux.guix.gnu.org (over 1,000,000 successful builds!) Date: Wed, 09 Feb 2022 18:34:18 +0000 Message-ID: <87h797eqnx.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1644438287; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=+2MOu6OEPwvAnHac+ZFQea/uBzmOGg4+nchwcCvV3IM=; b=FhwVCszoR3ay6rSDrWLLnSlKXe6E+xHIpyVpDi+pq9FDCNVRhU0AkwclUNKOav3nEO+xtr Wa5AutrSYe48XthB1+zlGAy4PwNNif/63oPHONIJ9HRw5ZFKtHJZraIKCS2iEyWY1m3UHb 8JSOuACfVwEZyDuerfZ62qpa8xCyqhhBmGpKNcULK+A19HiHqJzjfeGSkf+tEfAXTrxqrM vJ9JQg3EZY/W1+HqUORIcsVLTWJOS4rMlulkoNhNyPCW9FIcnndQnIKadpQw8+8sW7t4i7 u08A8x7JQvF3nVkc2SLbY8N9K8Ecb0OKXb84F63MNOrR5rWD6hUxVTaCHT4isQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1644438287; a=rsa-sha256; cv=none; b=BkphCA0HBKcKuYaET2UQ12vl1IpmlmHkhI69hciHxoUuI2Qux+yUOrdynj3gMiDNNFRwDB wYGXPFqQrRpXRFn9v6abSuMtjcIZ5po4sI4e79V96hLzjCq3n8HFIot6piqL5x3thldDJE Z+M2ng0emzU4ziaWftQ6X2i7A3qfQpZWMZZNsi5cBMDrmCjaEk3695mSFjhkH/yX3dAtdX i7FMwXWlOdQ7anD6NFBq3lExDxuNtPYsrzTtINQIRqv198x1cXS7wyk4r0QjWxHdq+Gz6d aRWX2lJ3ccqO1TUz4bM45yjJUJ2rQBgJiRJhHXRDXGRP96HQ86zDq4JTowL9wQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.73 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 3715D29C82 X-Spam-Score: -4.73 X-Migadu-Scanner: scn1.migadu.com X-TUID: e7WNBK91Cwui --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hey! It's not quite been a year since bordeaux.guix.gnu.org started operating, but I thought I'd send out an update email towards the start of 2022 looking back in to 2021 and looking forward to the rest of this year. If you're missing some context, this blog post should be useful, and has lots of useful links: https://guix.gnu.org/en/blog/2021/substitutes-now-also-available-from-bor= deauxguixgnuorg/ The last update was back in December [1] and talked about addressing the nar capacity, the machines building things, and IPv6 support (something I'm glad to see). 1: https://lists.gnu.org/archive/html/guix-devel/2021-12/msg00140.html ### Looking back First, some data. The first build was created back on the 9th of April, 2021. So far the build coordinator behind bordeaux.guix.gnu.org has been asked to build nearly 2 million things (> 1,939,000). Of those builds, around 824,100 haven't been processed, and > 1,114,900 have been processed. Of those processed builds, > 113,338 failed, and > 1,001,567 succeeded. From=20those successful builds, bordeaux.guix.gnu.org provides > 914586 nars. That's 3.6TiB of lzip compressed data (~11TiB's uncompressed). The Guix Build Coordinator project began with two use cases in mind, building things to provide substitutes, and building things to perform quality assurance tasks. The substitutes use case is the relevant one here, so I'm just focusing on that. Regarding substitutes, there were a few different aims in mind: things like making operating substitute servers easier, increasing hardware utilisation and improving substitute availability. I guess the baseline for comparison has changed since ci.guix.gnu.org no longer uses offloading to perform most builds, but I think there's reason to think that progress has been made and is being made on the above aims. Since bordeaux.guix.gnu.org came in to existence, not all that much has changed with the Guix Build Coordinator, I think most of the changes have been tweaks and optimisations. ### Looking forward I think the most important area for improvement is data.guix.gnu.org which has been struggling to keep up with new revisions lately. I think as Guix grows, both in terms of packages and supported architectures, this additional data has slowed the revision loading process. I've got some ideas on improving this, and I'm hoping to work on it soon. Something I've been working on recently is neatening up the configuration for the machines involved, there's a bit more to do on this. It would also be beneficial to add more machines and expand the supported architectures. Also in terms of the operation, getting regular automated database backups in place, and increasing the redundancy and capacity for the nar storage would be good. Specifically on the nar capacity issue, assuming that the retention is kept at 100%, I think it's likely that the available storage (currently there's a rough limit of 5.5TiB) will need expanding this year. Now that the nar-herder exists, this should be much less stressful than the previous work to expand the available storage (which involved writing the nar-herder from scratch). On the subject of the nar-herder, I think continuing to work towards having mirrors in different geographical locations is important to improve substitute performance for everyone. I sent out an email recently to resume discussion of the nar-herder design [2]. I also want to look at gathering metrics about substitute use, probably through the nar-herder, and that's somewhat dependent on mirrors, as I want any metrics gathered to be across several mirrors. 2: https://lists.gnu.org/archive/html/guix-devel/2022-02/msg00033.html It would also be nice to make the inner workings of the build coordinator behind bordeaux.guix.gnu.org more visible, so everyone can see what builds are happening. Finally, it would be good to work towards bordeaux.guix.gnu.org building packages for non-master branches. This would help when preparing for staging/core-updates merges. I think this mostly involves properly setting up a data.qa.guix.gnu.org data service and enabling substitutes from this on the build agents. Please let me know if you have any comments or questions. I think there will also be time for discussion at the upcoming Guix Days event. Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmIEIoJfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xf8tw//WTtvgidNx3PWhsoPibjlf1r4YgUtoSde FnEylHu0NSFbXrqsiwjY+6lSUdZdnEgz+K2e3Sn4g/DSbnDhXKW4ZMWthkd7H+Hg 0EAUTwuqEzH/aO9DXxP2Xj25vT8BYb9ydI2mkGl14u0jX+82aedxwDYal6jWaAAE NEbIpYVYrpY3tXZ3LiSNyrvhkkFb07hdbAs2/ZpyNK4b59DgZbcgL3Mz+WpdMIL6 7QMyk1oZcS+jdKs/tFrY11mFWwuTW/evFSHOzuSv2ngP9H4PKEFCnExvDTKZ1hKU PR1BJvazoshqp/OGj0Zc/TsMoJhl/GmLMtvE2ksxqb8faPd7M2TN1TzDEEq1y7eG WMY29hcWgbI7xigQIDLM2Tbi0dc6NGjLZ+E4b75QA6981G7rJv9TvdptKySgT4PR FY/74/bZBn8GaR+8gQClI7e991HiQDzP/CxMH6jL0R8cWv2t8BGNdJUrE5R08dWY kgx3BpzpHSoPcqAJO36mQ5dQf2fxFmhJ+5j1Fcg/FJJ/fI6QL4n0RNQnpiDhu1XZ MUAEqJhaDTYBhdnPQHKmM2Qe2hsQ/wHb+oBwmxiEJ8CkkWAB+wunKbMojk1aj/fS Lzy+5ZjIXe6h43eKugdmq8Y1UpohgbVsrijnOOivHdXQ+GbNE1V5PoW56QFZy2PD eTHsyIJDVK8= =4ggy -----END PGP SIGNATURE----- --=-=-=--