From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id ILgKLPKZxmNprgAAbAwnHQ (envelope-from ) for ; Tue, 17 Jan 2023 13:52:02 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id +DQaLPKZxmOjYQEA9RJhRA (envelope-from ) for ; Tue, 17 Jan 2023 13:52:02 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 609631035F for ; Tue, 17 Jan 2023 13:52:02 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pHlR4-0006yn-QS; Tue, 17 Jan 2023 07:51:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pHlQj-0006ow-TN for guix-devel@gnu.org; Tue, 17 Jan 2023 07:51:14 -0500 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pHlQf-000254-Cl for guix-devel@gnu.org; Tue, 17 Jan 2023 07:51:09 -0500 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:54d1:d5d4:280e:f699]) by mira.cbaines.net (Postfix) with ESMTPSA id 64AD327BBE9 for ; Tue, 17 Jan 2023 12:51:02 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id eb1e77f7 for ; Tue, 17 Jan 2023 12:51:01 +0000 (UTC) User-agent: mu4e 1.8.11; emacs 28.2 From: Christopher Baines To: guix-devel@gnu.org Subject: January (2023) update on bordeaux.guix.gnu.org Date: Tue, 17 Jan 2023 10:00:13 +0000 Message-ID: <87sfg92ukt.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1673959922; a=rsa-sha256; cv=none; b=r6Pb+zCRs3oQhFQ7cRpOPVjzD+sD1trYT/jziUypozUiPkqS8HqF6rGWVUo2Mjwn3HIe3f czQyoMH980YZ8ytX2c8PMzmnVwyIV38VHHqrlWJj9O6qrv81XjiNwc2k8I6zqh3LF+68Ke d0S7f4IXHbbZub0Q4EyxYnJy/oUEH5CszEtLz1XQV1efrMHAOTKbof63GPgsmbcEOj5N9f azvpOQc5YmL7Y8MR1n/W9FaNtfxaEEsQ9xFsF7ItYyfGBLGdC+FFcLWpNy1Nkb5wiVeAjL 8ng4qyGfkyLVVRcKSFyqtJ4gr2AtC2qc4A1G+tsUAvlqqN7n+UPqraR+YfeCZQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1673959922; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=C+7VPxsubyc/RVfUMTKzLm85Sd4NO5iOtnUu8W+j9w0=; b=Q/qWKGzTRd+VGsuB7fL/3Z26hPqyhmDR3NzOCc++3ldsKgXsV2nFho/SOW4nb9/KkGxCcK mtrGIVK1LOr6wRTIGtADZDHaI8VgH/vrm1uqd1ozWtVUZUgdsVv9Y3slbeIe6Lc6lI2WLz UzmbWho/9P5T0cXMCixW7Mig9C+vduvq5MvBRN9KjsQADX2PWVxxn5p2krX6d/KxyB+RHm WyMIAVyIyQ7oTyjDATvv6f8iysF7MI/NQMGWSxCZdmVWBkIWZrxyplSzhf+/f7mn1a7jtF Q1VwVNcGC1OzRLwQaLCvQ23YeCZcKC7KHJQDq0rUwF/78OZsnjl3aN4e6Mr07w== X-Migadu-Scanner: scn1.migadu.com X-Migadu-Spam-Score: -5.03 X-Spam-Score: -5.03 X-Migadu-Queue-Id: 609631035F Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-TUID: pJMFCFttSmf3 --=-=-= Content-Type: text/plain Hey! The last update was sent out in September [1], so 4 and a bit months ago. 1: https://lists.gnu.org/archive/html/guix-devel/2022-09/msg00038.html ## Numbers bordeaux.guix.gnu.org currently provides ~2 million nars, which take up ~8.3TiB to store. Substitute availability is still reasonable I think. Additionally, there's some substitutes for i586-gnu (the Hurd) now. ## Non master branch substitutes One gradual change that's been happening over the last few months is that there's more builds/substitutes on bordeaux.guix.gnu.org for non master branch things (so for patches and branches like staging). ## Mirrors No progress has happened in terms of mirrors. There's still the US mirror I host [2] plus bishan effectively acts as a mirror [3]. 2: https://bordeaux-us-east-mirror.cbaines.net/ 3: https://bishan.guix.gnu.org/ I think there's still the opportunity to sort out the DNS and certificates so that bordeaux.guix.gnu.org refers to multiple servers, increasing reliability and hopefully performance as well. ## Serving fixed output files by hash This has progressed recently as Guile Fibers now supports streaming responses without storing the whole response in memory [4]. 4: https://github.com/wingo/fibers/pull/63 I've enabled this in the bayfront NGinx config, so you can now make requests like [5] 5: https://bordeaux.guix.gnu.org/file/texlive-20210325-texmf.tar.xz/sha256/070gczcm1h9rx29w2f02xd3nhd84c4k28nfmm8qgp69yq8vd84pz This seems to work for me on a fast connection, but the requests drops part way through on a slower connection. I'm not quite sure why yet but it's probably some NGinx configuration issue. You can also try making these requests directly against bishan. ## Next steps There's still a whole lot to improve on in multiple areas. While bordeaux.guix.gnu.org is now a default substitute server, and there's been a release since this change was made, the substitute servers list is ordered, so the obvious question is what needs to change so that bordeaux.guix.gnu.org is the substitute server that users should try first? In my mind, the main blocking issue is the current lack of zstd substitutes, but that's something that's being addressed now. There's also the content addressed mirror issue (discussed above as serving fixed output files by hash). I think a little more testing is required, but then it should be possible to add bordeaux.guix.gnu.org to the list of content addressed mirrors. In terms of resource requirements, especially now that builds for patches and non-master branches are happening, there's a use for more build hardware. The more pressing issue though is storage space. Currently the two machines responsible for storing all the nars (bishan and hatysa) each have around 10TiB of storage for this purpose, which will probably be all used up by July this year. It should be possible to extend the storage in hatysa, but bishan will probably need replacing. There's also improvements to and around the build coordinator, particularly in the area of observability (seeing what builds are happening and queued). If you're interested in working on any of this, do let me know as while I don't have time to work on much of it myself, I should be able to make time to help others. Let me know if you have any comments or questions! Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmPGmbJfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XeniQ/+IGrWWb+Jtir7ww8wDLi1IUKPnjnLcSwF u+SPXKhAubu8AesTjPm8iqbC38YiEU8NWf9/JeLc800ZEzHNBQlCMwBfZtI/s2Cs 3eBmce0cRNSSFKBY7K6WdqF3fP9vRf1BEqbmVu/ZVTWWPgbROcxjb7QEbYJF9Zoi EVeb853QDX0IriUPJFjOmuqK5UsytNBA1FLe3/28ACeu533eW96g0zc3bceY+BPH RXP8Y1/leTIJY35hwaO+xi5rfZWASXuyR9EctdtUuOkLXZWVpZ508NqjDCnMLmy6 SA0bhnuqsmCrTTG32sBd2l/4UtjPZwgBeEaXG/u5BC65wB62CW6avj+Tczf8BrBh 7KFdxqFNYs30oFQQo1uJiTsyEPqg6GoavcsZy4Q3INDQQs4V6uXjq5+WqY6fW9Xd qaa0szXduX1VSfeqxj98c/hJ0vzefIWMqvozivPeR3mjX1EjzpemneKoVq6aCmtW XajrDi2Fu6cs0AYcaYI8kqJpHKrO9Pr44YbAZlyiudEps02UaFRPGCFVCywuSG7X TX5uTVruvaTf1h5Wzg1iQ8tQM1kh5yU8/1uWQDufJmokgQ8k3YsmIhzowRqGmz8A b6wpWzITA+veWvPyuDQ5ofQIWX5BlE5VG0TzyoUwozrtWusy217M3FV7rWySm8EY dZ6of4qMCcQ= =/+aG -----END PGP SIGNATURE----- --=-=-=--