From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id SSA7Ihzdg2CiOQAAgWs5BA (envelope-from ) for ; Sat, 24 Apr 2021 10:55:56 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 8E36HBzdg2B2GAAAB5/wlQ (envelope-from ) for ; Sat, 24 Apr 2021 08:55:56 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 14E351E742 for ; Sat, 24 Apr 2021 10:55:56 +0200 (CEST) Received: from localhost ([::1]:44602 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1laE4x-0003n7-89 for larch@yhetil.org; Sat, 24 Apr 2021 04:55:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35770) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1laE4Z-0003mz-Fd for guix-devel@gnu.org; Sat, 24 Apr 2021 04:55:33 -0400 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:52469) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1laE4X-0007p2-Cc for guix-devel@gnu.org; Sat, 24 Apr 2021 04:55:31 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id B35AD27BC7C for ; Sat, 24 Apr 2021 09:55:27 +0100 (BST) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id b7650ced for ; Sat, 24 Apr 2021 08:55:27 +0000 (UTC) User-agent: mu4e 1.4.15; emacs 27.1 From: Christopher Baines To: guix-devel@gnu.org Subject: Update on using the Guix Build Coordinator for substitutes Date: Sat, 24 Apr 2021 09:55:25 +0100 Message-ID: <875z0cgiyq.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1619254556; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=5PbHv+L4N7vizYUNE5vPJwgjYF0QCvQGo1pKIxdldpQ=; b=JafjTUMhpbH2CN0Iu1th7X3KvbBXZnO/kG00M02amatQ+ObCLieSfI8jDKErK5vBiztuAq ipFK/9FU+gwTTDHs33SoBeVyNlCyFULBLJqovt615FlEXfyagDfHIs0pziDSqzlqx4Qhrt FdSpumWfnapU3dptGWfXZZq53WtWNXUpE2FrOUr/PVQAxw5dbVUbmkjznl4yY9upDpcMqG TApLUmQSlyrQ3WdhSZR6zGhTtj8AWknUWmfXs1oq9FKuK4XsPQ+7jTl+Q9eBmsBhmInvc7 DhkBGl8yCMDcIkJk9azh8M4T9L89LE/DExD7dzCIt4UCfmeIw2QD762D/vJaxQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1619254556; a=rsa-sha256; cv=none; b=MIo7S/5Rruf/NDQ7oVR6WJ/0ezAspQepyj+6JUheiWwWXArwsLjDOuEsKRV8iwFgKON0FA I31ROqymGrkGt9IgurFu/c9WIRZn/BWxBbpkrbwz0C3pCbrwmb0fG8F2MGheuEPb/2rr0l IR1xHZFmCJEWjlSIjPcgJNXyqA75ATsyPwhNrCI6LxP2nfRtZOUc9kdRH1eJRnxvNAb5uC UOTC+NmZ2FEqLqZ802G5XddwqTUOdWgF4gCouPQqQD47yyBliAG+rLdernEdUzZTJEmvcB JZsCIOXa5Ee76tOA40+K1aD40ENgcYWFrbMkG1SLwphhq4ZP9hN8FO99mIPL7g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -4.54 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 14E351E742 X-Spam-Score: -4.54 X-Migadu-Scanner: scn0.migadu.com X-TUID: +VIb9C2cnOs6 --=-=-= Content-Type: text/plain Hey, This is probably something I haven't spoken enough about, but my test site (guix.cbaines.net) for building and providing substitutes has now got to a point where I've tested most things I set out to test. In particular, it's building the following things: - packages for: - x86_64-linux - i686-linux - aarch64-linux - armhf-linux - i586-gnu - cross build packages: - x86_64-linux -> arm-linux-gnueabihf - x86_64-linux -> aarch64-linux-gnu - x86_64-linux -> arm-linux-gnueabihf - channel instance derivations (guix pull) - x86_64-linux - i686-linux - aarch64-linux I can't remember when this started, but I think it was mid 2020, starting with a subset of the above, and I gradually got it building more things. Recent additions have been i586-gnu (the Hurd) support, as well as mixing QEMU and native/compatible builds for aarch64-linux and armhf-linux. One of the things I wanted to try out with the Guix Build Coordinator was retrying builds on failure, and I think that's a big part of why guix.cbaines.net often will have higher substitute availability than ci.guix.gnu.org. I wanted to build on this to try mixing builds through QEMU for aarch64-linux and armhf-linux with builds taking place on aarch64-linux (with hardware that has compatibility with armhf). I don't think using QEMU is a goal, as I think using QEMU can introduce problems, but suplementing some ARM hardware with x86_64+QEMU can be useful for building things for substitutes. As the Guix Build Coordinator has the ability to tag builds and agents, and match up tags so that specific builds only get allocated to specific agents, this allows for spotting when aarch64-linux or armhf-linux builds fail, and scheduling retry builds that are guarrenteed to take place without using QEMU. Like the general approach of retrying failing builds, this reduces the chance that a substitute won't be available because of a specific failure, like using QEMU. I think overall this experiment has been a success, specifically on the aarch64-linux and armhf-linux builds, I think most things that can be built, have been built (although I don't have a rigerous way of checking this). Back at the start of 2020, I think there was some optimism about using the Guix Build Coordinator for building substitutes for ci.guix.gnu.org, which hasn't happened yet, but I'll send a separate email about that. Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmCD3P1fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xfm2Q//daebTKJsErrzQYlIvYko9zN+enBO+Orn HEFDexX47i1TPMH9j/AvCgjAsJjFOryvnypSQYB2h7UvPVpBzy1O2Yp2EpnPFzyJ CDw2kM3wMnh8NhkUz/b5AuzkCWgvIFToZdBGo6+y+dTPgODRTnDEUomOgGd0jJ+K 6YfHYTABl+RVLnPt5fb2K9hg1E43W0NySlNb9U4xetYb7mjyhxQwwNx4ucrFq+5b MKurs1FU6+6zlJgkrje+FQffE3akZZW4BQW9gYxJCiYEDuI4AeZn1Rlz+e1Eaps7 uXhCuvSwkKEg9hc6Yu6Md+OlDbJIQz6wmYVNOSTSM8IWfT07gDJI5SYRVNQWeeOh nx9Q1K0wKE3AJy13AoZF0+MX+vdYMQcIK4wE+VJKwsqws4zxG63hnm26ATt/z0YY YjeUUXJ+jzsBe+yGIdlhYbJ+WNyoqs+sVJRFxNrwnDJDVVnQnrhAIteghKm+l9kd b/VGvypo3cRyZZ3GiOw65HY+2pgi60zkLkCMwrWftCJmUfBJpuFw5wnnNrjmGe4M oCoID6BhcbeJ6eOY1L7dfsHEUBR0zkwtvQUM84ciPG89gKjXGjLs5tcSq6t4lkvm SUQwQWEv5ozTIp+B1dpMpluXZgaj53ONLwbZwg734b8032y9aMhKUCb2vx8kZJTO y4JzRx7RC4Q= =cAGG -----END PGP SIGNATURE----- --=-=-=--