From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 4Hn6MkAgIF+NXgAA0tVLHw (envelope-from ) for ; Tue, 28 Jul 2020 12:55:28 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id cFrALkAgIF8ZawAAbx9fmQ (envelope-from ) for ; Tue, 28 Jul 2020 12:55:28 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 80CB294062E for ; Tue, 28 Jul 2020 12:55:28 +0000 (UTC) Received: from localhost ([::1]:38680 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0P8h-0007HN-HE for larch@yhetil.org; Tue, 28 Jul 2020 08:55:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60566) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k0P8Y-0007Gz-Lp for guix-devel@gnu.org; Tue, 28 Jul 2020 08:55:18 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49506) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k0P8Y-0004Ub-C2; Tue, 28 Jul 2020 08:55:18 -0400 Received: from [2a01:cb18:832e:5f00:cd85:33f5:211:292b] (port=45366 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1k0P8V-00050f-Pu; Tue, 28 Jul 2020 08:55:16 -0400 From: Mathieu Othacehe To: Jonathan Brielmaier Subject: Re: ARM build machines References: <3308cccb-0f9f-6499-b948-3062a8a81ec8@web.de> Date: Tue, 28 Jul 2020 14:55:13 +0200 Message-ID: <874kpriytq.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Guix-devel Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Spam-Score: -1.01 X-TUID: 5r8yjhBx92q4 Hello Jonathan, Thanks a lot for this nice report. Let me summarize a bit the situation first, before commenting on your findings. On our main "berlin" build farm, we have currently a few aarch64 machines: * overdrive.guixsd.org * dover.guix.info * dmitri.tobias.gr * sergei.tobias.gr We are also emulating aarch64 and armhf builds on around 20 x86_64 machines (hydra-guix-101 ... hydra-guix-120 roughly). Regarding the physical machines, the overdrive seems like a modest machine (in comparison to the x86 machines we have), and I don't know about the other 3 ones. For the emulation, even though it's way way slower than real hardware, those 20 machines are really powerful and should be able to bring us good substitutes coverage. The current situation is that due to Cuirass/offloading issues such as [1], our build farm is most of the time idle. Given our computation power, we should be able to bake much more substitutes I think. Maybe we could also take advantage of the build-coordinator Christopher is implementing (+ Guix daemon RPC's over HTTP) to make sure that we are able to deal with a distributed build farm efficiently. Now the question I'm asking myself is: could the ARM substitutes situation be solved by improving our CI software stack, or do we really need more hardware? Hard to answer right now, but maybe the best way to proceed would be to make sure that we are using the hardware we have close to its limit first and then consider buying more. > Neither VPS nor dedicated build machines for ARM are cheap, if they are > decent powerful. As a first step I could try to reach out Linaro if they > are willing to provide us some VMs. Yep, that would be great to explore this option! Concerning the other options you proposed, they are indeed expensive and as I said, at this point, our CI software stack would not be able to use it efficiently, which would be sad. What do other people think? Thanks, Mathieu [1]: https://issues.guix.gnu.org/34033