From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id CCXtJxDTEmGyDgEAgWs5BA (envelope-from ) for ; Tue, 10 Aug 2021 21:27:12 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id QN0yIxDTEmFwUwAA1q6Kng (envelope-from ) for ; Tue, 10 Aug 2021 19:27:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 49E7F4FEC for ; Tue, 10 Aug 2021 21:27:12 +0200 (CEST) Received: from localhost ([::1]:56226 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mDXP5-0007A8-1F for larch@yhetil.org; Tue, 10 Aug 2021 15:27:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37006) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mDXOv-00078l-Ew for guix-devel@gnu.org; Tue, 10 Aug 2021 15:27:01 -0400 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:35833) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mDXOt-0002o6-CS; Tue, 10 Aug 2021 15:27:01 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id A418E27BC78; Tue, 10 Aug 2021 20:26:56 +0100 (BST) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id a597d78b; Tue, 10 Aug 2021 19:26:55 +0000 (UTC) References: <87bl6hbd05.fsf@cbaines.net> <87a6lwoqrs.fsf@gnu.org> <87r1f2i82z.fsf@cbaines.net> <87im0d96ot.fsf@gnu.org> User-agent: mu4e 1.4.15; emacs 27.2 From: Christopher Baines To: Mathieu Othacehe Subject: Re: Project direction with testing changes (branches and patches) In-reply-to: <87im0d96ot.fsf@gnu.org> Date: Tue, 10 Aug 2021 20:26:53 +0100 Message-ID: <87czqlhzia.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org, guix-maintainers@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1628623632; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=vEa4SIFYHXDXgYbuj6AXHmdMq0J+dT+MGV0QlG9RH/4=; b=YqQ7NLdZ6gw2sqYozlcqC8+Rz/S4R6w55+7hqU0ZOOGBaoxUQhgtg5mR3Tgja7/6Xv1J2w IBpG/PPVR9kF7ZqA2hXWGqsWmhhOiQvklggXXeEXno4a6YSQJpL2U4ajqrsqZsUKdrUGRG US1SfrWSQ81dsm9H74hdIX/30PHUhzmFJWSz/4YqxQJAf4hFTH+4kSFYLufcJSbCG8inhP W6dQeHHA1B0cX+E1PTj9czZc4QBO7LnssDPnPUFDATH5hwKKkLYGxGVHDZmyDOhgmt0o6W aky4B7Qfb2DA4bOYEVhUpTAoa8mkhojLo51Q3V7s3BD50e6I4tY3TwqyvvQVSA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1628623632; a=rsa-sha256; cv=none; b=EhX9KYoXcDkQOWJl9lgHb7OVqDwlDg4IYS38snJ+6A76lTYqDkXwkfEWOS4MK/CK4xT0m+ 51Y/LrbklSNgrOZVxS8XQWG4WFOSywvglJdixWpRKY9NYU0V2X6ZK6TPdylTb+ZNf77J/s 4c2WXpnsJfobf6Td+3U+usNPPST3S248UwrXUT9lLhtiZlx6IEGIUbwYSxQtgnANAsv/Ki jLLfAv+Z5aJJQoR+ivS9c5zel+pYl43YQT9sJ8v2F7OVkxniip05bVdEZc6lumzLdywDv5 lX/GDhP6gdhs6wcWXGrcBZm4CmT69cz9TBEc76LMob+poyXjJMlrrEeleNnIQg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -4.51 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 49E7F4FEC X-Spam-Score: -4.51 X-Migadu-Scanner: scn0.migadu.com X-TUID: Q3flRnlN9g98 --=-=-= Content-Type: text/plain Mathieu Othacehe writes: >> I think trying to change up how branches (staging/core-updates) are >> tested is a good place to start. The concrete change I'm proposing is to >> use an instance of the Guix Data Service plus an instance of the Guix >> Build Coordinator to do the testing and builds, rather than Cuirass on >> ci.guix.gnu.org which is the current approach. >> >> The main advantages of that would be the comparison support from the >> Guix Data Service, and the build performance and reliability that the >> Guix Build Coordinator brings. The main disadvantage is probably the >> lack of an admin like interface similar to that of Cuirass (I think this >> can be remedied in the medium term though). > > We indeed desperately need some more automation. For each new patch > series, it would be great to have the following information: > > * Status of the linter. > * Status of the depending derivations. > * Status of the unit tests (in the tests/ directory). > * Status of the system tests (in the gnu/tests/ directory). > > I would like to stay focused on the existing, well adopted solutions and > build upon them. With Cuirass we already have most of the machinery to > provide those information. I was thinking of using Cuirass for building derivations when testing patches, but I gave up on that approach back in 2019 after trying to use it (I discussed trying to use it here [1]). 1: https://lists.gnu.org/archive/html/guix-devel/2019-03/msg00010.html I was specifically thinking about testing patches when I initially designed both the Guix Data Service and Guix Build Coordinator. For me at least, the focus has been on this direction for the last ~3 years. I realise that Cuirass now has some of the functionality that the Guix Data Service was written to provide, like tracking all packages/derivations in each revision. But from my perspective, Cuirass still lacks key things like being able to compare two revisions and tracking lint warnings. There's also things like testing derivations on different hardware, regularly testing fixed output derivations and automatically retrying failed builds that I think the Guix Build Coordinator is better setup to do compared to Cuirass. But this feedback is why I started this thread. I don't see the same option as was found for improving substitutes by setting up a new substitute server using the Guix Data Service and the Guix Build Coordinator. There's a much stronger need to have one approach as a project for testing changes, and if using the Guix Data Service and Guix Build Coordinator isn't looking like a convincing option at this point, that's better to know now, compared to later when more time and effort has been put in. Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmES0v1fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xc39w/+LPRb7BxWOsmbqf/v6uu7mwgBMK+WAdfN J7TL5/Xl12E5Cb3WxOb07ToWwhHZOk1bG/oQxukNYcCAhxC2WZ7c4u5ymUJ6PmyI azCGBSJK+fO7mlyAT0wINiQ7FVfSCkvWsJqak7jZTate5G4fbzs7gLgXe+wgTZ6D +beXEFxdshNrMjJDfNiG6psTUn5hhStP/J05rBg7shvsk+CcRNCTbWTCZNBKTnpt 1CPEvAT6kdmdUZJn3ZSfRJLJfsYQ1NHDfwk2QqUECo2OD3/hu7W1p/JtyafhrH2V 0MF12fPeeLG5E/i9bX42SzWVm45kTyDPs8/C8jGBtr3N0/Macf7LUna/VEVurtLc aAu891zScETz+kdd5LA21Bjgz6i1qblktnjmb+tPun/momHAiKdcMm/ReLNsK5uX SCph8CTrAPatXUnlKL5x/TDw45GFkxLUdgsoOgG9k3D9eGB5E+kBKU+UdqUR27j6 6bUj0gApLc+lXMVEWOoR+TvYuaC8cqG4JjLvsgTsgY+cwS4b3gRoQQMIy7FA6ZBE ZserAvEQrIqmCzQD2BZzkkI9RKD1r3aKTThC6MZ89PP9yQ5eaiWO9CIlcB/hXjzR +tatKNZvNOkA8rzeUzWGWyYzo0wzUqNx7VMn3fdXXwEog1AcvF3mF+4isHGylgTM kXL6pfxtUpw= =+MCj -----END PGP SIGNATURE----- --=-=-=--