From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id eHD/OpS5q2Wj4AAAqHPOHw:P1 (envelope-from ) for ; Sat, 20 Jan 2024 13:16:21 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id eHD/OpS5q2Wj4AAAqHPOHw (envelope-from ) for ; Sat, 20 Jan 2024 13:16:21 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1705752980; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=oj4AxSCQDP6vEAL7wrbj1fMMnYYobQMpntgYsSHXSF4=; b=T2mXzjrm/6kRAkiZoc/JG91ZshGGCnjBc16vS0TU3epohTH0JkV3jlx2nghGvnhi+pvhSt JqgaLBa7nAS5IYqRxo1nnkuwsBrm/DPkT6kHcwq7HZ1AHMHL7X+Ef5vDbbnayqbOIIqvd0 /MlhvGhJ5C9SjPIzmTOsR3SWJVqx+FBu7Yin9m7O/3J1Sw0hLSqrhHlpxHnOH+ic/M/DY4 lOfALODdVnzuSdeGcws0Myr82oa6WDJJBSiVRUu/+p3OVJbJREy9Yi9XETXy1F4T3S8wt3 7VdYh0OErLzZS7zanavNtsX8ncAOs/xvabSGLwmkgxNUoo2EZNn+s46SgM1yGg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1705752980; a=rsa-sha256; cv=none; b=p3RHnrnvYlD6UpxPE1hCzCKJ8WdaA/1Sbismv+8zLCOYR+JBnfEoBQlw4VTzg77eTU+mTd 3OEiSSSE6t7Q3pdzFyz4zBqJD7UVZnLEQMjnUa4QVzfXjfmbfDQpxMQTbHMYxdRzB1Z1J4 OwD67JCvCI/Jzxg6N7Y0c2NfFOLjCTjzN2thkwJ2CRVawRem0ktVHYbpQSjwwvK2xEdr3X Px2BsZBVtUESfYkJG6GOQFiamNO3d3FxQLABdAHUttPV4incyDYW0S2MZdPn0zVQ7qpXo9 vVM6ZAdI32P4GjGszZ1FHr2FYepYVCwUqWxN5HKRRThomBtGlwnSZpoThQQPiA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DAB7F74E98 for ; Sat, 20 Jan 2024 13:16:20 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rRAGH-0004tW-Ay; Sat, 20 Jan 2024 07:15:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rRAGB-0004sb-J7 for guix-devel@gnu.org; Sat, 20 Jan 2024 07:15:39 -0500 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rRAG9-0003md-T9 for guix-devel@gnu.org; Sat, 20 Jan 2024 07:15:39 -0500 Received: from localhost (unknown [89.207.171.157]) by mira.cbaines.net (Postfix) with ESMTPSA id 0737927BBE2 for ; Sat, 20 Jan 2024 12:15:33 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 17c3ca8e for ; Sat, 20 Jan 2024 12:15:32 +0000 (UTC) User-agent: mu4e 1.10.7; emacs 29.1 From: Christopher Baines To: guix-devel@gnu.org Subject: December/January update on QA and related things Date: Sat, 20 Jan 2024 06:34:09 +0000 Message-ID: <87plxwb3q9.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -2 X-Spam_score: -0.3 X-Spam_bar: / X-Spam_report: (-0.3 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_03_06=1.592, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -8.78 X-Spam-Score: -8.78 X-Migadu-Queue-Id: DAB7F74E98 X-TUID: On4ZqPktpGDS --=-=-= Content-Type: text/plain Hey, I sent out the last update [1] back near the start of December. 1: https://lists.gnu.org/archive/html/guix-devel/2023-12/msg00021.html In summary, QA has been not really working since mid December as data.qa.guix.gnu.org wasn't keeping up with processing revisions for patches and branches. I was also looking to shutdown data.qa.guix.gnu.org but this didn't happen, there still seems to be some interest in managing the infrastructure though the project (and shutting it down also takes more time and effort than leaving it running). Things are still running though, and I'm hopeful that QA will be back processing patches in the next few weeks. Specifically for the QA frontpage, little has changed in the last month. I have got around to making a diagram to put QA in context and it's now included in the README [2]. Let me know if any more diagrams would be useful as I've now got a process. 2: https://qa.guix.gnu.org/README#orgd169767 For the Guix Data Service, I've put some time in to speeding up the processing of revisions. I replaced all uses of delete-duplicates with a sort and pair-fold alternative and parallelised computing the guix derivations, computing the package derivations and a few other things that happen through inferiors. This still needs a bit more testing, but the changes are deployed on data.qa.guix.gnu.org and I think it's sped up processing individual revisions at least. Since data.qa.guix.gnu.org had less revisions in the database, I took advantage of this to do some maintenance and managed to reduce the size of the database considerably. Hopefully the frequent cleanup tasks will prevent it from getting this large again. Finally, I made various small fixes and speedups in the Guix Build Coordinator. I hopefully mitigated the port encoding issue [3] by switching from using the display procedure to log to using string->utf8 and put-bytevector. I opened a bug for a Guile segfault I hadn't seen before [4]. I also hopefully reduced the impact of the build coordinator stopping listening for connections by checking this internally and exiting if there's an issue. Unfortunately I've been quite slow in tracking down and trying to fix or at least mitigate these frequent but hard to reproduce bugs, but I think I've made some progress recently. 3: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=62590 4: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68221 Looking forward, there's still the issue of me hosting various parts of QA. Andreas has sent a proposal around on this in the last couple of days though. There's been some discussion in the past about using the data service to monitor performance related things, but maybe this is important for just keeping QA running as well. Failing to spot these issues before they're introduced can cause significant disruption, so maybe we need the data service to start monitoring and reporting on how particular performance characteristics change between revisions so that this can be reported by QA. While the bordeaux build farm is still doing well I think, I still haven't got around to implementing a way of pruning the nars that aren't for the master branch from being stored indefinitely. I've got some design ideas, but they need implementing and testing. There's also the ongoing issue of build hardware for current and up and coming architectures. Let me know if you have any comments or questions! I'm also planning to be at the Guix Days event and FOSDEM in a couple of weeks. Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWruV5fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xe49A/9HCU5W4o98jsK6NmZ1UhThSIV9H6rqUxM UYI9PKxEl/Zznj3rE5zUYeQ0cAHPaDsXSDCKl1XgIl8G0kCBvwfV3mWhpPHIuSqv 0Q7cWAZAgPDr7wh3GdB4aLJMcrDn5ywI+XCHUWcr0JNVo+bE65x2BuNFlyaiRoaO wohpa1lZK7J1PD8zASADDpuPKFzNnxNnIRMi8SpZKO/jsKBxVubNgMIK83Z3Wx27 vhhLUbLSQae1xGVL2lR+nrOKsRO+4l0j37UZI5jiGDx/eZXAFsojwGucrJLhjkhs yvRBvWxgekfAmoMRQYHVEpWRNJHWDRZFLM+zsRj97sPQ/DGioRafFlIVSHQkpxOo 6BbHoQQ4+plRGwj5u+1H0FbTT1ULjive0Irr1rTskq7cirjVUhvIEnbooKGOBVqZ mqwnXECL24ThsC3O0O6Ek/8VSoDxiCOhfGh0BASCL4dbAz2SLniZlAfE/pXBQCHQ 3C8LPcHUAEeeacJY+vbuTN+roLsjXipOHmjbqhfTOTJ0PWdwYTALsN2AzN3R+E9F PiV1grq6N7zqvxbvstWUAHyEkrp17GJu4tJ7DV05famHVMYWC1N2++GWgF+qVop3 eRbU75rI8MJb0FN9XwGosOzPzNURNsbmKjfrBefx/17wGqRgRVGETM+RimIaswyN cN60t8lOw3w= =263+ -----END PGP SIGNATURE----- --=-=-=--