From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id uEQpApNyIWDRQgAA0tVLHw (envelope-from ) for ; Mon, 08 Feb 2021 17:19:15 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id qLZ7OZJyIWBzHQAAbx9fmQ (envelope-from ) for ; Mon, 08 Feb 2021 17:19:14 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A70539402B1 for ; Mon, 8 Feb 2021 17:19:14 +0000 (UTC) Received: from localhost ([::1]:40954 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l9ABt-00074H-FT for larch@yhetil.org; Mon, 08 Feb 2021 12:19:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53420) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l93Lq-0002rV-88 for guix-devel@gnu.org; Mon, 08 Feb 2021 05:01:10 -0500 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:32975) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l93LN-0004Ow-7u for guix-devel@gnu.org; Mon, 08 Feb 2021 05:00:55 -0500 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 6F17727BC1E for ; Mon, 8 Feb 2021 10:00:27 +0000 (GMT) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 5fdab5aa for ; Mon, 8 Feb 2021 10:00:26 +0000 (UTC) User-agent: mu4e 1.4.14; emacs 27.1 From: Christopher Baines To: guix-devel@gnu.org Subject: Guix Day - Feb 2021 - Quality Assurance notes Date: Mon, 08 Feb 2021 10:00:23 +0000 Message-ID: <87eehqyj48.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -1.96 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: A70539402B1 X-Spam-Score: -1.96 X-Migadu-Scanner: scn0.migadu.com X-TUID: 7J/JBXB/TeGB --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Here are the notes from the Quality Assurance notes pad [1]. Thanks everyone for participating! 1: https://pubcryptpad.pep.foundation/pad/#/2/pad/edit/JPbMDykstUCuz2z95qxO= vne+/ ## What does QA mean?=20=20 * Non broken packages=20=20 * exists for python packages on core-updates=20=20 * Telegram package broke shortly after being added (probably due to the Staging merge/qt update) * pre-merge automated GNU Guix builds for all dependents affected by the change, a bot on the mailing list for example, make it soft/hard requirement for merge =20=20=20=20=20=20=20=20 * Non broken Guix (guix pull works)=20=20 * pull to latest version works from old release=20=20 =20=20=20=20=20=20=20=20 * Substitute availability=20=20 * guix pull works with substitutes (exists today! https://guix.gnu.org/manual/devel/en/html_node/Channels-with-Substitut= es.html#Channels-with-Substitutes * Usability on foreign distros =20 * Graphical installer/installer script =20 * Working guix pull from current released version to latest master commit =20 * Documentation quality=20=20 * Having up to date documentation=20=20 * document all new features + 'guix pull --news' all new features=20=20 =20=20=20=20=20=20=20=20 * Human testing first, then automated testing and fixing =20 * Up to date packages =20 * "guix lint" kind of things=20=20 * formatting=20=20 * spelling/grammar=20=20 =20=20=20=20=20=20=20=20 * Working Guix system configuration=20=20 * Users provide configuration for tests to happen against=20=20 =20=20=20=20=20=20=20=20 * Working Guix System services=20=20 * passing system tests for services=20=20 * guix lint style checks?=20=20 ### Things that are probably not QA=20=20 * User support=20=20 * How to get support when something breaks, and you're not on IRC or the mailing list * 'guix install' installs the most recent version which builds / is not broken * Recovery from bad states (when guix pull doesnt' work) * guix package --rollback without using 'guix' command=20=20 * Input validation (system services)=20=20 * Validate config upfront, to avoid errors deep in the config=20=20 * Config file validation, e.g. validating the NGinx config file=20=20 ### What to do about package quality (packages that build)?=20=20 * ci.guix.gnu.org build speed, it's good to find out quickly which packages fail to build * Testing prior to merging patches (patch review)=20=20 * Avoiding breaking packages=20=20 * Automatically build packages affected by patches, to spot breakages=20= =20 =20=20=20=20=20=20=20 * Merging branches in to master (staging/core-updates/...)=20=20 * How to spot breakages=20=20 =20=20=20=20=20=20=20 * List of broken packages=20=20 * Guix Data Service: https://data.guix-patches.cbaines.net/repository/2/= branch/master/latest-processed-revision/package-derivations?search_query=3D= &system=3Dx86_64-linux&target=3Dnone&minimum_builds=3D&maximum_builds=3D&bu= ild_status=3Dfailing&field=3Dsystem&field=3Dtarget&field=3Dbuilds&after_nam= e=3D&limit_results=3D&all_results=3Don =20=20=20=20=20=20=20 * Comparing two commits=20=20 * Through https://data.guix-patches.cbaines.net/compare-by-datetime/ or https://data.guix-patches.cbaines.net/compare/ =20=20=20=20=20=20=20 * Checking for whether it builds on ci.guix.gnu.org before updating locally * Manifest checking on ci.guix.gnu.org (Cuirass), to see if packages build * Distinguishing between "package not built" and "package known to fail to build" * Notifications (email) when packages fail to build=20=20=20=20 ### What to do about guix pull?=20=20 * Broken package references, more review/testing prior to merge=20=20 * Doing more checking in git pre push hooks?=20=20 * Server side Git hook checking * Doing more checking in git pre commit hooks? * guix lint * guix build * Breaking packages involved in \`make as-derivation\`=20=20 * guix pull checks if revision has been processed successfully on ci.guix.gnu.org prior to pulling * https://guix.gnu.org/manual/devel/en/html_node/Channels-with-Substitut= es.html#Channels-with-Substitutes ### What to do about substitute availability?=20=20 * Using remote build information locally (don't build packages without substitutes if there's evidence that they fail) * "Negitive substitutes" that share information about build failures=20=20 * Speed/latency of fetching nars+narinfos=20=20 * Mirroring substitutes=20=20 * People running their own substitute servers --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmAhC7dfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XcPjQ//fudRIMHWYkAjPF7llbYWCgI3o5OQ/SSG Q01gNeTHpFumSwQk/YBcUyVTos+kNqRGpZIb8Qb1uEsx8qbxH7Up0pj3xerJTjBD d3OWZYQ6Gn9194neu5Y9h5Ut4criEaZKrhDtc/cWJ8hJXf7Z1Zn1dZyU/5lyWKTc BAPW6zs9PirYrPP1G1sEOjTbqF8qOX3eCM1T/tit/oy/nNlYTada8kjFh96wm0PC Rd7aFKhVt911O66KrQLnO5Zqdf+got/IQAp3G8zvexgLGQ7LJ2qeRdIExeAHQuwi SXEh3TMpkVGVUEa6wpRvyN5JiC9btFsFDVU/MGKv1tI20dT3q1g27DNs/IYg9kkU P1XwUWcHVS0RWboM8fMtiDbEE69lma4rscW/KS9pT6QptWqWhUZj+FHKmPL5n7wa gAG7Sz099I4Apt9JHshn26l3iWwFr7xJXEVxzcZD5zG7trejy4t6vjtCLNJK+gHv CFPwYP+zoknVB1Q9ZOT1iyJsvzQuKxbFx1RYzXTVIUAnXfGjwnNWjkHjygxK/SQ0 vlNs30wodEYjHdW1sQsA9STzvEMNJbNIzmmvD3Rbeb/TpvVoC/3RDs1p/dISDilj 07LUMQZwQ6Jcbs1EKeoBLSInKY24AudeGto0ACu4hXOw0wSrVmss2qMNW8Xxza1c oGN+Dk1T8dA= =ZVWu -----END PGP SIGNATURE----- --=-=-=--