Guix Day - Feb 2021 - Quality Assurance notes

Guix Day - Feb 2021 - Quality Assurance notes

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 4367 bytes --]


Here are the notes from the Quality Assurance notes pad [1]. Thanks
everyone for participating!

1: https://pubcryptpad.pep.foundation/pad/#/2/pad/edit/JPbMDykstUCuz2z95qxOvne+/


## What does QA mean?  

 * Non broken packages  
   * exists for python packages on core-updates  
   * Telegram package broke shortly after being added (probably due to
     the Staging merge/qt update)
   * pre-merge automated GNU Guix builds for all dependents affected by
     the change, a bot on the mailing list for example, make it
     soft/hard requirement for merge
        
 * Non broken Guix (guix pull works)  
   * pull to latest version works from old release  
        
 * Substitute availability  
   * guix pull works with substitutes (exists today!
     https://guix.gnu.org/manual/devel/en/html_node/Channels-with-Substitutes.html#Channels-with-Substitutes

 * Usability on foreign distros
 
 * Graphical installer/installer script
 
 * Working guix pull from current released version to latest master
   commit
 
 * Documentation quality  
   * Having up to date documentation  
   * document all new features + 'guix pull --news' all new features  
        
 * Human testing first, then automated testing and fixing
 
 * Up to date packages
 
 * "guix lint" kind of things  
    * formatting  
    * spelling/grammar  
        
 * Working Guix system configuration  
   * Users provide configuration for tests to happen against  
        
 * Working Guix System services  
   * passing system tests for services  
   * guix lint style checks?  


### Things that are probably not QA  

 * User support  

 * How to get support when something breaks, and you're not on IRC or
   the mailing list

 * 'guix install' installs the most recent version which builds / is not
   broken

 * Recovery from bad states (when guix pull doesnt' work)
   * guix package --rollback without using 'guix' command  

 * Input validation (system services)  
   * Validate config upfront, to avoid errors deep in the config  
   * Config file validation, e.g. validating the NGinx config file  


### What to do about package quality (packages that build)?  

 * ci.guix.gnu.org build speed, it's good to find out quickly which
   packages fail to build

 * Testing prior to merging patches (patch review)  
   * Avoiding breaking packages  
   * Automatically build packages affected by patches, to spot breakages  
       
 * Merging branches in to master (staging/core-updates/...)  
   * How to spot breakages  
       
 * List of broken packages  
   * Guix Data Service: https://data.guix-patches.cbaines.net/repository/2/branch/master/latest-processed-revision/package-derivations?search_query=&system=x86_64-linux&target=none&minimum_builds=&maximum_builds=&build_status=failing&field=system&field=target&field=builds&after_name=&limit_results=&all_results=on
       
 * Comparing two commits  
   * Through https://data.guix-patches.cbaines.net/compare-by-datetime/
     or https://data.guix-patches.cbaines.net/compare/
       
 * Checking for whether it builds on ci.guix.gnu.org before updating
   locally

 * Manifest checking on ci.guix.gnu.org (Cuirass), to see if packages
   build

 * Distinguishing between "package not built" and "package known to fail
   to build"

 * Notifications (email) when packages fail to build    


### What to do about guix pull?  

 * Broken package references, more review/testing prior to merge  

 * Doing more checking in git pre push hooks?  
   * Server side Git hook checking

 * Doing more checking in git pre commit hooks?
   * guix lint
   * guix build

 * Breaking packages involved in \`make as-derivation\`  

 * guix pull checks if revision has been processed successfully on
   ci.guix.gnu.org prior to pulling
   * https://guix.gnu.org/manual/devel/en/html_node/Channels-with-Substitutes.html#Channels-with-Substitutes


### What to do about substitute availability?  

 * Using remote build information locally (don't build packages without
   substitutes if there's evidence that they fail)

 * "Negitive substitutes" that share information about build failures  

 * Speed/latency of fetching nars+narinfos  

 * Mirroring substitutes  

 * People running their own substitute servers

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]