From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id EBEbCrg8LWFKwgAAgWs5BA (envelope-from ) for ; Mon, 30 Aug 2021 22:16:56 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id YLemBbg8LWH8KwAAbx9fmQ (envelope-from ) for ; Mon, 30 Aug 2021 20:16:56 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id CEA3AE9BF for ; Mon, 30 Aug 2021 22:16:55 +0200 (CEST) Received: from localhost ([::1]:51210 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mKniA-0004Ej-M0 for larch@yhetil.org; Mon, 30 Aug 2021 16:16:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35012) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mKngM-0002zh-Qe for guix-patches@gnu.org; Mon, 30 Aug 2021 16:15:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:48850) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mKngM-0003F1-HS for guix-patches@gnu.org; Mon, 30 Aug 2021 16:15:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mKngM-0001yU-5w for guix-patches@gnu.org; Mon, 30 Aug 2021 16:15:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#49981] wip: Introduce unit-tests. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 30 Aug 2021 20:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 49981 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: Mathieu Othacehe Cc: 49981@debbugs.gnu.org Received: via spool by 49981-submit@debbugs.gnu.org id=B49981.16303544647519 (code B ref 49981); Mon, 30 Aug 2021 20:15:02 +0000 Received: (at 49981) by debbugs.gnu.org; 30 Aug 2021 20:14:24 +0000 Received: from localhost ([127.0.0.1]:60396 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mKnfg-0001x9-Rm for submit@debbugs.gnu.org; Mon, 30 Aug 2021 16:14:24 -0400 Received: from mail-qv1-f52.google.com ([209.85.219.52]:38413) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mKnfa-0001wk-AG for 49981@debbugs.gnu.org; Mon, 30 Aug 2021 16:14:18 -0400 Received: by mail-qv1-f52.google.com with SMTP id ew6so9094582qvb.5 for <49981@debbugs.gnu.org>; Mon, 30 Aug 2021 13:14:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=uDMvceqzfpVuxcNUXatDUhCVbJktSlA2X78LC0fIlZM=; b=bpt0bXkbHKmklVGHUc2OGz/o6aeszOv1NauLTFWU+h0Y2pc2SiUZtheZcEL0XqVoQl AGpYdhF/fMjtcSnp0PMP4LzcKj0ekU8VJk2GW+2X9L7z2Dx0jiwQ1a3T31MlKd2qFO34 cUNExnPGxsYzRLxQrMwyGKVatJx6hOetvvR5vRTemPWevRbzYGd+pVmaM7sMqGyZzbDD +DaXI6a0QhouDSeQqp4dLi33XOFQCHfogJ/u2ALT/1YDoS//Aft0Xgl/LPJsFG36JHOb GAlJGP4zuhFEabjtme3XOoymyvmpaYRkMYeO8hiAnhJyBcawdYisZ6FHfbIBHQM0o+WD a5VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=uDMvceqzfpVuxcNUXatDUhCVbJktSlA2X78LC0fIlZM=; b=UvluNfV+pGJ1l04iEdp/62euv/6FBQbyLzgFpocMQV5rXZYTYJiucP03227TjBUuuT /BFPmZWx3W2xYzZD5TCjy9bxgh/C3oVFNcfGeEF5b57gfaSp99l7eYGi9u5c7HAJJC1k TvYbueFkaadKExzAoWIfjHEUfIbyIgEYuM9N/EnLtOHSzdeI2+shRWOM4/7pP/NRKg9i opZLDYsyxnSLayxikfL5sGfAGrIU7D9VHVPaM6ES29HsI5ctzRwFTprgWgXakbBkUeob Fqlf99KLdcaMESGsw7lzVMZ0P4FUlB9l7LdY47E2vvHkaXCnva3MBMo/osPEOepMLHMp GqgA== X-Gm-Message-State: AOAM5335t2okozujTOaN6roPcGlB7Vb1Rj7fFrXVNY0Lom2Rqv3XWk5X U2l97pxGrPo4BvhN3tmKIPb2e7+F2MI3kg== X-Google-Smtp-Source: ABdhPJzQg2O5mCEd249Db0lw3DEK0nNDTEGj8C2xkO142QijZHd2Eh87X/9eiuGf11R9CVnioZT6+w== X-Received: by 2002:ad4:5651:: with SMTP id bl17mr18018539qvb.49.1630354448591; Mon, 30 Aug 2021 13:14:08 -0700 (PDT) Received: from hurd (dsl-10-141-212.b2b2c.ca. [72.10.141.212]) by smtp.gmail.com with ESMTPSA id l13sm12078415qkp.97.2021.08.30.13.14.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Aug 2021 13:14:08 -0700 (PDT) From: Maxim Cournoyer References: <87o8a5734b.fsf@gnu.org> Date: Mon, 30 Aug 2021 16:14:07 -0400 In-Reply-To: <87o8a5734b.fsf@gnu.org> (Mathieu Othacehe's message of "Tue, 10 Aug 2021 17:04:20 +0200") Message-ID: <87czpulmgg.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1630354615; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=uDMvceqzfpVuxcNUXatDUhCVbJktSlA2X78LC0fIlZM=; b=pCtnb4vxOcIcDFR7BPK6zvCrtjEtPDc/ahdcnvAJvLCpbhy/4NflJGz1YPS+c213Q7hunm PzeHX+B/n96j3WclIL/l+14NOHuTinkUV9UYGSZfdOOsR/1foaEPwLdZ+yc/HckpiqyJgR uBX+nUayoxY5WapvynnCeHRccy/7uXJiNhHiKUqFvIbWlEfo+mwsNg4wBCZD1RFKR2GyZr Bel3sq6Hu7ubnz5b3VmrQfUsRxbfEEGSv5jnjLFSmT8xdQbsO9SKyS83dm7MOF+860bb3Z pLfJbRfwOAd9FT77PzICanlSPX3Bi9yQ9gVEXWRWznB4NTaAoW1SxDDTLeYe1A== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1630354615; a=rsa-sha256; cv=none; b=X3Z+PPijBphi9P2kdLLlje267iNVhEj4GYcPf8dV5WcM0PbelLpkzmu7EIaczmQJEwwfYx 1DYeSTqPycqzjypH3tHog7KphV48zBOMav6O7f3VlbrbZPdFVB4n2DHWm09VR8xk9i6Ao4 TXbBLSkM0wgEUY7jicrcjBJqew9b9TncDRfVKJ+GT76vUQhVtGxDZZHA3Q6azrGcapp6r9 wN9lZBCXBHPAdGLt0gyPT1pDm9Dsae6ybK5cDgPXQ5QVkJNvrDs9BCKTl194ftMXdUZSG9 ry5F4MPAqLwXym3y83uMqQ3Z0EwS2qsAJKvkV/XUzml9TKLswRaYZbjh01yczQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20161025 header.b=bpt0bXkb; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: -1.32 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20161025 header.b=bpt0bXkb; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: CEA3AE9BF X-Spam-Score: -1.32 X-Migadu-Scanner: scn0.migadu.com X-TUID: noYRkUphu2eS Hi Mathieu, Mathieu Othacehe writes: > Hello, > > I would like to convert the Guix tests in the "tests/" directory to > derivations, in the exact same way as for the system tests in the > "gnu/tests/" directory. Perhaps it's because I spent some effort into improving our (srfi srfi-64) based test runner, but I have some reserves about the proposed change, that echoes what Chris and others have mentioned. 1. More in the way between the tests and the code, which may complicate test debugging. Unit tests are supposed to involve as little as possible, ideally; getting the daemon and the store for even the most trivial tests seems undesirable. 2. One gripe that I have for the check-system tests is that for flaky tests, if they pass, the success is cached (it's a derivation) and there's no easy way to re-run them. I wouldn't want that property to now apply to unit tests as well. > For that, I propose to introduce a new record. This would > allow us to select all the unit tests using the "all-unit-tests" > procedure, and add them to the (gnu ci) module. I'm not sure if that's a convenient API for the CI, but our unit test runner has had the [--select=REGEXP] and [--exclude=REGEXP] command line switches for a while, that provides the ability to select or exclude specific tests (at their individual level). > This way, we could have a Cuirass specification for the unit tests, as > we already have for the system tests, to spot regressions early on. Is there something with the current scheme that prevents us from doing so already? > Here's a patch that translates the "account.scm" test module to the new > proposed mechanism. If there are no objections, I plan to convert all > the remaining tests. I guess mine is an objection :-). But with more explanations perhaps I can better understand things. Thanks, Maxim