From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id OOVZGhh9OmX9WgEAauVa8A:P1 (envelope-from ) for ; Thu, 26 Oct 2023 16:52:08 +0200 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id OOVZGhh9OmX9WgEAauVa8A (envelope-from ) for ; Thu, 26 Oct 2023 16:52:08 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1B06032495 for ; Thu, 26 Oct 2023 16:52:08 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1698331928; a=rsa-sha256; cv=none; b=Sp1uLlOTXNlXx2t0AR2+FdRfIuZHPge77BSV59KqXauKHu2d8f1mZ2xCk2sPxS1beecF21 UMj7IrL6ek7ZNlu7AauqK7c0EOh4v7DTQ9tFB9QARpowjC9vEPfIhsI2jNDUGW6NjloIYM 05HbcBQ48UJ4pPOrcHasd8GKvO8wkPIhAT900WTvwLHOw3/qcSVGMH6Htsf+KH1Mo+NU8B K0R0oxobCRQBSrGMMI92ixFIoqtCda9W4YnQ0MqYgNB+hvHJ7YD1SxmDTdkhFjFcDzaOCC FfYmGgXiSjQq6FwuVJnr41aj6tPW5jCsXmQ7jtDu5NxzUhTfzufvpX6iaGwwKg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1698331928; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=i0oEkVi2bs8/w2utinzs/xNrhgSVPZtipeUzGtE3ZYY=; b=XoUsrob17U9G5lDj6H5ioq6ZShF10nRETRKt2+nmILeu3unJshSqU1W9n6/ns7JC1zUufb uYVcy8JFTlw+KjMtIz1w0oAF7so3w15HuwA2kwiImJL8ktNBYGQNQnFyedXgHA2t97mhAE I+daA3eg9kDLF6GN9uopEtQBSUF7MQF0ZKeaKSDi2EjpFpSQp6V/YjuIUkASyQ4BRvyKTn x6UzdMAyTjeh5NjAYeB9vKn2pUbxEYFJsjmZOIcFxiWGjoI+aJs0lQ5cMULHAE+QMPVX1m DtLAC/M6BKy53/PwcDGP1qajMRmBEpr7J29YT1CYt3wxwI3a/jfrBd+MzJPbPg== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qw1hz-00060M-8m; Thu, 26 Oct 2023 10:51:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qw1hx-000601-CX for guix-devel@gnu.org; Thu, 26 Oct 2023 10:51:37 -0400 Received: from smtpm3.myservices.hosting ([185.26.105.234]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qw1hv-0003ZU-D9 for guix-devel@gnu.org; Thu, 26 Oct 2023 10:51:37 -0400 Received: from mail1.netim.hosting (unknown [185.26.106.173]) by smtpm3.myservices.hosting (Postfix) with ESMTP id 76A4F20E20 for ; Thu, 26 Oct 2023 16:51:25 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail1.netim.hosting (Postfix) with ESMTP id 0A1A1800AB; Thu, 26 Oct 2023 16:41:32 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mail1.netim.hosting Received: from mail1.netim.hosting ([127.0.0.1]) by localhost (mail1-2.netim.hosting [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id IW8zQHSnP_wK; Thu, 26 Oct 2023 16:41:31 +0200 (CEST) Received: from guix-nuc (unknown [10.192.1.83]) (Authenticated sender: lumen@makinata.eu) by mail1.netim.hosting (Postfix) with ESMTPSA id 7C598800AA; Thu, 26 Oct 2023 16:41:31 +0200 (CEST) From: Bruno Victal To: guix-devel@gnu.org Subject: Expressing system test dependencies X-Hashcash: 1:26:231026:guix-devel@gnu.org::eUXfAhpnwUiiEFbr:0MJX/ Date: Thu, 26 Oct 2023 15:41:30 +0100 Message-ID: <87wmv9igyt.fsf@makinata.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.26.105.234; envelope-from=mirai@makinata.eu; helo=smtpm3.myservices.hosting X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Queue-Id: 1B06032495 X-Migadu-Scanner: mx12.migadu.com X-Migadu-Spam-Score: -6.10 X-Spam-Score: -6.10 X-TUID: DElf7N8X0475 Hi, There are system tests that would benefit from being able to express that their results are dependent on the result of other tests, when these constitute independent units, especially when there are configuration variants present. (e.g. gnu/tests/gdm.scm) Consider the following files/system-tests [1]: * gnu/tests/gdm.scm Right now this module implements two tests: %test-gdm-x11 and %test-gdm-wayland. (whose names are self-explanatory) * gnu/tests/vnc.scm This module implements one test, %test-xvnc, in particular it does so by testing the XDMCP feature using GDM. Note: this module predates gnu/tests/gdm.scm. Here's where we would benefit from being able to express system-test dependencies. Clearly some of the =E2=80=9Ctest-assert=E2=80=9D clauses in gnu/tests/vnc.scm would be better placed in gnu/tests/gdm.scm as they are GDM specific tests. Since they constitute configuration variants of the service, it would be placed into a %test-gdm-autosuspend variable. The rationale for a test-dependency mechanism is as follows: Suppose we split the GDM specific test to (gnu tests gdm). Now let's say that we run %test-xvnc and it fails. Is the failure due to GDM or is it caused by something else? (within the %test-xvnc) By splitting the test to (gnu tests gdm) we would have to run the GDM tests first, which isn't obvious. If we could express a dependency here, the debugging experience is improved as we now have a way to know that a integral component involved in the test failed. (or could be used to rule out other parts) Since the system-test results are expressed as derivations, successful tests shouldn't result in duplicate runs so perhaps we could make use of this fact for the effect? [1]: guix: 33f5b747b4a0a508e1ffc94a5bf425cff707c6e3 --=20 Thanks, Bruno.