From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ricardo Wurmus Subject: Re: Treating tests as special case Date: Fri, 06 Apr 2018 10:27:42 +0200 Message-ID: <874lkokadd.fsf@elephly.net> References: <20180405052439.GA30291@thebird.nl> <87tvsp4cxh.fsf@netris.org> <20180406060601.GA4343@thebird.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:53995) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f4MjF-0002qu-JP for guix-devel@gnu.org; Fri, 06 Apr 2018 04:28:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f4Mj9-0007nN-SR for guix-devel@gnu.org; Fri, 06 Apr 2018 04:28:13 -0400 Received: from sender-of-o51.zoho.com ([135.84.80.216]:21002) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f4Mj9-0007iL-JS for guix-devel@gnu.org; Fri, 06 Apr 2018 04:28:07 -0400 In-reply-to: <20180406060601.GA4343@thebird.nl> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Pjotr Prins Cc: guix-devel@gnu.org Pjotr Prins writes: > Ludo is correct that provisioning binary substitutes is one solution. > But not cheap. Can we guarantee keeping all substitutes? At least the > ones with long running tests ;). For berlin.guixsd.org we have an external storage array of a couple of TB, which currently isn=E2=80=99t attached (I=E2=80=99ll get around to it s= ome day). We can keep quite a few substitutes with that amount of space. > Even so, with my idea of test substitutes you don't have to opt out of > testing. And you would still have found that bug. Those who care can > test all they please. I am not sure there=E2=80=99s an easy implementation that allows us to make tests optional safely. They are part of the derivation. We could make execution dependent on an environment variable that is set or not by the daemon, I suppose. > One suggestion: let's also look at tests that are *not* about > integration or hardware/kernel configuration and allow for running them > optionally. Stupidly running all tests that people come up with is not > a great idea. We just run what authors decide that should be run. We=E2=80=99ve already trimmed some of the longer test suites. There are so= me libraries and applications that have different test suites for different purposes, and in those cases we picked something lighter and more appropriate for our purposes. -- Ricardo GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC https://elephly.net