From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Treating tests as special case Date: Thu, 05 Apr 2018 17:24:12 +0200 Message-ID: <87vad5y8v7.fsf@gnu.org> References: <20180405052439.GA30291@thebird.nl> <87efjuj6f2.fsf@elephly.net> <87efjtzqo4.fsf@gnu.org> <20180405145929.GA345@thebird.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:59767) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f46kN-0004ib-EJ for guix-devel@gnu.org; Thu, 05 Apr 2018 11:24:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f46kK-0006zf-9E for guix-devel@gnu.org; Thu, 05 Apr 2018 11:24:19 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:59330) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f46kK-0006yW-2e for guix-devel@gnu.org; Thu, 05 Apr 2018 11:24:16 -0400 In-Reply-To: <20180405145929.GA345@thebird.nl> (Pjotr Prins's message of "Thu, 5 Apr 2018 16:59:29 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Pjotr Prins Cc: guix-devel@gnu.org Pjotr Prins skribis: > I am *not* suggesting we stop testing and stop writing tests. They are > extremely important for integration (thought we could do with a lot > less and more focussed integration tests - ref Hickey). What I am > writing is that we don't have to rerun tests for everyone *once* they > succeed *somewhere*. If you have a successful reproducible build and > tests on a platform there is really no point in rerunning tests > everywhere for the exact same setup. It is a nice property of our FP > approach. Proof that it is not necessary is the fact that we > distribute substitute binaries without running tests there. What I am > proposing in essence is 'substitute tests'.=20 Understood. > If tests are so important to rerun: tell me why we are not running > tests when substituting binaries? Because you have a substitute if and only those tests already passed somewhere. This is exactly the property we=E2=80=99re interested in, right? That is why I was suggesting putting effort in improving substitute delivery rather than trying to come up with special mechanisms. >> Ricardo Wurmus skribis: >>=20 >> > An idea that came up on #guix several months ago was to separate the >> > building of packages from testing. Testing would be a continuation of >> > the build, like grafts could be envisioned as a continuation of the >> > build. >>=20 >> I agree it would be nice, but I think there=E2=80=99s a significant tech= nical >> issue: test suites usually expect to run from the build tree. > > What I understand is that Nix already does something like this. they > have split testing out to allow for network access. Do you have pointers to that? All I=E2=80=99m aware of is the =E2=80=98doC= heck=E2=80=99 variable that is unset (i.e., false) by default: https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/setup.sh= #L1192 Ludo=E2=80=99.