From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Treating tests as special case Date: Thu, 05 Apr 2018 16:14:19 +0200 Message-ID: <87efjtzqo4.fsf@gnu.org> References: <20180405052439.GA30291@thebird.nl> <87efjuj6f2.fsf@elephly.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:45813) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f45ek-0007s5-Al for guix-devel@gnu.org; Thu, 05 Apr 2018 10:14:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f45eh-0000BW-3y for guix-devel@gnu.org; Thu, 05 Apr 2018 10:14:26 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:58988) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1f45eg-0000B7-T3 for guix-devel@gnu.org; Thu, 05 Apr 2018 10:14:23 -0400 In-Reply-To: <87efjuj6f2.fsf@elephly.net> (Ricardo Wurmus's message of "Thu, 05 Apr 2018 12:26:09 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ricardo Wurmus Cc: guix-devel@gnu.org Hello! I sympathize with what you write about the inconvenience of running tests, when substitutes aren=E2=80=99t available. However, I do think runn= ing tests has real value. Of course sometimes we just spend time fiddling with the tests so they would run in the isolated build environment, and they do run flawlessly once we=E2=80=99ve done the usual adjustments (no networking, no /bin/sh, e= tc.) However, in many packages we found integration issues that we would just have missed had we not run the tests; that in turn can lead to very bad user experience. In other cases we found real upstream bugs and were able to report them (cf. for an example from today.) Back when I contributed to Nixpkgs, tests were not run by default and I think that it had a negative impact on QA. So to me, not running tests is not an option. The problem I=E2=80=99m more interested in is: can we provide substitutes m= ore quickly? Can we grow an infrastructure such that =E2=80=98master=E2=80=99,= by default, contains software that has already been built? Ricardo Wurmus skribis: > An idea that came up on #guix several months ago was to separate the > building of packages from testing. Testing would be a continuation of > the build, like grafts could be envisioned as a continuation of the > build. I agree it would be nice, but I think there=E2=80=99s a significant technic= al issue: test suites usually expect to run from the build tree. Also, would a test failure invalidate the previously-built store item(s)? Thanks, Ludo=E2=80=99.