From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: shortening the git test suite Date: Wed, 11 Jul 2018 14:46:13 +0200 Message-ID: <87k1q2kkiy.fsf@gnu.org> References: <87r2ko6pby.fsf@elephly.net> <87r2ki2hso.fsf@gmail.com> <87fu0y11ir.fsf@elephly.net> <87sh4wgrc3.fsf@netris.org> <87va9s9itb.fsf@gnu.org> <87a7r3gfmx.fsf@netris.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:43533) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fdEVc-0002tw-QR for guix-devel@gnu.org; Wed, 11 Jul 2018 08:46:17 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fdEVb-0005wP-RS for guix-devel@gnu.org; Wed, 11 Jul 2018 08:46:16 -0400 In-Reply-To: <87a7r3gfmx.fsf@netris.org> (Mark H. Weaver's message of "Sat, 07 Jul 2018 12:44:06 -0400") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Mark H Weaver Cc: guix-devel@gnu.org Hello, Mark H Weaver skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: [...] >> Currently if test fails, the whole derivation fails, and you can=E2=80= =99t >> install your package. If tests were run separately, this would no >> longer hold: you could get your package regardless of whether tests >> fail. > > Indeed, and I agree that we would need to address this. > >> How would you address this? I guess that calls for a new build >> model, no? > > I'm not sure what you mean by "build model", whether you are talking > about the daemon interface or something else, but I think the changes > could be confined to the Guix user interface. A field could be added to > , somewhat similar to 'replacement', but pointing to a package > object which runs tests, or perhaps a list of package objects. The guix > client could simply add the test packages to the list of derivations to > build. This could be inhibited via a "--no-tests" guix build option. > > In typical cases where "make check" expects to be run from a fully built > source directory, the main package would typically produce a tarball of > the built source directory as an additional output. The test package > would simply unpack this tarball and run "make check" in it. > > Regarding the build order: ideally, we would run the tests for package > FOO immediately after building FOO, or at least before building anything > that depends on FOO, to avoid wasted work in case the tests fail. I'm > not sure how best to accomplish this. > > What do you think? I=E2=80=99m afraid doing everything at the level of package objects rather = than derivations would lead to something clunky (grafts already suffer from that.) Namely, Guix clients would have to explicitly do a second =E2=80=98build-derivations=E2=80=99 RPC once they=E2=80=99re done building = packages. If they don=E2=80=99t do that, then tests are not run and you get to install a poss= ibly broken package. Besides, there are lots of details to get right in a proposal like yours. In particular, saving the build tree and for later reuse may prove to be more challenging than it seems. IOW, this looks like a radical change with lots of potential pitfalls, one I=E2=80=99d rather not make before 1.0. I agree that long test suites = can be a problem, but for now I think we should focus on package-specific solutions, to the extent possible, such as what we discussed for Git, as well as improvements to our build farm infrastructure. Thoughts? Ludo=E2=80=99.