From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark H Weaver Subject: Re: shortening the git test suite Date: Fri, 06 Jul 2018 14:19:08 -0400 Message-ID: <87sh4wgrc3.fsf@netris.org> References: <87r2ko6pby.fsf@elephly.net> <87r2ki2hso.fsf@gmail.com> <87fu0y11ir.fsf@elephly.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:34873) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fbVLG-0007Xg-6N for guix-devel@gnu.org; Fri, 06 Jul 2018 14:20:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fbVLD-00056k-1H for guix-devel@gnu.org; Fri, 06 Jul 2018 14:20:26 -0400 Received: from world.peace.net ([64.112.178.59]:33996) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fbVLC-00056X-ST for guix-devel@gnu.org; Fri, 06 Jul 2018 14:20:22 -0400 In-Reply-To: <87fu0y11ir.fsf@elephly.net> (Ricardo Wurmus's message of "Thu, 05 Jul 2018 11:21:16 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ricardo Wurmus Cc: guix-devel@gnu.org Hi Ricardo, Ricardo Wurmus writes: > Chris Marusich writes: > >> Ricardo Wurmus writes: >> >>> Hi Guix, >>> >>> git takes a very long time to build, because it has an extensive test >>> suite. Most of the time is spent in running the SVN interoperability >>> tests, though, which are not really all that interesting for most uses >>> of git. >>> >>> The Makefile says this: >>> >>> # Define NO_SVN_TESTS if you want to skip time-consuming SVN interope= rability >>> # tests. These tests take up a significant amount of the total test = time >>> # but are not needed unless you plan to talk to SVN repos. >>> >>> What do you think about disabling the SVN tests in the git package? >> >> This sounds similar to the discussion we had earlier about treating >> tests as a special case: >> >> https://lists.gnu.org/archive/html/guix-devel/2018-04/msg00071.html >> >> I felt that the conclusion of that thread was basically that if someone >> is concerned about the build time, then they ought to be able to use >> substitutes to speed things up, and we should continue to run as many >> tests as possible in order to discover problems sooner. > > What I=E2=80=99m worried about is availability of substitutes. When we k= eep > changing packages on the =E2=80=9Cmaster=E2=80=9D branch that have very e= xpensive test > suites then we accept that people won=E2=80=99t have substitutes for a wh= ile. > The duration of that while depends on how quickly the build of this > package is started by our continuous integration software, and how long > it takes to complete the build. > > Granted, disabling parts of the test suite in an attempt to shorten it > is really a technical fix to a social problem. > > Personally, I think that the SVN tests are non-essential (after all, > we=E2=80=99re building Git here, we keep running the individual test suit= es of > Git and Subversion, and git-svn interop seems like a thing that only > upstream need to worry about), which is why I made this proposal. Why do you say that only upstream needs to worry about it? I would think that you could say the same thing about almost any test suite, but there's always the possibility that our particular combination of input versions, or the unusual aspects of Guix, might break something that upstream doesn't know about. I would think that git-svn interop is something that any user of the git/svn integration needs to worry about. If we feel that very few of our users care about git-svn interop, another option would be to add a lighter variant of 'git' that does not include SVN support. It would probably be a good idea to have a 'git-minimal' package anyway, for use by our 'git-fetch' origin method. Naturally, only the 'git' package variant that includes SVN support would need to run the SVN tests. Also, looking ahead, I think it would be great if we could eventually move to a model where the tests of some packages are split off into separate derivations. Similarly, we could work toward splitting off documentation generation to separate derivation for selected packages. The most important advantage to this approach is that it would allow inputs needed only for tests or docs to be omitted from the inputs of the main package. I expect that this will in many cases be needed to prevent circular dependencies, and it could also greatly reduce the amount of rebuilding needed after updating certain packages. What do you think? Mark