From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Suggestion: disable offloading for texlive builds on hydra? Date: Mon, 27 Oct 2014 13:58:57 +0100 Message-ID: <87mw8h4qjy.fsf@gnu.org> References: <87ppdf1dwc.fsf@netris.org> <20141026074926.GA3937@intra> <877fzmncmf.fsf@gnu.org> <87a94irf0u.fsf@yeeloong.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:51109) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xijsi-0002vn-1e for guix-devel@gnu.org; Mon, 27 Oct 2014 08:58:48 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Xijsd-0001vg-92 for guix-devel@gnu.org; Mon, 27 Oct 2014 08:58:43 -0400 Received: from hera.aquilenet.fr ([2a01:474::1]:42379) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xijsc-0001vY-9A for guix-devel@gnu.org; Mon, 27 Oct 2014 08:58:39 -0400 In-Reply-To: <87a94irf0u.fsf@yeeloong.lan> (Mark H. Weaver's message of "Sun, 26 Oct 2014 12:07:13 -0400") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Mark H Weaver Cc: guix-devel@gnu.org Mark H Weaver skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: [...] >> The rationale was that, in general, you just slow everything down by >> sending several things at once. > > I have my doubts that it would slow things down very much, if at all. > The number of parallel transfers would still be limited to a small > number, typically 4 per build slave. The expense associated with > running multiple processes on a CPU is mainly due to cache effects, but > I wouldn't expect that to be an issue with network connections, > especially when those connections are between the same two hosts. The > practice of using multiple connections is well established in web > browsers and imap clients, as long as the number is not too large. > > We're losing a huge amount of available CPU capacity in our build farm > (probably over 30 machine-hours per texinfo rebuild) in exchange for a > dubious increase in network efficiency. > > The more I think about it, the more I agree with John that we've chosen > the wrong tradeoff here. I think we should remove those mutexes. Hmm OK. I=E2=80=99m happy to try that (it=E2=80=99s a two-line change plus= deployment.) I can do it one of the next few days, but I=E2=80=99m happy if you do it. = :-) >> diff --git a/gnu/packages/texlive.scm b/gnu/packages/texlive.scm >> index e562b02..bc0ece7 100644 >> --- a/gnu/packages/texlive.scm >> +++ b/gnu/packages/texlive.scm >> @@ -88,7 +88,7 @@ >> ("pkg-config" ,pkg-config) >> ("python" ,python-2) ; incompatible with Python 3 (print syntax) >> ("tcsh" ,tcsh))) >> - (outputs '("out" "data")) >> + (outputs '("out" "data" "doc")) >> (arguments >> `(#:out-of-source? #t >> #:configure-flags >> >> >> Data point: there=E2=80=99s 1.6 GiB in texmf-dist/doc (which the patch a= bove >> splits out), and 1.4 GiB in texmf-dist/fonts. > > I'd definitely be in favor of splitting out the docs. OK, I=E2=80=99ll test it locally and commit if nothing breaks. >> Another option Andreas and I discussed a while back would be to use a >> fixed-output derivation for the data, since it=E2=80=99s really what it = is. >> That=E2=80=99s a bit hacky though: we=E2=80=99d have to install it, comp= ute the hash of >> the installed files, and then use that as the derivation=E2=80=99s outpu= t hash. > > Hmm. It is indeed a hack, but maybe worth considering. When I think > about Guix users downloading over 3 GiB from our humble hydra quite > often just to have TeX, it makes me worry about our bandwidth > requirements. Agreed. Ludo=E2=80=99.