From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Suggestion: disable offloading for texlive builds on hydra? Date: Sun, 26 Oct 2014 15:12:40 +0100 Message-ID: <877fzmncmf.fsf@gnu.org> References: <87ppdf1dwc.fsf@netris.org> <20141026074926.GA3937@intra> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:32821) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XiOYa-00018S-5x for guix-devel@gnu.org; Sun, 26 Oct 2014 10:12:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1XiOYV-0004rB-8y for guix-devel@gnu.org; Sun, 26 Oct 2014 10:12:32 -0400 Received: from hera.aquilenet.fr ([2a01:474::1]:41525) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1XiOYU-0004r5-Q5 for guix-devel@gnu.org; Sun, 26 Oct 2014 10:12:27 -0400 In-Reply-To: <20141026074926.GA3937@intra> (John Darrington's message of "Sun, 26 Oct 2014 08:49:27 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: John Darrington Cc: guix-devel@gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable John Darrington skribis: > On Sun, Oct 26, 2014 at 03:36:03AM -0400, Mark H Weaver wrote: > When texlive is built on hydra, the build slave that built it is tie= d up > for 12 hours or more waiting for the build outputs (over 3 gigabytes= !) > to be transferred back to hydra. >=20=20=20=20=20=20 > By design, only one transfer can happen at a time from a given build > slave, so during those 12 hours, the build slave's CPU is left idle,= and > typically another 3 built-but-not-yet-transferred packages must wait > until the texlive transfer finishes. > > Why is it designed like that? It seems like a poor design to me. The rationale was that, in general, you just slow everything down by sending several things at once. TeX Live is a pathological case in that respect. As for disabling offloading, see my reply to Federico: we could expose #:local-build? to gnu-build-system, and use that here, but at the moment that also disables substitutes. WDYT? > I suggest that we arrange for hydra.gnu.org to build texlive locally= for > x86_64 and i686, to avoid this problem. > > Would it help if texlive was split into more outputs? For example, the d= ocs=20 > take up a lot of space, and not everyone needs them. I think it may help a bit, at least by leaving a small window during which other builds could get started. And it would also be more convenient for users, who could choose whether to install the whole thing or not. When you mentioned it some time ago on IRC, I gave it a try, but then failed to actually test the patch due to... ENOSPC. :-) Anyway, here=E2=80=99s the patch I had. I=E2=80=99d be happy if you or som= eone else could just confirm it works as expected: --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/gnu/packages/texlive.scm b/gnu/packages/texlive.scm index e562b02..bc0ece7 100644 --- a/gnu/packages/texlive.scm +++ b/gnu/packages/texlive.scm @@ -88,7 +88,7 @@ ("pkg-config" ,pkg-config) ("python" ,python-2) ; incompatible with Python 3 (print syntax) ("tcsh" ,tcsh))) - (outputs '("out" "data")) + (outputs '("out" "data" "doc")) (arguments `(#:out-of-source? #t #:configure-flags --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Data point: there=E2=80=99s 1.6 GiB in texmf-dist/doc (which the patch above splits out), and 1.4 GiB in texmf-dist/fonts. Another option Andreas and I discussed a while back would be to use a fixed-output derivation for the data, since it=E2=80=99s really what it is. That=E2=80=99s a bit hacky though: we=E2=80=99d have to install it, compute= the hash of the installed files, and then use that as the derivation=E2=80=99s output h= ash. Thanks, Ludo=E2=80=99. --=-=-=--