Re: Suggestion: disable offloading for texlive builds on hydra?

[ludo@gnu.org (Ludovic Courtès)]

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:

[...]

>> The rationale was that, in general, you just slow everything down by
>> sending several things at once.
>
> I have my doubts that it would slow things down very much, if at all.
> The number of parallel transfers would still be limited to a small
> number, typically 4 per build slave.  The expense associated with
> running multiple processes on a CPU is mainly due to cache effects, but
> I wouldn't expect that to be an issue with network connections,
> especially when those connections are between the same two hosts.  The
> practice of using multiple connections is well established in web
> browsers and imap clients, as long as the number is not too large.
>
> We're losing a huge amount of available CPU capacity in our build farm
> (probably over 30 machine-hours per texinfo rebuild) in exchange for a
> dubious increase in network efficiency.
>
> The more I think about it, the more I agree with John that we've chosen
> the wrong tradeoff here.  I think we should remove those mutexes.

Hmm OK.  I’m happy to try that (it’s a two-line change plus deployment.)

I can do it one of the next few days, but I’m happy if you do it.  :-)

>> diff --git a/gnu/packages/texlive.scm b/gnu/packages/texlive.scm
>> index e562b02..bc0ece7 100644
>> --- a/gnu/packages/texlive.scm
>> +++ b/gnu/packages/texlive.scm
>> @@ -88,7 +88,7 @@
>>        ("pkg-config" ,pkg-config)
>>        ("python" ,python-2) ; incompatible with Python 3 (print syntax)
>>        ("tcsh" ,tcsh)))
>> -   (outputs '("out" "data"))
>> +   (outputs '("out" "data" "doc"))
>>     (arguments
>>      `(#:out-of-source? #t
>>        #:configure-flags
>>
>>
>> Data point: there’s 1.6 GiB in texmf-dist/doc (which the patch above
>> splits out), and 1.4 GiB in texmf-dist/fonts.
>
> I'd definitely be in favor of splitting out the docs.

OK, I’ll test it locally and commit if nothing breaks.

>> Another option Andreas and I discussed a while back would be to use a
>> fixed-output derivation for the data, since it’s really what it is.
>> That’s a bit hacky though: we’d have to install it, compute the hash of
>> the installed files, and then use that as the derivation’s output hash.
>
> Hmm.  It is indeed a hack, but maybe worth considering.  When I think
> about Guix users downloading over 3 GiB from our humble hydra quite
> often just to have TeX, it makes me worry about our bandwidth
> requirements.

Agreed.

Ludo’.