Re: Parallel downloads

["Ludovic Courtès" <ludo@gnu.org>]

Hi Pierre!

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>>> - Can we configure the default value?
>>
>> Yup, just pass ‘--max-jobs=N’ to the daemon.
>
> So I suppose you mean to use the `extra-options` field, e.g.
>
> (guix-service-type config =>
>                    (guix-configuration
>                     (inherit config)
>                     (extra-options '("--max-jobs=4"))))

Yes.

> Ludo, what do you think of Tobias suggestion and have an extra knob to
> specifically configure the number of download jobs?

Like I wrote, it’s not that simple (we’d first need the daemon to
distinguish substitution jobs from other jobs, but note that there are
also “downloads” that are actually derivation builds), and it’s not
clear to me that it’s overall beneficial anyway: it’s not supposed to be
faster to download 10 things in parallel from ci.guix.gnu.org, than to
download them sequentially.

If it _is_ faster, then we need to investigate why that is the case.
For example, I’m aware that for some reason, nginx gives us low
bandwidth when downloading a nar that’s not already in its cache.  Its
probably an nginx misconfiguration issue but I couldn’t find out.  (This
has been discussed before on one of the mailing lists, I think.)

Does that make sense?  :-)

Thanks,
Ludo’.