From mboxrd@z Thu Jan  1 00:00:00 1970
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@gnu.org>
Subject: Re: Parallel downloads
Date: Sat, 09 Nov 2019 18:40:56 +0100
Message-ID: <87h83d7xsn.fsf@gnu.org>
References: <87pnid3qdh.fsf@ambrevar.xyz> <87k18kyjlm.fsf@nckx>
 <87wock97ye.fsf@ambrevar.xyz> <87k18h3tii.fsf@gnu.org>
 <87muddm110.fsf@ambrevar.xyz> <87eeyl10jq.fsf@gnu.org>
 <875zjxq978.fsf@ambrevar.xyz>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Return-path: <guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org>
Received: from eggs.gnu.org ([2001:470:142:3::10]:58799)
 by lists.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@gnu.org>) id 1iTUjM-0004Ex-9G
 for guix-devel@gnu.org; Sat, 09 Nov 2019 12:41:01 -0500
In-Reply-To: <875zjxq978.fsf@ambrevar.xyz> (Pierre Neidhardt's message of
 "Wed, 06 Nov 2019 17:08:11 +0100")
List-Id: "Development of GNU Guix and the GNU System distribution."
 <guix-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/guix-devel>,
 <mailto:guix-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/guix-devel>
List-Post: <mailto:guix-devel@gnu.org>
List-Help: <mailto:guix-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/guix-devel>,
 <mailto:guix-devel-request@gnu.org?subject=subscribe>
Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org
Sender: "Guix-devel" <guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org>
To: Pierre Neidhardt <mail@ambrevar.xyz>
Cc: guix-devel@gnu.org

Hi Pierre!

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> Ludovic Court=C3=A8s <ludo@gnu.org> writes:
>
>>> - Can we configure the default value?
>>
>> Yup, just pass =E2=80=98--max-jobs=3DN=E2=80=99 to the daemon.
>
> So I suppose you mean to use the `extra-options` field, e.g.
>
> (guix-service-type config =3D>
>                    (guix-configuration
>                     (inherit config)
>                     (extra-options '("--max-jobs=3D4"))))

Yes.

> Ludo, what do you think of Tobias suggestion and have an extra knob to
> specifically configure the number of download jobs?

Like I wrote, it=E2=80=99s not that simple (we=E2=80=99d first need the dae=
mon to
distinguish substitution jobs from other jobs, but note that there are
also =E2=80=9Cdownloads=E2=80=9D that are actually derivation builds), and =
it=E2=80=99s not
clear to me that it=E2=80=99s overall beneficial anyway: it=E2=80=99s not s=
upposed to be
faster to download 10 things in parallel from ci.guix.gnu.org, than to
download them sequentially.

If it _is_ faster, then we need to investigate why that is the case.
For example, I=E2=80=99m aware that for some reason, nginx gives us low
bandwidth when downloading a nar that=E2=80=99s not already in its cache.  =
Its
probably an nginx misconfiguration issue but I couldn=E2=80=99t find out.  =
(This
has been discussed before on one of the mailing lists, I think.)

Does that make sense?  :-)

Thanks,
Ludo=E2=80=99.