From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tobias Geerinckx-Rice Subject: Re: Parallel downloads Date: Thu, 31 Oct 2019 17:18:13 +0100 Message-ID: <87k18kyjlm.fsf@nckx> References: <87pnid3qdh.fsf@ambrevar.xyz> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:37049) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iQD9X-00054I-2t for guix-devel@gnu.org; Thu, 31 Oct 2019 12:18:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iQD9V-0006Tp-6B for guix-devel@gnu.org; Thu, 31 Oct 2019 12:18:26 -0400 Received: from tobias.gr ([2001:470:7405::1]:51152) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iQD9U-0006Od-Gw for guix-devel@gnu.org; Thu, 31 Oct 2019 12:18:25 -0400 Received: by tobias.gr (OpenSMTPD) with ESMTP id aea4992c for ; Thu, 31 Oct 2019 16:18:17 +0000 (UTC) Received: by submission.tobias.gr (OpenSMTPD) with ESMTPSA id 71147e9e (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO) for ; Thu, 31 Oct 2019 16:18:16 +0000 (UTC) In-reply-to: <87pnid3qdh.fsf@ambrevar.xyz> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel@gnu.org --=-=-= Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Hullo Pierre! Pierre Neidhardt =E5=86=99=E9=81=93=EF=BC=9A > Is there any plan to support parallel downloads? Guix already downloads sources and substitutes in parallel with=20 other builds/downloads through the --max-jobs option. You could add a separate knob for downloads that defaults to=20 =2D-max-jobs. Or even (* max-jobs cores). No plans for that AFAIK.=20 I don't think it's a trivial tweak. I'm interested in the numbers behind this claim: > Currently downloads are a bottleneck for `guix install /=20 > upgrade`, > parallel downloads could reduce the operation duration by an=20 > order > of magnitude. =E2=80=A6because on a substitute-only workload, my default --max-jobs=3D4=20 connections give me 4 MiB/s versus 1.5 MiB/s on a single job. That's not even a linear increase, let alone an order of magnitude=20 (base 2 doesn't count :-). Kind regards, T G-R --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEfo+u0AlEeO9y5k0W2Imw8BjFSTwFAl27CUUACgkQ2Imw8BjF STylmw//Wr0hKVzCO6Ud4kkZHu6Am9yMTUXIWZQ0zk0DD58mKVBXPisCKwcsZ8TY l/IL/GVnDIKB5EpcYpKXZJakyhgIA9Fbt+orAR/wMyTl5Z2aQoj/yXMQtBDgB6sd Tc+rf/nsj85xG+jhQDU2SV7Sfos3F9PuHtxCn2l4whHooPUyxvhXSAgVOhuyLVf8 O1yNl/8tLV4KyXMPC/BOwQpODEo5ikXclQy/qKfyM5o53uCeEFWC+ZJD4uNxtZsw vhlvQ/kYlfwKl+vJbmLbpKvIwykcRB4sNoRYaDxkZeS912GQUjvfsf9UecPiSzWB 6TjVle9Xi+URm6iicpUQdXTYbhykNV/qa0nIPn5MpQxq4akLHPaVk3nLcs6lHqia 6gSq+9q33H9cBFQOcF28wwHFd2I5l+dy+7wOll4EsQFoRO7obxkt9Q4RqG/b9sZg FamkdEXnRY6OOpTyHVQBiDWqFoOMlf7AMD4c4O/vd/a8gYjlELpty8zbYvmCqwVZ 0Db3YtzxAc+vUCtQkFCSuLxGkudh29vJPAcbhLcdmuZvSOpT6+kK266Mi7ifJbL/ aYMdkpya+E8dkfIjT+sXiPsT+4SNK9j9+el0QdijBoTNdRYZyo7piq040aVuRlGG BRV1OdPV3khDzyxLPKb/RHpL3Ldw5xenrUjmGR/c4n898JqSSrA= =D5SJ -----END PGP SIGNATURE----- --=-=-=--