From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: Re: Parallel downloads Date: Sun, 17 Nov 2019 18:15:24 +0100 Message-ID: <87zhgumnkj.fsf@gnu.org> References: <87pnid3qdh.fsf@ambrevar.xyz> <87k18kyjlm.fsf@nckx> <87wock97ye.fsf@ambrevar.xyz> <87k18h3tii.fsf@gnu.org> <87muddm110.fsf@ambrevar.xyz> <87eeyl10jq.fsf@gnu.org> <875zjxq978.fsf@ambrevar.xyz> <87h83d7xsn.fsf@gnu.org> <20191112174442.GA25397@jasmine.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:38626) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iWO91-0003HL-8w for guix-devel@gnu.org; Sun, 17 Nov 2019 12:15:28 -0500 In-Reply-To: <20191112174442.GA25397@jasmine.lan> (Leo Famulari's message of "Tue, 12 Nov 2019 12:44:42 -0500") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Leo Famulari Cc: guix-devel@gnu.org Hi, Leo Famulari skribis: > On Sat, Nov 09, 2019 at 06:40:56PM +0100, Ludovic Court=C3=A8s wrote: >> Like I wrote, it=E2=80=99s not that simple (we=E2=80=99d first need the = daemon to >> distinguish substitution jobs from other jobs, but note that there are >> also =E2=80=9Cdownloads=E2=80=9D that are actually derivation builds), a= nd it=E2=80=99s not >> clear to me that it=E2=80=99s overall beneficial anyway: it=E2=80=99s no= t supposed to be >> faster to download 10 things in parallel from ci.guix.gnu.org, than to >> download them sequentially. > > Parallel downloading is not faster in terms of overall transfer rate > from ci.guix.gnu.org. > > However, installing things with Guix involves downloading a lot of very > small files like derivations, and Guix spends a lot of time initiating > these downloads. Good point. Note that .drv files are never downloaded, but nevertheless it=E2=80=99s tr= ue that there are often small files like the =E2=80=9Cmodule-import-compiled= =E2=80=9D things. This happens when building a system, but not so much when building a package, though. > For example, I can download things at 100 megabits, but when Guix needs > to sequentially download 50 10-kilobyte files, it may take an entire > minute. > > So there is a huge speedup with parallel downloading. One thing I=E2=80=99d like to get rid of is the initial HTTP GET for /nix-cache-info which is completely useless now. That could help a bit. Thanks, Ludo=E2=80=99.