From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id +BXyNlJ+GGD7EQAA0tVLHw (envelope-from ) for ; Mon, 01 Feb 2021 22:18:58 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 8OWwMlJ+GGA2CAAAB5/wlQ (envelope-from ) for ; Mon, 01 Feb 2021 22:18:58 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9EBF0940366 for ; Mon, 1 Feb 2021 22:18:58 +0000 (UTC) Received: from localhost ([::1]:37932 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l6hX7-00035W-K8 for larch@yhetil.org; Mon, 01 Feb 2021 17:18:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:52556) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l6hWy-00035B-T2 for guix-devel@gnu.org; Mon, 01 Feb 2021 17:18:48 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:34521) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l6hWy-0002LI-0G; Mon, 01 Feb 2021 17:18:48 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=52240 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l6hWx-0007p5-IV; Mon, 01 Feb 2021 17:18:47 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Guillaume Le Vaillant Subject: Re: When substitute download + decompression is CPU-bound References: <87im94qbby.fsf@gnu.org> <94405d66-b13c-e6e6-e8d5-df23b93e5d97@web.de> <87im92voqw.fsf@dismail.de> <87ft3d2fge.fsf@yamatai> <87bldr191v.fsf@gnu.org> <87h7ni62nt.fsf@ambrevar.xyz> <87bld9j651.fsf@gnu.org> <87eei4rs8x.fsf@ambrevar.xyz> <874kj0x9yv.fsf@yamatai> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 13 =?utf-8?Q?Pluvi=C3=B4se?= an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 01 Feb 2021 23:18:46 +0100 Message-ID: <875z3bh1o9.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.86 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 9EBF0940366 X-Spam-Score: -2.86 X-Migadu-Scanner: scn0.migadu.com X-TUID: 1N6X/LJImMz2 Hi, Guillaume Le Vaillant skribis: > Pierre Neidhardt skribis: [...] >>> It=E2=80=99s not as nice as the ability to choose a download strategy, = as we >>> discussed earlier, but implementing that download strategy sounds >>> tricky. >> >> If the user can choose their favourite substitute compression, I believe >> it's usually enough since they are the best judge of their bandwidth / >> hardware requirements. As should be clear with what Guillaume and Nico posted, it=E2=80=99s pretty= hard to determine whether you need one compression algorithm or the other, and it changes as you move your laptop around (different networking, different CPU frequency scaling strategy, etc.). > Here are a few numbers for the installation time in seconds (download > time + decompression time) when fetching 580 MB of substitutes for > download speeds between 0.5 MB/s and 20 MB/s. > > | Download speed | gzip -9 | lzip -9 | zstd -19 | > |----------------+---------+---------+----------| > | 0.5 | 287 | 151 | 181 | > | 1.0 | 144 | 78 | 91 | > | 1.5 | 97 | 54 | 61 | > | 2.0 | 73 | 42 | 46 | > | 2.5 | 59 | 35 | 37 | > | 3.0 | 49 | 30 | 31 | > | 3.5 | 42 | 27 | 26 | > | 4.0 | 37 | 24 | 23 | > | 4.5 | 33 | 22 | 21 | > | 5.0 | 30 | 21 | 19 | > | 5.5 | 28 | 19 | 17 | > | 6.0 | 25 | 18 | 16 | > | 6.5 | 24 | 17 | 14 | > | 7.0 | 22 | 17 | 14 | > | 7.5 | 21 | 16 | 13 | > | 8.0 | 20 | 15 | 12 | > | 8.5 | 18 | 15 | 11 | > | 9.0 | 18 | 14 | 11 | > | 9.5 | 17 | 14 | 10 | > | 10.0 | 16 | 13 | 10 | > | 11.0 | 15 | 13 | 9 | > | 12.0 | 14 | 12 | 8 | > | 13.0 | 13 | 12 | 8 | > | 14.0 | 12 | 11 | 7 | > | 15.0 | 11 | 11 | 7 | > | 16.0 | 11 | 11 | 6 | > | 17.0 | 10 | 10 | 6 | > | 18.0 | 10 | 10 | 6 | > | 19.0 | 9 | 10 | 5 | > | 20.0 | 9 | 10 | 5 | > > When the download speed is lower than 3.5 MB/s, Lzip is better, and > above that speed Zstd is better. > > As Gzip is never the best choice, it would make sense to drop it, even > if we have to wait a little until everyone has updated their Guix daemon > to a version with at least Lzip support. Right. We can drop it eventually, maybe soon since only 1% of our downloads pick gzip. > I think there are many people (like me) with a download speed slower > than 3 MB/s, so like Pierre I would prefer keeping "lzip -9" and > "zstd -19". Understood. To me, that means we need to implement something smart. Ludo=E2=80=99.