From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id gE+NKkj9EmCLKAAA0tVLHw (envelope-from ) for ; Thu, 28 Jan 2021 18:07:04 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id MDxeJkj9EmANWwAA1q6Kng (envelope-from ) for ; Thu, 28 Jan 2021 18:07:04 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4F8D8940276 for ; Thu, 28 Jan 2021 18:07:04 +0000 (UTC) Received: from localhost ([::1]:44116 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l5Bh9-0002Wm-2T for larch@yhetil.org; Thu, 28 Jan 2021 13:07:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:41846) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l5BYT-0004kG-NN for guix-devel@gnu.org; Thu, 28 Jan 2021 12:58:06 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:48645) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l5BYT-0005Ye-31; Thu, 28 Jan 2021 12:58:05 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=58668 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1l5BYR-00053c-Ns; Thu, 28 Jan 2021 12:58:04 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Pierre Neidhardt Subject: Re: When substitute download + decompression is CPU-bound References: <87im94qbby.fsf@gnu.org> <94405d66-b13c-e6e6-e8d5-df23b93e5d97@web.de> <87im92voqw.fsf@dismail.de> <87ft3d2fge.fsf@yamatai> <87bldr191v.fsf@gnu.org> <87h7ni62nt.fsf@ambrevar.xyz> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 9 =?utf-8?Q?Pluvi=C3=B4se?= an 229 de la =?utf-8?Q?R?= =?utf-8?Q?=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 28 Jan 2021 18:58:02 +0100 In-Reply-To: <87h7ni62nt.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Fri, 15 Jan 2021 09:10:46 +0100") Message-ID: <87bld9j651.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.85 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 4F8D8940276 X-Spam-Score: -2.85 X-Migadu-Scanner: scn0.migadu.com X-TUID: wMMjJoqNuhy8 Pierre Neidhardt skribis: > On Guillaume's graph, the compression speed at the default level 3 is > about 110 MB/s, while at level 10 it's about 40 MB/s, which is > approximately the gzip speed. > > If server compression time does not matter, then I agree, level >=3D 10 > would be a good option. > > What about zstd level 19 then? It's as slow as lzip to compress, but > decompresses still blazingly fast, which is what we are trying to > achieve here, _while_ offering a compression ration in the ballpark of > lzip level 6 (but still not that of lzip level 9). We could do that. I suppose a possible agenda would be: 1. Start providing zstd susbstitutes anytime. However, most clients will keep choosing lzip because it usually compresses better. 2. After the next release, stop providing lzip substitutes and provide only gzip + zstd-19. This option has the advantage that it wouldn=E2=80=99t break any installati= on. It=E2=80=99s not as nice as the ability to choose a download strategy, as we discussed earlier, but implementing that download strategy sounds tricky. Thoughts? Ludo=E2=80=99.