From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id CMd3HRtOAWCrfQAA0tVLHw (envelope-from ) for ; Fri, 15 Jan 2021 08:11:07 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id i/ItGRtOAWAeLAAAbx9fmQ (envelope-from ) for ; Fri, 15 Jan 2021 08:11:07 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C72E694043B for ; Fri, 15 Jan 2021 08:11:06 +0000 (UTC) Received: from localhost ([::1]:51176 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l0KCH-0007iS-Nj for larch@yhetil.org; Fri, 15 Jan 2021 03:11:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:59806) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0KC8-0007iK-El for guix-devel@gnu.org; Fri, 15 Jan 2021 03:10:56 -0500 Received: from relay8-d.mail.gandi.net ([217.70.183.201]:57903) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l0KC6-0004wy-4B; Fri, 15 Jan 2021 03:10:56 -0500 X-Originating-IP: 86.247.16.87 Received: from mimimi (lfbn-idf2-1-709-87.w86-247.abo.wanadoo.fr [86.247.16.87]) (Authenticated sender: mail@ambrevar.xyz) by relay8-d.mail.gandi.net (Postfix) with ESMTPSA id D92451BF20A; Fri, 15 Jan 2021 08:10:46 +0000 (UTC) From: Pierre Neidhardt To: Ludovic =?utf-8?Q?Court=C3=A8s?= , Guillaume Le Vaillant Subject: Re: When substitute download + decompression is CPU-bound In-Reply-To: <87bldr191v.fsf@gnu.org> References: <87im94qbby.fsf@gnu.org> <94405d66-b13c-e6e6-e8d5-df23b93e5d97@web.de> <87im92voqw.fsf@dismail.de> <87ft3d2fge.fsf@yamatai> <87bldr191v.fsf@gnu.org> Date: Fri, 15 Jan 2021 09:10:46 +0100 Message-ID: <87h7ni62nt.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Received-SPF: pass client-ip=217.70.183.201; envelope-from=mail@ambrevar.xyz; helo=relay8-d.mail.gandi.net X-Spam_score_int: 0 X-Spam_score: -0.1 X-Spam_bar: / X-Spam_report: (-0.1 / 5.0 requ) BAYES_00=-1.9, FROM_SUSPICIOUS_NTLD=0.498, PDS_OTHER_BAD_TLD=1.997, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -4.46 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: C72E694043B X-Spam-Score: -4.46 X-Migadu-Scanner: scn0.migadu.com X-TUID: AJCATMWDLXkt --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > One takeaway for me is that zstd decompression remains an order of > magnitude faster than the others, regardless of the compression level. > > Another one is that at level 10 and higher zstd achieves compression > ratios that are more in the ballpark of lzip. Hmmm, this is roughly true for lzip < level 6, but as soon as lzip hits lev= el 6 (the default!) it compresses up to twice as much! > If we are to change the compression methods used at ci.guix.gnu.org, we > could use zstd >=3D 10. On Guillaume's graph, the compression speed at the default level 3 is about 110 MB/s, while at level 10 it's about 40 MB/s, which is approximately the gzip speed. If server compression time does not matter, then I agree, level >=3D 10 would be a good option. What about zstd level 19 then? It's as slow as lzip to compress, but decompresses still blazingly fast, which is what we are trying to achieve here, _while_ offering a compression ration in the ballpark of lzip level 6 (but still not that of lzip level 9). > We could also drop gzip, but there are probably pre-1.1 daemons out > there that understand nothing but gzip=C2=B9, so perhaps that=E2=80=99ll = have to > wait. Now, compressing substitutes three times may be somewhat > unreasonable. Agreed, maybe release an announcement and give it a few months / 1 year? Cheers! =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmABTgYSHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/hkAIAKI/kM+2/eXO6fZPTPF1o8EmQDb6zJDo W4yreWhJsZ+n2yzI5kOGjH8XV3PjunUS+VuCU6dI0EE+04/gpU8igvRiYgdama+e +uNQpp7gEah7/2naA0cwMkXzQERoAcpSIm1OBIVvpBWqNzCnr3B6VyCl2vez9Qbz qE7fp8vv+5o4xhfdAlrLNJshbvP8+7pqJ17g8qOIh7InS0rRIlRUG6HM2Qghbq04 ClYkjYq6AQsTnRvHvJT26DhtSEDu6Y1+FSEihV1GpLlSOLKEUmoGo9/u2CUcPz7l c/aNLVLzC5n2VlKAO06hSzu8dEutzdwpnvucrXHxOcsvhJ5ZVwx4zYk= =NKXF -----END PGP SIGNATURE----- --=-=-=--