From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id uO6VDlnZE2D2EAAA0tVLHw (envelope-from ) for ; Fri, 29 Jan 2021 09:46:01 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 4JNFClnZE2A0AgAAbx9fmQ (envelope-from ) for ; Fri, 29 Jan 2021 09:46:01 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5FB2E9402C8 for ; Fri, 29 Jan 2021 09:46:00 +0000 (UTC) Received: from localhost ([::1]:51888 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l5QLm-0008G9-Ho for larch@yhetil.org; Fri, 29 Jan 2021 04:45:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:48690) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l5QLb-0008F2-5V for guix-devel@gnu.org; Fri, 29 Jan 2021 04:45:47 -0500 Received: from relay9-d.mail.gandi.net ([217.70.183.199]:38049) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l5QLX-0008DM-32; Fri, 29 Jan 2021 04:45:45 -0500 X-Originating-IP: 86.247.16.87 Received: from bababa (lfbn-idf2-1-709-87.w86-247.abo.wanadoo.fr [86.247.16.87]) (Authenticated sender: mail@ambrevar.xyz) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 05266FF876; Fri, 29 Jan 2021 09:45:34 +0000 (UTC) From: Pierre Neidhardt To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: When substitute download + decompression is CPU-bound In-Reply-To: <87bld9j651.fsf@gnu.org> References: <87im94qbby.fsf@gnu.org> <94405d66-b13c-e6e6-e8d5-df23b93e5d97@web.de> <87im92voqw.fsf@dismail.de> <87ft3d2fge.fsf@yamatai> <87bldr191v.fsf@gnu.org> <87h7ni62nt.fsf@ambrevar.xyz> <87bld9j651.fsf@gnu.org> Date: Fri, 29 Jan 2021 10:45:34 +0100 Message-ID: <87eei4rs8x.fsf@ambrevar.xyz> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Received-SPF: pass client-ip=217.70.183.199; envelope-from=mail@ambrevar.xyz; helo=relay9-d.mail.gandi.net X-Spam_score_int: 0 X-Spam_score: -0.1 X-Spam_bar: / X-Spam_report: (-0.1 / 5.0 requ) BAYES_00=-1.9, FROM_SUSPICIOUS_NTLD=0.499, PDS_OTHER_BAD_TLD=1.999, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -4.45 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 5FB2E9402C8 X-Spam-Score: -4.45 X-Migadu-Scanner: scn1.migadu.com X-TUID: U1n5EOwGx2SE --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo! Ludovic Court=C3=A8s writes: >> On Guillaume's graph, the compression speed at the default level 3 is >> about 110 MB/s, while at level 10 it's about 40 MB/s, which is >> approximately the gzip speed. >> >> If server compression time does not matter, then I agree, level >=3D 10 >> would be a good option. >> >> What about zstd level 19 then? It's as slow as lzip to compress, but >> decompresses still blazingly fast, which is what we are trying to >> achieve here, _while_ offering a compression ration in the ballpark of >> lzip level 6 (but still not that of lzip level 9). > > We could do that. I suppose a possible agenda would be: > > 1. Start providing zstd susbstitutes anytime. However, most clients > will keep choosing lzip because it usually compresses better. > > 2. After the next release, stop providing lzip substitutes and provide > only gzip + zstd-19. > > This option has the advantage that it wouldn=E2=80=99t break any installa= tion. But why would we keep gzip since it offers no benefits compared to zstd? It feels like continuing to carry a (huge) burden forever... Besides, dropping Lzip seems like a step backward in my opinion. Users with lower bandwidth (or simply further away from Berlin) will be impacted a lot. I would opt for dropping gzip instead, only to keep zstd-19 and lzip-9 (possibly plzip-9 if we update the bindings). > It=E2=80=99s not as nice as the ability to choose a download strategy, as= we > discussed earlier, but implementing that download strategy sounds > tricky. If the user can choose their favourite substitute compression, I believe it's usually enough since they are the best judge of their bandwidth / hardware requirements. Wouldn't this simple enough? =2D-=20 Pierre Neidhardt https://ambrevar.xyz/ --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQFGBAEBCAAwFiEEUPM+LlsMPZAEJKvom9z0l6S7zH8FAmAT2T4SHG1haWxAYW1i cmV2YXIueHl6AAoJEJvc9Jeku8x/jqUH/j6qqsd88dlYpwZhHyu4/VmdrtkBRh4b qbvepDvk1KT8C89FLGCGpBeYfGFsWG08VkI2xGnpsn+HDbn/Aht+lKQWmgEyx/5e vYCcxSmyopYmfXPCPa11LHIc/Icr7J2NAzNto2220zNEXfTi53pmGDCrKPnNn/jR iRggnY3zeBCtVkbUj4SLaHL9yDpPE0jYI56Lze1sdFVsGKjtvQc7D0z00X0BRvK+ //ususbr5f2is3LkZdoTcDITN20YPl6Uxrtv7dmR7oIm+vJBaW9uodw5nbz4I0L7 evPQLmBKLc/CnsU31fG+ERnQgRcQuRuN39jv+/NuD1pWAjlBr3wSEK8= =Ce5h -----END PGP SIGNATURE----- --=-=-=--