From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id QJ0gDCKh2F+pOAAA0tVLHw (envelope-from ) for ; Tue, 15 Dec 2020 11:42:26 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id OAAGCCKh2F+gFAAA1q6Kng (envelope-from ) for ; Tue, 15 Dec 2020 11:42:26 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id EBBC2940466 for ; Tue, 15 Dec 2020 11:42:25 +0000 (UTC) Received: from localhost ([::1]:44606 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kp8im-0002Uc-Sf for larch@yhetil.org; Tue, 15 Dec 2020 06:42:24 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:58250) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kp8ic-0002TV-D6 for guix-devel@gnu.org; Tue, 15 Dec 2020 06:42:14 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49611) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kp8ia-0007HH-Mw; Tue, 15 Dec 2020 06:42:12 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=44648 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kp8ia-0004Oe-4R; Tue, 15 Dec 2020 06:42:12 -0500 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Pierre Neidhardt Subject: Re: When substitute download + decompression is CPU-bound References: <87im94qbby.fsf@gnu.org> <87sg88ngd1.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> <871rfrld59.fsf@ambrevar.xyz> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 25 Frimaire an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 15 Dec 2020 12:42:10 +0100 In-Reply-To: <871rfrld59.fsf@ambrevar.xyz> (Pierre Neidhardt's message of "Tue, 15 Dec 2020 08:52:18 +0100") Message-ID: <87blevmh2l.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel , =?utf-8?Q?Nicol=C3=B2?= Balzarotti Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.81 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: EBBC2940466 X-Spam-Score: -2.81 X-Migadu-Scanner: scn0.migadu.com X-TUID: GuqNA421Fb/s Hi, Pierre Neidhardt skribis: > Another option is plzip (parallel Lzip, an official part of Lzip). > >> decompression of ungoogled-chromium from the LAN completes in 2.4s for >> gzip vs. 7.1s for lzip. On a low-end ARMv7 device, also on the LAN, I >> get 32s (gzip) vs. 53s (lzip). > > With four cores, plzip would beat gzip in the first case. > With only 2 cores, plzip would beat gzip in the second case. > > What's left to do to implement plzip support? That's the good news: > almost nothing! > > - On the Lzip binding side, we need to add support for multi pages. > It's a bit of work but not that much. > - On the Guix side, there is nothing to do. Well, =E2=80=98guix publish=E2=80=99 would first need to create multi-membe= r archives, right? Also, lzlib (which is what we use) does not implement parallel decompression, AIUI. Even if it did, would we be able to take advantage of it? Currently =E2=80=98restore-file=E2=80=99 expects to read an archive stream sequential= ly. Even if I=E2=80=99m wrong :-), decompression speed would at best be doubled= on multi-core machines (wouldn=E2=80=99t help much on low-end ARM devices), and that=E2=80=99s very little compared to the decompression speed achieved by = zstd. Ludo=E2=80=99.