From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id mO4XDHzu11+7JAAA0tVLHw (envelope-from ) for ; Mon, 14 Dec 2020 23:00:12 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id uLsGCHzu118UNAAAbx9fmQ (envelope-from ) for ; Mon, 14 Dec 2020 23:00:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E3204940396 for ; Mon, 14 Dec 2020 23:00:11 +0000 (UTC) Received: from localhost ([::1]:56276 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kowp8-0005lq-Lj for larch@yhetil.org; Mon, 14 Dec 2020 18:00:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33018) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kowoy-0005lj-II for guix-devel@gnu.org; Mon, 14 Dec 2020 18:00:00 -0500 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]:33914) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kowow-0002dX-VC; Mon, 14 Dec 2020 18:00:00 -0500 Received: by mail-wm1-x32b.google.com with SMTP id g25so9903779wmh.1; Mon, 14 Dec 2020 14:59:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:in-reply-to:references:date:message-id:mime-version :content-transfer-encoding; bh=0l9xvEbpMsmf2De3LmPWbfTe8FbYZPTRWz6IHxtn9/Y=; b=FB+qSbKqoGb1ob2IAuOMUNFsFmMbP8k0MtgBDxWJV/EcNmh23GprtNdgsE/CV5noku bYqrhp2Kl63VegXfHu9CVcDFHRZayLUwC44uD9+xfxu+1ctOWYFg7BFWbnbixUljrFoe CHGkkUpXVjS358q4yE2o2ik8rOiJd627oTSrgrBRGp1q2NuhjL/U9ZeJjqajbXd4Z9iu /5f5U771tfGdp5NPqFOwbuky83qpW6a3V9htxAjPrp5x3k1AECGmU8Pch1WkJhgk8fuO jKQc9czfNLafItUV3Je9EkZMMa9cFVtmvi9iVs0UJVhk+mEf28zpzzgEsQsi4P1JbOVI aYGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=0l9xvEbpMsmf2De3LmPWbfTe8FbYZPTRWz6IHxtn9/Y=; b=WpnD8D0qvYH4Gv3/YbnVpAGhxY4Pb5ucdnieq3wbT3Yup58gPsaZGGZV616NSxRWQG 2BLnEAVIXTT1k7tjgt8GME2A9raH99ieiHO9WBu0+lYcmgwyKH2parIEUaJ+7dvT5mfZ csnPSD7TlNggjP7rm5pswP8TNKV8b//b4l5ynN9HV2lD12rH2Q/LHOnbBL05vBSGnUdT Buz2Im4CIylGEg22PvfB7zuTVp6+AhUSjh6E/NgmmZBqyUqkKpmbYLuBXtblXJhT/Mln WHKiLHx8+4iBPEa9R4hVKMWKO0cnaSVrJbRv3DEkmGXs8E5QXTGBqKd10K47M5tWpRo6 mkjA== X-Gm-Message-State: AOAM533IrBR5BU/5cVjvBkcTx2WVc8Q4PW8V+Cvu6IZ02XpF23f2voNf pZn4u+UJPJSTOJJgbV5Y6JbqkH05Xfo= X-Google-Smtp-Source: ABdhPJxyDGE4FJ4R5CiY64u4uGKT0xVlDduueiMRWRc3/sE/9O8D3FlVa5oQBF0xxNWQLinRFuJxvQ== X-Received: by 2002:a1c:9ecf:: with SMTP id h198mr29791379wme.104.1607986796287; Mon, 14 Dec 2020 14:59:56 -0800 (PST) Received: from guixSD (host-95-244-155-253.retail.telecomitalia.it. [95.244.155.253]) by smtp.gmail.com with ESMTPSA id z15sm35580708wrv.67.2020.12.14.14.59.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 14 Dec 2020 14:59:55 -0800 (PST) From: =?utf-8?Q?Nicol=C3=B2?= Balzarotti To: Ludovic =?utf-8?Q?Court=C3=A8s?= , guix-devel Subject: Re: When substitute download + decompression is CPU-bound In-Reply-To: <87im94qbby.fsf@gnu.org> References: <87im94qbby.fsf@gnu.org> Date: Mon, 14 Dec 2020 23:59:54 +0100 Message-ID: <87sg88ngd1.fsf@guixSD.i-did-not-set--mail-host-address--so-tickle-me> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::32b; envelope-from=anothersms@gmail.com; helo=mail-wm1-x32b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -1.51 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=FB+qSbKq; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: E3204940396 X-Spam-Score: -1.51 X-Migadu-Scanner: scn0.migadu.com X-TUID: YtzfvwM0ieQk Ludovic Court=C3=A8s writes: > Hi Guix! > Hi Ludo > Quick decompression bench: I guess this benchmark follows the distri talk, doesn't it? :) File size with zstd vs zstd -9 vs current lzip: - 71M uc.nar.lz - 87M uc.nar.zst-9 - 97M uc.nar.zst-default > Where to go from here? Several options: > 1. Since ci.guix.gnu.org still provides both gzip and lzip archives, > =E2=80=98guix substitute=E2=80=99 could automatically pick one or th= e other > depending on the CPU and bandwidth. Perhaps a simple trick would > be to check the user/wall-clock time ratio and switch to gzip for > subsequent downloads if that ratio is close to one. How well would > that work? I'm not sure using heuristics (i.e., guessing what should work better, like in 1.) is the way to go, as temporary slowdowns to the network/cpu will during the first download would affect the decision. > 2. Use Zstd like all the cool kids since it seems to have a much > higher decompression speed: . > 630=C2=A0MB/s on ungoogled-chromium on my laptop. Woow. I know this means more work to do, but it seems to be the best alternative. However, if we go that way, will we keep lzip substitutes? The 20% difference in size between lzip/zstd would mean a lot with slow (mobile) network connections. Nicol=C3=B2