From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id 2MUdDcpzNWTCCAEASxT56A (envelope-from ) for ; Tue, 11 Apr 2023 16:50:50 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id cEDADMpzNWTtTAAAauVa8A (envelope-from ) for ; Tue, 11 Apr 2023 16:50:50 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A68B32A80A for ; Tue, 11 Apr 2023 16:50:49 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pmFJe-00064I-TP; Tue, 11 Apr 2023 10:49:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pmFJZ-00063h-FH for guix-devel@gnu.org; Tue, 11 Apr 2023 10:49:45 -0400 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pmFJX-0006xj-Hs; Tue, 11 Apr 2023 10:49:45 -0400 Received: by mail-wr1-x431.google.com with SMTP id ffacd0b85a97d-2ef1e98d6bfso398988f8f.0; Tue, 11 Apr 2023 07:49:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1681224581; x=1683816581; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=hpcgRZPhychQPmwfaTz3Se7tF8PDWXkdh9u9cJ69/iQ=; b=IsPSGHvwTE5JF95RdRgu9eiPz+aPYsn/4KvGWvl3szmMfRtlucRUSCtg4E/1H9uYUU WNwYoToqrMX+/muQthPzt4dqsjdbxAxIWHVEUqtI76a5DM+FUKPGNjJblmEg4YZnsvnl 6jZw3/pwPEFAJct9Q7ZqqMDwJEPAiBn5MLi1XtVe5mwTn4Fl/gf6fyPEUaQc489RqK1c iGIJGIZhtX73nmE8bNB8iVV26Wn2Fo4tOq/NdqSYV/v7z53ir0mWpx8LMlvmpclRuokL Cr5CDkt/O3AaFnqvAI/f9MjZKZXF/ZLlNb3cJ4MPPKpEoTWh1O2Pf7rKENsVJzccljL8 iYbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1681224581; x=1683816581; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hpcgRZPhychQPmwfaTz3Se7tF8PDWXkdh9u9cJ69/iQ=; b=tCnaJVqmqFROAUKyv37KnJ1GvHvV4oQ/9u7dO34FJ0DQBOdXkV04XrSidJFIP7eK19 eelDbyW27Uj28iUZWnH0idEWnnpDMQAvLJ3bTdycgwBNw/d+I4YARxyAPjH/pQNRP5Hv Tdn9Nk/+Zt+xU3VXcSA0jOtEmjL53j929OSPeqxYAYFgRbCbtdZrWMWNxjzdmKSo0SYx epDnThsjHGAIgvhXopoCUczdd+yF5YcQTRlQKXN7TmJsoRKDwWnHrQaTdZDA5Ch7B4tI LGgzyBaovXvMqSWo8BHsk0j9sCu0MpH33XMRmhwdfPufQHZQWB4mDX/IPdS63VOVq0Zp bNag== X-Gm-Message-State: AAQBX9c7PCWoOy5UStzgcIWDnlOWFzAnoYer40rRG1d5H8zATXyk7mAM /ecO7g0NmM4TnzEqfqY+xxqe4NRqFM0= X-Google-Smtp-Source: AKy350b4OO/M222HGUxyOLGFLe/GeIToFWswuy3LoX6pr1/FUo1CSZIjPH819MWwj9oJTNvfEkmu4Q== X-Received: by 2002:a5d:4d4d:0:b0:2d3:b1fa:302c with SMTP id a13-20020a5d4d4d000000b002d3b1fa302cmr5744005wru.5.1681224580539; Tue, 11 Apr 2023 07:49:40 -0700 (PDT) Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id c3-20020adffb43000000b002de99432fc8sm14739240wrs.49.2023.04.11.07.49.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Apr 2023 07:49:40 -0700 (PDT) From: Simon Tournier To: Maxim Cournoyer , Ludovic =?utf-8?Q?Court?= =?utf-8?Q?=C3=A8s?= Cc: Christopher Baines , guix-devel@gnu.org Subject: Re: What's the state of (guix build download-nar)? In-Reply-To: <87a61mhqkp.fsf@gmail.com> References: <871qn2erqp.fsf@cbaines.net> <87o7q4zfr4.fsf@gnu.org> <87a61mhqkp.fsf@gmail.com> Date: Tue, 11 Apr 2023 15:34:31 +0200 Message-ID: <87edoqk0ew.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::431; envelope-from=zimon.toutoune@gmail.com; helo=mail-wr1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1681224649; a=rsa-sha256; cv=none; b=hNyn2xoS1VInzAPyWpME7Nj0K+GJjr8FUxb1lgcWsBKidrPSCHogqBi/ABendqYfQxxtgG 4h4aUBE7E5mzvfvO8MNP4wIUN/K2nmTC9J3rZNNzpHIQlgab53K+C0Irg56JJaIiKAPNGr Rk7G+qbex19ACacvN4EuZ8taGvlgfOHRmExSAjwURdRl43k80IRISMlvbHNBgwMmrahOYm 13Rb2Fh5UXN7YjiK/vfUPWleoFfFv7nCcsQxbRu6kB0jMs7J/U8GabvwFNzH3raqSyi1Xh NrFsnuLY8u+3ht/yg2ELq/PhagzKzC6izKnt5b0KLY2Kcy58T/nReziP5z6tBw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=IsPSGHvw; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1681224649; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=hpcgRZPhychQPmwfaTz3Se7tF8PDWXkdh9u9cJ69/iQ=; b=dEl0KCPniq+2m2kB705kyTkTMkykcvLxk2QtME5zCaOzVX8lzbzQ2a8anbMnfUdR+Ght6C 7DO6zoiqID6W9rd4mTXTuiMgZVG0/zvEd8u19CBkuzwKu+J6o4v9Pr0jz/aoAL4KHHHAM/ ud/Xt2ele7rVZDpnjnm+jxsGtRrgFWg5uveqfPZlxwVyCfhzyBUJpqC/ABVBbbeCIIBXrg TULo1HTT6iT2oR6p9y7YuvbTt6AH5hhIGCL8SHDfPS5bP43dA8xImwZocVvd0KbuaPUfbn HebrReHaQW+oQ+aGJhNUHq1C5jNDM6JWwVDE5RVoKJAMtxLKgL7Pom6f/phdQg== X-Migadu-Spam-Score: -3.95 X-Migadu-Scanner: scn1.migadu.com Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=IsPSGHvw; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Spam-Score: -3.95 X-Migadu-Queue-Id: A68B32A80A X-TUID: 37QJJqUg1fEJ Hi, On jeu., 09 f=C3=A9vr. 2023 at 09:16, Maxim Cournoyer wrote: >> Currently it=E2=80=99s used by some of the (guix VCS-download) modules. = I think >> we should just update to (1) use lzip instead of gzip, and (2) have it >> check ci.guix.gnu.org + bordeaux.guix.gnu.org. > > How about using zstd? I'm proposing it instead of lzip, because long > term, I think we want to reduce the size of our storage requirements and > offer a single compression type for our NARs, which zstd would be ideal > (it's faster and compresses close enough to lzip). I guess it=E2=80=99s the same direction as this thread [1], When substitute download + decompression is CPU-bound Ludovic Court=C3=A8s Mon, 14 Dec 2020 23:20:17 +0100 id:87im94qbby.fsf@gnu.org especially this Guillaume=E2=80=99s message [2] comparing various methods. = The possible agenda [3] concluding the thread was, quoting: We could do that. I suppose a possible agenda would be: 1. Start providing zstd susbstitutes anytime. However, most clie= nts will keep choosing lzip because it usually compresses better. 2. After the next release, stop providing lzip substitutes and pr= ovide only gzip + zstd-19. This option has the advantage that it wouldn=E2=80=99t break any in= stallation. It=E2=80=99s not as nice as the ability to choose a download strate= gy, as we discussed earlier, but implementing that download strategy sounds tricky. Well, to be honest, I am a bit lost about the compression methods; especially when also considering this old blog post [4]. As the subject of this thread is asking: =E2=80=9CWhat's the state of (guix build download-nar)?=E2=80=9D ;-) 1: https://yhetil.org/guix/87im94qbby.fsf@gnu.org/#r 2: https://yhetil.org/guix/87ft3d2fge.fsf@yamatai 3: https://yhetil.org/guix/87bld9j651.fsf@gnu.org 4: https://guix.gnu.org/en/blog/2019/substitutes-are-now-available-as-lzip/ Cheers, simon