From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id wBTeH4+WCmIYHAEAgWs5BA (envelope-from ) for ; Mon, 14 Feb 2022 18:51:11 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id UFCSGI+WCmJepAAAG6o9tA (envelope-from ) for ; Mon, 14 Feb 2022 18:51:11 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 2434E38EBF for ; Mon, 14 Feb 2022 18:51:11 +0100 (CET) Received: from localhost ([::1]:51896 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nJfVG-0005os-8v for larch@yhetil.org; Mon, 14 Feb 2022 12:51:10 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58048) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nJfV3-0005oS-QN; Mon, 14 Feb 2022 12:50:57 -0500 Received: from [2a0c:e300::1] (port=48788 helo=hera.aquilenet.fr) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nJfV0-0004BG-Pc; Mon, 14 Feb 2022 12:50:57 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 6B2D0B57; Mon, 14 Feb 2022 18:50:50 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rNT7UCJ1zpLj; Mon, 14 Feb 2022 18:50:49 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 1728C416; Mon, 14 Feb 2022 18:50:49 +0100 (CET) From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Maxim Cournoyer Subject: Re: Dropping gzip-compressed substitutes References: <878rum1pph.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 26 =?utf-8?Q?Pluvi=C3=B4se?= an 230 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 14 Feb 2022 18:50:48 +0100 In-Reply-To: <878rum1pph.fsf@gmail.com> (Maxim Cournoyer's message of "Mon, 07 Feb 2022 13:44:42 -0500") Message-ID: <87h791s5fr.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: / X-Rspamd-Server: hera X-Rspamd-Queue-Id: 6B2D0B57 X-Spamd-Result: default: False [0.61 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FREEMAIL_ENVRCPT(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_ALL(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; R_MIXED_CHARSET(0.71)[subject]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a0c:e300::1 (failed) Received-SPF: softfail client-ip=2a0c:e300::1; envelope-from=ludo@gnu.org; helo=hera.aquilenet.fr X-Spam_score_int: -4 X-Spam_score: -0.5 X-Spam_bar: / X-Spam_report: (-0.5 / 5.0 requ) BAYES_00=-1.9, RDNS_NONE=0.793, SPF_HELO_PASS=-0.001, SPF_SOFTFAIL=0.665, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel , guix-sysadmin Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1644861071; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ALkmM527DmBg8bHpUBEmeqGEjdr2gKGD690Abwx5HN4=; b=BGF6Tr5sqwYcjUxL3N7XyOaz5B/RqMf+BVCDA9oZS+ScnuDDJ2kCm3w8tu7HY88ByJ5y+j c0llsrQNy5FKftRJmEHmtgjJmnLZpTXmbfPBCuZUKVAoq5al6FlTlivLIpxFu6Asq6hfs3 CCfS6U+0TUfjB+ecZep93/Vy7eosv9NhOU37YRzbFEoe/kr7aXoN9TilGUOCcg18HkiYod qeXeLss0fo8EWMiacBBjCz9t8tV2VzdNtVy6mVi9YtqgrUO0ZoMc/a1qw2aLljr2mqOIYP PAZMX9UG8TWYj8LcRLuq/GM16WKolqS8MwUAKpuD4mR0+tvWGRbkNm1k4uMwVw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1644861071; a=rsa-sha256; cv=none; b=M55MtCaeN/xq8vHl9EeK8qtEI2i+0uPGJ0HS3tijisxy48qbsLGRBlvEI9+2Hx63NE10Oj ZYbDnmvS+I9xO65aFE8tYVDcwROsEL4EpsU6hQk2YLa6u5Jh8xqCtqV0yv30kd1kUW3PdQ un53A3zVAjjmhzikAW3xkPQdpKZy83MKpfc0FYry2V2zRj1ps+sj4xozdpSwcZcs9G43Gg c5hP+EUtx5KqDFTma9dF3DnUm4O4FJX9M8hYGZW4igoIZ0v6Cv5J5cEd0TCC4dkLlvvWQd 28uCvOJLGmwzrqAKS+fs7S73dtvxckvkZgbCc/HXiqM/YEu1sYVWip0PJePPiA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.73 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 2434E38EBF X-Spam-Score: -4.73 X-Migadu-Scanner: scn0.migadu.com X-TUID: xe43Aaoto6Tu Hi! Thumbs up for the progress made! Maxim Cournoyer skribis: > 3. Come up with a Guile script that is able to > > a) Strip gzip-related metadata from the .narinfo guix-publish metadata > files > b) recompute and update their 'Signature' field. > > 4. Finally, 'rm -r /var/guix/publish/gzip' and free about 6.5 TiB of data. As discussed on IRC, I=E2=80=99m skeptical about this because: 1. It requires the development and testing of a custom tool that=E2=80=99s easy to get wrong=E2=80=94e.g., it removes a gzipped nar for something= that had nothing but gzip available, etc. 2. That code would have to run with privileges that give it access to the signing key on berlin. 3. Those 6.5=C2=A0TB are an initial constant factor; growth of the storage requirements going forward probably matters more and will give us more flexibility on that. I like Chris Baines=E2=80=99 idea of decoupling nar distribution from nar building. If we want to keep nars long enough so that =E2=80=98time-machin= e=E2=80=99 is usable, then storage requirements will keep growing. Perhaps that means we can regularly copy nars =E2=80=9Celsewhere=E2=80=9D f= or long-term storage, using nar-herder, rsync, or whatever. The machine that stores nars long-term has low requirements compared to the build farm because we don=E2=80=99t need to trust it for anything other than storage. If that makes things easier (and financially viable), a VPS is good enough. Thoughts? Ludo=E2=80=99.