From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:bcc0::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id OASNIooLX2C9EAAAgWs5BA (envelope-from ) for ; Sat, 27 Mar 2021 11:40:10 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id uEBeHIoLX2AwRAAAbx9fmQ (envelope-from ) for ; Sat, 27 Mar 2021 10:40:10 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 62E211ACF0 for ; Sat, 27 Mar 2021 11:40:09 +0100 (CET) Received: from localhost ([::1]:59800 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lQ6MS-0004zk-CB for larch@yhetil.org; Sat, 27 Mar 2021 06:40:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54940) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lQ6MM-0004zN-52 for guix-patches@gnu.org; Sat, 27 Mar 2021 06:40:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:60144) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lQ6ML-0004QV-Tb for guix-patches@gnu.org; Sat, 27 Mar 2021 06:40:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lQ6ML-0002R6-Rv for guix-patches@gnu.org; Sat, 27 Mar 2021 06:40:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#47336] Disarchive as a fallback for downloads Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 27 Mar 2021 10:40:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47336 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: Timothy Sample Cc: Mathieu Othacehe , 47336@debbugs.gnu.org, zimoun Received: via spool by 47336-submit@debbugs.gnu.org id=B47336.16168415729325 (code B ref 47336); Sat, 27 Mar 2021 10:40:01 +0000 Received: (at 47336) by debbugs.gnu.org; 27 Mar 2021 10:39:32 +0000 Received: from localhost ([127.0.0.1]:43457 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lQ6Ls-0002QL-Ay for submit@debbugs.gnu.org; Sat, 27 Mar 2021 06:39:32 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33608) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lQ6Lp-0002Q7-Hf for 47336@debbugs.gnu.org; Sat, 27 Mar 2021 06:39:30 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:34494) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lQ6Lj-00046H-Lv; Sat, 27 Mar 2021 06:39:23 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41636 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lQ6Lh-0003Y1-1l; Sat, 27 Mar 2021 06:39:22 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87eeg6o50b.fsf@ngyro.com> <86sg4mnreu.fsf@gmail.com> <87sg4mt00c.fsf@ngyro.com> Date: Sat, 27 Mar 2021 11:39:18 +0100 In-Reply-To: <87sg4mt00c.fsf@ngyro.com> (Timothy Sample's message of "Tue, 23 Mar 2021 10:31:15 -0400") Message-ID: <87eeg0284p.fsf_-_@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1616841609; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=6Ri/wl8KX+MfQAPsRcU6FBXIG03rbkrib+LluLByFck=; b=YBBvbEZWxlsMpNWQG7vmfnEHpv9ZLtTjSMEjvC+/B6fuMuEjYSoNB5EphRG1A73VvZOL7c Nv59k2FzEG5k7M/1FQCD2sdAabQTgoDB7/67hRpKuO/buwoQPTuvMUh+zBjTki45Skr2Da gK0eFMWKVFoUUQ7InC7/ivCTtu3TDN1sQB62rPpFiIJo3anw0MbGkat9xrnz98J2mO0LPN UFMzmSgYV/67HStONKkSgR0NMC8O4ppDsLFBMeGDgwuPtMqddrOyZ0fZCD57HkVtdYXXqI Ua4ZRkybJzCsPWGm7YB0chgvzv6ouU3I9gYz2q249fpPPtZvucfUlOg/1OZw9g== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1616841609; a=rsa-sha256; cv=none; b=qlUu977k6M5jSIVZhY1+nFLV7KgiRc6WrkYSA07IyIY41GHaMaRXnYPCM3TxxN+Kk3wAyf 3wxgZ6sECLQFAf4KioeCodElWG3WVtT7LOqUEDPFrKTdwwwt1WWaSc0gYuqxph77Av6qiU TDtYMGhBcuemZePstxxxCB/Nlif16nw+Ur54mTQiN7C3cBWSQjEryyHfj0eHKrqmXNULE/ 5Ige1Hpf+ZrFhWLZoEZdXXkT83M6zILDQPXjCaxk5gzwBKKxFyVjScZG7+Yout4Jxv7iBw xL11iCR/go2XoMF19g793yHOFV0t3nCgQeRJBvdm8R1q8RRa/+ybtVOXaQcNYA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: 0.08 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 62E211ACF0 X-Spam-Score: 0.08 X-Migadu-Scanner: scn0.migadu.com X-TUID: SUNhHttO4duy Hi! Timothy Sample skribis: > With =E2=80=9Cgzip -9=E2=80=9D, the average metadata file is 6.8KiB. It= =E2=80=99s pretty > manageable. There=E2=80=99s room for improvement on the Disarchive side,= too. > It still stores some redundant information. Uncompressed, it=E2=80=99s m= ore > like 112KiB per file. This is still pretty okay, really. It means we > might hit tens of GiB over a couple years. (It would take just over > 100GiB to store a million uncompressed metadata files.) The compression > ratio is what drove me to skip Git for now. If needed, the sexp serialization could still be made more compact: using =E2=80=98write=E2=80=99 instead of =E2=80=98pretty-print=E2=80=99, sh= ortening field names (but that=E2=80=99d be incompatible). We could also use CBOR or canonical sexp serialization, though maybe gzipped sexps are more compact than what we could achieve? Anyway, these are surface syntax optimizations that can always be made at a later point in time when we feel a need for them. Ludo=E2=80=99.