From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id +MKSG5J6bmXTWgEA9RJhRA:P1 (envelope-from ) for ; Tue, 05 Dec 2023 02:19:14 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id +MKSG5J6bmXTWgEA9RJhRA (envelope-from ) for ; Tue, 05 Dec 2023 02:19:14 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4EBEC5D944 for ; Tue, 5 Dec 2023 02:19:14 +0100 (CET) Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=kK8aETXS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1701739154; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=JjF/h4xrwBV5tGZa6K/z65J/LSKlpIZjLTpa2rktBMM=; b=Ju8c9gnBxm8hGid4RFS24iIino5hxrbB2iFhy9yXwR80YgoHgiv4aj4Iw0sCUe+7JRl8hM vL/Bj86nWEFLsujuU0fMdHUgz1kvDMx38uTB9KAlWNyFdoeCfZX9Vrfz3lF79G7Jl3CMOp SjE9fFdZF7uUGWw2qfcixFwItxS0k92lsGGp2By0E8hF1mdUhZ5EVOZt8kX/kQN65IAM+p 7t3mDbcZMIWytr51+z7vHKmuKf2EePHPOtKkq2MWMGAsS6EiQuPdimngXNU/FZ+BFaP+C6 rpe4FpZojKi9g0ThwGsC3DKggLDoDchkMOQA+hsMw729sLkCeHg2GLt9Mx1vfQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1701739154; a=rsa-sha256; cv=none; b=BByPSxj3BfUDrnDrK25sX3k5ZgbOf+JdMn+4AihUAvlpKG1qFqYFuaTywr3WWrw71fTyIi zLKmcIKxoMoc3K8ge6HYtphQ1JPpMAU1JSMiAa4Bi+Gj568jlQ/qSeq3ocEs4ao8+DXqMT 9tQs2kD8W4Pc4KoSR0gMEjRqq+ta+KpVxZNFY8ievPwR04+y6CtxuzefwPMm8P+uYKXzLh qTa71zBs0NrewqcEqcIhqOBVOgN0+jm7SCT7cChR7ISozOnPr6EI7DfBjHwaVfLvS12y/R 3CqHR32Qz/Coqkfxvk3EhyYi4cspUw1gL8ZB/JyVNzx6rl9/1b+bT1EOWNJYEA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=kK8aETXS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rAK52-0003V3-Rv; Mon, 04 Dec 2023 20:18:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rAK51-0003Uv-Ri for guix-devel@gnu.org; Mon, 04 Dec 2023 20:18:31 -0500 Received: from mail-oi1-x231.google.com ([2607:f8b0:4864:20::231]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rAK50-0001Tk-8O; Mon, 04 Dec 2023 20:18:31 -0500 Received: by mail-oi1-x231.google.com with SMTP id 5614622812f47-3b8b1a50889so749635b6e.3; Mon, 04 Dec 2023 17:18:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701739107; x=1702343907; darn=gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JjF/h4xrwBV5tGZa6K/z65J/LSKlpIZjLTpa2rktBMM=; b=kK8aETXSmwRk5PnhAnWI2z5xSwuGntic+Lfxytmd9n158IO6nKuO6VvWKMj/P6xy+1 4pzc2J4arDj5Qy+toteTx97gj1ksFAerhOrg7+1mvmC5LqB5bjN5W1SGfkxS/VGgEIMJ 5xs9qkfa+9O2HuxwcNW6Ejudp7mxvybM/MFsxHR9h8zV3EI1uPDX5ed5Rzif+iakJxhy +EFj3BUbghK4IbegsMV00EjxOlSOfdsgGKfZX6MWWDeecgR1typpaxin2fWnY6pMlJcX qce//45LWhQuOEPg3+OTCfgTEv3UoM5TfEcaFyVcK15l0mo0Bwpzxv2LikXAxeZRzSkI +L8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701739107; x=1702343907; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=JjF/h4xrwBV5tGZa6K/z65J/LSKlpIZjLTpa2rktBMM=; b=KWQ6cT/YhOBUT6QMSmPO4BCIm2mJYKEvxTUoWuY3uBpLhSqwAxTDJELvXTfPQmBssZ v/hFTV5Mqx7jgHQxMMqU3XdOqrBAf9cb7wH1hYFNXleAazR3iqQuwfYEiJ6pSNJZNWCt q3IxWuk4bCnkucenuTSAvURP4hpN+0QvRHE2tFUXj1ZtF3dMo53DzBPuKjKx5DoJsTlN 7I44ik77pmDDG3mY7YkRLPTm14+IA8uCzkG2eK2SSzWjU0MlwXeKW+kzo7+vkwlbGkUU F/kziC9qc0dCLDZ5Ze26oEljZioQ6YPr4hI3vakEolDuA+twjsFbQWdPEWzrtBwQ1sPi ULjg== X-Gm-Message-State: AOJu0YwHmRSrEH+0ZXmU7ePWsejGzeBukKM83i7LkKiAN0M4ZP500jhI 34gAkCK1pgx0VBRFEj/X6oCD3Fi8mF84fA== X-Google-Smtp-Source: AGHT+IE/4MUgC7qfjUTBWM77FdSrXoPUGeTzxYizSDZVOTjO+/mctsPi2yh/3Kw+hF1xBUeI5gmv7Q== X-Received: by 2002:a05:6808:1391:b0:3a9:b9eb:998e with SMTP id c17-20020a056808139100b003a9b9eb998emr2842524oiw.51.1701739107641; Mon, 04 Dec 2023 17:18:27 -0800 (PST) Received: from hurd (dsl-141-198.b2b2c.ca. [66.158.141.198]) by smtp.gmail.com with ESMTPSA id bz17-20020a05622a1e9100b0042542160fd0sm2856497qtb.20.2023.12.04.17.18.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 17:18:26 -0800 (PST) From: Maxim Cournoyer To: Guillaume Le Vaillant Cc: Simon Tournier , Ludovic =?utf-8?Q?Court?= =?utf-8?Q?=C3=A8s?= , Ricardo Wurmus , guix-devel@gnu.org Subject: Re: Building and caching old Guix derivations for a faster time machine In-Reply-To: <877clztj13.fsf@kitej> (Guillaume Le Vaillant's message of "Thu, 30 Nov 2023 14:05:38 +0000") References: <87o7g29c94.fsf@elephly.net> <878r6xu2pe.fsf@gnu.org> <877cmfd6tm.fsf@gmail.com> <87h6ldiqyh.fsf@gnu.org> <87plzsr019.fsf@gmail.com> <87zfyv4bgc.fsf@gmail.com> <877clztj13.fsf@kitej> Date: Mon, 04 Dec 2023 20:18:25 -0500 Message-ID: <87wmtto3a6.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::231; envelope-from=maxim.cournoyer@gmail.com; helo=mail-oi1-x231.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: mx11.migadu.com X-Migadu-Spam-Score: -5.39 X-Spam-Score: -5.39 X-Migadu-Queue-Id: 4EBEC5D944 X-TUID: J+8o4wBE891a Hi Guillaume, Guillaume Le Vaillant writes: > Maxim Cournoyer skribis: > >> Hi Simon, >> >> Simon Tournier writes: >> >>> Hi, >>> >>> On mer., 22 nov. 2023 at 19:27, Ludovic Court=C3=A8s wro= te: >>> >>>> For long-term storage though, we could choose to keep lzip only (becau= se >>>> it compresses better). Not something we can really do with the current >>>> =E2=80=98guix publish=E2=80=99 setup though. >>> >>> It looks good to me. For me, the priority list looks like: >> >> I'd like to have a single archive type as well in the future, but I'd >> settle on Zstd, not lzip, because it's faster to compress and >> decompress, and its compression ratio is not that different when using >> its highest level (19). > > Last time I checked, zstd with max compression (zstd --ultra -22) was > a little slower and had a little lower compression ratio than lzip with > max compression (lzip -9). > Zstd is however much faster for decompression. I think when we talk about performance of NARs, we mean it in the context of a Guix user installing them (decompressing) more than in the context of the CI producing them, so zstd beats lzip here. > Another thing that could be useful to consider is that lzip was designed > for long term storage, so it has some redundancy allowing fixing/recoveri= ng > a corrupt archive (e.g. using lziprecover) if there has been some bit > rot in the hardware storing the file. > Whereas as far as I know zstd will just tell you "error: bad checksum" > and will have no way to fix the archive. That's an interesting aspect of lzip, but in this age of CRC-check file systems like Btrfs, we have other means on ensuring data integrity (and recovery, assuming we have backups available). I'm still of the opinion that carrying a single set of zstd-only NARs makes the most sense in the long run. --=20 Thanks, Maxim