From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id ALvuI3mupGWiZQEA62LTzQ:P1 (envelope-from ) for ; Mon, 15 Jan 2024 05:03:05 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id ALvuI3mupGWiZQEA62LTzQ (envelope-from ) for ; Mon, 15 Jan 2024 05:03:05 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ngvlM3Jr; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1705291385; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=QOdjSq+TI+kZeBSopSMiLYN7p1MvRMYg/dVrEuRl+Zs=; b=sNKUH2XPTSOJgeWFIxTnPQCWr9KZZjRVLVunmmV3kmiAaRQrxxiRZUwN6H47LXhhMZRFtE MeX3vHWbOYle3QzivH5QEWB2xcbIDCJBmJPAGq2kLpIzBWmrU8t4O+lQUCu9i9zEvRxlDe ca65WpFx3IUSfZfAzv8+IGHO7aU/+seZCZIVfhwHTwOalpoNCLKXWpP8BsGe1HAJvyIGs0 zeZsLMqP9dK2GZ9EE7G9ZKq5MCRKpwmNasW6/QPc5sHeR5GgM9oN5nbSF0E3tqDqQ1X7Z/ nxzQ9m9Sad+owCq6e8DTnmYKhzSfP+19DIobZ7cVnuTkTr64Nb+uOfAcSRn1cw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=ngvlM3Jr; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=key1; d=yhetil.org; t=1705291385; a=rsa-sha256; cv=none; b=Fp9FsSo/VJiR6W0Aoln7aArDSiJhWi9rry4LDOzXcKxCo8TOquaruTecIASkkMBDzgzFAQ 5WY+SY+BOW4h5x3iCMTcXD487Gznl8GM9d6iLShUxltOIXxkM99AZgO84hfseuS96EFPN+ 18xAxp1kyS9aGla3ZdAemdBy4l58IHUgcd+jwKmv86s53h8BPQ+LqYAHnp7AILF3679qSD Nr4bSL0cybnR6Elv2bKZwqIFszylq7dnFyr5TRwEWN3pGeD2MJOhLWMi3V9Kv8YPU6eDQO 6mtJKp+aGnhnkq3nfG24n+aFrLkNP9U31Ylr13wLqGHUMFMS63jDpqz89u0sjw== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 643C447D69 for ; Mon, 15 Jan 2024 05:03:05 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rPEAv-00034V-Fv; Sun, 14 Jan 2024 23:02:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rPEAt-00032b-QU for guix-devel@gnu.org; Sun, 14 Jan 2024 23:02:11 -0500 Received: from mail-qt1-x834.google.com ([2607:f8b0:4864:20::834]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rPEAq-0002sz-E8; Sun, 14 Jan 2024 23:02:11 -0500 Received: by mail-qt1-x834.google.com with SMTP id d75a77b69052e-429bd0f2768so30835771cf.0; Sun, 14 Jan 2024 20:02:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1705291326; x=1705896126; darn=gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QOdjSq+TI+kZeBSopSMiLYN7p1MvRMYg/dVrEuRl+Zs=; b=ngvlM3Jrm6VpGhiGl7y1J9UyE2ZVbskuJ5VJRtUBBQ2BjslsChSPFk20/T1pjRnRfi O8YjKfdPthvsAnj4n+HC7zU2n/hSR1TetLpsrzDJQhwbhTwDRyk54mabu9hDiAaKchFN SdcdZQGFQxEptzC426eD0yXxQissd7pXP5nRiG4bED/Rbh/Ykb0HrcpFSi+5OBc1C6SW ab19OZJnOX0Lwll6N7YzV7DwfcIgWVAesItA3hkmgbgAP7mX0ABFTrbwIItNa5zl1VzM A5Vzd31zmoLdmoq4af0i/G1+9nVEAETo6YfoFjZp1j2grkn9C38HRbDwtOAbJNeG41zQ Z9Rg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705291326; x=1705896126; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=QOdjSq+TI+kZeBSopSMiLYN7p1MvRMYg/dVrEuRl+Zs=; b=DnZCjxRonM53dBHDcT6f6pJ9s7UAvj8mp6FfOo3wKvmugi4rfNAzHbzXlfy3onYeue zqds5GUM7ow912YDesauDY31Hah14P2UpSPHpm57aDsI3HzmXP3fvQsIqaluhdW+Ok6S 8MJ9lBY3T6r/FPHw4DZr5IlCZOpFvOztLo2dG8SYr3g1rIYR2/iacla6pyrx4H1gKsnG obSh4fW5EvOa4diRtMlKDzYC+ZN5BlX8V2MNT4b1sYwtYcbNZbycdEv5Op3CZKxpUEk5 XeyjISDsKkBbXOsa7/n80LJg7AuoJ8SaAOzVjJ6qjY3oq1vk4NBF6gtjQLjl1Ey7ZEuJ gwwQ== X-Gm-Message-State: AOJu0YwAhSzMnytwH2Z5Ed7gCr+JbfPPbvc1UG+5r3mB1NereYv2o87Z 1egiGK9yLbugaxDtB4Dhdqzwo86mcg8= X-Google-Smtp-Source: AGHT+IGltS5f0HmggNRWzHZF5nFmadQMxoqbGzV/YBo5Fc9+A/ps9sh+JcwZ63NybIJojJFI9Gp8SA== X-Received: by 2002:ac8:57ce:0:b0:429:b8cf:587f with SMTP id w14-20020ac857ce000000b00429b8cf587fmr7611245qta.7.1705291325786; Sun, 14 Jan 2024 20:02:05 -0800 (PST) Received: from hurd (dsl-10-136-68.b2b2c.ca. [72.10.136.68]) by smtp.gmail.com with ESMTPSA id fb18-20020a05622a481200b004281ce041f6sm3601361qtb.21.2024.01.14.20.02.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 Jan 2024 20:02:05 -0800 (PST) From: Maxim Cournoyer To: Simon Tournier Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , Ricardo Wurmus , guix-devel Subject: Re: Building and caching old Guix derivations for a faster time machine In-Reply-To: <87jzoeyiwi.fsf@gmail.com> (Simon Tournier's message of "Fri, 12 Jan 2024 10:56:29 +0100") References: <87o7g29c94.fsf@elephly.net> <878r6xu2pe.fsf@gnu.org> <877cmfd6tm.fsf@gmail.com> <87h6ldiqyh.fsf@gnu.org> <87plzsr019.fsf@gmail.com> <87zfyv4bgc.fsf@gmail.com> <87jzoeyiwi.fsf@gmail.com> Date: Sun, 14 Jan 2024 23:02:04 -0500 Message-ID: <87v87vqm6b.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::834; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qt1-x834.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -8.50 X-Migadu-Queue-Id: 643C447D69 X-Spam-Score: -8.50 X-Migadu-Scanner: mx11.migadu.com X-TUID: m63CHzVzlhqU Hi Simon, Simon Tournier writes: > Hi Maxim, > > On Thu, 30 Nov 2023 at 08:28, Maxim Cournoyer = wrote: > >> I'd like to have a single archive type as well in the future, but I'd >> settle on Zstd, not lzip, because it's faster to compress and >> decompress, and its compression ratio is not that different when using >> its highest level (19). > > When running an inferior (past revision), some past Guile code as it was > in this past revision is launched. Hum, I have never checked: the > substitution mechanism depends on present revision code (Guile and > daemon) or on past revision? > > Other said, what are the requirements for the backward compatibility? > Being able to run past Guix from a recent Guix, somehow. We're only impacting the future, not the past, I think. The inferior mechanism still relies on the same daemon, as far as I know, and the currently available gzipped nars would remain available according to their current retention policy (6 months when unused). >>> 1. Keep for as longer as we can all the requirements for running Guix >>> itself, e.g., =E2=80=9Cguix time-machine=E2=80=9D. Keep all the depen= dencies and all >>> the outputs of derivations. At least, for all the ones the build farms >>> are already building. >>> >>> 2. Keep for 3-5 years all the outputs for specific Guix revision, as >>> v1.0, v1.1, v1.2, v1.3, v1.4. And some few others. >> >> That'd be nice, but not presently doable as we can't fine tune retention >> for a particular 'derivation' and its inputs in the Cuirass >> configuration, unless I've missed it. > > That=E2=80=99s an implementation detail, a bug or a feature request, pick= the > one you prefer. ;-) I'd say it's a feature request :-). > We could imagine various paths for these next steps, IMHO. For > instance, we could move these outputs to some specific stores > independent of the current ones (ci.guix and bordeaux.guix). For > instance, we could have =E2=80=9Ccold=E2=80=9D storage with some cooking = bakery for > making hot again, instead of keeping all hot. For instance, we could > imagine etc. :-) > > Well, I do not have think much and I just speak loud: Cuirass (and Build > Coordinator) are the builders, and I would not rely on them for some NAR > =E2=80=9Carchiving=E2=80=9C, instead maybe =E2=80=9Cwe=E2=80=9D could put= some love into the tool > nar-herder. Somehow, extract specific NAR that the project would like > to keep longer than the unpredictable current mechanism. It seems the nar-herder would perhaps be well suited for this, if someone is inclined to implement it, given it keeps each nars in a database, which should make it fast to query for all the 'guix' packages substitutes. Perhaps it even has (or could have) hooks when registering a new nars which could define what is done to it (send to another server). Otherwise good old 'find' could be used to rsync the 'guix' named nars and their .narinfo metadata files to a different location, but that'd probably be less efficient (IO-intensive) on the huge multi-terabytes collection of nars we carry. --=20 Thanks, Maxim