From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id cMv4JVo9WGW+ZwAA9RJhRA:P1 (envelope-from ) for ; Sat, 18 Nov 2023 05:28:10 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id cMv4JVo9WGW+ZwAA9RJhRA (envelope-from ) for ; Sat, 18 Nov 2023 05:28:10 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4F16514DCA for ; Sat, 18 Nov 2023 05:28:10 +0100 (CET) Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Gfk3AYEI; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1700281690; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=bV81/KYi98gIb9/14iCdEwfmAEc9GoNxkNnD++B0GMA=; b=VNUPfxi/ysPSV1ywYRTMeuQULQ/kl908CC7RP0HgYjxNZbqAw2gpbn7+g2+11ATMSUU8Tv 7KjMggATGdOwnnzt0TwhrofmX4LoVHczC3BYVCpnH0JGw48c4GYAYTKZuZ3LKCoyW/9pWm Zpy2Ge0fEsEAhUUWlCoZGCVwkAeh27/XV28OCdtFGOK94fFMsCfKMEuLouJlRjzuwv5mRH X2HStBfWwan74FMfxhnNVSzm7ZzjiAbvID8pwq6OQPX3thi19oY5XEYuBGoA/82YI40QCf WD0z8hfXpMCKT3YDV3Mt+dexfBrkWb3OWsfWhg6k2nCQCmEQrDdrFirzP3OfMw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Gfk3AYEI; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=key1; d=yhetil.org; t=1700281690; a=rsa-sha256; cv=none; b=CpeGqHsSVYA0szfrT3HZtGsTU4k2NFzjeV3CEkJcbAHKbcrDKJSlN6MoGftBAty5AksuLJ c5nqSkKee8GIKxmfZI/hg3ISLgaZ5hZnxs2v+UefC8cbQABAnZ4UA8P7SYAoVnDcRlAY6b NdnCXii8D5BZoA7Ok2U4v+Ze+t1OdvkSGtHPREKJB7sPX4IgJSgVqItlepntOCm3Yd2xMd m/tmuXLP/Mz7QAb8oIe/qvH55C/lMusEtFZFdhLL5irhyxY5Y55tnxvX6d1xjYuvqHgWKP oFeAMM/OPXcIxfc0pVrTBihOf0HkGkrW9BCM5BX9M7mJf+DnTFT8TH+o6ZwECg== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1r4Cvl-0003LO-36; Fri, 17 Nov 2023 23:27:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1r4Cvj-0003LF-Tk for guix-devel@gnu.org; Fri, 17 Nov 2023 23:27:39 -0500 Received: from mail-qv1-xf2c.google.com ([2607:f8b0:4864:20::f2c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1r4Cvi-0008Pb-AP; Fri, 17 Nov 2023 23:27:39 -0500 Received: by mail-qv1-xf2c.google.com with SMTP id 6a1803df08f44-66d76904928so964496d6.2; Fri, 17 Nov 2023 20:27:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700281656; x=1700886456; darn=gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bV81/KYi98gIb9/14iCdEwfmAEc9GoNxkNnD++B0GMA=; b=Gfk3AYEIvXExDHL81LjrxrXZEtn0uaTQGHWV7Q0gS/CJyz9v+tPlvDuWNJX9kNXQq/ 62OzTnktFbsjQ7VN51o9qk1PwFdyPoILS7dda67b7fh7MH+u32raFdy6qLw43mFtMzBK nDGiQjrN6RBEIbTN5E1hNGTElX7yXxm9j7knaZVaSPdbbF5etlk/4nuMdhnSe77dXP+I YsC4qnpYDa9a1Eciqf8bzVyBsnTvt888VKlrBLqAe4V5FlTuk3PmkQD8NcUX8ki81zGe KdCXUGzL/PWzeFnpI3cIxm+xaptW0Q/ya+NPSuwoQHY9eNjr8Wn4aKAMytcxxiY7XV1G 9+7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700281656; x=1700886456; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=bV81/KYi98gIb9/14iCdEwfmAEc9GoNxkNnD++B0GMA=; b=LhMPCRFodW5mbCXFONAuex6lt0GpbrCaoNBZZe56O9PgNe9UPs5g5rNWahH+wQuDAR z/oT12reeSL1oN32cVG4sE0DRUDv9sVUOy53AtuFZrBJv9dkOahvE5dZA1VMb6vliKnx 1gTOwWSBXmVYU/U7fiY8GQc+lM6JGZG5kJ9GJcjh4NkQ1IhGuAvQ3EAHOkKF1FYU4p2R tRPZ2bQel2R006gly4ursvFb5VFr8EYhwxBCNe02/It7Tjs5GyhUKgcnKs2wbxABYHSF JpAEMksmSfJz42LLbwdbbDR2DoRWA5RzjcraT0561IF/enQxPjHylwii0wcq9hSOj0iy Bcvw== X-Gm-Message-State: AOJu0YwgjUcFzDckGleO6xLtAsda02+WnsYYXcs5Xmngx9LkmGcx95Tw VjoxYm/Ob8k4jd9NgJci//XDfDNYhzY= X-Google-Smtp-Source: AGHT+IFNJb0RHk/hdOSc1aJeXBDNgQnPiP25sX6qTWMwE5SL68NdGpSMnRdnqriXHXlXT09RgrDAUA== X-Received: by 2002:a05:6214:d6a:b0:66d:6439:4383 with SMTP id 10-20020a0562140d6a00b0066d64394383mr1360172qvs.58.1700281655716; Fri, 17 Nov 2023 20:27:35 -0800 (PST) Received: from hurd (dsl-154-55.b2b2c.ca. [66.158.154.55]) by smtp.gmail.com with ESMTPSA id m9-20020a0ce6e9000000b00677a12f11bcsm1134096qvn.24.2023.11.17.20.27.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Nov 2023 20:27:35 -0800 (PST) From: Maxim Cournoyer To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: Ricardo Wurmus , guix-devel Subject: Re: Building and caching old Guix derivations for a faster time machine In-Reply-To: <878r6xu2pe.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Thu, 16 Nov 2023 16:39:57 +0100") References: <87o7g29c94.fsf@elephly.net> <878r6xu2pe.fsf@gnu.org> Date: Fri, 17 Nov 2023 23:27:33 -0500 Message-ID: <877cmfd6tm.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::f2c; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qv1-xf2c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -6.41 X-Spam-Score: -6.41 X-Migadu-Queue-Id: 4F16514DCA X-Migadu-Scanner: mx10.migadu.com X-TUID: j5vwECCw9kH/ Hi, Ludovic Court=C3=A8s writes: > Hi, > > Ricardo Wurmus skribis: > >> to me the biggest downside of using =E2=80=9Cguix time-machine=E2=80=9D = is that it has >> to do a lot of boring work before the interesting work begins. The >> boring work includes building Guix derivations for the given channels, >> most of which have long been collected as garbage on ci.guix.gnu.org. >> >> It would be helpful, I think, to more aggressively cache these >> derivations and their outputs, and to go back in time and build the >> derivatinons for past revisions of Guix. I would expect there to be a >> lot of overlap in the produced files, so perhaps it won=E2=80=99t cost a= ll that >> much in terms of storage. >> >> What do you think? > > I agree. The =E2=80=98guix publish=E2=80=99 TTL=C2=B9 at ci.guix was inc= reased to 180 days > following in 2021. That=E2=80=99s st= ill not > that much and these days and right now we have 84=C2=A0TiB free at ci.gui= x. > > I guess we can afford increasing the TTL, probably starting with, say, > 300 days, and monitoring disk usage. > > WDYT? While the 84 TiB we have at our disposal is indeed lot, I'd rather we keep the TTL at 180 days, to keep things more manageable for backup/sync purposes. Our current TTL currently yields 7 TiB of compressed NARs, which fits nicely into the hydra-guix-129 10 TiB slice available for local/simple redundancy (it's still on my TODO, missing the copy bit). I've been meaning to document an easy mirroring setup for that /var/cache/guix/publish directory, and having 14 TiB instead of 7 TiB there would hurt such setups. Perhaps a compromise we could do is drop yet another compression format? We carry both Zstd and LZMA for Berlin, which I see little value in; if we carried only ZSTD archives we could probably continue having < 10 TiB of NARs for a TTL of 360 days (although having only 3.5 TiB of NARs to sync around for mirrors would be great too!). What do you think? --=20 Thanks, Maxim