From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id aWG/KfrlZWB+5AAAgWs5BA (envelope-from ) for ; Thu, 01 Apr 2021 17:25:46 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id IBV2IvrlZWAPBgAAB5/wlQ (envelope-from ) for ; Thu, 01 Apr 2021 15:25:46 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E1FBF1EE33 for ; Thu, 1 Apr 2021 17:25:45 +0200 (CEST) Received: from localhost ([::1]:51164 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lRzCZ-0005Mq-Rv for larch@yhetil.org; Thu, 01 Apr 2021 11:25:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49492) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRurz-00071O-QK for guix-science@gnu.org; Thu, 01 Apr 2021 06:48:11 -0400 Received: from flashner.co.il ([178.62.234.194]:46970) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lRuq3-0004Fw-Ay for guix-science@gnu.org; Thu, 01 Apr 2021 06:48:11 -0400 Received: from localhost (unknown [31.210.177.71]) by flashner.co.il (Postfix) with ESMTPSA id E32EA402D4; Thu, 1 Apr 2021 10:46:09 +0000 (UTC) Date: Thu, 1 Apr 2021 13:45:37 +0300 From: Efraim Flashner To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-science@gnu.org Subject: Re: GC strategy on clusters Message-ID: References: <87v9969uml.fsf@inria.fr> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="8wbxJonQkwZVtmsy" Content-Disposition: inline In-Reply-To: <87v9969uml.fsf@inria.fr> X-PGP-Key-ID: 0x41AAE7DCCA3D8351 X-PGP-Key: https://flashner.co.il/~efraim/efraim_flashner.asc X-PGP-Fingerprint: A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Received-SPF: pass client-ip=178.62.234.194; envelope-from=efraim@flashner.co.il; helo=flashner.co.il X-Mailman-Approved-At: Thu, 01 Apr 2021 11:25:34 -0400 X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-science-bounces+larch=yhetil.org@gnu.org Sender: "Guix-Science" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1617290746; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=oWIgWac5RkXuWSfStqz9s6DvfiIwjRPmjgTAPecihEc=; b=jRbvmsg8CR74Cu3xeFSSCmxytbQWFceIo7AIDJa42gkdN5ljjdOYrYjf/krZdYFT2Cj6j4 IOmtyKAli3Jm6W8D6GZ4cGo/5zNiZXQ0W2LdMRMb/NftxnW2rcZcUeQy/aIcggkLQFiVdI rO/U7zW78HonNOYtMr4Hfhp1damvC3cF6bpgupPrYU5JFT/obWFXJAEqdRdOAucwcuxuue 1tWZCOnMNmz8DCW/ra8NJggs07hXa3NSELhE+pVHjFi9BsuhNTuzAuswluzTyj7+duv6Bw DKyw87JX7ZL5xGKy4bARMczmh+MOzOUy/WOQbpjIv6BfEJY4VbamqNXCDXf0Mg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1617290746; a=rsa-sha256; cv=none; b=qwhNX3PJYTLzBSw3wemkinfhNh9GGq+J+TT7yTYTAK7jWPifGQhIGIOzpJXbdpKjCxRoum OyEihumc/J+PtLNqPRBHzTGHxEHLru69tXY9RyRWroMcl0Gby5jfUjzuE+iFBoEzeYEZ/J eC7dXuDawY0xoaOpQ3FknP15kYdtc2pTH53YT1GYBIi8lHE0HVaaPGcFDJhWPPXlYPx4Pw zRXMCskmbBN5MZ3UN6de19Fdi2RvGn+oPAh7ohRkVHA5cJp2Mlb6VK4Tazz89dr6ryMKMf 6aKLGBW3QJ29y53xSW4FhpBcuaQMPTWeHX+83KxRwYVsuLAfjK8RcCjNHqVr/g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Spam-Score: -4.03 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Queue-Id: E1FBF1EE33 X-Spam-Score: -4.03 X-Migadu-Scanner: scn0.migadu.com X-TUID: xOXhKTEK5hj+ --8wbxJonQkwZVtmsy Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Apr 01, 2021 at 12:17:22PM +0200, Ludovic Court=C3=A8s wrote: > Hi there! >=20 > Recently the Guix head node of our cluster at Inria was getting short on > disk space, despite running =E2=80=98guix gc -F20G=E2=80=99 (or similar) = twice a day. >=20 > Turns out that some users had accumulated many profile generations and > that was getting in the way. So we kindly asked them to run: >=20 > guix package --delete-generations=3D4m >=20 > or similar, which was enough to free more space. I feel like 4-6 months should be plenty for anything active. Even if it were run automatically for them it wouldn't remove the last generation. > We=E2=80=99re now considering setting up automatic user notification by e= mail, > as is commonly done for disk quotas, asking them to remove old > generations. That way, users remain in control and choose what GC roots > or generations they want to remove. >=20 > How do people on this list deal with that? I like the idea of asking people to remove old generations. It's not something that we've come up against yet. It doesn't feel that different than reminding them that their $HOME is for code and smaller things and the storage space is for their large data collections. > Longer term, I think Guix should automatically delete old generations > and instead store the channels + manifest to reproduce them, when > possible. >=20 This seems to help a bit less when we run into issues about dates being wrong on SSL tests, or when sources go missing. How much storage and people are you working with? Our initial multiuser system has 188GB for /gnu and I think 30-40 people and some people have profiles going back almost 3 years. Not many people have multiple profiles and the experiments we tried with shared profiles in /usr/local/guix-profiles don't see a lot of use or get updated frequently. I guess I'm not really sure if its a technology problem or a people problem. Figuring out if someone is the only one pulling in a copy of glibc-2.25 is doable but how many copies of diffoscope is too many? On a practical note, 'guix package --list-profiles' as root currently lists everyone's profiles so it can be easier to see who has older profiles hanging around. --=20 Efraim Flashner =D7=90=D7=A4=D7=A8=D7=99=D7=9D = =D7=A4=D7=9C=D7=A9=D7=A0=D7=A8 GPG key =3D A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted --8wbxJonQkwZVtmsy Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmBlpE8ACgkQQarn3Mo9 g1HHyRAAuYjmwQ1cpCosAMuksIzT5+egLOO2b/WKQapORcZIQcAdNyJAn4bMlNA1 mE73RiuCm3dGK6rGgZbiYHihPGOard2kS8xYS+kYr9r9oeAE7QYeoUj5OgtCgdbb /Oc19rb8EkbUXy1QGmzzglIkTBrgTDPqJD1k51aUluwATREFIgQW/E0GUTUXL/Hm kkiYwbb9o1SuUPQzokq2IIn/QEnnr25BQiDD/m52jatDktpBE24MwiOWzc9i/JRy 84dfo3xoQE0a7n+2W7pfEkXkwMf3PJtpLICeqVrAS+yCGE35KF6wYUVoCzQvfVl0 3tqvgnV+oHNFcelAgiVIdF1Dep6q0kCrMoz5AOfEDX1CX1ReUd8IhbjKataGeqqq rFQ4ebuecESOy5fIkDiFRAXBRVCVXTiWgdDn4RYR4KfcjjJJ37sDyseJxhoerGXR aAcmeAZ/lMpxTgi9V5fjilU37/wn2M+BrIxCsGJG7kWXZb+EQY+KTNweT/+Mj5Nn 58bbcvjH/VRjL5VIhC7vY9+xFye0c2LOIlzwrifuo0AV0nLr0M9CK4vRsRgi8T/7 RjuH+SXlvn1lQWnyPv5HfYeHKyN16VIUVPSHVg4/qefby0zZPlzD77L7l74A7KBL 4urQzvnZ64NW+n3GHkx5y8MCdYTtzoFDeDUbnMJEbhZfdNatmGs= =l2Fp -----END PGP SIGNATURE----- --8wbxJonQkwZVtmsy--