From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 4LOxBW6+ZWCbkAAAgWs5BA (envelope-from ) for ; Thu, 01 Apr 2021 14:37:02 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 0NwwO22+ZWDUDgAAB5/wlQ (envelope-from ) for ; Thu, 01 Apr 2021 12:37:01 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 44E5317A49 for ; Thu, 1 Apr 2021 14:37:01 +0200 (CEST) Received: from localhost ([::1]:45946 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lRwZI-00060p-G3 for larch@yhetil.org; Thu, 01 Apr 2021 08:37:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52528) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRwZE-0005z2-Eh for guix-science@gnu.org; Thu, 01 Apr 2021 08:36:56 -0400 Received: from mail3-relais-sop.national.inria.fr ([192.134.164.104]:4309) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lRwZC-0008Fd-2n for guix-science@gnu.org; Thu, 01 Apr 2021 08:36:55 -0400 IronPort-HdrOrdr: =?us-ascii?q?A9a23=3AGHwtO6qQ2S/uKFsd9/cx48EaV5s6LNV00zAX?= =?us-ascii?q?/kB9WHVpW+SCncGvg/gXkTfo4QxhGU0Is9aGJaWGXDfg5Yd4iLNhRYuKcQH6tA?= =?us-ascii?q?KTXeNfxKf42Dv6AWnC8IdmpNJdWodkDtmYNykese/e4BOkV/46ytiG76zAv5aR?= =?us-ascii?q?815JTRt2L4Bt6h4RMHfRLmRSRBNaQbo1faDsrvZvgjq7ZDAwcMO2HRA+Lqf+jv?= =?us-ascii?q?jKjo+jWBgdGnccmUWzpBap8qO/KQSSxB0EUzhCqI1Su1Tttgzi++GetOumwQXX?= =?us-ascii?q?zG/Z4/1t6Zbc4/9iIODJtcQPMDXrjW+TFf1ccpmPpio8ru3qyHtCqqirnz4YJM?= =?us-ascii?q?h4537ceWu4yCGdr3iW7B8U53XvyUCVjBLYyKSTLl5UN+N7iZtdYl/l7SMbzbVB?= =?us-ascii?q?+ZhG2G+UrPNsfGX9tR7g7NvFXQwCrDvFnVMel4co/htieLpbQKNNrIAZ4UNeGI?= =?us-ascii?q?oBGiW/1og9F+loAqjnicp+QBe+aXbB11Mfs+CEbzAWMlOvXlUC0/b17xFm2FRU?= =?us-ascii?q?i2YR2cB3pBc93aN4b7UBxujANM1T+o1mf4szQeZcPY46MKiKI12IZwnNPmKUKU?= =?us-ascii?q?nmE69CG0ulke+N3JwFoNunYIAFy5E/8a6xIm9whCoJYkrrTeeO0JdI/guIf2mn?= =?us-ascii?q?RzTqzahllulEk4y5fqPiNWmnRE8jlqKb0osiK/yeYe+6PNZtHvf/MXGGI/c34y?= =?us-ascii?q?TOH6dvDT0wa+sw0+xLIW6mk4b0FqmvjcDjVcbvAtPWYEoZc1K6PEcmGAXUTf8w?= =?us-ascii?q?k3yDazv+hlzYQHntelPylKgAWZTnww=3D=3D?= X-IronPort-AV: E=Sophos;i="5.81,296,1610406000"; d="scan'208";a="377511581" Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201]) by mail3-relais-sop.national.inria.fr with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Apr 2021 14:36:50 +0200 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: Efraim Flashner Cc: guix-science@gnu.org Subject: Re: GC strategy on clusters References: <87v9969uml.fsf@inria.fr> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 12 Germinal an 229 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 01 Apr 2021 14:36:50 +0200 In-Reply-To: (Efraim Flashner's message of "Thu, 1 Apr 2021 13:45:37 +0300") Message-ID: <87wntm89lp.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=192.134.164.104; envelope-from=ludovic.courtes@inria.fr; helo=mail3-relais-sop.national.inria.fr X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-science-bounces+larch=yhetil.org@gnu.org Sender: "Guix-Science" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1617280621; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=5iTEAoX2FWaLrP09SXWko2OgiPwHYtwEPm8eXRs7jHQ=; b=eXlR77rX6ZYzd3SKS9tg4rpR2+hMaDEqOBCwjGkMI9mz35Yy5fXO5eppPb1KqIjXVYaays Wp3YdsUtBOSwZ/igZCvYjmdiXKJrlupWuO3NuCLFBGAG5iTbUNXW6CcrayORac9RqMIBR7 7Y1VG84YZ7CeE04VtwEgRdE3RjFY6+ZqV9znEmPKLepmlHqgN9D0RLQzXU8IWRyKBt3J+3 PDxORoKxEOwX0yZeWy4kn9fui4BTUpOTLvpsodZHugqzLylkdpUxubct7vR1mRzWUPTf+a 4jBS4F/d3WOqf9z5JAVuXkIC6pUQYryiHqQON9m6zaINayUtPbQyY9TB+NLuZQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1617280621; a=rsa-sha256; cv=none; b=RM/+XUAz0zWc0AgjI2MziZl9iFxNRlrQV39autdJhth9BGGPByo564hVGcxMSjxb+Qm5hi JNLGbRiBd9jiGaTMn1KqYME0UKORHfVsq5D3EVpprqP9joEsBf8vc76pFgIg3tY1sRcFqW cydXgZ93OCBQyNYEG6gvgR04QO/pKBWaE6kzNkOyXAHJbc7jk9pOmrnd9rUNyUiPQbAEJA H8/mA6evZ9eW3Gi6Pzo0sak9GdixPwjjDL2WGUJBoCVFOQvV8AgBPfPeuB+6/GgzoeQ7ou rnhbIBcEhNxEVC1fdSBwhf6sY9200HTvCntNX1q9l2hhyrCqLq1obQbGX7ikKA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Spam-Score: -2.43 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Queue-Id: 44E5317A49 X-Spam-Score: -2.43 X-Migadu-Scanner: scn0.migadu.com X-TUID: 0BMZCApZqllN Hello! Efraim Flashner skribis: > On Thu, Apr 01, 2021 at 12:17:22PM +0200, Ludovic Court=C3=A8s wrote: [...] >> guix package --delete-generations=3D4m >>=20 >> or similar, which was enough to free more space. > > I feel like 4-6 months should be plenty for anything active. Even if it > were run automatically for them it wouldn't remove the last generation. It depends. A practical use case I have in mind: you run experiments, you submit a paper including its results, you get initial reviews months later, and even later it=E2=80=99s published and you get to present it. At= that point, you want to answer questions and to reproduce it. 4=E2=80=936 month= s is not a lot in that context. (Though of course, ideally you=E2=80=99d save channels.scm + manifest.scm a= nd share it with reviewers and readers in the first place=E2=80=A6) Besides, I think the whole point of Guix is that users on the cluster can remain in control, unlike what happens with =E2=80=9Cenvironment module= s=E2=80=9D. >> Longer term, I think Guix should automatically delete old generations >> and instead store the channels + manifest to reproduce them, when >> possible. >>=20 > > This seems to help a bit less when we run into issues about dates being > wrong on SSL tests, or when sources go missing. Good points. Hopefully =E2=80=9Csources go missing=E2=80=9D can soon be co= nsidered addressed. Really, failing TLS tests is the most worrisome issue to me because we don=E2=80=99t have any idea on how to address it systematically. > How much storage and people are you working with? Our initial multiuser > system has 188GB for /gnu and I think 30-40 people and some people have > profiles going back almost 3 years. Not many people have multiple > profiles and the experiments we tried with shared profiles in > /usr/local/guix-profiles don't see a lot of use or get updated > frequently. I=E2=80=99m not sure how much storage the Guix head node has (I=E2=80=99m n= ot an admin), but the number of users and duration is in the same ballpark. > I guess I'm not really sure if its a technology problem or a people > problem. Figuring out if someone is the only one pulling in a copy of > glibc-2.25 is doable but how many copies of diffoscope is too many? > > On a practical note, 'guix package --list-profiles' as root currently > lists everyone's profiles so it can be easier to see who has older > profiles hanging around. Actually, as non-root, I walked /var/guix/profiles/per-user on the cluster to see the number of generations per user, which allowed us to target those with a lot of generations. :-) It would be nice to provide a documented approach sysadmins could follow! Ludo=E2=80=99.