From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id wJI5N2QHmWBVXQAAgWs5BA (envelope-from ) for ; Mon, 10 May 2021 12:13:56 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id KC3wMmQHmWA6EAAA1q6Kng (envelope-from ) for ; Mon, 10 May 2021 10:13:56 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8ADDB150DB for ; Mon, 10 May 2021 12:13:56 +0200 (CEST) Received: from localhost ([::1]:37100 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lg2vD-00058f-OG for larch@yhetil.org; Mon, 10 May 2021 06:13:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38492) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lg2jp-0000kU-JS for guix-devel@gnu.org; Mon, 10 May 2021 06:02:10 -0400 Received: from sender4-of-o51.zoho.com ([136.143.188.51]:21154) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lg2jm-0001IB-Sf for guix-devel@gnu.org; Mon, 10 May 2021 06:02:09 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1620640921; cv=none; d=zohomail.com; s=zohoarc; b=DWoZ4WTPLk+bIgH4//HiI5a1Jx1zJrBcKu9TIn/FkI8KchOWlA26xq0fxbM27GaHZWzTPB6t7eZy4Uu+WDtQcJqOAIUuODw+PfXJijXPHJLXltujsBx4//+kic8cpR2jW2Wk67u8lNVMYgjWHf6N8YOV8KXKn2N4K6CwfX3l8QQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1620640921; h=Content-Type:Content-Transfer-Encoding:Date:From:MIME-Version:Message-ID:Subject:To; bh=1avoegxU8fU+hBXuf+rzgi8RcV2OzyIE+Gnsx7SXE20=; b=W7EJUg7L/jG6sgeueOuk6sxqSUf9NC2iuR2Okpl1QxglU3l+gfBfwirYQs6XUfBzX/nacMf9WnNlrezCosZQK7fm94P4JTKI+KPLOkRz5dq4tXdWED/wqJeD1qOGWtKps6VdUyTu8BNdsHMf8tIAMqr/h9IUcbzxjvdIRw1zA8s= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1620640921; s=zoho; d=elephly.net; i=rekado@elephly.net; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=1avoegxU8fU+hBXuf+rzgi8RcV2OzyIE+Gnsx7SXE20=; b=fVMA3a+n+zIeeL2tOOTG92oBif+EStXCZlU4u/WECuAmZL6Jp4paNT9GkwkvG/FJ c3bMS9ezm+ohSEToAhznPJFTlBhyFUsr3rCKScz6kT8SP6M0UQbDYrQLgc6CS1q5pmB mJbneRFf1tHIB8oRC6FGDPp4+XcYw2u2f12R8A1Y= Received: from localhost (p54ad4f8b.dip0.t-ipconnect.de [84.173.79.139]) by mx.zohomail.com with SMTPS id 1620640788961185.8264506554558; Mon, 10 May 2021 02:59:48 -0700 (PDT) User-agent: mu4e 1.4.15; emacs 27.2 From: Ricardo Wurmus To: guix-devel@gnu.org Subject: =?utf-8?B?4oCcZ3VpeCBnY+KAnSw=?= auto gcroots, cluster deployments X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Date: Mon, 10 May 2021 11:59:44 +0200 Message-ID: <87im3qq57z.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.51; envelope-from=rekado@elephly.net; helo=sender4-of-o51.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1620641636; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=1avoegxU8fU+hBXuf+rzgi8RcV2OzyIE+Gnsx7SXE20=; b=Mh8JFK5JEr7vpbe0vddzGUXEUZo7HqL27/NTRUHd6BJbWNbW4hUMlKSOI0Ue1AFxCu0H3w 6leessTVZ1P0LzlLgSWakYnT1d1rTpnD2zQPVB0Z7vdF0Vq4ndQR5O/Q6L6HgA3MBLzQ0W VBaHCzEZDrqUxSIPjNf2IBzj8thRkPmFHtMy6GMopbG4wqH11QSxIfgIB/5XweTZY18AFP yI+tKG6Tlajhn1n82u13Li149hqbu9cxJsisxTUk5WSwuEyqPnDLYST2tgmGrJFg+twO/5 3yMnOyfcwzJ9FwQb3c6V5BUt30N2AE7yn+8lVzcxv3BKUM647xSzhtrn3O7LmA== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1620641636; a=rsa-sha256; cv=pass; b=VMlublrIY5F3tb4Q9DMmeWqxSSgpqnDde9LqBZAre6ytH8SmpMEoPBHPta8TJ1zezSqaFg lU8xkjbn/N9JeWBr6wtlpbB88tXUg8CNH3RUKm0Lr5e77DFb/6tu23FJLxxBg6MEPYO54u dxjGpJbAeQQWMUQqrgSkNnKYdmTdvH1HMUn+RKdC5oQlJQN4Hepor4m9j3XDglAox0G7F5 OaUiGYXqqnY4z3wxAmN1lkOexDaKHJPpoJPZMPrEh/3DE9QQ4jOn1wCOcJU8VS2YXXxC5a Givv3+coLgn/reRdjOJIjLQlKSAWAqAE4B5sOvwCXe+Un2HomxuCMMu8C4Gs/w== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b=fVMA3a+n; arc=pass ("zohomail.com:s=zohoarc:i=1"); spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -3.65 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b=fVMA3a+n; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 8ADDB150DB X-Spam-Score: -3.65 X-Migadu-Scanner: scn0.migadu.com X-TUID: ejVwhZOH/7eY Hi Guix, On my cluster installation I ran =E2=80=9Cguix gc --list-dead=E2=80=9D out = of=20 curiosity. When finding roots, the daemon also removes what it=20 considers stale links. On my cluster installation not all links=20 always resolve, because the target files reside on remote file=20 systems. These remote locations are not readable by the root user=20 on the server where guix-daemon runs (ignoring local root=20 privileges is pretty common for NFS servers), so they cannot=20 possibly be resolved. So the daemon ends up deleting all these links from=20 /var/guix/gcroots/auto/. This may not be too bad on its own, but it also means that the=20 next time around =E2=80=9Cguix gc=E2=80=9D would consider the eventual targ= et=20 profiles to be garbage. There are two problems here: 1) I don=E2=80=99t think =E2=80=9Cguix gc --list-dead=E2=80=9D (or =E2=80= =9C--list-live=E2=80=9D, or more=20 generally =E2=80=9CfindRoots=E2=80=9D in nix/libstore/gc.cc) should delete= =20 anything. It should just list and not clean up. 2) For cluster installations with remote file systems perhaps=20 there=E2=80=99s something else we can do to record gcroots. We now have=20 this excursion into unreadable space because we use a symlink, but=20 the start ($localstatedir/gcroots/auto) and endpoints=20 (/gnu/store/=E2=80=A6) are both accessible by the daemon. Since these=20 intermediate locations are tied to user accounts, could we not=20 store them in a per-user directory? This problem does not exist for user profiles, because the link in=20 unreadable home directories is not all that important; it merely=20 points to $localstatedir, which is always readable by the daemon.=20 Perhaps we could do the same for temporary roots and let *users*=20 decide when to let go of them by giving them a command to erase=20 the important links in $localstatedir. So instead of having a link from=20 /gnu/var/guix/gcroots/auto/8ypp8dmwnydgbsgjcms2wyb32mng0wri to=20 /home/me/projects/mrg1_chipseq/.guix-profile-1-link pointing to=20 /gnu/store/ap0vrfxjdj57iqdapg8q83l4f7aylqzm-profile, we would=20 record=20 /var/guix/profiles/per-user/me/auto/8ypp8dmwnydgbsgjcms2wyb32mng0wri=20 pointing to /gnu/store/ap0vrfxjdj57iqdapg8q83l4f7aylqzm-profile,=20 and then point /home/me/projects/mrg1_chipseq/.guix-profile-1-link=20 at that. Yes, removing=20 /home/me/projects/mrg1_chipseq/.guix-profile-1-link would no=20 longer free up the profile for garbage collection, but removing=20 $(readlink /home/me/projects/mrg1_chipseq/.guix-profile-1-link)=20 would. This change would pretty much solve the problem for cluster=20 deployments, which otherwise leads to =E2=80=9Cguix gc=E2=80=9D avoidance. What do you think? --=20 Ricardo