From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id +M6iGBcSmWC3dAAAgWs5BA (envelope-from ) for ; Mon, 10 May 2021 12:59:35 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id leNJFBcSmWCaYAAAB5/wlQ (envelope-from ) for ; Mon, 10 May 2021 10:59:35 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 10F34167E0 for ; Mon, 10 May 2021 12:59:35 +0200 (CEST) Received: from localhost ([::1]:34712 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lg3dO-0007UH-5t for larch@yhetil.org; Mon, 10 May 2021 06:59:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48992) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lg3d5-0007U7-Bx for guix-devel@gnu.org; Mon, 10 May 2021 06:59:15 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:60020) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lg3d4-0001na-RZ; Mon, 10 May 2021 06:59:14 -0400 Received: from 2001-1c02-0b18-2900-222e-248e-9586-e52d.cable.dynamic.v6.ziggo.nl ([2001:1c02:b18:2900:222e:248e:9586:e52d]:33406) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lg3d3-0004Cx-5S; Mon, 10 May 2021 06:59:13 -0400 Message-ID: <3795a0268fd5e90b6e1bac171bca97693d8a721d.camel@gnu.org> Subject: Re: =?UTF-8?Q?=E2=80=9Cguix?= =?UTF-8?Q?_gc=E2=80=9D=2C?= auto gcroots, cluster deployments From: Roel Janssen To: Ricardo Wurmus , guix-devel@gnu.org Date: Mon, 10 May 2021 12:59:10 +0200 In-Reply-To: <87im3qq57z.fsf@elephly.net> References: <87im3qq57z.fsf@elephly.net> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.40.1 (3.40.1-1.fc34) MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1620644375; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=CozCfOkenerZUP4ZuqLPCMA9YbKCGbOX+Liq7/rxBVo=; b=ahuZK90sYnp+gfAoKwnTfXvFGCM+YxJRzR/2qisvmHzfLI/vjNdgSuVyod7Ck2Z5euX5a7 1+lBv4BOQXxLscKr4834a5orOONRytp09+1QXGRUs7xmywaHIEFS9G2/RrfwpyQjPpawUu lvGD6hTTCdJGigDMtuVhEq/naIVkdDIql753Vm9VPPAQX+FU6x/grae0trxnOUiniRXuEd JW4hX5ngSAgNSLdtowdIl2yvFQrGzRNSPMVgetj0R/xKNf0FC3CIaZyRkdt9bUd9emigf6 Xr+StQt79nGMeYyfmIdO1Ty6vnDXqROZPrzOM+KLRxCCKoKs9vDSD8oEsxjtuQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1620644375; a=rsa-sha256; cv=none; b=oijoOekt9YZ497/3IivQdskU/+oJLhn8G4T0vgz3CTA8dLUe76MWPdwURCk6DsCHN3Pxq5 wENlS0kkNESTN45emWq+IPMTCc3QZi4zuzOH6WobVNyq+puZxJqCktfvcZvIt59bcSaufq E7MwbKLsrweHlK6qMe6qYzCe3xc4Ge/ulLf6Aox9N2ftQiTCd70dPPRQUXUdGVPz705qh4 Zs1++X9Da2ei5x1TLVGTRTNVdVbCcpVKoyBYOO1D8IYsv7RtZa+8lJ6aGTbC2V24wmnvRt /LmyH4nQYjsnRCdDwEGwp0YG+92txnFNWZlU3zN1Ij6D8TGlq0sLPcsVjFrOeg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -2.95 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 10F34167E0 X-Spam-Score: -2.95 X-Migadu-Scanner: scn0.migadu.com X-TUID: GCv29Po/6fsy On Mon, 2021-05-10 at 11:59 +0200, Ricardo Wurmus wrote: > Hi Guix, > > On my cluster installation I ran “guix gc --list-dead” out of > curiosity.  When finding roots, the daemon also removes what it > considers stale links.  On my cluster installation not all links > always resolve, because the target files reside on remote file > systems.  These remote locations are not readable by the root user > on the server where guix-daemon runs (ignoring local root > privileges is pretty common for NFS servers), so they cannot > possibly be resolved. > > So the daemon ends up deleting all these links from > /var/guix/gcroots/auto/. > > This may not be too bad on its own, but it also means that the > next time around “guix gc” would consider the eventual target > profiles to be garbage. > > There are two problems here: > > 1) I don’t think “guix gc --list-dead” (or “--list-live”, or more > generally “findRoots” in nix/libstore/gc.cc) should delete > anything.  It should just list and not clean up. > I agree it would be better if --list-* options don't remove store items and/or database entries. > 2) For cluster installations with remote file systems perhaps > there’s something else we can do to record gcroots.  We now have > this excursion into unreadable space because we use a symlink, but > the start ($localstatedir/gcroots/auto) and endpoints > (/gnu/store/…) are both accessible by the daemon.  Since these > intermediate locations are tied to user accounts, could we not > store them in a per-user directory? > > This problem does not exist for user profiles, because the link in > unreadable home directories is not all that important; it merely > points to $localstatedir, which is always readable by the daemon. > Perhaps we could do the same for temporary roots and let *users* > decide when to let go of them by giving them a command to erase > the important links in $localstatedir. > > So instead of having a link from > /gnu/var/guix/gcroots/auto/8ypp8dmwnydgbsgjcms2wyb32mng0wri to > /home/me/projects/mrg1_chipseq/.guix-profile-1-link pointing to > /gnu/store/ap0vrfxjdj57iqdapg8q83l4f7aylqzm-profile, we would > record > /var/guix/profiles/per-user/me/auto/8ypp8dmwnydgbsgjcms2wyb32mng0wri > pointing to /gnu/store/ap0vrfxjdj57iqdapg8q83l4f7aylqzm-profile, > and then point /home/me/projects/mrg1_chipseq/.guix-profile-1-link > at that.  Yes, removing > /home/me/projects/mrg1_chipseq/.guix-profile-1-link would no > longer free up the profile for garbage collection, but removing > $(readlink /home/me/projects/mrg1_chipseq/.guix-profile-1-link) > would. > > This change would pretty much solve the problem for cluster > deployments, which otherwise leads to “guix gc” avoidance. > > What do you think? > We are facing a similar issue on our cluster deployment. In our case, directories are only readable to certain users but not root. So the garbage collector can't read the symlinks, and therefore cannot determine whether a profile is still in use or not. So this seems like a good idea to me. Would it be possible to add an option to retrospectively apply this transformation? Maybe that could work somewhat like this: $ ls -lha ... /home/me/projects/mrg1_chipseq/.guix-profile-1-link -> /gnu/store/ap0vrfxjdj57iqdapg8q83l4f7aylqzm-profile $ ls -lh $localstatedir/gcroots/auto $ guix gc --synchronize-profiles-to-gcroots $ ls -lh $localstatedir/gcroots/auto ... 8ypp8dmwnydgbsgjcms2wyb32mng0wri -> /gnu/store/ap0vrfxjdj57iqdapg8q83l4f7aylqzm-profile $ rm /home/me/projects/mrg1_chipseq/.guix-profile-1-link $ guix gc --synchronize-profiles-to-gcroots $ ls -lh $localstatedir/gcroots/auto Kind regards, Roel Janssen