unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* “guix gc”, auto gcroots, cluster deployments
@ 2021-05-10  9:59 Ricardo Wurmus
  2021-05-10 10:59 ` Roel Janssen
                   ` (2 more replies)
  0 siblings, 3 replies; 11+ messages in thread
From: Ricardo Wurmus @ 2021-05-10  9:59 UTC (permalink / raw)
  To: guix-devel

Hi Guix,

On my cluster installation I ran “guix gc --list-dead” out of 
curiosity.  When finding roots, the daemon also removes what it 
considers stale links.  On my cluster installation not all links 
always resolve, because the target files reside on remote file 
systems.  These remote locations are not readable by the root user 
on the server where guix-daemon runs (ignoring local root 
privileges is pretty common for NFS servers), so they cannot 
possibly be resolved.

So the daemon ends up deleting all these links from 
/var/guix/gcroots/auto/.

This may not be too bad on its own, but it also means that the 
next time around “guix gc” would consider the eventual target 
profiles to be garbage.

There are two problems here:

1) I don’t think “guix gc --list-dead” (or “--list-live”, or more 
generally “findRoots” in nix/libstore/gc.cc) should delete 
anything.  It should just list and not clean up.

2) For cluster installations with remote file systems perhaps 
there’s something else we can do to record gcroots.  We now have 
this excursion into unreadable space because we use a symlink, but 
the start ($localstatedir/gcroots/auto) and endpoints 
(/gnu/store/…) are both accessible by the daemon.  Since these 
intermediate locations are tied to user accounts, could we not 
store them in a per-user directory?

This problem does not exist for user profiles, because the link in 
unreadable home directories is not all that important; it merely 
points to $localstatedir, which is always readable by the daemon. 
Perhaps we could do the same for temporary roots and let *users* 
decide when to let go of them by giving them a command to erase 
the important links in $localstatedir.

So instead of having a link from 
/gnu/var/guix/gcroots/auto/8ypp8dmwnydgbsgjcms2wyb32mng0wri to 
/home/me/projects/mrg1_chipseq/.guix-profile-1-link pointing to 
/gnu/store/ap0vrfxjdj57iqdapg8q83l4f7aylqzm-profile, we would 
record 
/var/guix/profiles/per-user/me/auto/8ypp8dmwnydgbsgjcms2wyb32mng0wri 
pointing to /gnu/store/ap0vrfxjdj57iqdapg8q83l4f7aylqzm-profile, 
and then point /home/me/projects/mrg1_chipseq/.guix-profile-1-link 
at that.  Yes, removing 
/home/me/projects/mrg1_chipseq/.guix-profile-1-link would no 
longer free up the profile for garbage collection, but removing 
$(readlink /home/me/projects/mrg1_chipseq/.guix-profile-1-link) 
would.

This change would pretty much solve the problem for cluster 
deployments, which otherwise leads to “guix gc” avoidance.

What do you think?

-- 
Ricardo


^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2021-06-08 12:55 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-10  9:59 “guix gc”, auto gcroots, cluster deployments Ricardo Wurmus
2021-05-10 10:59 ` Roel Janssen
2021-05-10 11:59   ` Ricardo Wurmus
2021-05-10 15:45     ` Roel Janssen
2021-05-10 16:23       ` Ricardo Wurmus
2021-05-11 20:42     ` Ludovic Courtès
2021-05-10 13:40 ` Sébastien Lerique
2021-05-10 13:59   ` Guix on NFS Ricardo Wurmus
2021-05-11 20:50 ` “guix gc”, auto gcroots, cluster deployments Ludovic Courtès
2021-05-14 10:21   ` Ricardo Wurmus
2021-06-08 12:55     ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).