Re: “guix gc”, auto gcroots, cluster deployments

[Roel Janssen <roel@gnu.org>]

On Mon, 2021-05-10 at 13:59 +0200, Ricardo Wurmus wrote:
> 
> Hi Roel,
> 
> thanks for your feedback!
> 
> > Would it be possible to add an option to retrospectively apply 
> > this
> > transformation?  Maybe that could work somewhat like this:
> > 
> > $ ls -lha
> > ... /home/me/projects/mrg1_chipseq/.guix-profile-1-link ->
> > /gnu/store/ap0vrfxjdj57iqdapg8q83l4f7aylqzm-profile
> 
> This wouldn’t work, because we can’t read 
> /home/me/projects/mrg1_chipseq/.guix-profile-1-link centrally.  In 
> this particular case only the user “me” could resolve the link and 
> thus migrate the link.  (I would do this semi-manually by 
> impersonating the users to read their links.)
> 

Indeed we cannot resolve /home/me/projects/mrg1_chipseq/.guix-profile-
1-link as another user than "me".  So "me" would have to run the
hypothetical "guix gc --synchronize-profiles-to-gcroots".

From my point of view, that would be fine.  We can simply ask the users
on the cluster to execute that command once.

This could work if we have a mechanism to determine how complete the
garbage-collection picture for "root" is, and only continue GCing when
the picture is complete.  A starting point for this is 
$localstatedir/gcroots/auto which provides an overview of all profiles
that have been created. Perhaps we can reuse the filename of the
symlink.  So in the example given in the previous e-mail we could add
another link like so, but in a different directory:
8ypp8dmwnydgbsgjcms2wyb32mng0wri -> /gnu/store/...-profile

For which this already existed in $localstatedir/gcroots/auto:
8ypp8dmwnydgbsgjcms2wyb32mng0wri ->
/home/me/projects/mrg1_chipseq/.guix-profile-1-link

And then a one-time transition can be made by looking up 
8ypp8dmwnydgbsgjcms2wyb32mng0wri in both places.

Maybe I'm making things needlessly complex, but I would really like a
solution that can be applied to an existing cluster setup. :)

So, your initial proposal sounds good to me already. How it could be
made more useful for the retrospective matching of unreachable symlinks
to /gnu/store paths can be considered a separate issue to solve.

> ~ ~ ~
> 
> Another related problem I found is that the names of the links in 
> unreadable space may have names that only make sense on the system 
> where they were created.  For example, on the server “beast” we 
> may mount the cluster home directory as “/clusterhome/me”, whereas 
> on cluster nodes it would be mounted as “/home/me”.  When I have 
> Guix record a gcroot while working on “beast” I would get a link 
> that is no longer valid when I work on the cluster.
> 

Yes, I agree that this is another reason to implement your suggested
change. :)

Kind regards,
Roel Janssen