Re: Guix on clusters and in HPC

[Ricardo Wurmus <rekado@elephly.net>]

myglc2 <myglc2@gmail.com> writes:

> While SGE is dated and can be a bear to use, it provides a useful
> yardstick for HPC/Cluster functionality. So it is useful to consider how
> Guix(SD) might impact this model. Presumably a defining characteristic
> of GuixSD clusters is that the software configuration of compute hosts
> no longer needs to be fixed and the user can "dial in" a specific SW
> configuration for each job step.  This is in many ways a good thing. But
> it also generates new requirements. How does one specify the SW config
> for a given job or recipe step:
>
> 1) VM image?
>
> 2) VM?
>
> 3) Installed System Packages?
>
> 4) Installed (user) packages?

At the MDC we’re using SGE and users specify their software environment
in the job script.  The software environment is a Guix profile, so the
job script usually contains a line to source the profile’s
“etc/profile”, which has the effect of setting up the required
environment variables.

I don’t know of anyone who uses VMs or VM images to specify software
environments.

> Based on my experiments with Guix/Debian, GuixSD, VMs, and VM images it
> is not obvious to me which of these levels of abstraction is
> appropriate.

FWIW we’re using Guix on top of CentOS 6.8.  The store is mounted
read-only on all cluster nodes.

> The most forward-thinking group that I know discarded their cluster
> hardware a year ago to replace it with starcluster
> (http://star.mit.edu/cluster/). Starcluster automates the creation,
> care, and feeding of a HPC clusters on AWS using the Grid Engine
> scheduler and AMIs. The group has a full-time "starcluster jockey" who
> manages their cluster and they seem quite happy with the approach. So
> you may want to consider starcluster as a model when you think of
> cluster management requirements.

When using starcluster are software environments transferred to AWS on
demand?  Does this happen on a per-job basis?  Are any of the
instantiated machines persistent or are they discarded after use?

~~ Ricardo