From mboxrd@z Thu Jan 1 00:00:00 1970 From: myglc2 Subject: Re: Guix on clusters and in HPC Date: Mon, 31 Oct 2016 18:01:11 -0400 Message-ID: <86mvhkfaag.fsf@gmail.com> References: <87r37divr8.fsf@gnu.org> <86vawh9lvw.fsf@gmail.com> <877f8vgvns.fsf@elephly.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:44712) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c1KcY-0006Ii-V3 for guix-devel@gnu.org; Mon, 31 Oct 2016 18:00:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c1KcV-0000aq-TI for guix-devel@gnu.org; Mon, 31 Oct 2016 17:59:59 -0400 In-Reply-To: <877f8vgvns.fsf@elephly.net> (Ricardo Wurmus's message of "Wed, 26 Oct 2016 14:08:23 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ricardo Wurmus Cc: Guix-devel On 10/26/2016 at 14:08 Ricardo Wurmus writes: > At the MDC we=E2=80=99re using SGE and users specify their software envir= onment > in the job script. The software environment is a Guix profile, so the > job script usually contains a line to source the profile=E2=80=99s > =E2=80=9Cetc/profile=E2=80=9D, which has the effect of setting up the req= uired > environment variables. Cool. How do you deal with the tendency of user's profiles to be "moving targets?" IOW, I am wondering how one would reproduce a result at a later date when one's profile has "changed"? > I don=E2=80=99t know of anyone who uses VMs or VM images to specify softw= are > environments. One rationale I can think of for VM images is to "archive" them along with the analysis result to provide brute-force reproducibility. An example I know of is a group whose cluster consists of VMs on VMware. The VMs run a mix of OSes provisioned with varying levels of resource (e.g. #CPUs, amount of memory, installed software).=20 >> Based on my experiments with Guix/Debian, GuixSD, VMs, and VM images it >> is not obvious to me which of these levels of abstraction is >> appropriate. > > FWIW we=E2=80=99re using Guix on top of CentOS 6.8. The store is mounted > read-only on all cluster nodes. Nice. Do you attempt to "protect" your users from variations in the CentOS config? >> The most forward-thinking group that I know discarded their cluster >> hardware a year ago to replace it with starcluster >> (http://star.mit.edu/cluster/). Starcluster automates the creation, >> care, and feeding of a HPC clusters on AWS using the Grid Engine >> scheduler and AMIs. The group has a full-time "starcluster jockey" who >> manages their cluster and they seem quite happy with the approach. So >> you may want to consider starcluster as a model when you think of >> cluster management requirements. > > When using starcluster are software environments transferred to AWS on > demand? Does this happen on a per-job basis? Are any of the > instantiated machines persistent or are they discarded after use? In the application I refer to the cluster is kept spun up. I am not sure if they have built a custom Amazon VM-image (AMI) or if they start with a "stock" AMI and configure the compute hosts during the spin up.