From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ricardo Wurmus Subject: Re: Guix on clusters and in HPC Date: Tue, 01 Nov 2016 08:15:59 +0100 Message-ID: <8737jbk6vk.fsf@elephly.net> References: <87r37divr8.fsf@gnu.org> <86vawh9lvw.fsf@gmail.com> <877f8vgvns.fsf@elephly.net> <86mvhkfaag.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:42336) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c1TIr-000429-Kh for guix-devel@gnu.org; Tue, 01 Nov 2016 03:16:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c1TIq-0000VV-Nw for guix-devel@gnu.org; Tue, 01 Nov 2016 03:16:13 -0400 In-reply-to: <86mvhkfaag.fsf@gmail.com> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: myglc2 Cc: Guix-devel myglc2 writes: > On 10/26/2016 at 14:08 Ricardo Wurmus writes: > >> At the MDC we’re using SGE and users specify their software environment >> in the job script. The software environment is a Guix profile, so the >> job script usually contains a line to source the profile’s >> “etc/profile”, which has the effect of setting up the required >> environment variables. > > Cool. How do you deal with the tendency of user's profiles to be "moving > targets?" IOW, I am wondering how one would reproduce a result at a > later date when one's profile has "changed"? I strongly encourage users to do two things: - use manifests - record the current version of Guix and our local package repository when instantiating a manifest. It only takes these two pieces of information to reproduce a software environment >>> Based on my experiments with Guix/Debian, GuixSD, VMs, and VM images it >>> is not obvious to me which of these levels of abstraction is >>> appropriate. >> >> FWIW we’re using Guix on top of CentOS 6.8. The store is mounted >> read-only on all cluster nodes. > > Nice. Do you attempt to "protect" your users from variations in the > CentOS config? I’m not sure I understand the question. With Guix we’re not relying on the host system software with the exception of the kernel at runtime. Users have one profile that they use on the cluster nodes (running CentOS) as well as on their workstations (Ubuntu or a later version of CentOS). When it comes to building software outside of Guix (e.g. when using “pip install” for Python or “install.packages()” in R) there’s little I can do. I’m considering to provide a “runtime” environment in which the Guix toolchain and very common libraries are available, which can be used to build software in a traditional fashion. I’m hacking on Rstudio server now to make it usable running inside a container where the system toolchain is essentially swapped out for a toolchain from Guix. This is needed because mixing binaries that are dynamically loaded at runtime (e.g. some R modules from Guix with bindings to system libraries) cannot possibly work due to ABI incompatibility. This is actually the most common problem I’m facing here, because users are used to install stuff on their own. Mixing with Guix binaries only works as long as the applications run as separate processes. ~~ Ricardo