unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Using a shared Guix store (was RE: [Bio-packaging] testing out guix)
@ 2015-06-18 20:22 Cook, Malcolm
  2015-06-19  8:06 ` Ricardo Wurmus
  0 siblings, 1 reply; 28+ messages in thread
From: Cook, Malcolm @ 2015-06-18 20:22 UTC (permalink / raw)
  To: 'Ricardo Wurmus', 'Pjotr Prins'
  Cc: Guix-devel, 'bio-packaging@mailman.open-bio.org'

Ricardo, Pj, et al,


... Hi - I am resending this message as I omitted Guix-devel <guix-devel@gnu.org> at first and would like to cast a broader net....


I am interested in understanding details behind Ricardo's observation: "Guix is not designed to be run in a centralised manner. A Guix daemon is supposed to run on each system as root and it listens to RPCs from local users only. In an environment with multiple clusters and multiple workstations this approach requires considerable effort to make it work correctly and securely. Instead we opted to run the Guix daemon on a single dedicated server, writing profile data and store items onto an NFS share. . The cluster nodes and workstations mount this share read-only." (http://elephly.net/posts/2015-04-17-gnu-guix.html)

Can anyone elaborate a little on what are the obstacles to having  `/gnu` mounted read-write network wide?

Can it be partially characterized as:

	Multi-process contention over un-coordinated access to the store (especially write access necessitated by supporting the `build` action)

If so, might this be mitigated using a variant off of "Using the Offload Facility" (http://www.gnu.org/software/guix/manual/guix.html#Daemon-Offload-Setup) in which builds would still be offloaded (and thus subject to coordination), with the elimination of the need for " Missing prerequisites for the build are copied over SSH to the target machine, which then proceeds with the build; upon success the output(s) of the build are copied back to the initial machine" since they would be done through the shared file system?

Do I understand correctly that in your setup, Ricardo, that absolutely no `guix` commands are executed on any host other than the "single dedicated server".   What about `guix environment p1 p2 p3` when p1 p2 p3 are already available in /gnu.  If I understand correctly, in such a case, nothing need be written to /gnu... and so should not present any challenge to running guix off a shared mount.   Or am I missing an aspect of what is going on?

Lacking the ability to, from any host, dynamically alter the runtime environment to include _already installed_  scientific applications seems like the most important aspect of guix that would be untolerable to my users were I to user guix.    I do hope I am mis-understanding the trade-off you made at your site.

I think the second-worst trade-off this compromise takes is that a user cannot even alter their own profile unless connected to the master guix host.  For instance, a user switching her default emacs  between two already built & installed versions only requires editing the profiles/per-user symlink farm; couldn't such commands be put behind a per-user semaphore,  allowing guix/profiles/per-user to be made available +rw on a shared network mount?

I really appreciate everyone's assistance in helping me understand these trade-offs, how guix works, and if there is any new or different thinking about strategies for deploying guix.

Thanks,

Malcolm Cook



Hi Malcolm,

> Pleased to e-meet you.  It was your
> http://elephly.net/posts/2015-04-17-gnu-guix.html that got on this 
> path.  I'm so glad I found it.

Oh, great to read that it reached someone who might benefit from it :)

> Great.  I have a trove of recipes (in either bash and a
> brew-derivative) which I intend to move into guix over time, unless 
> I'm beaten to any of them (I'm sure I already have been for some):>

[...]

I just went through the list and found a few that looked familiar.
Below are some comments.

Non-free, won't package:

> gatk
> jimKentUtils

On my list (possibly non-free):

> igv
> igvtools
> meme
> picard (requires more Java toolchain stuff to build from source)
> soapdenovo2
> trimmomatic

Non-free stuff, packaged in my private guix-nonfree repo:

> macs14
> tophat
> viennaRNA

Free stuff, packaged and available through Guix:

> bamtools
> bedtools
> blast+  (WIP, it's really big)
> bowtie2
> cufflinks
> fastqc
> fastx_toolkit
> graphviz
> ncbi_sra_toolkit
> ngsutils
> rsem
> star
> tabix

These look familiar but I'm not sure if I already have them:

> bam2fastq
> bcftools

[...]

> Gonna be a party!

Indeed!  I would be very happy if you decide for Guix in the end.  There isn't a lot of bioinfo stuff there yet, but that can be fixed by having more people contribute recipes.

Cheers,
Ricardo

^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2015-07-23 22:52 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-06-18 20:22 Using a shared Guix store (was RE: [Bio-packaging] testing out guix) Cook, Malcolm
2015-06-19  8:06 ` Ricardo Wurmus
2015-06-19 11:34   ` Ludovic Courtès
2015-06-25  6:40     ` Ricardo Wurmus
2015-06-19 11:40   ` Ludovic Courtès
2015-07-08 19:20     ` Cook, Malcolm
2015-07-08 19:43       ` Ricardo Wurmus
2015-06-19 17:48   ` Cook, Malcolm
2015-06-24 19:57     ` Ludovic Courtès
2015-07-08 18:03       ` Cook, Malcolm
2015-07-08 19:53         ` Ricardo Wurmus
2015-07-10  8:39         ` Ludovic Courtès
2015-07-11  0:48           ` Cook, Malcolm
2015-07-13 16:45             ` Test suite failures Ludovic Courtès
2015-07-18  3:04               ` Cook, Malcolm
2015-07-18 15:02                 ` Ludovic Courtès
2015-07-11  0:54           ` Using a shared Guix store (was RE: [Bio-packaging] testing out guix) Cook, Malcolm
2015-07-15 15:45             ` Ricardo Wurmus
2015-07-15 19:49               ` Cook, Malcolm
2015-07-15 20:28                 ` Pjotr Prins
2015-07-18  9:26                 ` Cook, Malcolm
2015-07-18 15:13                   ` Ludovic Courtès
2015-07-19  9:18                     ` Claes Wallin (韋嘉誠)
2015-07-19  9:33                       ` Andreas Enge
2015-07-20 22:37                     ` Cook, Malcolm
2015-07-21 20:23                       ` Cook, Malcolm
2015-07-21 20:29                         ` Ricardo Wurmus
2015-07-23 22:52                         ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).