unofficial mirror of gwl-devel@gnu.org
 help / color / mirror / Atom feed
* Next steps for the GWL
@ 2019-05-29 13:47 Ricardo Wurmus
  2019-06-03 15:16 ` zimoun
                   ` (2 more replies)
  0 siblings, 3 replies; 16+ messages in thread
From: Ricardo Wurmus @ 2019-05-29 13:47 UTC (permalink / raw)
  To: gwl-devel

Hi,

I’m going to use the GWL in the next few days to rewrite the PiGx RNAseq
pipeline from Snakemake.  This will likely show me what features are
still missing from the GWL and what implemented features are awkward to
use.

My goals for the future (in no particular order) are as follows:

* add support for running workflows from a file (without
  GUIX_WORKFLOW_PATH)

* replace the “qsub”-based Grid Engine execution engine with proper
  DRMAA bindings

* add support for running remote jobs via Guile SSH

* add support for spawning jobs on AWS or Azure or via Kubernetes

* tighter integration with Guix features, e.g. to export a container
  image per process via “guix pack” or to pack up the whole workflow as
  a relocatable executable.

* inversion of control: enable workflow designers to use the GWL as a
  library, so that the “guix workflow” user interface does not need to
  be used at all (see PiGx for an example).

* explore the use of inferiors — the GWL should be usable with any
  version of Guix that may be installed, not just the version that was
  used at compilation time.  Can we use “guix repl” and inferiors,
  perhaps?

* add support for executing processes in isolated environments
  (containers) — this requires a better understanding of process inputs.

What do you think about this?  Who would like to help flesh out these
ideas?

--
Ricardo

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2019-06-12  9:46 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-29 13:47 Next steps for the GWL Ricardo Wurmus
2019-06-03 15:16 ` zimoun
2019-06-03 16:18   ` Ricardo Wurmus
2019-06-06 11:07     ` zimoun
2019-06-06 12:19       ` Ricardo Wurmus
2019-06-06 13:23         ` Pjotr Prins
2019-06-06  3:19 ` Kyle Meyer
2019-06-06 10:11   ` Ricardo Wurmus
2019-06-06 10:55     ` zimoun
2019-06-06 11:59       ` Ricardo Wurmus
2019-06-06 13:44       ` Pjotr Prins
2019-06-06 14:06         ` Pjotr Prins
2019-06-06 15:07     ` Kyle Meyer
2019-06-06 20:29       ` Ricardo Wurmus
2019-06-07  4:11         ` Kyle Meyer
2019-06-12  9:46 ` Ricardo Wurmus

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).