From: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
To: gwl-devel@gnu.org
Subject: Next steps for the GWL
Date: Wed, 29 May 2019 15:47:18 +0200 [thread overview]
Message-ID: <87a7f5l6e1.fsf@mdc-berlin.de> (raw)
Hi,
I’m going to use the GWL in the next few days to rewrite the PiGx RNAseq
pipeline from Snakemake. This will likely show me what features are
still missing from the GWL and what implemented features are awkward to
use.
My goals for the future (in no particular order) are as follows:
* add support for running workflows from a file (without
GUIX_WORKFLOW_PATH)
* replace the “qsub”-based Grid Engine execution engine with proper
DRMAA bindings
* add support for running remote jobs via Guile SSH
* add support for spawning jobs on AWS or Azure or via Kubernetes
* tighter integration with Guix features, e.g. to export a container
image per process via “guix pack” or to pack up the whole workflow as
a relocatable executable.
* inversion of control: enable workflow designers to use the GWL as a
library, so that the “guix workflow” user interface does not need to
be used at all (see PiGx for an example).
* explore the use of inferiors — the GWL should be usable with any
version of Guix that may be installed, not just the version that was
used at compilation time. Can we use “guix repl” and inferiors,
perhaps?
* add support for executing processes in isolated environments
(containers) — this requires a better understanding of process inputs.
What do you think about this? Who would like to help flesh out these
ideas?
--
Ricardo
next reply other threads:[~2019-05-29 13:47 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-29 13:47 Ricardo Wurmus [this message]
2019-06-03 15:16 ` Next steps for the GWL zimoun
2019-06-03 16:18 ` Ricardo Wurmus
2019-06-06 11:07 ` zimoun
2019-06-06 12:19 ` Ricardo Wurmus
2019-06-06 13:23 ` Pjotr Prins
2019-06-06 3:19 ` Kyle Meyer
2019-06-06 10:11 ` Ricardo Wurmus
2019-06-06 10:55 ` zimoun
2019-06-06 11:59 ` Ricardo Wurmus
2019-06-06 13:44 ` Pjotr Prins
2019-06-06 14:06 ` Pjotr Prins
2019-06-06 15:07 ` Kyle Meyer
2019-06-06 20:29 ` Ricardo Wurmus
2019-06-07 4:11 ` Kyle Meyer
2019-06-12 9:46 ` Ricardo Wurmus
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.guixwl.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87a7f5l6e1.fsf@mdc-berlin.de \
--to=ricardo.wurmus@mdc-berlin.de \
--cc=gwl-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).