* RFC: Support for DRMAA backend
@ 2021-05-03 9:24 Ricardo Wurmus
0 siblings, 0 replies; only message in thread
From: Ricardo Wurmus @ 2021-05-03 9:24 UTC (permalink / raw)
I pushed a new branch “wip-drmaa”, which lets GWL use an HPC
scheduler via DRMAA. Set the GWL_USE_DRMAA variable and use the
default execution engine. (In the future this variable will
disappear once we have agreed on a nicer way to select DRMAA
I only tested this with a local Slurm cluster on my laptop, but it
seems to work fine. I’d be happy to hear test reports!
Before this can make its way into the main branch we have to
figure out if the concept of an execution engine still makes sense
the way it is implemented.
Execution engines currently are only used to build custom scripts
(or script wrappers) — but using DRMAA does not require any new
kind of script. The old Grid Engine backend builds the usual
workflow scripts and a set of job scripts that call those scripts
after passing Grid Engine settings. For DRMAA we only need
I wonder if we should scrap execution engines as they currently
exist; we would only ever generate the same kind of workflow
scripts and move all execution-related work (like submitting jobs
to a scheduler) to, well, execution time. This also means that
the workflow scripts would not need to be modified when picking a
different execution backend, which I think is good.
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2021-05-03 9:28 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-03 9:24 RFC: Support for DRMAA backend Ricardo Wurmus
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).