Re: Guix Workflow Language ?

[Pjotr Prins <pjotr.public12@thebird.nl>]

On Thu, Jan 25, 2018 at 11:04:36PM +0100, Roel Janssen wrote:
> I get that point.  Maybe it's then a better idea to write the workflow
> in CWL (like you would do), and use Guix to generate Docker containers.
> 
> Then you do get the benefit of Guix's strong reproducibility and
> composability forscientific software, plus you get to keep writing the
> workflow in CWL. :-)

I think the CWL is not particularly great, mostly because its design
is too open ended (they include a Javascript interpreter - so it is
YAML + JS) and 'complex'. Nor does it help/enforce analysis to be
deterministic.

CWL has community backing. But it could well implode on itself. So
many people are working on it (nominally) for years, and so little has
it got to show for. The promise is truly shared pipelines - and, so
far, it has not happened.  But maybe by sheer grit they will get
there. I know people who are working hard to make it happen.

I think that if you are not drinking the CWL cool-aid, GWL is a great
alternative. But it needs LISP and it may need a bit more development
to make it a smooth experience. If more people help out I am sure we
can get there.  What would be a great pipeline that has general
interest? How about using GWL on the build farm, for example?

I don't think the way forward is adding CWL-style YAML support to GWL.
I think the way forward is to be able to run CWL workflows inside GWL,
i.e., a CWL runner is treated as a single step inside GWL. That way we
get the best of both worlds. Treat CWL as a piece of software on its
own and provision the Guix container to match. Interestingly, this is
pretty much possible today. (Note that there will be some complexity
in error handling because both GWL and CWL will submit jobs to a
cluster environment).

Pj.