Re: How do I support building a guix package over multiple machines in a cloud environment?

[pjotr.public12@thebird.nl]

Yeah, if you read my BLOG below you can also simply split packaging
(read deployment) from the pipeline (runner). It would already be
worthwile to show how Guix can be mixed with Nextflow and how you
could use the same method to create reproducible containers.

That is a real win!

In the next step try to package Nextflow itself.

Or, alternatively, port pipelines to GWL. Mind, GWL needs development.
Nextflow is an amazing (Cloud) tool that can benefit from
reproducible deployment.

Pj.


On Mon, Dec 02, 2019 at 08:00:07PM +0100, zimoun wrote:
> Hi (again) Josh :-)
> 
> On Mon, 2 Dec 2019 at 19:38, Josh Marshall
> <joshua.r.marshall.1991@gmail.com> wrote:
> 
> > He uses it as a bioinformatics workflow to generate some analysis.  It
> 
> GWL should work for this use case. o/
> 
> > Is this kind of use case supported?  If so, how so?  Is nextflow not
> > practical to keep?  Please, someone catch me up here so I can start to
> > write code to help him out.  If this goes well, my company could
> > integrate for gwl/guix in our work, which would be amazing.
> 
> Netxflow [1] is a Domain Specific Language (DSL): you write "rules"
> and how these rules are combined together. In the bioinformatics
> field, Snakemake [2] seems more popular. Other alternatives are CWL
> [3], WDL [4], etc.
> 
> Basically, you describe:
>  - what is the inputs
>  - what is the outputs
>  - how to process the inputs to produce the outputs
> 
> You can find examples there [*]. It uses the WISP syntax [5] but it
> perfectly works with a Scheme-syntax if you prefer parenthesis. ;-)
> 
> [*] https://guixwl.org/
> 
> 
> However, you should be interested by this blog post [#] by Pjotr using
> Guix and CWL and other niceties!
> 
> [#] https://hpc.guix.info/blog/2019/01/creating-a-reproducible-workflow-with-cwl/
> 
> AFAIK, Nextflow is not yet packaged in Guix. One direction is to
> package it and then use the workflow described in Nextflow DSL in the
> spirit of [#]. One other direction is to rewrite the workflow using
> the GWL DSL. It depends a bit on what is your final aim.
> 
> 
> Hope that helps.
> simon
> 
> [1] https://www.nextflow.io/
> [2] https://snakemake.readthedocs.io/en/stable/
> [3] https://www.commonwl.org/
> [4] http://www.openwdl.org/
> [5] https://srfi.schemers.org/srfi-119/srfi-119.html
>