Re: How do I support building a guix package over multiple machines in a cloud environment?

[Josh Marshall <joshua.r.marshall.1991@gmail.com>]

I'll start with packaging nextflow.  That seems like the easy starting
point, will be needed if I want to switch my personal work systems to
guix, and I still shouldn't get deep into guix development challenges
since it is all still new.

On Mon, Dec 2, 2019 at 3:02 PM <pjotr.public12@thebird.nl> wrote:
>
> Yeah, if you read my BLOG below you can also simply split packaging
> (read deployment) from the pipeline (runner). It would already be
> worthwile to show how Guix can be mixed with Nextflow and how you
> could use the same method to create reproducible containers.
>
> That is a real win!
>
> In the next step try to package Nextflow itself.
>
> Or, alternatively, port pipelines to GWL. Mind, GWL needs development.
> Nextflow is an amazing (Cloud) tool that can benefit from
> reproducible deployment.
>
> Pj.
>
>
> On Mon, Dec 02, 2019 at 08:00:07PM +0100, zimoun wrote:
> > Hi (again) Josh :-)
> >
> > On Mon, 2 Dec 2019 at 19:38, Josh Marshall
> > <joshua.r.marshall.1991@gmail.com> wrote:
> >
> > > He uses it as a bioinformatics workflow to generate some analysis.  It
> >
> > GWL should work for this use case. o/
> >
> > > Is this kind of use case supported?  If so, how so?  Is nextflow not
> > > practical to keep?  Please, someone catch me up here so I can start to
> > > write code to help him out.  If this goes well, my company could
> > > integrate for gwl/guix in our work, which would be amazing.
> >
> > Netxflow [1] is a Domain Specific Language (DSL): you write "rules"
> > and how these rules are combined together. In the bioinformatics
> > field, Snakemake [2] seems more popular. Other alternatives are CWL
> > [3], WDL [4], etc.
> >
> > Basically, you describe:
> >  - what is the inputs
> >  - what is the outputs
> >  - how to process the inputs to produce the outputs
> >
> > You can find examples there [*]. It uses the WISP syntax [5] but it
> > perfectly works with a Scheme-syntax if you prefer parenthesis. ;-)
> >
> > [*] https://guixwl.org/
> >
> >
> > However, you should be interested by this blog post [#] by Pjotr using
> > Guix and CWL and other niceties!
> >
> > [#] https://hpc.guix.info/blog/2019/01/creating-a-reproducible-workflow-with-cwl/
> >
> > AFAIK, Nextflow is not yet packaged in Guix. One direction is to
> > package it and then use the workflow described in Nextflow DSL in the
> > spirit of [#]. One other direction is to rewrite the workflow using
> > the GWL DSL. It depends a bit on what is your final aim.
> >
> >
> > Hope that helps.
> > simon
> >
> > [1] https://www.nextflow.io/
> > [2] https://snakemake.readthedocs.io/en/stable/
> > [3] https://www.commonwl.org/
> > [4] http://www.openwdl.org/
> > [5] https://srfi.schemers.org/srfi-119/srfi-119.html
> >