From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:43602) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ibs0z-0007gK-RZ for gwl-devel@gnu.org; Mon, 02 Dec 2019 15:09:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ibs0y-0006JO-G3 for gwl-devel@gnu.org; Mon, 02 Dec 2019 15:09:49 -0500 Received: from mail.thebird.nl ([94.142.245.5]:54476) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ibs0y-0006J9-9m for gwl-devel@gnu.org; Mon, 02 Dec 2019 15:09:48 -0500 Date: Mon, 2 Dec 2019 14:02:36 -0600 From: pjotr.public12@thebird.nl Subject: Re: How do I support building a guix package over multiple machines in a cloud environment? Message-ID: <20191202200236.ghnn3cbirecgjfnj@thebird.nl> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gwl-devel-bounces+kyle=kyleam.com@gnu.org Sender: "gwl-devel" To: zimoun Cc: gwl-devel@gnu.org, Josh Marshall Yeah, if you read my BLOG below you can also simply split packaging (read deployment) from the pipeline (runner). It would already be worthwile to show how Guix can be mixed with Nextflow and how you could use the same method to create reproducible containers. That is a real win! In the next step try to package Nextflow itself. Or, alternatively, port pipelines to GWL. Mind, GWL needs development. Nextflow is an amazing (Cloud) tool that can benefit from reproducible deployment. Pj. On Mon, Dec 02, 2019 at 08:00:07PM +0100, zimoun wrote: > Hi (again) Josh :-) > > On Mon, 2 Dec 2019 at 19:38, Josh Marshall > wrote: > > > He uses it as a bioinformatics workflow to generate some analysis. It > > GWL should work for this use case. o/ > > > Is this kind of use case supported? If so, how so? Is nextflow not > > practical to keep? Please, someone catch me up here so I can start to > > write code to help him out. If this goes well, my company could > > integrate for gwl/guix in our work, which would be amazing. > > Netxflow [1] is a Domain Specific Language (DSL): you write "rules" > and how these rules are combined together. In the bioinformatics > field, Snakemake [2] seems more popular. Other alternatives are CWL > [3], WDL [4], etc. > > Basically, you describe: > - what is the inputs > - what is the outputs > - how to process the inputs to produce the outputs > > You can find examples there [*]. It uses the WISP syntax [5] but it > perfectly works with a Scheme-syntax if you prefer parenthesis. ;-) > > [*] https://guixwl.org/ > > > However, you should be interested by this blog post [#] by Pjotr using > Guix and CWL and other niceties! > > [#] https://hpc.guix.info/blog/2019/01/creating-a-reproducible-workflow-with-cwl/ > > AFAIK, Nextflow is not yet packaged in Guix. One direction is to > package it and then use the workflow described in Nextflow DSL in the > spirit of [#]. One other direction is to rewrite the workflow using > the GWL DSL. It depends a bit on what is your final aim. > > > Hope that helps. > simon > > [1] https://www.nextflow.io/ > [2] https://snakemake.readthedocs.io/en/stable/ > [3] https://www.commonwl.org/ > [4] http://www.openwdl.org/ > [5] https://srfi.schemers.org/srfi-119/srfi-119.html >