From mboxrd@z Thu Jan 1 00:00:00 1970 From: zimoun Subject: Re: Guix Workflow Language ? Date: Thu, 25 Jan 2018 17:16:33 +0100 Message-ID: References: <874lnbqauw.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:37205) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eekCc-0004Dq-14 for guix-devel@gnu.org; Thu, 25 Jan 2018 11:16:39 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eekCb-0007SB-1U for guix-devel@gnu.org; Thu, 25 Jan 2018 11:16:37 -0500 In-Reply-To: <874lnbqauw.fsf@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Roel Janssen Cc: guix-devel@gnu.org Dear Roel, Thank you for your comments. I was imaging your point 2. And the softwares come from Guix. The added benefit was: a controlled and reproducible environment. In other words, the added benefit came from the GuixWorkflow (the engine of workflow), and not from the Language (lisp EDSL). But maybe it is a wrong way. >From my experience, the classical strategy of writing pipelines is to adapt an already existing workflow for one another particular question. We fetch bits here and there, do some ugly and dirty hacks to have some results; then depending on them, a cleaner pipeline is written (or not! :-) or other pieces are tested. Again from my experience, there is (at least) 3 issues: the number of tools to learn and know enough to be able to adapt; the bits/pieces already available; the environment/dependencies and how they are managed. In this context, since 'lispy' syntax is not mainstream (and will never be), it appears to me as a hard position. That's why I asked if a Guix-backend workflow engine for CWL specs is doable. Run CWL specs workflow on the top of the GWL engine. However, I got your point, I guess. You mean: it is a lot of work with unclear benefits over existing engines. Therefore, your point 1. reverses "my issue". Once the pipeline is well-established, write it with GWL! :-) Next, if it is possible to convert this GWL specs pipeline to CWL one [+ Docker] (with softwares coming from Guix), then we can enjoy the CWL-world engine capabilities. The benefit of that is from two sides: run the pipeline with different engines; and produce a clean docker image. So , instead of working on improving the GWL engine (adding features about efficiency, Grid, Amazon, etc.) which is a very tough task, the doable plan would be to add an "exporter". Right ? Another question, do you think it is doable to write "importers" ? I am not sure that the metaphor is good enough, but do you think it is a feasible goal from the existing GWL to go towards a kind of `Pandoc of workflows` ? also packing the softwares. And a start should be: - write a parser for (subset of) CWL yaml file and obtain the GWL representation of the workflow - write a exporter to CWL + Docker image What do you think ? About the parser, I haven't found yet an easy-to-use Guile lib for parsing YAML-like files. Any pointer ? Adapt some Racket ones ? Thank you for your insights. All the best, simon