From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pjotr Prins Subject: Re: Guix Workflow Language ? Date: Fri, 26 Jan 2018 14:05:20 +0100 Message-ID: <20180126130520.GB15888@thebird.nl> References: <874lnbqauw.fsf@gnu.org> <87tvv9lhmz.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:41106) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ef3kQ-0008Lh-29 for guix-devel@gnu.org; Fri, 26 Jan 2018 08:08:51 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ef3kL-0002LW-2v for guix-devel@gnu.org; Fri, 26 Jan 2018 08:08:50 -0500 Content-Disposition: inline In-Reply-To: <87tvv9lhmz.fsf@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Roel Janssen Cc: guix-devel@gnu.org On Thu, Jan 25, 2018 at 11:04:36PM +0100, Roel Janssen wrote: > I get that point. Maybe it's then a better idea to write the workflow > in CWL (like you would do), and use Guix to generate Docker containers. > > Then you do get the benefit of Guix's strong reproducibility and > composability forscientific software, plus you get to keep writing the > workflow in CWL. :-) I think the CWL is not particularly great, mostly because its design is too open ended (they include a Javascript interpreter - so it is YAML + JS) and 'complex'. Nor does it help/enforce analysis to be deterministic. CWL has community backing. But it could well implode on itself. So many people are working on it (nominally) for years, and so little has it got to show for. The promise is truly shared pipelines - and, so far, it has not happened. But maybe by sheer grit they will get there. I know people who are working hard to make it happen. I think that if you are not drinking the CWL cool-aid, GWL is a great alternative. But it needs LISP and it may need a bit more development to make it a smooth experience. If more people help out I am sure we can get there. What would be a great pipeline that has general interest? How about using GWL on the build farm, for example? I don't think the way forward is adding CWL-style YAML support to GWL. I think the way forward is to be able to run CWL workflows inside GWL, i.e., a CWL runner is treated as a single step inside GWL. That way we get the best of both worlds. Treat CWL as a piece of software on its own and provision the Guix container to match. Interestingly, this is pretty much possible today. (Note that there will be some complexity in error handling because both GWL and CWL will submit jobs to a cluster environment). Pj.