From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:55151) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ibsn7-0006hZ-Cv for gwl-devel@gnu.org; Mon, 02 Dec 2019 15:59:34 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ibsn6-0000Hk-2C for gwl-devel@gnu.org; Mon, 02 Dec 2019 15:59:33 -0500 Received: from mail-qk1-x729.google.com ([2607:f8b0:4864:20::729]:36614) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ibsn5-0000GY-PH for gwl-devel@gnu.org; Mon, 02 Dec 2019 15:59:31 -0500 Received: by mail-qk1-x729.google.com with SMTP id v19so1118772qkv.3 for ; Mon, 02 Dec 2019 12:59:31 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: zimoun Date: Mon, 2 Dec 2019 21:59:20 +0100 Message-ID: Subject: Re: How do I support building a guix package over multiple machines in a cloud environment? Content-Type: text/plain; charset="UTF-8" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gwl-devel-bounces+kyle=kyleam.com@gnu.org Sender: "gwl-devel" To: Josh Marshall Cc: gwl-devel@gnu.org On Mon, 2 Dec 2019 at 21:03, Josh Marshall wrote: > > Looking at https://lists.gnu.org/archive/html/gwl-devel/2019-01/msg00034.html > the use case I'm looking at explicitly requires the input files to be > hashed and tracked manually, as if a package. Currently, how to track inputs/outputs is still a work in progress. As you can see, the last email of the mailing list was back on July. Since then, bit of life intervened and I do not have enough time to contribute/improve until next January. (I will not speak for Ricardo but I know he is currently a bit busy by real life. :-)) > The actual pipeline > doesn't change much if at all, but those large data files must be > tracked. Nextflow is the current fad pipeline, but it would be nice > to have some fully magical reproducible way to just re-use any DSL, as It is an hard topic... Back on January 2018, I was thinking to write CWL front end and the original author of GWL did this insighted answer [1]. [1] https://lists.gnu.org/archive/html/guix-devel/2018-01/msg00390.html Thank you to keep alive the interest in this project. :-) I am sure there is room in bioinformatics field for such tool. All the best, simon