From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([209.51.188.92]:59529) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hXpfh-0004s6-8O for gwl-devel@gnu.org; Mon, 03 Jun 2019 12:18:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hXpff-0007wx-6b for gwl-devel@gnu.org; Mon, 03 Jun 2019 12:18:53 -0400 Received: from a2062.mx.srv.dfn.de ([194.95.232.172]:45755) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hXpfe-0007qm-8j for gwl-devel@gnu.org; Mon, 03 Jun 2019 12:18:51 -0400 References: <87a7f5l6e1.fsf@mdc-berlin.de> From: Ricardo Wurmus In-Reply-To: Date: Mon, 3 Jun 2019 18:18:40 +0200 Message-ID: <87o93eiqvz.fsf@mdc-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Subject: Re: Next steps for the GWL List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gwl-devel-bounces+kyle=kyleam.com@gnu.org Sender: "gwl-devel" To: zimoun Cc: gwl-devel@gnu.org Hi simon, >> * tighter integration with Guix features, e.g. to export a container >> image per process via =E2=80=9Cguix pack=E2=80=9D or to pack up the wh= ole workflow as >> a relocatable executable. > > Yes! Awesome. > Relocatable tarballs. Docker images. Singularity one. > > And maybe generate one pack (docker) per process and something to glue > together, e.g., > http://www.genouest.org/godocker/ Generating one =E2=80=9Ccontaine image=E2=80=9D per process is a desirable = goal (even though it seems a little wasteful). I don=E2=80=99t know how godocker fits= into this. The home page says; It is a batch computing/cluster management tool using Docker as execution/isolation system. It can be seen like Sun Grid Engine/Torque/... The software does not manage however itself the dispatch of the commands on the remote nodes. For this, it integrates with container management tools (Docker Swarm, Apache Mesos, ...) It acts as an additional layer above those tools on multiple user systems where users do not have Docker priviledges or knowledge. Can we directly support these container management tools? I=E2=80=99d like= to make GWL workflows very portable, so that there are only few runtime requirements. Depending on a cluster management tool to be configured would be counter to this goal. > Talking about ideas: > - what about the Content Adressable Store? This already exists, but I=E2=80=99m not sure it=E2=80=99s sufficient. > - what about a bridge with CWL? I=E2=80=99m open to this idea, but it would need to be well-defined. What = does it really mean? Generating CWL files from GWL workflows? That really shouldn=E2=80=99t be too hard. Anything else, however, is hard for me to imagine. -- Ricardo