From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1goQ63-0005Wz-Tr for mharc-gwl-devel@gnu.org; Tue, 29 Jan 2019 04:54:23 -0500 Received: from eggs.gnu.org ([209.51.188.92]:45772) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1goQ61-0005Vh-OX for gwl-devel@gnu.org; Tue, 29 Jan 2019 04:54:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1goQ60-0003ta-UK for gwl-devel@gnu.org; Tue, 29 Jan 2019 04:54:21 -0500 Received: from sender-of-o51.zoho.com ([135.84.80.216]:21067) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1goQ60-0003ro-DH for gwl-devel@gnu.org; Tue, 29 Jan 2019 04:54:20 -0500 References: <87bm40qta0.fsf@elephly.net> From: Ricardo Wurmus In-reply-to: <87bm40qta0.fsf@elephly.net> Date: Tue, 29 Jan 2019 10:38:53 +0100 Message-ID: <875zu7refm.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: support for containers List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: gwl-devel@gnu.org Ricardo Wurmus writes: > the GWL could already support execution in containers with this patch: [=E2=80=A6] > The directories to be mounted in the container are derived from the > declared inputs and outputs. The only problem is that inputs are > read-only in this implementation. I like it this way, actually, but it > means that the extended example workflow won=E2=80=99t work as it tries to > delete its inputs. > > Should data inputs be declared as (mutable-file =E2=80=A6) or (file =E2= =80=A6) instead > of being plain strings? Taking a step back I think it is worth remembering that ultimately we=E2=80= =99d like to have output caching via an immutable data store. I think declaring files as mutable would be a mistake. Garbage collection is better than modifying output files. Some thoughts on how the data store should work: it=E2=80=99s easy to add s= tuff to the store (we just hash the inputs leading up to the output, excluding the output itself); but how do we make store items available to a process? When using containers we can bind mount the file to the declared input location; when not using containers we need to link or copy the file from I also want to have finer control over inputs. Only declared input *files* should be available in the container, not whole *directories*. -- Ricardo