From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from list by lists.gnu.org with archive (Exim 4.71)
	id 1goQ63-0005Wz-Tr
	for mharc-gwl-devel@gnu.org; Tue, 29 Jan 2019 04:54:23 -0500
Received: from eggs.gnu.org ([209.51.188.92]:45772)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <rekado@elephly.net>) id 1goQ61-0005Vh-OX
	for gwl-devel@gnu.org; Tue, 29 Jan 2019 04:54:22 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <rekado@elephly.net>) id 1goQ60-0003ta-UK
	for gwl-devel@gnu.org; Tue, 29 Jan 2019 04:54:21 -0500
Received: from sender-of-o51.zoho.com ([135.84.80.216]:21067)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
	(Exim 4.71) (envelope-from <rekado@elephly.net>) id 1goQ60-0003ro-DH
	for gwl-devel@gnu.org; Tue, 29 Jan 2019 04:54:20 -0500
References: <87bm40qta0.fsf@elephly.net>
From: Ricardo Wurmus <rekado@elephly.net>
In-reply-to: <87bm40qta0.fsf@elephly.net>
Date: Tue, 29 Jan 2019 10:38:53 +0100
Message-ID: <875zu7refm.fsf@elephly.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Subject: Re: support for containers
List-Id: <gwl-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/gwl-devel>,
	<mailto:gwl-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/gwl-devel/>
List-Post: <mailto:gwl-devel@gnu.org>
List-Help: <mailto:gwl-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/gwl-devel>,
	<mailto:gwl-devel-request@gnu.org?subject=subscribe>
To: gwl-devel@gnu.org


Ricardo Wurmus <rekado@elephly.net> writes:

> the GWL could already support execution in containers with this patch:
[=E2=80=A6]
> The directories to be mounted in the container are derived from the
> declared inputs and outputs.  The only problem is that inputs are
> read-only in this implementation.  I like it this way, actually, but it
> means that the extended example workflow won=E2=80=99t work as it tries to
> delete its inputs.
>
> Should data inputs be declared as (mutable-file =E2=80=A6) or (file =E2=
=80=A6) instead
> of being plain strings?

Taking a step back I think it is worth remembering that ultimately we=E2=80=
=99d
like to have output caching via an immutable data store.  I think
declaring files as mutable would be a mistake.  Garbage collection is
better than modifying output files.

Some thoughts on how the data store should work: it=E2=80=99s easy to add s=
tuff
to the store (we just hash the inputs leading up to the output,
excluding the output itself); but how do we make store items available
to a process?  When using containers we can bind mount the file to the
declared input location; when not using containers we need to link or
copy the file from

I also want to have finer control over inputs.  Only declared input
*files* should be available in the container, not whole *directories*.

--
Ricardo