Re: Next steps for the GWL

[Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>]

Kyle Meyer <kyle@kyleam.com> writes:

> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:
>
>>> One of the things I'd love to do
>>> with GWL is to make it play well with git-annex, something that would
>>> almost certainly be too specific for GWL itself.
>
> [...]
>
>> I wonder what the protocol should look like.  Should a workflow
>> explicitly request a “git annex” file or should it be up to the person
>> running the workflow, i.e. when “git annex” has been configured to be
>> the cache backend it would simply look up the declared input/output
>> files there.
>
> The latter is what I had in mind.  One benefit I see of leaving it up to
> the configured backend is that it makes it easier to share a workflow
> with someone that doesn't have/want the requirements for a particular
> backend.

I agree, this would be convenient.
I’m not familiar with git annex.  Would you be interested in drafting
this feature, e.g. by writing a patch or specifying how it should work
in detail?

>>>> * add support for executing processes in isolated environments
>>>>   (containers) — this requires a better understanding of process inputs.
>
> [...]
>
>> This means that it can map file systems into the container and then run
>> the process expression in that environment.
>>
>> One thing I’m not happy about is that I can only mount directories, and
>> not individual files that have been declared as inputs.  I’d like to
>> have more fine-grained access.
>
> Right, limiting to the declared files makes sense.
>
> With `docker run', you can give files to -v:
>
>   % ls /tmp/ | wc -l
>   121
>   % file /tmp/scratch
>   /tmp/scratch: ASCII text
>   % docker run -it --rm -v /tmp/scratch:/tmp/scratch busybox ls /tmp
>   scratch
>
> It looks like using files works for `guix environment' too, which makes
> me think that call-with-container can handle receiving files in MOUNT.
>
>   % guix environment -C --ad-hoc coreutils -- ls /tmp | wc -l
>   0
>   % guix environment -C --expose=/tmp/scratch=/tmp/scratch --ad-hoc coreutils -- ls /tmp
>   scratch

Oh, neat.  I’ll give this a try later.  Thanks!

-- 
Ricardo