From mboxrd@z Thu Jan 1 00:00:00 1970 References: <87a7f5l6e1.fsf@mdc-berlin.de> <8736knieo3.fsf@kyleam.com> <87v9xjja6b.fsf@mdc-berlin.de> <87o93apx9w.fsf@kyleam.com> From: Ricardo Wurmus Subject: Re: Next steps for the GWL In-Reply-To: <87o93apx9w.fsf@kyleam.com> Date: Thu, 6 Jun 2019 22:29:08 +0200 Message-ID: <87h892jw4r.fsf@mdc-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable To: Kyle Meyer Cc: gwl-devel@gnu.org List-ID: Kyle Meyer writes: > Ricardo Wurmus writes: > >>> One of the things I'd love to do >>> with GWL is to make it play well with git-annex, something that would >>> almost certainly be too specific for GWL itself. > > [...] > >> I wonder what the protocol should look like. Should a workflow >> explicitly request a =E2=80=9Cgit annex=E2=80=9D file or should it be up= to the person >> running the workflow, i.e. when =E2=80=9Cgit annex=E2=80=9D has been con= figured to be >> the cache backend it would simply look up the declared input/output >> files there. > > The latter is what I had in mind. One benefit I see of leaving it up to > the configured backend is that it makes it easier to share a workflow > with someone that doesn't have/want the requirements for a particular > backend. I agree, this would be convenient. I=E2=80=99m not familiar with git annex. Would you be interested in drafti= ng this feature, e.g. by writing a patch or specifying how it should work in detail? >>>> * add support for executing processes in isolated environments >>>> (containers) =E2=80=94 this requires a better understanding of proce= ss inputs. > > [...] > >> This means that it can map file systems into the container and then run >> the process expression in that environment. >> >> One thing I=E2=80=99m not happy about is that I can only mount directori= es, and >> not individual files that have been declared as inputs. I=E2=80=99d lik= e to >> have more fine-grained access. > > Right, limiting to the declared files makes sense. > > With `docker run', you can give files to -v: > > % ls /tmp/ | wc -l > 121 > % file /tmp/scratch > /tmp/scratch: ASCII text > % docker run -it --rm -v /tmp/scratch:/tmp/scratch busybox ls /tmp > scratch > > It looks like using files works for `guix environment' too, which makes > me think that call-with-container can handle receiving files in MOUNT. > > % guix environment -C --ad-hoc coreutils -- ls /tmp | wc -l > 0 > % guix environment -C --expose=3D/tmp/scratch=3D/tmp/scratch --ad-hoc c= oreutils -- ls /tmp > scratch Oh, neat. I=E2=80=99ll give this a try later. Thanks! --=20 Ricardo