all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: zimoun <zimon.toutoune@gmail.com>
To: Ricardo Wurmus <rekado@elephly.net>
Cc: gwl-devel@gnu.org
Subject: Re: support for containers
Date: Tue, 29 Jan 2019 11:39:24 +0100	[thread overview]
Message-ID: <CAJ3okZ2TU+mWt72CYOJ9cMA6kR1muXDTG86ejyQ5Tyo5DQvHGg@mail.gmail.com> (raw)
In-Reply-To: <875zu7refm.fsf@elephly.net>

Hi Ricardo,

On Tue, 29 Jan 2019 at 10:54, Ricardo Wurmus <rekado@elephly.net> wrote:
> Ricardo Wurmus <rekado@elephly.net> writes:
>
> > the GWL could already support execution in containers with this patch:
> […]
> > The directories to be mounted in the container are derived from the
> > declared inputs and outputs.  The only problem is that inputs are
> > read-only in this implementation.  I like it this way, actually, but it
> > means that the extended example workflow won’t work as it tries to
> > delete its inputs.
> >
> > Should data inputs be declared as (mutable-file …) or (file …) instead
> > of being plain strings?
>
> Taking a step back I think it is worth remembering that ultimately we’d
> like to have output caching via an immutable data store.  I think
> declaring files as mutable would be a mistake.  Garbage collection is
> better than modifying output files.

I agree that Garbage Collection is a better design, from my opinion.
However, as an user I would like to garbage collect only the
inputs/outputs of a specific workflow. Not all inside the store.


> Some thoughts on how the data store should work: it’s easy to add stuff
> to the store (we just hash the inputs leading up to the output,
> excluding the output itself); but how do we make store items available
> to a process?  When using containers we can bind mount the file to the
> declared input location; when not using containers we need to link or
> copy the file from

By store, do you the Guix store? or another store? as GWL store?

By inputs, do you mean data-inputs and package-inputs?


I do not know if below makes sense.
What happens if a kind of record managing the data-inputs is added?
I mean, there are: package, process, workflow and we add data.
Then the data is manage as other symbols, with define-public, etc.
However, these data need to live a special store, not in the store of packages.

>
> I also want to have finer control over inputs.  Only declared input
> *files* should be available in the container, not whole *directories*.

Hum? I make sense.
Especially for references used by aligners.
Even if the size of the container will significantly growth. :-)


All the best,
simon

  reply	other threads:[~2019-01-29 10:39 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-01-28 23:03 support for containers Ricardo Wurmus
2019-01-29  9:38 ` Ricardo Wurmus
2019-01-29 10:39   ` zimoun [this message]
2019-01-29 11:46     ` Ricardo Wurmus
2019-01-29 14:29       ` zimoun
2019-01-29 17:19         ` Ricardo Wurmus
2019-01-29 21:52           ` zimoun
2019-01-29 23:16             ` Ricardo Wurmus
2019-01-30 10:17               ` zimoun
2019-01-30 12:46                 ` Ricardo Wurmus
2019-01-29 10:22 ` zimoun
2019-01-29 11:44   ` Ricardo Wurmus

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAJ3okZ2TU+mWt72CYOJ9cMA6kR1muXDTG86ejyQ5Tyo5DQvHGg@mail.gmail.com \
    --to=zimon.toutoune@gmail.com \
    --cc=gwl-devel@gnu.org \
    --cc=rekado@elephly.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.