From: Kyle Meyer <kyle@kyleam.com>
To: zimoun <zimon.toutoune@gmail.com>
Cc: Ricardo Wurmus <rekado@elephly.net>, gwl-devel@gnu.org
Subject: Re: [PATCH] workflow: Consider unspecified free inputs when checking cache.
Date: Tue, 25 Jun 2019 21:31:07 -0400 [thread overview]
Message-ID: <87ef3hceuc.fsf@kyleam.com> (raw)
In-Reply-To: <CAJ3okZ3tLoq-YcXoMvpjtRQTKV+W0DVDeHAH0h8-kChQ7fkjWQ@mail.gmail.com>
Hi simon,
zimoun <zimon.toutoune@gmail.com> writes:
> Hi,
>
> On Tue, 25 Jun 2019 at 06:30, Kyle Meyer <kyle@kyleam.com> wrote:
>>
>> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> > I’m not sure if we should keep picking
>> > inputs from the environment silently and by default, but your patch is
>> > anyway more correct than what we had before.
>>
>> Hmm, for my use case, taking free inputs from the file system based on
>> the current directory is the only method that I'm actually interested in
>> (i.e. I don't see myself having any use for --input). Perhaps my
>> thinking is too shaped by make/snakemake, and I don't fully grasp the
>> approach GWL is trying to take.
>
> I am not sure to fully understand the issue and all the recent changes.
No need to understand the patch :] The quoted discussion is pretty
tangential to the issue resolved by the patch. It's only related in
that it's about unspecified free inputs.
> One idea of GWL is to have a functional workflow: the
> multi-composition of functions/processes. And free inputs
> are--say--the argument of this function. Therefore, if you have many
> samples and you need to apply the same workflow, then you just apply
> the function to each sample with --input. I mean it is my
> understanding of the approach. Maybe I have wrong...
Thanks, that matches my understanding.
To expand on my original comment that --input doesn't fit my desired use
case: I want the workflow to be fully specified. For me, this
translates to
(1) all scripts that aren't included in Guix packages are tracked in a
Git repository
(2) software dependencies are completely specified via a manifest and
Guix inferiors
(3) all input data files are tracked in the repository (via git-annex)
(4) the workflow itself is tracked in the repository
(5) the workflow unambiguously specifies how to generate each output.
I don't want to have to document that "to generate THIS, you
should run `guix workflow --input=THIS=THAT'". I want to make a
blanket statement that "you can run `<workflow cmd> TARGET' to
generate any desired output, which will do whatever is needed to
make that happen". (I know this isn't currently possible with
GWL.)
Of course none of that is to say above is _the_ way to do it; I'm just
elaborating on what my use case and focus is.
next prev parent reply other threads:[~2019-06-26 1:31 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-13 3:48 [PATCH] workflow: Consider unspecified free inputs when checking cache Kyle Meyer
2019-06-24 13:54 ` Ricardo Wurmus
2019-06-25 4:30 ` Kyle Meyer
2019-06-25 17:33 ` zimoun
2019-06-25 18:33 ` Ricardo Wurmus
2019-06-26 1:31 ` Kyle Meyer [this message]
2019-06-26 7:05 ` Ricardo Wurmus
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87ef3hceuc.fsf@kyleam.com \
--to=kyle@kyleam.com \
--cc=gwl-devel@gnu.org \
--cc=rekado@elephly.net \
--cc=zimon.toutoune@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.