From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1glb2f-0006s1-RF for mharc-gwl-devel@gnu.org; Mon, 21 Jan 2019 09:59:13 -0500 Received: from eggs.gnu.org ([209.51.188.92]:38792) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1glb2e-0006qh-Cz for gwl-devel@gnu.org; Mon, 21 Jan 2019 09:59:13 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1glb2d-0007cW-6C for gwl-devel@gnu.org; Mon, 21 Jan 2019 09:59:12 -0500 Received: from sender-of-o53.zoho.com ([135.84.80.218]:21774) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1glb2c-0007al-Kc for gwl-devel@gnu.org; Mon, 21 Jan 2019 09:59:11 -0500 References: <87bm4df2ld.fsf@elephly.net> <878szgg9bi.fsf@elephly.net> From: Ricardo Wurmus In-reply-to: <878szgg9bi.fsf@elephly.net> Date: Mon, 21 Jan 2019 15:43:42 +0100 Message-ID: <87a7judqap.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: =?UTF-8?B?UmU6IG1lcmdpbmcg4oCccHJvY2Vzc2Vz4oCdIGFuZCDigJxyZXN0?= =?UTF-8?B?cmljdGlvbnPigJ0=?= List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: zimoun Cc: gwl-devel@gnu.org Ricardo Wurmus writes: > Hi simon, > >> I am not clear yet about how to manage the inputs/outputs >> (fixed in the definition of the process or fixed in the workflow) > [=E2=80=A6] >> I am still failing to write a macro that inplements my "view": >> - write the graph >> - collect the inputs/outputs > > This is interesting and it might be a solution to this conundrum. If > the processes can declare their inputs without refering to other > processes then we have a solution: the graph can be built from the > inputs and outputs of the provided processes without having to specify > any dependencies manually. > > We need a procedure that takes any number of processes as inputs and > matches inputs with outputs to generate an adjacency list of processes. > This shouldn=E2=80=99t be difficult. This works now: --8<---------------cut here---------------start------------->8--- define-module : simple-wisp use-modules gwl workflows gwl processes gwl sugar process: hello package-inputs list "hello" synopsis "Run hello" procedure '(system* "hello") process: python-test package-inputs list "python2" data-inputs list "sample.bam" "hg38.fa" "abc" synopsis "Run Python" description . "Run Python and demonstrate that it can access process information vi= a environment variables." procedure ## python import os def hello(): print "hello from python 2" print os.environ["_GWL_PROCESS_DATA_INPUTS"] print os.environ["_GWL_PROCESS_NAME"] hello() ## process: bash-test package-inputs : list "bash" synopsis "Run Bash" description . "Run Bash and demonstrate that it can access process information via = environment variables." procedure ## /bin/bash -c echo "${_GWL_PROCESS_DATA_INPUTS}" echo "${_GWL_PROCESS_NAME}" ## workflow: simple-wisp processes graph python-test -> hello bash-test -> hello --8<---------------cut here---------------end--------------->8--- With a little more macrology the =E2=80=9Cgraph=E2=80=9D call could disappe= ar, but that would make it impossible to specify an adjacency list as a simple alist, which I=E2=80=99d like to keep supporting as it is useful when combining workflows. -- Ricardo