Re: merging “processes” and “restrictions”

[Ricardo Wurmus <rekado@elephly.net>]

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi simon,
>
>> I am not clear yet about how to manage the inputs/outputs
>> (fixed in the definition of the process or fixed in the workflow)
> […]
>> I am still failing to write a macro that inplements my "view":
>>  - write the graph
>>  - collect the inputs/outputs
>
> This is interesting and it might be a solution to this conundrum.  If
> the processes can declare their inputs without refering to other
> processes then we have a solution: the graph can be built from the
> inputs and outputs of the provided processes without having to specify
> any dependencies manually.
>
> We need a procedure that takes any number of processes as inputs and
> matches inputs with outputs to generate an adjacency list of processes.
> This shouldn’t be difficult.

This works now:

--8<---------------cut here---------------start------------->8---
define-module : simple-wisp

use-modules
  gwl workflows
  gwl processes
  gwl sugar

process: hello
  package-inputs
    list "hello"
  synopsis "Run hello"
  procedure '(system* "hello")

process: python-test
  package-inputs
    list "python2"
  data-inputs
    list "sample.bam" "hg38.fa" "abc"
  synopsis "Run Python"
  description
    . "Run Python and demonstrate that it can access process information via environment variables."
  procedure ## python
import os

def hello():
  print "hello from python 2"
  print os.environ["_GWL_PROCESS_DATA_INPUTS"]
  print os.environ["_GWL_PROCESS_NAME"]

hello()
##

process: bash-test
  package-inputs : list "bash"
  synopsis "Run Bash"
  description
    . "Run Bash and demonstrate that it can access process information via environment variables."
  procedure ## /bin/bash -c
echo "${_GWL_PROCESS_DATA_INPUTS}"
echo "${_GWL_PROCESS_NAME}"
##

workflow: simple-wisp
  processes
    graph
      python-test -> hello
      bash-test   -> hello
--8<---------------cut here---------------end--------------->8---

With a little more macrology the “graph” call could disappear, but that
would make it impossible to specify an adjacency list as a simple alist,
which I’d like to keep supporting as it is useful when combining
workflows.

--
Ricardo