From mboxrd@z Thu Jan 1 00:00:00 1970 References: From: Ricardo Wurmus Subject: Re: [GWL] (random) next steps? In-reply-to: Date: Sat, 15 Dec 2018 10:09:45 +0100 Message-ID: <874lbfxijq.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-devel" To: zimoun Cc: Guix Devel , gwl-devel@gnu.org List-ID: Hi simon, > Here, I would like to collect some discussions or ideas about the Guix > Workflow Language (GWL) and the next steps of this awesome tool! thanks for kicking off this discussion! > With wisp, the workflow description seems close to CWL, which is an argum= ent ;-) > Last time, I check, CWL files seems flat yaml-like files and they lack > programmable extension; as Snakemake provides with Python for example. > And GWL-wisp will have both: easy syntax and programmable stuff, > because it is still lisp under the hood. I=E2=80=99m working on updating the GWL manual to show a simple wispy examp= le. > 4. > Some GWL scripts are already there. > Could we centralize them to one repo? > Even if they are not clean. I mean something in this flavor: > https://github.com/common-workflow-language/workflows I only know of Roel=E2=80=99s ATACseq workflow[1], but we could add a few m= ore independent process definitions for simple tasks such as sequence alignment, trimming, etc. This could be a git subtree that includes an independent repository. [1]: https://github.com/UMCUGenetics/gwl-atacseq/ > 5. > I recently have discovered the ELisp package `s.el` via the blog post: > http://kitchingroup.cheme.cmu.edu/blog/2018/05/14/f-strings-in-emacs-lisp/ > or other said: > https://github.com/alphapapa/elexandria/blob/master/elexandria.el#L224 > > Does it appear to you a right path to write a "formater" in this > flavour instead of the `string-append` ? > I mean, e.g., > `(system ,(string-command "gzip ${data-inputs} -c > ${outputs}")) > instead of e.g., > `(system ,(string-append "gzip " data-inputs " -c > " outputs)) > > It seems more on the flavour of Snakemake. Scheme itself has (format #f "=E2=80=A6" foo bar) for string interpolation. With a little macro we could generate the right =E2=80=9Cformat=E2=80=9D in= vocation, so that the user could do something similar to what you suggested: (shell "gzip ${data-inputs} -c > ${outputs}") =E2=80=93> (system (format #f "gzip ~a -c > ~a" data-inputs outputs)) String concatenation is one possibility, but I hope we can do better than that. scsh offers special process forms that would allow us to do things like this: (shell (gzip ,data-inputs -c > ,outputs)) or (run (gzip ,data-inputs -c) (> 1 ,outputs)) Maybe we can take some inspiration from scsh. > 6. > The graph of dependencies between the processes/units/rules is written > by hand. What should be the best strategy to capture it ? By files "=C3=A0 > la" Snakemake ? Other ? The GWL currently does not use the input information provided by the user in the data-inputs field. For the content addressible store we will need to change this. The GWL will then be able of determining that data-inputs are in fact the outputs of other processes. > 7. > Does it appear to you a good idea to provide a command `guix workflow pac= k` ? > to produce an archive with the binaries or the commit hashes of the > channels, etc. This shouldn=E2=80=99t be difficult to implement as all the needed pieces already exist. > Last, the webpage [1] points to gwl-devel mailing list which seems broken. > Does the gwl-devel need to be activated and the GWL discussion will > append there or everything stay here to not scattered too much. > > [1] https://www.guixwl.org/community Hmm, looks like the mailing list exists, but has never been used. That=E2=80=99s why there is no archive. Let=E2=80=99s see if this email cr= eates this archive. -- Ricardo