Hi, I've been hacking around for the past weeks on a build system that uses Guix. The more I write it, the more I find similarities with what GWL already does/provides. So I'm thinking, one could easily do something like this for a simple Guile project: --8<---------------cut here---------------start------------->8--- process compile-guile-source (with dot-scm) package "guile" inputs : file dot-scm outputs : file : string-replace-substring dot-scm ".scm" ".go" # { guild compile --output {{outputs}} {{inputs}} } --8<---------------cut here---------------end--------------->8--- And this could also be done for other projects, e.g.: --8<---------------cut here---------------start------------->8--- process compile-c-source (with dot-c) package "gcc-toolchain" inputs : file dot-c outputs : file : string-replace-substring dot-c ".c" ".o" # { gcc -c -o {{outputs}} {{inputs}} } --8<---------------cut here---------------end--------------->8--- I know that GWL focuses on scientific workflows, but I think that it can go beyond its original goal. Thoughts? -- Olivier Dion oldiob.dev
Hi Olivier, > So I'm thinking, one could easily do something like this for a simple > Guile project: > > process compile-guile-source (with dot-scm) > package "guile" > inputs : file dot-scm > outputs : file : string-replace-substring dot-scm ".scm" ".go" > # { > guild compile --output {{outputs}} {{inputs}} > } > > > And this could also be done for other projects, e.g.: > > process compile-c-source (with dot-c) > package "gcc-toolchain" > inputs : file dot-c > outputs : file : string-replace-substring dot-c ".c" ".o" > # { > gcc -c -o {{outputs}} {{inputs}} > } It is not entirely surprising to me that the GWL can express this, because it has really simple abstractions: that of a process and that of a workflow consisting of processes. What I do find a little surprising, though, is that there is an apparent need for declaring processes like this. The Scheme Shell (scsh), for example, takes on a similar problem, but the abstraction is different; less focused on inputs and outputs and instead more geared to integrating Shell script idioms with plain Scheme. Personally, I find the GWL too complex for mere process abstractions and lacking in features that are common in Shell scripts. Notably, it punts on improving the syntax of the command; it shrugs and just uses a string to express the whole command (with a little variable interpolation). Perhaps there is space for a different tool that takes lessons from the GWL and Scsh alike, with a focus on command composition and shell abstractions. Perhaps that tool already exists and is called Metabash: https://github.com/artyom-poptsov/metabash :) -- Ricardo
Hi, I am late to the party. :-) On Mon, 06 Jun 2022 at 10:50, Ricardo Wurmus <rekado@elephly.net> wrote: > It is not entirely surprising to me that the GWL can express this, > because it has really simple abstractions: that of a process and that of > a workflow consisting of processes. [...] > Perhaps there is space for a different tool that takes lessons from the > GWL and Scsh alike, with a focus on command composition and shell > abstractions. Perhaps that tool already exists and is called Metabash: > > https://github.com/artyom-poptsov/metabash From my understanding, metabash allows to remotely run processes, i.e., distribute the pipeline. Somehow, it could be see as an extension of Scsh. However, a pipeline is a linear sequence of processes. When a workflow is a DAG of processes. Therefore, it would appear difficult to me to be able to express a build-system using only pipelines. Last, it appears to me expected that GWL could be considered as a build-system. A scientific workflow system [1] (as GWL) is just a specialized implementation to deal with a graph of dependencies. Software folks speak about the venerable Make as build automation workflow, while bioinfo folks speak about a specific Python implementation SnakeMake as data analysis workflow. Just the same concepts but viewed by different communities. :-) If I might, an interesting analysis of different strategies for dealing with the graph of dependencies is done in the paper «Build systems à la carte» [2]. It presents the various abstractions using Haskell notations. 1: <https://en.wikipedia.org/wiki/Scientific_workflow_system> 2: <https://doi.org/10.1017/S0956796820000088> Cheers, simon