unofficial mirror of gwl-devel@gnu.org
 help / color / mirror / Atom feed
* Preparing for a new release
@ 2020-02-08 12:39 Ricardo Wurmus
  2020-02-08 20:38 ` Kyle Meyer
  0 siblings, 1 reply; 14+ messages in thread
From: Ricardo Wurmus @ 2020-02-08 12:39 UTC (permalink / raw)
  To: gwl-devel

Hi,

I’ve pushed a bunch of the previously discussed improvements to the
repository and also added more documentation.  Please also note the
TODO.org file, which is for ideas and plans that may not necessarily be
flully fleshed out.

I’ll leave the load-path + inferior magic for a later release.  This is
hairy stuff and I’d rather have us mull this over for a little while
longer before committing to an implementation.

Your comments on the code and the documentation, and also your
experience reports are very welcome!

--
Ricardo

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-08 12:39 Preparing for a new release Ricardo Wurmus
@ 2020-02-08 20:38 ` Kyle Meyer
  2020-02-08 21:50   ` Ricardo Wurmus
  0 siblings, 1 reply; 14+ messages in thread
From: Kyle Meyer @ 2020-02-08 20:38 UTC (permalink / raw)
  To: Ricardo Wurmus, gwl-devel

Ricardo Wurmus <rekado@elephly.net> writes:

> I’ve pushed a bunch of the previously discussed improvements to the
> repository and also added more documentation.  Please also note the
> TODO.org file, which is for ideas and plans that may not necessarily be
> flully fleshed out.

Thanks!

> I’ll leave the load-path + inferior magic for a later release.  This is
> hairy stuff and I’d rather have us mull this over for a little while
> longer before committing to an implementation.

Sounds like a good idea.

> Your comments on the code and the documentation, and also your
> experience reports are very welcome!

The added documentation looks very nice.  Here are a few minor
suggestions.

-- >8 --
Subject: [PATCH] doc: Touch up recent changes to the manual.

* doc/gwl.texi: Revise recently added text.
---
 doc/gwl.texi | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/doc/gwl.texi b/doc/gwl.texi
index 62eb3bc..e1ebdd2 100644
--- a/doc/gwl.texi
+++ b/doc/gwl.texi
@@ -93,7 +93,7 @@ Installation
 @chapter Installation
 
 There really is no point in using the GWL without Guix.  If you
-already have a Guix installation you can install the GWL with
+already have a Guix installation, you can install the GWL with
 @code{guix install gwl}.
 
 The Guix Workflow Language uses the GNU build system.  To install it
@@ -112,7 +112,7 @@ Installation
 then perform the above steps.
 
 Note that in order for Guix to learn about the ``workflow''
-sub-command provided by the GWL the Guile module @code{(guix scripts
+sub-command provided by the GWL, the Guile module @code{(guix scripts
 workflow)} must be found in a directory on the @code{GUILE_LOAD_PATH}.
 
 @c *********************************************************************
@@ -167,7 +167,7 @@ Defining a Process
      ' display "hello"
 @end example
 
-This creates a process with name ``hello'', which will print the
+This creates a process with the name ``hello'', which will print the
 string @code{"hello"} once the process is executed.  The
 @code{procedure} field holds the Scheme code that does all the work of
 saying ``hello''.  We will talk about the @code{procedure} field a
@@ -211,7 +211,7 @@ @code{process} Fields
 @item name
 The readable name of the process as a string.  This is used for
 display purposes and to select processes by name.  When the
-@code{process} constructor is used the @code{name} field need not be
+@code{process} constructor is used, the @code{name} field need not be
 provided explicitly.
 
 @cindex version, process field
@@ -234,7 +234,7 @@ @code{process} Fields
 available when executing the process.  Packages can either be Guix
 package specifications --- such as the string @code{"guile@@3.0"} for
 Guile version 3.0 --- or package variable names.  When using package
-variable names you need to make sure to import the appropriate Guix
+variable names, you need to make sure to import the appropriate Guix
 module at the top of your workflow file, e.g. @code{(import (gnu
 packages guile))} for the variable @code{guile}.
 
@@ -343,7 +343,7 @@ @code{process} Fields
 @item run-time
 This field is used to specify run-time resource estimates, such as the
 memory requirement of the process or the maximum time it should run.
-This is especially useful when submitting jobs to a HPC cluster
+This is especially useful when submitting jobs to an HPC cluster
 scheduler such as Grid Engine, as these schedulers may give higher
 priority to jobs that declare a short run time.
 
@@ -365,7 +365,7 @@ @code{process} Fields
   outputs "haiku.txt"
   synopsis "Write a haiku to a file"
   description
-    . "This process writes a haiku by Gary Hotham\
+    . "This process writes a haiku by Gary Hotham \
 to the file \"haiku.txt\"."
   procedure
     ` with-output-to-file ,outputs
@@ -573,14 +573,14 @@ Defining a Workflow
     . something-else
 @end example
 
-This defines a workflow with name ``do-stuff'', binds it to a variable
+This defines a workflow with the name ``do-stuff'', binds it to a variable
 @code{do-stuff}, and declares that it consists of the three processes
 @code{this}, @code{that}, and @code{something-else}.  All of these
 processes will be run at the same time.  This may not be what you want
 when the processes depend on each other.
 
 @cindex auto-connect, workflow order
-If the processes all declare inputs and outputs the GWL can connect
+If the processes all declare inputs and outputs, the GWL can connect
 the processes and ensure that only independent processes are run
 simultaneously.  Use the @code{auto-connect} procedure on your
 processes:

base-commit: 86a19d120cf6b90bab2a24a9cd3a7793395c3222
-- 
2.25.0

^ permalink raw reply related	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-08 20:38 ` Kyle Meyer
@ 2020-02-08 21:50   ` Ricardo Wurmus
  2020-02-09 13:00     ` Ricardo Wurmus
  0 siblings, 1 reply; 14+ messages in thread
From: Ricardo Wurmus @ 2020-02-08 21:50 UTC (permalink / raw)
  To: Kyle Meyer; +Cc: gwl-devel


Hi Kyle,

>> Your comments on the code and the documentation, and also your
>> experience reports are very welcome!
>
> The added documentation looks very nice.  Here are a few minor
> suggestions.

Applied, thank you!

I just realized that the manual does not say at all how to run a
workflow or how to specify inputs…  I’ll add another section in the
coming days.

--
Ricardo

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-08 21:50   ` Ricardo Wurmus
@ 2020-02-09 13:00     ` Ricardo Wurmus
  2020-02-09 23:33       ` zimoun
  0 siblings, 1 reply; 14+ messages in thread
From: Ricardo Wurmus @ 2020-02-09 13:00 UTC (permalink / raw)
  To: gwl-devel

While playing with a real-world workflow I found a few problems:

* inputs and outputs are not validated

  When a process declares that it produces an output, but then doesn’t do
  that, the next process will fail with a nasty error message.  This is
  especially nasty when using containerization as the error is about
  failing to map the input into the container.

  Processes should automatically validate their inputs and outputs.
  Since inputs and outputs could technically be something other than
  files I’m not sure exactly how to do this.

  @Roel: can you give an example of inputs / outputs that are not files?
  I remember that you suggested that inputs might be database queries,
  for example.  I wonder if we should mark inputs and outputs with
  types, so that the GWL can know if something is supposed to be a file
  or something else.  …just how would “something else” be used in a
  process?

* The --output option has no effect

  I think the “--output” option should cause all generated files to end
  up somewhere in the given directory.  I wonder if this should affect
  *all* generated files or just the final output.  If all outputs should
  be affected then all *inputs* must be adjusted as well.  Maybe
  “--output” is the wrong name.  Should it be “--prefix” instead?

* It’s not possible to select more than one tagged item

  In my test workflow I’m generating a bunch of inputs by mapping over
  an argument list.  Now the problem is that I can’t select all of these
  inputs easily in a code snippet.  With the syntax we have I can only
  select the first item following a tag.

  To address this I’ve extended the accessor syntax, so this works now:

--8<---------------cut here---------------start------------->8---
process frobnicate
  packages "frobnicator"
  inputs
    . genome: "hg19.fa"
    . samples: "a" "b" "c"
  outputs
    . "result"
  # {
    frobnicate -g {{inputs:genome}} --files {{inputs::samples}} > {{outputs}}
  }
--8<---------------cut here---------------end--------------->8---

  Note how {{inputs::samples}} is substituted with “a b c”.  With just a
  single colon it would be just “a”.  Single colon = single item; double
  colon = more than one item.

* Containerization and directories

  Containers for processes that create output files in directories that
  don’t exist yet cannot be created.  That’s because the “containerize”
  procedure tries to map directories of input and output files into the
  container — and the output directory doesn’t exist yet.

  How should this be handled?  We could ignore non-existing output
  directories when creating containers, I suppose.  I think that’s the
  best option, because we can’t just create them lest we break
  procedures that don’t deal well with existing directories.


--
Ricardo

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-09 13:00     ` Ricardo Wurmus
@ 2020-02-09 23:33       ` zimoun
  2020-02-10  1:25         ` Kyle Meyer
  2020-02-10  6:31         ` Ricardo Wurmus
  0 siblings, 2 replies; 14+ messages in thread
From: zimoun @ 2020-02-09 23:33 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: gwl-devel

Hi,

Thank you for pushing forward.


On Sun, 9 Feb 2020 at 14:01, Ricardo Wurmus <rekado@elephly.net> wrote:

> * inputs and outputs are not validated
>
>   When a process declares that it produces an output, but then doesn’t do
>   that, the next process will fail with a nasty error message.  This is
>   especially nasty when using containerization as the error is about
>   failing to map the input into the container.
>
>   Processes should automatically validate their inputs and outputs.
>   Since inputs and outputs could technically be something other than
>   files I’m not sure exactly how to do this.

From my understanding, Snakemake uses only files as inputs/outputs.
But I do not know what happens if 'rule 1' claims to output 'file' and
'rule 2' says its input is 'file' and then 'rule 1' never produces
this file 'file'; the 'rule 2' is never processed, I guess. Hum? need
to check...


>   @Roel: can you give an example of inputs / outputs that are not files?

Since UNIX considers everything as file.. ;-)

>   I remember that you suggested that inputs might be database queries,
>   for example.  I wonder if we should mark inputs and outputs with
>   types, so that the GWL can know if something is supposed to be a file
>   or something else.  …just how would “something else” be used in a
>   process?

I am not able to imagine other thing than files. But often, I would
like that these files stay in memory instead of be written on disk and
then re-read by the next process. When the IO is slow (disk full using
RAID6), it kills the workflow performance.



> * It’s not possible to select more than one tagged item
>
>   In my test workflow I’m generating a bunch of inputs by mapping over
>   an argument list.  Now the problem is that I can’t select all of these
>   inputs easily in a code snippet.  With the syntax we have I can only
>   select the first item following a tag.
>
>   To address this I’ve extended the accessor syntax, so this works now:
>
> --8<---------------cut here---------------start------------->8---
> process frobnicate
>   packages "frobnicator"
>   inputs
>     . genome: "hg19.fa"
>     . samples: "a" "b" "c"
>   outputs
>     . "result"
>   # {
>     frobnicate -g {{inputs:genome}} --files {{inputs::samples}} > {{outputs}}
>   }
> --8<---------------cut here---------------end--------------->8---
>
>   Note how {{inputs::samples}} is substituted with “a b c”.  With just a
>   single colon it would be just “a”.  Single colon = single item; double
>   colon = more than one item.

I am confused by the syntax.
Well how to select the second element "b"?

Naively, I would tempt to write {{inputs:samples:2}} or {{inputs::samples:2}}.



All the best,
simon

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-09 23:33       ` zimoun
@ 2020-02-10  1:25         ` Kyle Meyer
  2020-02-10  7:34           ` zimoun
  2020-02-10  6:31         ` Ricardo Wurmus
  1 sibling, 1 reply; 14+ messages in thread
From: Kyle Meyer @ 2020-02-10  1:25 UTC (permalink / raw)
  To: zimoun, Ricardo Wurmus; +Cc: gwl-devel

zimoun <zimon.toutoune@gmail.com> writes:

> On Sun, 9 Feb 2020 at 14:01, Ricardo Wurmus <rekado@elephly.net> wrote:
>
>> * inputs and outputs are not validated
>>
>>   When a process declares that it produces an output, but then doesn’t do
>>   that, the next process will fail with a nasty error message.  This is
>>   especially nasty when using containerization as the error is about
>>   failing to map the input into the container.
>>
>>   Processes should automatically validate their inputs and outputs.
>>   Since inputs and outputs could technically be something other than
>>   files I’m not sure exactly how to do this.
>
> From my understanding, Snakemake uses only files as inputs/outputs.
> But I do not know what happens if 'rule 1' claims to output 'file' and
> 'rule 2' says its input is 'file' and then 'rule 1' never produces
> this file 'file'; the 'rule 2' is never processed, I guess. Hum? need
> to check...

In that case, an exception is raised:

    Waiting at most 5 seconds for missing files.
    MissingOutputException in line 2 of /tmp/snakemake-hAlqJ/Snakefile:
    Missing files after 5 seconds:
    foo
    This might be due to filesystem latency. If that is the case, consider
    to increase the wait time with --latency-wait.
    Shutting down, this might take some time.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-09 23:33       ` zimoun
  2020-02-10  1:25         ` Kyle Meyer
@ 2020-02-10  6:31         ` Ricardo Wurmus
  2020-02-10  7:43           ` zimoun
  1 sibling, 1 reply; 14+ messages in thread
From: Ricardo Wurmus @ 2020-02-10  6:31 UTC (permalink / raw)
  To: zimoun; +Cc: gwl-devel


zimoun <zimon.toutoune@gmail.com> writes:

>> * It’s not possible to select more than one tagged item
>>
>>   In my test workflow I’m generating a bunch of inputs by mapping over
>>   an argument list.  Now the problem is that I can’t select all of these
>>   inputs easily in a code snippet.  With the syntax we have I can only
>>   select the first item following a tag.
>>
>>   To address this I’ve extended the accessor syntax, so this works now:
>>
>> --8<---------------cut here---------------start------------->8---
>> process frobnicate
>>   packages "frobnicator"
>>   inputs
>>     . genome: "hg19.fa"
>>     . samples: "a" "b" "c"
>>   outputs
>>     . "result"
>>   # {
>>     frobnicate -g {{inputs:genome}} --files {{inputs::samples}} > {{outputs}}
>>   }
>> --8<---------------cut here---------------end--------------->8---
>>
>>   Note how {{inputs::samples}} is substituted with “a b c”.  With just a
>>   single colon it would be just “a”.  Single colon = single item; double
>>   colon = more than one item.
>
> I am confused by the syntax.
> Well how to select the second element "b"?
>
> Naively, I would tempt to write {{inputs:samples:2}} or {{inputs::samples:2}}.

There’s no syntax for that because you can use good ol’ list processing
(let’s call it “Listp”, or perhaps “Lisp”…) to work on this outside of
the code snippet.

To me these two cases — accessing a single item and accessing multiple
items — are just really common enough to deserve their own syntax,
especially because the plain Scheme way to do this is rather dense:

   (and=> (memq inputs #:hello) cadr)

and

   (and=> (memq inputs #:hello)
          (lambda (sublist)
            (break keyword? (cdr sublist))))

I wouldn’t want to see bioinformaticians paste these kind of things all
over the place.

My hope is that in-list access is much less common and could either be
rewritten to place the item of interest in the first place, or could be
solved with a let binding.

--
Ricardo

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-10  1:25         ` Kyle Meyer
@ 2020-02-10  7:34           ` zimoun
  0 siblings, 0 replies; 14+ messages in thread
From: zimoun @ 2020-02-10  7:34 UTC (permalink / raw)
  To: Kyle Meyer; +Cc: Ricardo Wurmus, gwl-devel

On Mon, 10 Feb 2020 at 02:25, Kyle Meyer <kyle@kyleam.com> wrote:
>
> zimoun <zimon.toutoune@gmail.com> writes:
>
> > On Sun, 9 Feb 2020 at 14:01, Ricardo Wurmus <rekado@elephly.net> wrote:
> >
> >> * inputs and outputs are not validated
> >>
> >>   When a process declares that it produces an output, but then doesn’t do
> >>   that, the next process will fail with a nasty error message.  This is
> >>   especially nasty when using containerization as the error is about
> >>   failing to map the input into the container.
> >>
> >>   Processes should automatically validate their inputs and outputs.
> >>   Since inputs and outputs could technically be something other than
> >>   files I’m not sure exactly how to do this.
> >
> > From my understanding, Snakemake uses only files as inputs/outputs.
> > But I do not know what happens if 'rule 1' claims to output 'file' and
> > 'rule 2' says its input is 'file' and then 'rule 1' never produces
> > this file 'file'; the 'rule 2' is never processed, I guess. Hum? need
> > to check...
>
> In that case, an exception is raised:
>
>     Waiting at most 5 seconds for missing files.
>     MissingOutputException in line 2 of /tmp/snakemake-hAlqJ/Snakefile:
>     Missing files after 5 seconds:
>     foo
>     This might be due to filesystem latency. If that is the case, consider
>     to increase the wait time with --latency-wait.
>     Shutting down, this might take some time.

Thank you for checking.


All the best,
simon

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-10  6:31         ` Ricardo Wurmus
@ 2020-02-10  7:43           ` zimoun
  2020-02-10 21:28             ` Ricardo Wurmus
  0 siblings, 1 reply; 14+ messages in thread
From: zimoun @ 2020-02-10  7:43 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: gwl-devel

On Mon, 10 Feb 2020 at 07:31, Ricardo Wurmus <rekado@elephly.net> wrote:
> zimoun <zimon.toutoune@gmail.com> writes:


> >> * It’s not possible to select more than one tagged item
> >>
> >>   In my test workflow I’m generating a bunch of inputs by mapping over
> >>   an argument list.  Now the problem is that I can’t select all of these
> >>   inputs easily in a code snippet.  With the syntax we have I can only
> >>   select the first item following a tag.
> >>
> >>   To address this I’ve extended the accessor syntax, so this works now:
> >>
> >> --8<---------------cut here---------------start------------->8---
> >> process frobnicate
> >>   packages "frobnicator"
> >>   inputs
> >>     . genome: "hg19.fa"
> >>     . samples: "a" "b" "c"
> >>   outputs
> >>     . "result"
> >>   # {
> >>     frobnicate -g {{inputs:genome}} --files {{inputs::samples}} > {{outputs}}
> >>   }
> >> --8<---------------cut here---------------end--------------->8---
> >>
> >>   Note how {{inputs::samples}} is substituted with “a b c”.  With just a
> >>   single colon it would be just “a”.  Single colon = single item; double
> >>   colon = more than one item.
> >
> > I am confused by the syntax.
> > Well how to select the second element "b"?
> >
> > Naively, I would tempt to write {{inputs:samples:2}} or {{inputs::samples:2}}.
>
> There’s no syntax for that because you can use good ol’ list processing
> (let’s call it “Listp”, or perhaps “Lisp”…) to work on this outside of
> the code snippet.

If I understand correctly, for such cases, 3 solutions:

1. manually split

 inputs
   . sample-1: "a"
   . sample-2: "b"
   . sample-3: "c"

or 2. split elsewhere 'samples' using piece of Lisp

or 3. use Lisp features

 inputs
   . sample-1: `(list-ref ,samples 1)
   . sample-2: `(list-ref ,samples 2)
   . sample-3: `(list-ref ,samples 3)


Right?



> My hope is that in-list access is much less common and could either be
> rewritten to place the item of interest in the first place, or could be
> solved with a let binding.

It makes sense to me.


All the best,
simon

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-10  7:43           ` zimoun
@ 2020-02-10 21:28             ` Ricardo Wurmus
  2020-02-10 23:43               ` zimoun
  2020-02-11  9:34               ` Ricardo Wurmus
  0 siblings, 2 replies; 14+ messages in thread
From: Ricardo Wurmus @ 2020-02-10 21:28 UTC (permalink / raw)
  To: zimoun; +Cc: gwl-devel


zimoun <zimon.toutoune@gmail.com> writes:

> On Mon, 10 Feb 2020 at 07:31, Ricardo Wurmus <rekado@elephly.net> wrote:
>> zimoun <zimon.toutoune@gmail.com> writes:
>
>
>> >> * It’s not possible to select more than one tagged item
>> >>
>> >>   In my test workflow I’m generating a bunch of inputs by mapping over
>> >>   an argument list.  Now the problem is that I can’t select all of these
>> >>   inputs easily in a code snippet.  With the syntax we have I can only
>> >>   select the first item following a tag.
>> >>
>> >>   To address this I’ve extended the accessor syntax, so this works now:
>> >>
>> >> --8<---------------cut here---------------start------------->8---
>> >> process frobnicate
>> >>   packages "frobnicator"
>> >>   inputs
>> >>     . genome: "hg19.fa"
>> >>     . samples: "a" "b" "c"
>> >>   outputs
>> >>     . "result"
>> >>   # {
>> >>     frobnicate -g {{inputs:genome}} --files {{inputs::samples}} > {{outputs}}
>> >>   }
>> >> --8<---------------cut here---------------end--------------->8---
>> >>
>> >>   Note how {{inputs::samples}} is substituted with “a b c”.  With just a
>> >>   single colon it would be just “a”.  Single colon = single item; double
>> >>   colon = more than one item.
>> >
>> > I am confused by the syntax.
>> > Well how to select the second element "b"?
>> >
>> > Naively, I would tempt to write {{inputs:samples:2}} or {{inputs::samples:2}}.
>>
>> There’s no syntax for that because you can use good ol’ list processing
>> (let’s call it “Listp”, or perhaps “Lisp”…) to work on this outside of
>> the code snippet.
>
> If I understand correctly, for such cases, 3 solutions:
>
> 1. manually split
>
>  inputs
>    . sample-1: "a"
>    . sample-2: "b"
>    . sample-3: "c"
>
> or 2. split elsewhere 'samples' using piece of Lisp
>
> or 3. use Lisp features
>
>  inputs
>    . sample-1: `(list-ref ,samples 1)
>    . sample-2: `(list-ref ,samples 2)
>    . sample-3: `(list-ref ,samples 3)
>
>
> Right?

That’s correct, but to directly access values in the “inputs” field,
though, you’d have to use a let binding around the definition of the
code snippet, which would be uglier.

So ugly in fact that I decided to write this little procedure:

--8<---------------cut here---------------start------------->8---
;; Simplify access to tagged items in lists.
(define pick
  (case-lambda
    ;; First item.
    ((key collection)
     (and=> (memq key collection) cadr))
    ;; Nth item
    ((n key collection)
     (let ((sub
            (and=> (memq key collection)
                   (lambda (sublist)
                     (break keyword? (cdr sublist))))))
       (cond
        ((number? n)
         (and (> (length sub) n)
              (list-ref sub n)))
        ;; All items
        ((and (procedure? n)
              (eq? (procedure-name n) '*))
         sub)
        ;; SRFI-1 accessors like "first"
        ((procedure? n)
         (n sub))
        (else (error "pick: Selector not supported.")))))))
--8<---------------cut here---------------end--------------->8---

Then I noticed that it’s really inconvenient to use let bindings inside
of fields, so I changed the “make-process” macro to allow for
definitions inside of the fields of a process.

So you can do this now:

--8<---------------cut here---------------start------------->8---
process foo
  inputs
    . "something"
    . samples: "a" "b" "c"
  procedure
    define second-sample
      pick second samples: inputs
    . # { echo {{second-sample}} }
--8<---------------cut here---------------end--------------->8---

Unfortunately, the leading dot sticks out like a wart here.  We could
only drop it if I manage to rewrite the code-snippet value so that it
really is a procedure that will return itself when applied to zero
arguments…  It might be possible with some tinkering.

(Note that you do need the “procedure” field name here; it can be left
off only when the last value is a code-snippet.)

--
Ricardo

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-10 21:28             ` Ricardo Wurmus
@ 2020-02-10 23:43               ` zimoun
  2020-02-11  9:39                 ` Ricardo Wurmus
  2020-02-11  9:34               ` Ricardo Wurmus
  1 sibling, 1 reply; 14+ messages in thread
From: zimoun @ 2020-02-10 23:43 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: gwl-devel

Hi,

On Mon, 10 Feb 2020 at 22:28, Ricardo Wurmus <rekado@elephly.net> wrote:

> > Right?
>
> That’s correct, but to directly access values in the “inputs” field,
> though, you’d have to use a let binding around the definition of the
> code snippet, which would be uglier.
>
> So ugly in fact that I decided to write this little procedure:

Wow! :-)


> --8<---------------cut here---------------start------------->8---
> ;; Simplify access to tagged items in lists.
> (define pick

[...]

>         ;; SRFI-1 accessors like "first"
>         ((procedure? n)
>          (n sub))

Cool!

>         (else (error "pick: Selector not supported.")))))))
> --8<---------------cut here---------------end--------------->8---
>
> Then I noticed that it’s really inconvenient to use let bindings inside
> of fields, so I changed the “make-process” macro to allow for
> definitions inside of the fields of a process.

Ok. Even I am a bit lost with all the changes.


> So you can do this now:
>
> --8<---------------cut here---------------start------------->8---
> process foo
>   inputs
>     . "something"
>     . samples: "a" "b" "c"
>   procedure
>     define second-sample
>       pick second samples: inputs
>     . # { echo {{second-sample}} }
> --8<---------------cut here---------------end--------------->8---
>
> Unfortunately, the leading dot sticks out like a wart here.  We could
> only drop it if I manage to rewrite the code-snippet value so that it
> really is a procedure that will return itself when applied to zero
> arguments…  It might be possible with some tinkering.

Hum? I trust you.

> (Note that you do need the “procedure” field name here; it can be left
> off only when the last value is a code-snippet.)

Ok. It makes sense.

Does this work?

 --8<---------------cut here---------------start------------->8---
 process foo
   inputs
     . "something"
     . samples: "a" "b" "c"
     # { echo {{(pick second inputs:samples)}} }
 --8<---------------cut here---------------end--------------->8---

or   # { echo {{(pick second samples: inputs)}} }

Because it is enough to usual needs, I guess.


Cheers,
simon

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-10 21:28             ` Ricardo Wurmus
  2020-02-10 23:43               ` zimoun
@ 2020-02-11  9:34               ` Ricardo Wurmus
  2020-02-11 15:37                 ` zimoun
  1 sibling, 1 reply; 14+ messages in thread
From: Ricardo Wurmus @ 2020-02-11  9:34 UTC (permalink / raw)
  To: zimoun; +Cc: gwl-devel


Ricardo Wurmus <rekado@elephly.net> writes:

> So you can do this now:
>
> --8<---------------cut here---------------start------------->8---
> process foo
>   inputs
>     . "something"
>     . samples: "a" "b" "c"
>   procedure
>     define second-sample
>       pick second samples: inputs
>     . # { echo {{second-sample}} }
> --8<---------------cut here---------------end--------------->8---
>
> Unfortunately, the leading dot sticks out like a wart here.  We could
> only drop it if I manage to rewrite the code-snippet value so that it
> really is a procedure that will return itself when applied to zero
> arguments…  It might be possible with some tinkering.

It is now possible to leave off the leading period as code snippets can
now be in procedure application position returning themselves.

--8<---------------cut here---------------start------------->8---
process foo
  inputs
    . "something"
    . samples: "a" "b" "c"
  procedure
    define second-sample
      pick second samples: inputs
    # { echo {{second-sample}} }
--8<---------------cut here---------------end--------------->8---

Ah, that’s better!

--
Ricardo

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-10 23:43               ` zimoun
@ 2020-02-11  9:39                 ` Ricardo Wurmus
  0 siblings, 0 replies; 14+ messages in thread
From: Ricardo Wurmus @ 2020-02-11  9:39 UTC (permalink / raw)
  To: zimoun; +Cc: gwl-devel


zimoun <zimon.toutoune@gmail.com> writes:

> Does this work?
>
>  --8<---------------cut here---------------start------------->8---
>  process foo
>    inputs
>      . "something"
>      . samples: "a" "b" "c"
>      # { echo {{(pick second inputs:samples)}} }
>  --8<---------------cut here---------------end--------------->8---
>
> or   # { echo {{(pick second samples: inputs)}} }
>
> Because it is enough to usual needs, I guess.

No, this won’t work because {{…}} is only for simple variable
interpolation; it does not support anything with a space in it.  That’s
on purpose to avoid conflicts with the embedded language syntax.

You will need to define a local variable first and reference that:

--8<---------------cut here---------------start------------->8---
process foo
  inputs
    . "something"
    . samples: "a" "b" "c"
  procedure
    define my-selection
      pick second samples: inputs
    # { echo {{my-selection}} }
--8<---------------cut here---------------end--------------->8---

--
Ricardo

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Preparing for a new release
  2020-02-11  9:34               ` Ricardo Wurmus
@ 2020-02-11 15:37                 ` zimoun
  0 siblings, 0 replies; 14+ messages in thread
From: zimoun @ 2020-02-11 15:37 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: gwl-devel

On Tue, 11 Feb 2020 at 10:34, Ricardo Wurmus <rekado@elephly.net> wrote:

> It is now possible to leave off the leading period as code snippets can
> now be in procedure application position returning themselves.
>
> --8<---------------cut here---------------start------------->8---
> process foo
>   inputs
>     . "something"
>     . samples: "a" "b" "c"
>   procedure
>     define second-sample
>       pick second samples: inputs
>     # { echo {{second-sample}} }
> --8<---------------cut here---------------end--------------->8---
>
> Ah, that’s better!

Really cool!
Thank you

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2020-02-11 15:38 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-08 12:39 Preparing for a new release Ricardo Wurmus
2020-02-08 20:38 ` Kyle Meyer
2020-02-08 21:50   ` Ricardo Wurmus
2020-02-09 13:00     ` Ricardo Wurmus
2020-02-09 23:33       ` zimoun
2020-02-10  1:25         ` Kyle Meyer
2020-02-10  7:34           ` zimoun
2020-02-10  6:31         ` Ricardo Wurmus
2020-02-10  7:43           ` zimoun
2020-02-10 21:28             ` Ricardo Wurmus
2020-02-10 23:43               ` zimoun
2020-02-11  9:39                 ` Ricardo Wurmus
2020-02-11  9:34               ` Ricardo Wurmus
2020-02-11 15:37                 ` zimoun

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).