Re: [PATCH] tests/examples: Add running of workflow examples

[Ricardo Wurmus <rekado@elephly.net>]

Olivier Dion <olivier.dion@polymtl.ca> writes:

>>> +  (define scenarios
>>> +       (list "extended-example-workflow.w"))
>>
>> Should these better be discovered automatically via SCANDIR?
>
> Well some example are not full workflow.  Only processes.  So I think that
> yes, if we set a common prefix for full workflow:
>
> (scandir "." (cut string-prefix "exemple-workflow" <>))
>
> However, it's nice -- even more true when debugging -- to be able to
> cherry-pick the tests.  This is easier with the list above by commenting
> tests that are passing.

That makes sense.

>> Why shell out to RM when we have DELETE-FILE and its recursive friend in
>> Guix?  I’d also rather move clean-up work to a DYNAMIC-UNWIND handler.
>
> I hesitated to include Guix's module in test.  If you're okay with it, I
> will use the helpers available in Guix.  I concur that the cleanup
> should be in dynamic-unwind.

Yes, using the helpers from Guix is fine.  The GWL doesn’t make much
sense without Guix anyway :)

>>> +                      (format (error-output-port)
>>> +                              "Example directory: ~a\n" tmp-dir))
>>
>> Nitpick: ~% instead of \n.
>
> Is there a reason why?  I don't mind I just never really understood why
> scheme has this special format rule for newline.

I don’t know if there’s a reason for it, but I prefer it for consistency.

>>> +                  success?)))
>>> +            scenarios))
>>
>> This FOR-EACH loop combines test definition with test running, which
>> seems wrong to me.  Maybe SRFI-64 is not the best fit for tests that
>> only care about whether a shell command was run successfully.  Perhaps
>> we should do as Guix does and just have a shell script to run these
>> tests.
>
> What do you find wrong about it?  We could re-write it as:
>
> (define (run-example path)
>   ...)
>
> (test-assert (run-example "extended-workflow.w"))
> (test-assert (run-example "..."))
>
> if you like it better.  However, we lose the power of SCANDIR mentioned
> above.

Hmm, you’re right about that.  If you could split this up into smaller
procedures (one definition of RUN-EXAMPLE and another that loops
TEST-ASSERT and RUN-EXAMPLE on the test files) I’d already be happy :)

> Let me know, I will send a v2 with your above recommendations and answers
> to my questions.

Thank you!

-- 
Ricardo