unofficial mirror of
 help / color / mirror / Atom feed
From: Olivier Dion via <>
Subject: auto-connect fails to parallelize independent processes
Date: Thu, 09 Jun 2022 20:56:27 -0400	[thread overview]
Message-ID: <87edzxuyis.fsf@laura> (raw)


Given the following example:
--8<---------------cut here---------------start------------->8---
process proc (with output)
  packages "coreutils"
  outputs : file output
  # {
    for i in $(seq 10); do
      date >> {{outputs}}
      sleep 1

workflow parallel
      proc "A.out"
      proc "B.out"
--8<---------------cut here---------------end--------------->8---

We can see that both processes are completely independent, and yet they
are batched linearly.

If we re-write the workflow with `graph' instead:
--8<---------------cut here---------------start------------->8---

define A : proc "A.out"
define B : proc "B.out"

workflow parallel
      A ->
      B ->
--8<---------------cut here---------------end--------------->8---

we do get parallel execution of the processes.

Olivier Dion

             reply	other threads:[~2022-06-10  0:56 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-06-10  0:56 Olivier Dion via [this message]
2022-06-10 13:13 ` auto-connect fails to parallelize independent processes Ricardo Wurmus
2022-06-13 20:02   ` Olivier Dion via
2022-06-14 19:04   ` Olivier Dion via
2022-06-14 19:53 ` [PATCH 1/2] gwl/workflows: Execute independent processes in parallel Olivier Dion
2022-06-14 19:53   ` [PATCH 2/2] gwl/ui: Protect format to currnt-error-port with mutex Olivier Dion
2022-06-15 23:28     ` Ricardo Wurmus
2022-06-15 23:27   ` [PATCH 1/2] gwl/workflows: Execute independent processes in parallel Ricardo Wurmus
2022-06-16  1:23     ` Olivier Dion via

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87edzxuyis.fsf@laura \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).