all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Christopher Baines <mail@cbaines.net>
To: guix-devel@gnu.org
Subject: [PATCH 3/4] Enable make-worker-thread-channel to create multiple worker threads.
Date: Fri, 24 Jan 2020 19:44:05 +0000	[thread overview]
Message-ID: <20200124194406.29638-3-mail@cbaines.net> (raw)
In-Reply-To: <20200124194406.29638-1-mail@cbaines.net>

This will allow running multiple threads, that all listen on the same channel,
enabling processing multiple jobs at one time.

* src/cuirass/utils.scm (make-worker-thread-channel): Add a #:parallelism
argument, and create as many threads as the given parallelism.
---
 src/cuirass/utils.scm | 24 ++++++++++++++----------
 1 file changed, 14 insertions(+), 10 deletions(-)

diff --git a/src/cuirass/utils.scm b/src/cuirass/utils.scm
index dfed4a9..f3ba18d 100644
--- a/src/cuirass/utils.scm
+++ b/src/cuirass/utils.scm
@@ -99,20 +99,24 @@ delimited continuations and fibers."
 (define %worker-thread-args
   (make-parameter #f))
 
-(define (make-worker-thread-channel initializer)
+(define* (make-worker-thread-channel initializer
+                                     #:key (parallelism 1))
   "Return a channel used to offload work to a dedicated thread.  ARGS are the
 arguments of the worker thread procedure."
   (parameterize (((@@ (fibers internal) current-fiber) #f))
     (let ((channel (make-channel)))
-      (let ((args (initializer)))
-        (call-with-new-thread
-         (lambda ()
-           (parameterize ((%worker-thread-args args))
-             (let loop ()
-               (match (get-message channel)
-                 (((? channel? reply) . (? procedure? proc))
-                  (put-message reply (apply proc args))))
-               (loop))))))
+      (for-each
+       (lambda _
+         (let ((args (initializer)))
+           (call-with-new-thread
+            (lambda ()
+              (parameterize ((%worker-thread-args args))
+                (let loop ()
+                  (match (get-message channel)
+                    (((? channel? reply) . (? procedure? proc))
+                     (put-message reply (apply proc args))))
+                  (loop)))))))
+       (iota parallelism))
       channel)))
 
 (define (call-with-worker-thread channel proc)
-- 
2.24.1

  parent reply	other threads:[~2020-01-24 19:44 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-24 19:44 [PATCH 1/4] utils: Change critical section terminology to worker threads Christopher Baines
2020-01-24 19:44 ` [PATCH 2/4] Adjust make-worker-thread-channel to take an initializer Christopher Baines
2020-01-24 19:44 ` Christopher Baines [this message]
2020-01-24 19:44 ` [PATCH 4/4] database: Enable running up to 4 database queries at once Christopher Baines
2020-01-25 17:47   ` Ludovic Courtès
2020-01-25 17:46 ` [PATCH 1/4] utils: Change critical section terminology to worker threads Ludovic Courtès
2020-01-25 22:59   ` Christopher Baines
2020-01-28 10:27     ` Ludovic Courtès
2020-01-28 17:19       ` Christopher Baines
2020-01-29 16:26         ` Ludovic Courtès

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200124194406.29638-3-mail@cbaines.net \
    --to=mail@cbaines.net \
    --cc=guix-devel@gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.