From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Chris Vine Newsgroups: gmane.lisp.guile.user Subject: Re: Passing objects between threads Date: Tue, 13 Sep 2016 00:10:58 +0100 Message-ID: <20160913001058.1cf071ba@bother.homenet> References: <87mvjeb80n.fsf@pobox.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1473721905 30109 195.159.176.226 (12 Sep 2016 23:11:45 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 12 Sep 2016 23:11:45 +0000 (UTC) To: guile-user@gnu.org Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Tue Sep 13 01:11:41 2016 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bjaO1-0006yg-Lz for guile-user@m.gmane.org; Tue, 13 Sep 2016 01:11:37 +0200 Original-Received: from localhost ([::1]:45608 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bjaNz-0006Wl-RQ for guile-user@m.gmane.org; Mon, 12 Sep 2016 19:11:35 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:49149) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bjaNb-0006Uq-23 for guile-user@gnu.org; Mon, 12 Sep 2016 19:11:12 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bjaNU-00009M-0K for guile-user@gnu.org; Mon, 12 Sep 2016 19:11:09 -0400 Original-Received: from smtpout2.wanadoo.co.uk ([80.12.242.42]:56854 helo=smtpout.wanadoo.co.uk) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bjaNT-000082-L9 for guile-user@gnu.org; Mon, 12 Sep 2016 19:11:03 -0400 Original-Received: from bother.homenet ([95.146.108.17]) by mwinf5d17 with ME id inAy1t0060NZNDr03nAyB0; Tue, 13 Sep 2016 01:10:59 +0200 X-ME-Helo: bother.homenet X-ME-Date: Tue, 13 Sep 2016 01:10:59 +0200 X-ME-IP: 95.146.108.17 Original-Received: from bother.homenet (localhost [127.0.0.1]) by bother.homenet (Postfix) with ESMTP id 737E6120C78 for ; Tue, 13 Sep 2016 00:10:58 +0100 (BST) In-Reply-To: X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.31; i686-pc-linux-gnu) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 80.12.242.42 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:12905 Archived-At: On Mon, 12 Sep 2016 21:58:02 +0200 Panicz Maciej Godek wrote: [snip] > Hah, both look rather complicated. I eventually followed Chris' > advice and =20 > developed "my own" queues, which turned out to be a bit simplified > variant of his code: >=20 > (use-modules (ice-9 nice-9) (ice-9 q) (ice-9 threads)) >=20 > (define (make-channel) > `(,(make-q) ,(make-mutex) ,(make-condition-variable))) >=20 > (define (channel? c) > (and-let* (((queue mutex guard) c) > ((q? queue)) > ((mutex? mutex)) > ((condition-variable? guard))))) >=20 > (define (send! data #;through channel) > (let (((queue mutex guard) channel)) > (with-mutex mutex > (enq! queue data) > (signal-condition-variable guard)))) >=20 > (define (receive! #;from channel) > (let (((queue mutex guard) channel)) > (with-mutex mutex > (while (q-empty? queue) > (wait-condition-variable guard mutex)) > (deq! queue)))) All I would say about this, if you want to use an asynchronous queue of this kind as a "channel", is that "channels" often provide for throttling of the write side so the producer(s) can't overwhelm the consumer(s). This might comprise a hard limit to the size of the queue with a condition variable applied to the writer as well as the reader (Andy's solution), but there are other ways of doing it, such as inserting writer waits of increasing duration where the size of the queue starts to grow too large. > > I think more generally, now is a time for a lot of experimentation > > in the concurrency side of Guile; I look forward to seeing what > > people settle on :) =20 >=20 >=20 > I came up with an interface for thread communication, but I don't > know how to implement it -- maybe you (or someone else) would be able > to help, or at least warn not to go that way. >=20 > I am trying to extend my SLAYER framework to support reactive > programming. I came up with the following architecture: the event > system gathers events and puts them into a queue, for example, > pressing key k causes a '(key-down k) event to appear. >=20 > (define EVENT-QUEUE (make-channel)) >=20 > (set-event-handler! (lambda (event) (send! event #;through channel))) >=20 > Then there is this update function, which takes the current state of > the world, and an event, and returns an updated event. >=20 > Thus, the evolution of the world can be understood as the fold-left > on the event queue: >=20 > (define (channel-fold update state channel) > (let ((message (receive! #;from channel))) > (channel-fold update (update state message) channel))) >=20 > However, if I put things that way, I have no access to the current > state, because it is only a transient variable being passed to the > tail-recursive call. >=20 > I could of course communicate between the threads using a global > variable and assignment, but this is nasty. Instead I thought of the > following interface: I would like to have two functions, let's call > them "exposed" and "snapshot". The first one would behave like > identity (or values), except that its evaluation would cause the > snapshot value of the current thread to be set. The second function > would be used to retrieve the snapshot value, so that it could be > accessed from other threads. >=20 > For example, I could define the evolution thread in the following way: >=20 > (define evolution > (call-with-new-thread > (=CE=BB () > (channel-fold > (=CE=BB (state action) > (let ((state* (update state #;with action))) > (exposed state*))) > initial-state > EVENT-QUEUE)))) >=20 > Then, I could access the state variable using the snapshot procedure, > so that -- for example -- I could use it in the drawing routine: >=20 > (set-display-procedure! > (=CE=BB () > (draw! (visualization (snapshot evolution))))) >=20 > What I would like about that solution is that even if the world was > updated much faster than the drawing routine could handle, the system > could remain responsive. (I also think that this interface would be > nice with time-constrained optimization algorithms, so that they > could make the best known solution available at any time) >=20 > However, the problem is that -- while procedures have their > properties -- there seems to be nothing similar in the realm of > threads. Normally in similar cases I could use a thread-indexed hash > table, but in the context of multi-threading it would probably need > to be protected by a mutex, and it would likely slow things down. >=20 > Any suggestions? >=20 > Also, if the data is immutable, and only one thread does assignments, > is it OK to assume that variable reference and assignment are atomic > operations, or do I need mutices anyway? I don't have sufficient grasp of what you want and how slayer operates to help you with this. When I want to multiplex a number of "channels" onto a single receiver/event-queue thread, I use this: https://github.com/ChrisVine/guile-a-sync/wiki/coroutines https://github.com/ChrisVine/guile-a-sync/wiki/event-loop (On the second reference, scroll down and see in particular the await-task-in-thread! and await-task! procedures, and the other await* procedures). Each a-sync or compose-a-sync block constructs a coroutine which operates as a kind of channel or fibre with 'await' semantics - a single thread can wait concurrently on any number of them. However, this doesn't look as if it is what you are after. Chris