Re: Limiting parallelism using futures, parallel forms and fibers

[Chris Vine <vine35792468@gmail.com>]

On Wed, 8 Jan 2020 08:56:11 +0100
Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> wrote:
[snip]
> So my questions are:
> 
> - Is there a default / recommended way to limit parallelism for
> recursive calls to parallel forms?
> 
> - Is there a better way than a global counter with locking, to limit the
> number of futures created during recursive calls? I would dislike very
> much to have to do something like global state + mutex.
> 
> - What do you recommend in general to solve this?

I think you have it wrong, and that futures use a global queue and a
global set of worker threads.  I don't see how futures could work
without at least a global set of worker threads.  Have a look at the
futures source code.

If you want more control over the thread pool than guile's futures
provide, you could consider something like this:
https://github.com/ChrisVine/guile-a-sync2/blob/master/a-sync/thread-pool.scm
But then you would have to make your own futures if you want a graph
of futures rather than a graph of coroutines (which is what you would
get if you use the associated event loop).

The main thing is to get the parallel algorithm right, which can be
tricky.