Re: Delimited continuations to the rescue of futures

[ludo@gnu.org (Ludovic Courtès)]

Hi Mark,

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> As was reported recently by Mark and others, ‘par-map’ would only use
>> ncores - 1, because the main thread was stuck in a
>> ‘wait-condition-variable’ while touching one of the futures.
>>
>> The obvious fix is to write ‘par-map’ like this (as can be seen from
>> Chapter 2 of Marc Feeley’s PhD thesis):
>>
>>   (define (par-mapper mapper cons)
>>     (lambda (proc . lists)
>>       (let loop ((lists lists))
>>         (match lists
>>           (((heads tails ...) ...)
>>            (let ((tail (future (loop tails)))
>>                  (head (apply proc heads)))
>>              (cons head (touch tail))))
>>           (_
>>            '())))))
>
> Am I correct in believing that the above code would use the main thread
> only for applying 'proc' to the first element of each list?

Yes.

> In other words, if you have 4 cores and call 'par-map' on a list of 1000
> elements, the main thread will only be used to process 1 out of 1000
> elements, and only 3 cores will be used to process the other 999.
> Is that right?

No.  :-)

That’s what would happen with current stable-2.0.  But in
‘wip-nested-futures’, the main thread behaves as a worker while waiting
for the tail future to complete (which addresses your main concern, I
think, as we had discussed on IRC.)  Thus, all cores are always used.

Thanks,
Ludo’.