Re: Whats' the proper senario of par-map? (Was Re: bug#13188: par-map causes VM stack overflow)

[ludo@gnu.org (Ludovic Courtès)]

Mark H Weaver <mhw@netris.org> skribis:

> Nala Ginrut <nalaginrut@gmail.com> writes:
>
>> But I'm still puzzled with the performance of par-map:
>> --------------------cut-------------------
>> scheme@(guile-user)> ,time (define a (map (lambda (x) (expt x 5)) (iota
>> 10000)))
>> ;; 0.008019s real time, 0.007979s run time.  0.000000s spent in GC.
>> scheme@(guile-user)> ,time (define a (par-map (lambda (x) (expt x 5))
>> (iota 10000)))
>> ;; 6.596471s real time, 6.579375s run time.  1.513880s spent in GC.
>> --------------------end-------------------
>>
>> So my question is, what's the proper scenario to use par-map?
>
> It only makes sense to use 'par-map' when the procedure is fairly
> expensive to compute.

Indeed.

> There is inevitably a lot of overhead in creating and joining the
> threads.

We use a thread pool, so there’s no such cost.

But there are other costs.  When delimited continuations are used, we’re
on the slow path.  Also, Guile’s fat mutexes & co. are terribly
inefficient.  And finally, there may be contention on the futexes mutex
(esp. when the computations is too small.)

So yes, there’s room for improvement.  Yet, it should be fruitful,
provided you use it for reasonably long computations, as Mark outlines.

Ludo’.