bug#14756: threads - par-map - multicore issue

[Andy Wingo <wingo@pobox.com>]

On Tue 21 Jun 2016 10:33, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> skribis:
>
>> I see this, but I'm not quite sure what's going on.  What I do see is
>> that par-map of 1+ on a list is horribly slow, both on 2.0 and master.
>> Ludovic do you know what's going on here?
>
> As David put it, only one core is being used, which is clearly a bug.
>
> I believe the bug was introduced by
> 8a177d316c0062afe74f9a761ef460e297435e59 (however, before that commit,
> you would hit a stack overflow when doing ‘par-map’ on a large-enough
> list.)

Given that Guile 2.2. doesn't have a stack limit problem, I have
reverted this commit on master (though I kept the tests).

FWIW Guile 2.0 with this test

   $ time ../guile-2.0/meta/guile -c '(begin (use-modules (ice-9 threads)) (par-map 1+ (iota 40000)))'

   real	1m45.282s
   user	1m45.208s
   sys	0m0.036s


Guile 2.1.x with the stack-limit stuff:

   $ time /opt/guile/bin/guile -c '(begin (use-modules (ice-9 threads)) (par-map 1+ (iota 40000)))'

   real	0m51.738s
   user	1m2.720s
   sys	0m0.116s

Guile 2.1.x after reverting the patch:

   $ time meta/guile -c '(begin (use-modules (ice-9 threads)) (par-map 1+ (iota 40000)))'

   real	0m1.403s
   user	0m1.396s
   sys	0m0.024s

Note that I took a zero off the original test in all examples above.
However!  I still have the problem that mostly only one core is used.  I
would imagine that is because the thread that builds the spine is more
costly than the threads that actually do the workload (the 1+ in this
case).  But maybe that is wrong.  Certainly there are improvements that
can be made in the futures implementation in 2.2 with atomic boxes.

Andy