problem with futures?

problem with futures?

[Andy Wingo]

Hi,

With current CVS on PPC (because I still haven't investigated enough to
figure what's wrong on x86-64),

(define (nsum n)
  (apply + (let ((x 1))
             (par-map (lambda (y) (set! x (1+ x)) (+ y x))
                      (make-list n 0)))))

I get this:

guile> (nsum 100)
$1 = 5150
guile> (nsum 100)
$2 = 5150
guile> (nsum 100)
$3 = 5150
guile> (nsum 100)

Backtrace:
In current input:
   5: 0* [nsum 100]
   1: 1  [apply #<primitive-generic +> ...
   1: 2*  (let* ((x 1)) (par-map (lambda (y) (set! x #) (+ y x))
(make-list n 0)))
   1: 3   [par-map #<procedure #f (y)> (0 0 0 0 0 0 0 0 0 ...)]
In /opt/guile/share/guile/1.7/ice-9/threads.scm:
  52: 4   [map #<primitive-procedure future-ref> ...
  53: 5*   [apply #<primitive-generic map> #<procedure #f args> ...]
In unknown file:
   ?: 6    [map]
   ?: 7*   [map #<procedure #f args> (0 0 0 0 0 0 0 0 0 ...)]
   ?: 8*   [#<procedure #f args> 0]
In /opt/guile/share/guile/1.7/ice-9/threads.scm:
  55: 9*   (future (apply mapper (proc . arglists)))

/opt/guile/share/guile/1.7/ice-9/threads.scm:55:20: In expression
(future (apply mapper #)):
/opt/guile/share/guile/1.7/ice-9/threads.scm:55:20: Cannot allocate
memory

>From then on out I can't call nsum any more. Perhaps threads assigned to
futures are not being gc'd properly.

Regards,
-- 
Andy Wingo
http://wingolog.org/



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: problem with futures?

[Mikael Djurfeldt]

Andy,

I started implementing futures, but have not been doing any Guile
development since some time.

The basic mechanism is there, but the implementation is incomplete. 
The major omissions are associated with exactly the two problems you
bring up.  These known bugs are: the handling of the future cache
needs to be better, and, (even more important) the initialization of
the dynamic context for a new futures is currently not done at all.

The last point is a bug (the implementation is incomplete) which has
to be fixed before any release.  I've pointed this out before.

On 12/21/05, Andy Wingo <wingo@pobox.com> wrote:
> (define (nsum n)
>   (apply + (let ((x 1))
>              (par-map (lambda (y) (set! x (1+ x)) (+ y x))
>                       (make-list n 0)))))
>
> I get this:
>
> guile> (nsum 100)
> $1 = 5150
> guile> (nsum 100)
> $2 = 5150
> guile> (nsum 100)
> $3 = 5150
> guile> (nsum 100)
>
> Backtrace:
> In current input:
>    5: 0* [nsum 100]
>    1: 1  [apply #<primitive-generic +> ...
>    1: 2*  (let* ((x 1)) (par-map (lambda (y) (set! x #) (+ y x))
> (make-list n 0)))
>    1: 3   [par-map #<procedure #f (y)> (0 0 0 0 0 0 0 0 0 ...)]
> In /opt/guile/share/guile/1.7/ice-9/threads.scm:
>   52: 4   [map #<primitive-procedure future-ref> ...
>   53: 5*   [apply #<primitive-generic map> #<procedure #f args> ...]
> In unknown file:
>    ?: 6    [map]
>    ?: 7*   [map #<procedure #f args> (0 0 0 0 0 0 0 0 0 ...)]
>    ?: 8*   [#<procedure #f args> 0]
> In /opt/guile/share/guile/1.7/ice-9/threads.scm:
>   55: 9*   (future (apply mapper (proc . arglists)))
>
> /opt/guile/share/guile/1.7/ice-9/threads.scm:55:20: In expression
> (future (apply mapper #)):
> /opt/guile/share/guile/1.7/ice-9/threads.scm:55:20: Cannot allocate
> memory

This is probably simply that your OS/memory can't handle more than 100
pthreads (including the memory space for the stack of each thread).

> >From then on out I can't call nsum any more. Perhaps threads assigned to
> futures are not being gc'd properly.

Futures are cached for re-use.  This makes overhead of futures lighter
than that of ordinary threads.  Currently, there is no mechanism for
clearing the future cache.   As long as the application is using a
reasonable number of thread (relative to what the OS can cope with),
there is no big reason to clear it.  However, for robustness against
bugs and extreme use, there should be a set of rules for clearing it. 
For example, it could be cleared if no futures have been created
within the last 10 ms.  Also, one could have some upper limit on the
number of futures being cached.

For now, I will not work further on that code.  To the Guile
developers I suggest that you do what you want.  Fix it, or back out
the future feature entirely.

M


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

Re: problem with futures?

[Marius Vollmer]

Mikael Djurfeldt <mdjurfeldt@gmail.com> writes:

> For now, I will not work further on that code.  To the Guile
> developers I suggest that you do what you want.  Fix it, or back out
> the future feature entirely.

Hmm.  Semantically, 'begin-thread' and 'thread-join' should be
(nearly?) identical to 'future' and 'future-ref', so we are not losing
critical functionality when removing futures.

However, it would still be good to have some kind of thread pool for
the case where you want to use many short lived threads.

So, I'll remove futures from the API, but leave the code there.

-- 
GPG: D5D4E405 - 2F9B BCCC 8527 692A 04E3  331E FAF8 226A D5D4 E405


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel