[pthreads] performance test using fib.scm

[pthreads] performance test using fib.scm

[Mikael Djurfeldt]

In the announcement of the pthreads support, I attached a program
fib.scm:

(define (par-fib n n-forks)
  (cond ((<= n 1) 1)
	((> n-forks 0)
	 (letpar ((f1 (par-fib (- n 1) (- n-forks 1)))
		  (f2 (par-fib (- n 2) (- n-forks 1))))
	   (+ f1 f2)))
	(else
	 (let ((f1 (mono-fib (- n 1)))
	       (f2 (mono-fib (- n 2))))
	   (+ f1 f2)))))

(define (mono-fib n)
  (let iter ((f1 1) (f2 1) (i 1))
    (if (>= i n)
	f2
	(iter f2 (+ f1 f2) (+ 1 i)))))

When I run this with a Guile compiled with COPT threads, it invariably
takes around 12 s to run (par-fib 40000 1) on my dual CPU machine.
"Top" reports 50% of CPU capacity usage during calculation.

With the new pthreads support (which now allows for real SMP), it
takes 2.6 s and top says 91%!

Still, for large fibonnaci numbers and large branch depths (many
threads), Guile often crashes in realloc.  That function should be
thread safe, shouldn't it?  Maybe we're trashing up the heap somewhere
else...

Anyone keen on some debugging? :-)

Mikael


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: [pthreads] performance test using fib.scm

[Wolfgang Jaehrling]

On Tue, Dec 10, 2002 at 07:16:04PM +0100, Mikael Djurfeldt wrote:
> Still, for large fibonnaci numbers and large branch depths (many
> threads), Guile often crashes in realloc.  That function should be
> thread safe, shouldn't it?

malloc() in glibc is thread-save, but not reentrant.  Since the
behaviour of realloc() is unlikely to differ from malloc() in this
respect, you most certainly have to protect calls to it with a lock.

Cheers,
GNU/Wolfgang


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: [pthreads] performance test using fib.scm

[Mikael Djurfeldt]

Wolfgang Jaehrling <pro-linux@gmx.de> writes:

> malloc() in glibc is thread-save, but not reentrant.  Since the
> behaviour of realloc() is unlikely to differ from malloc() in this
> respect, you most certainly have to protect calls to it with a lock.

2002-12-10  Mikael Djurfeldt  <mdj@kvast.blakulla.net>

	* gc-malloc.c (malloc_mutex): New mutex.
	(scm_gc_init_malloc): Initialize it.
	(scm_realloc): Serialize call to realloc
	(scm_calloc): Same for calloc.
	Thanks to Wolfgang Jaehrling!
	(Now we have to make sure all calls to malloc/realloc are made
	through scm_malloc.)

 How about "free"?

 Best regards,
 Mikael D.
 


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: [pthreads] performance test using fib.scm

[Christopher Cramer]

On Tue, Dec 10, 2002 at 09:10:37PM +0100, Mikael Djurfeldt wrote:
> Wolfgang Jaehrling <pro-linux@gmx.de> writes:
> 
> > malloc() in glibc is thread-save, but not reentrant.  Since the
> > behaviour of realloc() is unlikely to differ from malloc() in this
> > respect, you most certainly have to protect calls to it with a lock.
> 
> 2002-12-10  Mikael Djurfeldt  <mdj@kvast.blakulla.net>
> 
> 	* gc-malloc.c (malloc_mutex): New mutex.
> 	(scm_gc_init_malloc): Initialize it.
> 	(scm_realloc): Serialize call to realloc
> 	(scm_calloc): Same for calloc.
> 	Thanks to Wolfgang Jaehrling!
> 	(Now we have to make sure all calls to malloc/realloc are made
> 	through scm_malloc.)

Every pthreads implementation is required to have a thread-safe malloc,
calloc, realloc and free (along with many other functions). The fact
that malloc etc. are non-reentrant only means that you shouldn't call
them from a signal handler.

Any pthreads implementation that requires you to lock a mutex to call
malloc is broken.

-- 
Christopher Cramer <crayc@pyro.net> <http://www.pyro.net/~crayc/>
"Gore would have finished ahead by the barest of margins had he pursued
and gained a complete statewide recount." -- Associated Press, 9/6/2002


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: [pthreads] performance test using fib.scm

[Maciej Stachowiak]

On 10Dec2002 10:10PM (+0100), Wolfgang Jaehrling wrote:
> On Tue, Dec 10, 2002 at 07:16:04PM +0100, Mikael Djurfeldt wrote:
> > Still, for large fibonnaci numbers and large branch depths (many
> > threads), Guile often crashes in realloc.  That function should be
> > thread safe, shouldn't it?
> 
> malloc() in glibc is thread-save, but not reentrant.  Since the
> behaviour of realloc() is unlikely to differ from malloc() in this
> respect, you most certainly have to protect calls to it with a lock.
> 

Huh? Doesn't malloc do it's own locking when modifying global data
structures?

 - Maciej



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: [pthreads] performance test using fib.scm

[Mikael Djurfeldt]

Maciej Stachowiak <mjs@noisehavoc.org> writes:

> On 10Dec2002 10:10PM (+0100), Wolfgang Jaehrling wrote:
>> malloc() in glibc is thread-save, but not reentrant.  Since the
>> behaviour of realloc() is unlikely to differ from malloc() in this
>> respect, you most certainly have to protect calls to it with a lock.
>> 
>
> Huh? Doesn't malloc do it's own locking when modifying global data
> structures?

This is what I would have expected.

Can someone please verify that malloc is reentrant?

Best regards,
Mikael D.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: [pthreads] performance test using fib.scm

[Maciej Stachowiak]

On 11Dec2002 08:11AM (+0100), Mikael Djurfeldt wrote:
> Maciej Stachowiak <mjs@noisehavoc.org> writes:
> 
> > On 10Dec2002 10:10PM (+0100), Wolfgang Jaehrling wrote:
> >> malloc() in glibc is thread-save, but not reentrant.  Since the
> >> behaviour of realloc() is unlikely to differ from malloc() in this
> >> respect, you most certainly have to protect calls to it with a lock.
> >> 
> >
> > Huh? Doesn't malloc do it's own locking when modifying global data
> > structures?
> 
> This is what I would have expected.
> 
> Can someone please verify that malloc is reentrant?

According to the best info I can find on Google, glibc's malloc can be
called safely from multiple threads but cannot necessarily safely be
called from a signal handler (another possible definition of
re-entrant). A mutex would not help with calling it from a signal
handler anyway, though. So don't bother with locking.

If you truly suspect heap corruption I suggest running with
MALLOC_CHECK_=2 set in your environment. This should detect many kinds
of heap corruption.

Regards,

Maciej





_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: [pthreads] performance test using fib.scm

[Han-Wen Nienhuys]

mjs@noisehavoc.org writes:
> If you truly suspect heap corruption I suggest running with
> MALLOC_CHECK_=2 set in your environment. This should detect many kinds
> of heap corruption.

for that matter, valgrind is even better: it detects other kinds of
errors.

--
Han-Wen Nienhuys   |   hanwen@cs.uu.nl   |   http://www.cs.uu.nl/~hanwen 


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: [pthreads] performance test using fib.scm

[Wolfgang Jaehrling]

On Wed, Dec 11, 2002 at 08:11:50AM +0100, Mikael Djurfeldt wrote:
> Can someone please verify that malloc is reentrant?

<http://sources.redhat.com/ml/libc-alpha/2002-11/msg00325.html>

Cheers,
GNU/Wolfgang


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: [pthreads] performance test using fib.scm

[Wolfgang Jaehrling]

On Wed, Dec 11, 2002 at 11:02:14AM +0900, NIIBE Yutaka wrote:
> Wolfgang Jaehrling wrote:
>  > malloc() in glibc is thread-save, but not reentrant.  Since the
>  > behaviour of realloc() is unlikely to differ from malloc() in this
>  > respect, you most certainly have to protect calls to it with a lock.
> 
> I think that we need some definition of terminology to avoid
> confusion.  You mean signal protection, with the word "lock", not a
> mutex, don't you?

Yes.  However, that malloc() in glibc is thread-save does not imply
that this is the case for other libc:s as well, so a mutex _might_
still be necessary for some systems.  If guile does not call malloc()
(or realloc()) from a signal handler, then it is supposed to not be a
problem for GNU/Linux and GNU/Hurd (and other systems using glibc).

Cheers,
GNU/Wolfgang


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel