Guile + pthreads + reentrancy?

Guile + pthreads + reentrancy?

[bindej]

I'm trying to get Guile working in a pthreaded program.  I ran up
against the stack overflow problem, as discussed here:

http://sources.redhat.com/ml/guile/1999-10/msg00114.html
http://mail.gnu.org/archive/html/guile-user/2001-08/msg00082.html
http://www.red-bean.com/guile/guile/old/1790.html

I did the message-queueing kludge.  It works, although it is messy.
But it's not reentrant.  If I tell Guile's pthread to call
scm_eval_string, which in turn calls a gsubr, and that gsubr spawns a
new pthread which queues a call to scm_makfrom0str, we have a
deadlock.

This situtation seems obscure, but it happens in the program I'm
writing.

The only solution I could come up with is to spawn a new Guile thread
for each item popped off the queue, so the loop doesn't stop while a
function is being called.  But Guile threads are cooperative, and I
can't figure out any way to have scm_yield called at the right time in
Guile's pthread.

Has anyone else succeeded at doing this?  Am I missing something
obvious?

Jeff Binder



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[Mikael Djurfeldt]

bindej@rpi.edu writes:

> I'm trying to get Guile working in a pthreaded program.  I ran up
> against the stack overflow problem, as discussed here:
>
> http://sources.redhat.com/ml/guile/1999-10/msg00114.html
> http://mail.gnu.org/archive/html/guile-user/2001-08/msg00082.html
> http://www.red-bean.com/guile/guile/old/1790.html
>
> I did the message-queueing kludge.  It works, although it is messy.
> But it's not reentrant.  If I tell Guile's pthread to call
> scm_eval_string, which in turn calls a gsubr, and that gsubr spawns a
> new pthread which queues a call to scm_makfrom0str, we have a
> deadlock.
>
> This situtation seems obscure, but it happens in the program I'm
> writing.
>
> The only solution I could come up with is to spawn a new Guile thread
> for each item popped off the queue, so the loop doesn't stop while a
> function is being called.  But Guile threads are cooperative, and I
> can't figure out any way to have scm_yield called at the right time in
> Guile's pthread.
>
> Has anyone else succeeded at doing this?  Am I missing something
> obvious?

The thread support in Guile has undergone two revisions since the
version you are using.  Do you have constraints for which version of
Guile to use?  Current CVS HEAD have full pthreads support, which
should make you able to remove the message queue and access Guile
directly from each thread.

M



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[Mikael Djurfeldt]

Mikael Djurfeldt <mdj@mit.edu> writes:

> Current CVS HEAD have full pthreads support, which should make you
> able to remove the message queue and access Guile directly from each
> thread.

Each thread which has been spawned by scm_spawn_thread, that is.

scm_spawn_thread calls pthread_create and sets up Guile things.  If,
for some reason, you need to create the pthread yourself and want to
later make it able to use Guile, that could be arranged.  Please tell
me and I'll add this to the API (this is on the TODO list).

M


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[bindej]

Mikael Djurfeldt writes:

 > Each thread which has been spawned by scm_spawn_thread, that is.
 > 
 > scm_spawn_thread calls pthread_create and sets up Guile things.  If,
 > for some reason, you need to create the pthread yourself and want to
 > later make it able to use Guile, that could be arranged.  Please tell
 > me and I'll add this to the API (this is on the TODO list).
 > 
 > M

That would probably be best.  My project is a library, and its Guile
support is optional.  Being able to still use pthread_create would be
cleaner (fewer conditionals).

Thanks.



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[bindej]

Mikael Djurfeldt writes:
 > Each thread which has been spawned by scm_spawn_thread, that is.
 > 
 > scm_spawn_thread calls pthread_create and sets up Guile things.  If,
 > for some reason, you need to create the pthread yourself and want to
 > later make it able to use Guile, that could be arranged.  Please tell
 > me and I'll add this to the API (this is on the TODO list).
 > 
 > M

I'm now converting my code to use scm_spawn_thread instead of
pthread_create.  This is not a problem, but there doesn't seem to be
any public interface to get a scm_t_thread out of the SCM object.
This is a serious problem because even functions such as
scm_thread_join take scm_t_thread arguments, and are thus unusable
outside of Guile.

What should I do about this?



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[Mikael Djurfeldt]

bindej@rpi.edu writes:

> Mikael Djurfeldt writes:
>  > Each thread which has been spawned by scm_spawn_thread, that is.
>  > 
>  > scm_spawn_thread calls pthread_create and sets up Guile things.  If,
>  > for some reason, you need to create the pthread yourself and want to
>  > later make it able to use Guile, that could be arranged.  Please tell
>  > me and I'll add this to the API (this is on the TODO list).
>  > 
>  > M
>
> I'm now converting my code to use scm_spawn_thread instead of
> pthread_create.  This is not a problem, but there doesn't seem to be
> any public interface to get a scm_t_thread out of the SCM object.

You're right that there isn't currently any public interface for
getting the scm_t_thread value out of the SCM object.

> This is a serious problem because even functions such as
> scm_thread_join take scm_t_thread arguments, and are thus unusable
> outside of Guile.
>
> What should I do about this?

Would it be difficult to use scm_thread_self ()?

We should probably add a selector for this, though.  I'll try to add
it to CVS HEAD today.  (Do you need it in 1.6?)

BTW, while the "old" parts of the thread API are pretty stable, the
new parts added in HEAD are still under development, so while there
probably won't be any major changes, some things *will* change.

M


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[bindej]

Mikael Djurfeldt writes:
 > bindej@rpi.edu writes:
 > 
 > > Mikael Djurfeldt writes:
 > >  > Each thread which has been spawned by scm_spawn_thread, that is.
 > >  > 
 > >  > scm_spawn_thread calls pthread_create and sets up Guile things.  If,
 > >  > for some reason, you need to create the pthread yourself and want to
 > >  > later make it able to use Guile, that could be arranged.  Please tell
 > >  > me and I'll add this to the API (this is on the TODO list).
 > >  > 
 > >  > M
 > >
 > > I'm now converting my code to use scm_spawn_thread instead of
 > > pthread_create.  This is not a problem, but there doesn't seem to be
 > > any public interface to get a scm_t_thread out of the SCM object.
 > 
 > You're right that there isn't currently any public interface for
 > getting the scm_t_thread value out of the SCM object.
 > 
 > > This is a serious problem because even functions such as
 > > scm_thread_join take scm_t_thread arguments, and are thus unusable
 > > outside of Guile.
 > >
 > > What should I do about this?
 > 
 > Would it be difficult to use scm_thread_self ()?

Somewhat.  The value is supposed to be available to the parent thread
immediately, so to be safe I would need to use a condition variable
and a global variable to communicate.

 > We should probably add a selector for this, though.  I'll try to add
 > it to CVS HEAD today.  (Do you need it in 1.6?)

My program requires 1.7 anyway, so no.



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[Mikael Djurfeldt]

bindej@rpi.edu writes:

> there doesn't seem to be any public interface to get a scm_t_thread
> out of the SCM object.

OK, CVS HEAD now has:

SCM_API scm_t_thread scm_c_scm2thread (SCM thread);

M


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[bindej]

Mikael Djurfeldt writes:
 > bindej@rpi.edu writes:
 > 
 > > there doesn't seem to be any public interface to get a scm_t_thread
 > > out of the SCM object.
 > 
 > OK, CVS HEAD now has:
 > 
 > SCM_API scm_t_thread scm_c_scm2thread (SCM thread);

Thanks.



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[Marius Vollmer]

bindej@rpi.edu writes:

> I'm now converting my code to use scm_spawn_thread instead of
> pthread_create.  This is not a problem, but there doesn't seem to be
> any public interface to get a scm_t_thread out of the SCM object.

This shouldn't be necessary, scm_t_thread is 'internal'.

> This is a serious problem because even functions such as
> scm_thread_join take scm_t_thread arguments, and are thus unusable
> outside of Guile.

Use scm_join_thread, which takes a SCM thread object.

The docs need some work, probably...

-- 
GPG: D5D4E405 - 2F9B BCCC 8527 692A 04E3  331E FAF8 226A D5D4 E405


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[bindej]

Mikael Djurfeldt writes:

 > The thread support in Guile has undergone two revisions since the
 > version you are using.  Do you have constraints for which version of
 > Guile to use?  Current CVS HEAD have full pthreads support, which
 > should make you able to remove the message queue and access Guile
 > directly from each thread.

Thanks for your reply.  I would prefer not to require my users to
install a CVS version of Guile.  On the other hand my package is in
very early development right now, so the next version of Guile may be
released before we release anything.

Is there any chance of a backport?  I would imagine not, because the
changes seem extensive.



_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[Mikael Djurfeldt]

bindej@rpi.edu writes:

> Is there any chance of a backport?  I would imagine not, because the
> changes seem extensive.

Your right.  That would probably be very burdensome.

Sorry.

M


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[Thien-Thi Nguyen]

   From: bindej@rpi.edu
   Date: Tue, 24 Feb 2004 16:19:24 -0500

   Is there any chance of a backport?

   I would imagine not, because the changes seem extensive.

aside from the constraint that changes do not break old programs,
it does not matter how extensive they are.  it only matters that
someone finds time and inclination to do the work.

the news that things have been revised twice leads me to wonder
if they will be revised yet again.  if so, why?  if not, why not?

thi


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[Mikael Djurfeldt]

Thien-Thi Nguyen <ttn@surf.glug.org> writes:

> the news that things have been revised twice leads me to wonder
> if they will be revised yet again.

Count on it.

> if so, why?

Well, it's called "progress".  :)

(And, no, there weren't any substantial changes to the API:s---old
programs are safe.)

M


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[Thien-Thi Nguyen]

   From: Mikael Djurfeldt <djurfeldt@nada.kth.se>
   Date: Tue, 24 Feb 2004 22:06:01 -0500

   Well, it's called "progress".  :)

   (And, no, there weren't any substantial changes to the API:s---old
   programs are safe.)

it all depends on the precise set of changes involved.  one
programmer's insubstantial changes is another's massive rework
requirement.

thi


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user

Re: Guile + pthreads + reentrancy?

[Mikael Djurfeldt]

bindej@rpi.edu writes:

> I did the message-queueing kludge.  It works, although it is messy.
> But it's not reentrant.  If I tell Guile's pthread to call
> scm_eval_string, which in turn calls a gsubr, and that gsubr spawns a
> new pthread which queues a call to scm_makfrom0str, we have a
> deadlock.

One thing you could do is to insert a conditional into the
queue_insert routine which checks whether the caller is the Guile
thread.  If it is, you evaluate the form immediately instead of
queuing it up.

M


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-user