unofficial mirror of guile-devel@gnu.org 
 help / color / mirror / Atom feed
* About cleaning up the thread stuff
@ 2005-01-18 15:54 Marius Vollmer
  2005-01-18 17:00 ` Marius Vollmer
                   ` (2 more replies)
  0 siblings, 3 replies; 13+ messages in thread
From: Marius Vollmer @ 2005-01-18 15:54 UTC (permalink / raw)


Hi,

I am now turning to go over the thread stuff in Guile, first to
understand what we have, and then to maybe clean it up and document it
authoritatively.

I'd like to give you advance warning of where I am heading at the
moment so that you can talk me out of doing the wrong things.  All of
this might change as I make up my mind, but please comment anyway.

I do the work on the 'branch_mvo-thread-cleanup' branch.

 - First, I want to rip out the plugin interface since it confuses me
   a bit.  Maybe I'll put it back in later, but to get started, I only
   want to deal with POSIX pthreads.  Guile will be able to use
   pthreads, but can also be compiled without pthreads.

   I consider the pthread API to be the plugin interface where
   different threading implementations can be selected.  Maybe it is
   useful to make the names configurable that Guile uses for the
   pthread API, but it should always be a pthread API.

 - Then I want to specify clearly the two modes that a program can be
   in: 'guile mode' and 'non-guile mode'.  In guile mode, a program
   can use functions from the Guile API and must play by the rules of
   Guile, in non-guile mode, it can not call Guile functions (safe a
   very few exceptions) but does not need to play by the rules.

   The rules are basically that a thread must periodically execute
   SCM_TICK and must only block in functions provided by Guile.  If
   you want to block in other ways, you must first leave guile mode
   (and enter again after blocking).  Stack frames that are created in
   non-guile mode are not scanned by the GC so you can not store SCM
   values in them.

   You can enter guile mode in any thread, but Guile offers
   convenience functions for creating threads that start out in Guile
   mode.  When a thread first enters guile mode, it gets a 'guile
   personality' and shows up in (all-threads), for example.  When it
   leaves guile mode, it keeps this personality and is only removed
   from (all-threads) when it terminates.

 - For Scheme (and also for C code that wants to use them), I want to
   offer 'fat' mutexes and condition variables.

   These are mutexes and condvars that are as nice as reasonable: they
   are fair, you can query easily where threads are blocking, asyncs
   are handled while blocking, dead-locks might be detected and
   mutexes can be recursive, for example.

   No other kinds of mutexes are offered, for example, all mutexes
   used by Scheme code will be of this 'fat' kind.  (The low level C
   API will disappear.)  If you want to use pthread mutexes, you can,
   but you must leave guile mode while blocking on them.

   To make this easy, there will be functions like
   scm_pthread_mutex_lock that is just like pthread_mutex_lock but
   will properly leave guile mode while blocking.

   (The only advantage that pthread mutexes have over these 'fat' ones
   is that they are slightly faster and slightly smaller.  I don't
   consider these reasons good enough to lose the advantages of the
   fat ones.)


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-01-18 15:54 About cleaning up the thread stuff Marius Vollmer
@ 2005-01-18 17:00 ` Marius Vollmer
  2005-01-18 18:03   ` Julian Graham
  2005-01-18 18:08   ` Andy Wingo
  2005-01-18 22:02 ` Kevin Ryde
  2005-03-02 21:09 ` Marius Vollmer
  2 siblings, 2 replies; 13+ messages in thread
From: Marius Vollmer @ 2005-01-18 17:00 UTC (permalink / raw)
  Cc: guile-devel

Marius Vollmer <marius.vollmer@uni-dortmund.de> writes:

> All of this might change as I make up my mind, but please comment
> anyway.

First radical thing: dynamic roots will go... their function will be
taken over by thread structure directly.  Good idea?


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-01-18 17:00 ` Marius Vollmer
@ 2005-01-18 18:03   ` Julian Graham
  2005-01-18 21:03     ` Marius Vollmer
  2005-01-18 18:08   ` Andy Wingo
  1 sibling, 1 reply; 13+ messages in thread
From: Julian Graham @ 2005-01-18 18:03 UTC (permalink / raw)


I like that, but only because I could never figure out how to use
dynamic roots.  ;)

Re: your other points,

So what will happen if you build Guile without the threading stuff? 
Are the thread procedures (e.g., 'make-thread',
'call-with-new-thread', etc.) simply not made visible?

When you say that any thread may enter guile-mode, do you mean that
the prohibition against more than one thread of C code making calls to
Guile's API is being lifted?

Also, let me know if you'd like me to tweak that thread-cancellation
patch I sent in a few months ago to reflect the changes you're talking
about.  (Or maybe you'd prefer to write it from scratch yourself?)


Cheers,
Julian

On Tue, 18 Jan 2005 18:00:46 +0100, Marius Vollmer
<marius.vollmer@uni-dortmund.de> wrote:

> First radical thing: dynamic roots will go... their function will be
> taken over by thread structure directly.  Good idea?


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-01-18 17:00 ` Marius Vollmer
  2005-01-18 18:03   ` Julian Graham
@ 2005-01-18 18:08   ` Andy Wingo
  2005-01-18 21:04     ` Marius Vollmer
  1 sibling, 1 reply; 13+ messages in thread
From: Andy Wingo @ 2005-01-18 18:08 UTC (permalink / raw)


On Tue, 2005-01-18 at 18:00 +0100, Marius Vollmer wrote:
> First radical thing: dynamic roots will go... their function will be
> taken over by thread structure directly.  Good idea?

What about protecting against capturing a continuation from within a
callback from C? No need for a thread there...

Wingo.



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-01-18 18:03   ` Julian Graham
@ 2005-01-18 21:03     ` Marius Vollmer
  0 siblings, 0 replies; 13+ messages in thread
From: Marius Vollmer @ 2005-01-18 21:03 UTC (permalink / raw)
  Cc: guile-devel

Julian Graham <joolean@gmail.com> writes:

> I like that, but only because I could never figure out how to use
> dynamic roots.  ;)

(I have decided to leave them in for a while... :-)

> So what will happen if you build Guile without the threading stuff? 
> Are the thread procedures (e.g., 'make-thread',
> 'call-with-new-thread', etc.) simply not made visible?

No, all functions will be there, but they wont do much.  I.e., you can
create mutexes and lock them, but you can't create new threads.

> When you say that any thread may enter guile-mode, do you mean that
> the prohibition against more than one thread of C code making calls to
> Guile's API is being lifted?

Yes, this restriction has been lifted already.  The remaining
restriction is that only threads created by Guile itself are allowed
to call Guile.  I hope to remove that restriction as well.  Also, I
hope to make thigns so that any thread can initialize Guile.

> Also, let me know if you'd like me to tweak that thread-cancellation
> patch I sent in a few months ago to reflect the changes you're talking
> about.  (Or maybe you'd prefer to write it from scratch yourself?)

I'll come back to you about this.  Is your patch finished?


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-01-18 18:08   ` Andy Wingo
@ 2005-01-18 21:04     ` Marius Vollmer
  0 siblings, 0 replies; 13+ messages in thread
From: Marius Vollmer @ 2005-01-18 21:04 UTC (permalink / raw)
  Cc: guile-devel

Andy Wingo <wingo@pobox.com> writes:

> On Tue, 2005-01-18 at 18:00 +0100, Marius Vollmer wrote:
>> First radical thing: dynamic roots will go... their function will be
>> taken over by thread structure directly.  Good idea?
>
> What about protecting against capturing a continuation from within a
> callback from C? No need for a thread there...

Yep.  I have left dynamic roots in for the time being, which was
simpler after all.  We have a nice proposal in
workbook/extensions/dynamic-root.text that might replace dynamic
roots, tho.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-01-18 15:54 About cleaning up the thread stuff Marius Vollmer
  2005-01-18 17:00 ` Marius Vollmer
@ 2005-01-18 22:02 ` Kevin Ryde
  2005-01-18 23:05   ` Marius Vollmer
  2005-03-02 21:09 ` Marius Vollmer
  2 siblings, 1 reply; 13+ messages in thread
From: Kevin Ryde @ 2005-01-18 22:02 UTC (permalink / raw)
  Cc: guile-user, guile-devel

Marius Vollmer <marius.vollmer@uni-dortmund.de> writes:
>
>    dead-locks might be detected

I wouldn't worry about that.  I think the combinations of things a
thread can block on will make it too hard to be sure a deadlock is
occurring.  There might even be some weird apparent deadlock that's
meant to be broken by a signal handler cancelling one thread or
something.

>    mutexes can be recursive

Yep, I think that's friendliest.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-01-18 22:02 ` Kevin Ryde
@ 2005-01-18 23:05   ` Marius Vollmer
  2005-01-19  0:02     ` Kevin Ryde
  0 siblings, 1 reply; 13+ messages in thread
From: Marius Vollmer @ 2005-01-18 23:05 UTC (permalink / raw)
  Cc: guile-devel

Kevin Ryde <user42@zip.com.au> writes:

> Marius Vollmer <marius.vollmer@uni-dortmund.de> writes:
>>
>>    dead-locks might be detected
>
> I wouldn't worry about that.  I think the combinations of things a
> thread can block on will make it too hard to be sure a deadlock is
> occurring.  There might even be some weird apparent deadlock that's
> meant to be broken by a signal handler cancelling one thread or
> something.

Yes, true.  Maybe the debugger could still try to figure out whether
there is a dead-lock, or the program could explicitely ask for it.
Etc.

>>    mutexes can be recursive
>
> Yep, I think that's friendliest.

I think there is use for both kinds of mutexes.  Sometimes you want a
recursive one, sometimes locking twice from the same thread is an
error.

-- 
GPG: D5D4E405 - 2F9B BCCC 8527 692A 04E3  331E FAF8 226A D5D4 E405


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-01-18 23:05   ` Marius Vollmer
@ 2005-01-19  0:02     ` Kevin Ryde
  2005-02-01 16:02       ` Marius Vollmer
  0 siblings, 1 reply; 13+ messages in thread
From: Kevin Ryde @ 2005-01-19  0:02 UTC (permalink / raw)
  Cc: guile-user, guile-devel

Marius Vollmer <mvo@zagadka.de> writes:
>
> Maybe the debugger could still try to figure out whether
> there is a dead-lock,

That'd be nice, to see what each is apparently getting up to.

Speaking of threads, there's more low level things that need to be
reviewed for safety.

	- gethostbyname and friends definitely
	- string->symbol I think needs protection
	- symbol->keyword might be ok, but the SCM_DEFER_INTS
	  could become some explicit mutex

I was doing some of the C bits with mutexes, but I'm thinking it may
be better just to demand the "_r" functions whenever threading is
enabled.  If the posix threading funcs exist then so should the _r's,
and those _r's will give maximum concurrency.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-01-19  0:02     ` Kevin Ryde
@ 2005-02-01 16:02       ` Marius Vollmer
  0 siblings, 0 replies; 13+ messages in thread
From: Marius Vollmer @ 2005-02-01 16:02 UTC (permalink / raw)
  Cc: guile-devel

Kevin Ryde <user42@zip.com.au> writes:

> Speaking of threads, there's more low level things that need to be
> reviewed for safety.

Yes, definitely.  I want to do a complete code review...


_______________________________________________
Guile-user mailing list
Guile-user@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-user


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-01-18 15:54 About cleaning up the thread stuff Marius Vollmer
  2005-01-18 17:00 ` Marius Vollmer
  2005-01-18 22:02 ` Kevin Ryde
@ 2005-03-02 21:09 ` Marius Vollmer
  2005-03-04  0:46   ` Kevin Ryde
  2 siblings, 1 reply; 13+ messages in thread
From: Marius Vollmer @ 2005-03-02 21:09 UTC (permalink / raw)


Marius Vollmer <marius.vollmer@uni-dortmund.de> writes:

> I do the work on the 'branch_mvo-thread-cleanup' branch.

Ok, I have merged this branch now back into HEAD, which probably
causes a lot of instability.

I will say more about this here, once I am satisfied with updating
NEWS and manual, which I haven't done yet completely.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-03-02 21:09 ` Marius Vollmer
@ 2005-03-04  0:46   ` Kevin Ryde
  2005-03-04 11:33     ` Marius Vollmer
  0 siblings, 1 reply; 13+ messages in thread
From: Kevin Ryde @ 2005-03-04  0:46 UTC (permalink / raw)
  Cc: guile-devel

Marius Vollmer <marius.vollmer@uni-dortmund.de> writes:
>
> Ok, I have merged this branch now back into HEAD, which probably
> causes a lot of instability.

With glibc 2.3.2, in eval.c I had to #define _GNU_SOURCE to get
PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP from <pthread.h>.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: About cleaning up the thread stuff
  2005-03-04  0:46   ` Kevin Ryde
@ 2005-03-04 11:33     ` Marius Vollmer
  0 siblings, 0 replies; 13+ messages in thread
From: Marius Vollmer @ 2005-03-04 11:33 UTC (permalink / raw)


Kevin Ryde <user42@zip.com.au> writes:

> Marius Vollmer <marius.vollmer@uni-dortmund.de> writes:
>>
>> Ok, I have merged this branch now back into HEAD, which probably
>> causes a lot of instability.
>
> With glibc 2.3.2, in eval.c I had to #define _GNU_SOURCE to get
> PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP from <pthread.h>.

Yes, thanks.  This is actually one of the portability
regressions... we had code to (con)figure out how to get a recursive
mutex portably, and I should probably reinstall it.


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel


^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2005-03-04 11:33 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-01-18 15:54 About cleaning up the thread stuff Marius Vollmer
2005-01-18 17:00 ` Marius Vollmer
2005-01-18 18:03   ` Julian Graham
2005-01-18 21:03     ` Marius Vollmer
2005-01-18 18:08   ` Andy Wingo
2005-01-18 21:04     ` Marius Vollmer
2005-01-18 22:02 ` Kevin Ryde
2005-01-18 23:05   ` Marius Vollmer
2005-01-19  0:02     ` Kevin Ryde
2005-02-01 16:02       ` Marius Vollmer
2005-03-02 21:09 ` Marius Vollmer
2005-03-04  0:46   ` Kevin Ryde
2005-03-04 11:33     ` Marius Vollmer

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).