Using define in multiple threads?

Using define in multiple threads?

[Linas Vepstas]

Is it "safe" or "legal" to use define in multiple threads?

I am currently looking at an infinite loop in guile, it looks
like there's a pair with cdr pointing to itself.  The pair is
a hashtable bucket. I'm reading through hashtab.c
and see no locking at all, and so it seems probable that
this circular bucket list was created in a race between
two threads inserting into the same hash table.

Now, the test case which triggers this does nothing
other than defines --  it loads a set of files, one per thread,
and, for good measure, I reloaded these over and over.
So typically, one will have the same function being
redefined over and over, typically in different threads.
The same function might be getting defined "simultaneously"
in different threads, even.

So-- I was assuming that this would be an inherently
"safe" operation ... but maybe its not? Is there some funky
semantics to define in a multi-threaded environment?

--linas

Re: Using define in multiple threads?

[Han-Wen Nienhuys]

Linas Vepstas escreveu:
> Is it "safe" or "legal" to use define in multiple threads?

I guess not.  Someone -I forgot who- put in the pthreads without thinking 
through the consequences.

Look through eval.c, you´ll see

  SCM_SETCAR (expr, SCM_IM_DO);
  // *
  SCM_SETCDR (expr, tail);

which is very dubious if a thread switch happens at (*)

-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

Re: Using define in multiple threads?

[Ludovic Courtès]

Hey!

Han-Wen Nienhuys <hanwen@xs4all.nl> writes:

> Linas Vepstas escreveu:
>> Is it "safe" or "legal" to use define in multiple threads?
>
> I guess not.  Someone -I forgot who- put in the pthreads without thinking 
> through the consequences.
>
> Look through eval.c, you´ll see
>
>   SCM_SETCAR (expr, SCM_IM_DO);
>   // *
>   SCM_SETCDR (expr, tail);
>
> which is very dubious if a thread switch happens at (*)

You're answering a more general question: "is it safe to evaluate code
in multiple threads?".  I'm afraid your answer is correct, but, if it
happens that the code to be evaluated has already been memoized, then
everything's alright.  Thankfully, the VM will soon save us from this.
:-)

Besides, as Linas noted, the hash table implementation isn't
thread-safe, which makes it "unsafe" to `define' in parallel.

Thanks,
Ludo'.

Re: Using define in multiple threads?

[Andy Wingo]

Hello!

On Mon 17 Nov 2008 15:57, ludo@gnu.org (Ludovic Courtès) writes:

> Han-Wen Nienhuys <hanwen@xs4all.nl> writes:
>
>> Linas Vepstas escreveu:
>>> Is it "safe" or "legal" to use define in multiple threads?
>
> Besides, as Linas noted, the hash table implementation isn't
> thread-safe, which makes it "unsafe" to `define' in parallel.

Ah, this is why, then. I was baffled.

IMO this is a bug; Guile can claim that "wierd behavior is expected"
when using a data structure from multiple threads in without locking,
but in this case the data structure is only being used implicitly.
Modules should contain a lock that is taken when they are mutated.

This isn't the first time I've wanted a functional hash table
implementation...

Andy
-- 
http://wingolog.org/

Re: Using define in multiple threads?

[Ludovic Courtès]

Andy Wingo <wingo@pobox.com> writes:

> On Mon 17 Nov 2008 15:57, ludo@gnu.org (Ludovic Courtès) writes:

>> Besides, as Linas noted, the hash table implementation isn't
>> thread-safe, which makes it "unsafe" to `define' in parallel.
>
> Ah, this is why, then. I was baffled.
>
> IMO this is a bug; Guile can claim that "wierd behavior is expected"
> when using a data structure from multiple threads in without locking,
> but in this case the data structure is only being used implicitly.
> Modules should contain a lock that is taken when they are mutated.

Right.

I submitted a bug: https://savannah.gnu.org/bugs/index.php?24867 .  I'm
not sure whether this can easily be fixed in 1.8, since that would
require adding a mutex to `module-type' and access it from C, which
would break the ABI.  Ideas?

> This isn't the first time I've wanted a functional hash table
> implementation...

I wrote long ago an implementation of functional FIFOs and one of
Bagwell's "VLists" [0]; VLists can be used at a building block for "hash
lists", which is a functional data structure akin to hash tables [0].
I've been willing to polish it and push it into Guile for some time, so
if you're motivated, that's even more motivating.  :-)

Thanks,
Ludo'.

[0] http://en.wikipedia.org/wiki/VList

Re: Using define in multiple threads?

[Neil Jerram]

2008/11/18 Ludovic Courtès <ludo@gnu.org>:
>
> I submitted a bug: https://savannah.gnu.org/bugs/index.php?24867 .  I'm
> not sure whether this can easily be fixed in 1.8, since that would
> require adding a mutex to `module-type' and access it from C, which
> would break the ABI.  Ideas?

Where exactly is the troublesome hash table?  (There's a lot of code
underneath scm_m_define(), so if you've already tracked it down, that
would save a little time.)

Regards,
      Neil

Re: Using define in multiple threads?

[Andy Wingo]

On Thu 20 Nov 2008 00:36, "Neil Jerram" <neiljerram@googlemail.com> writes:

> 2008/11/18 Ludovic Courtès <ludo@gnu.org>:
>>
>> I submitted a bug: https://savannah.gnu.org/bugs/index.php?24867 .  I'm
>> not sure whether this can easily be fixed in 1.8, since that would
>> require adding a mutex to `module-type' and access it from C, which
>> would break the ABI.  Ideas?
>
> Where exactly is the troublesome hash table?  (There's a lot of code
> underneath scm_m_define(), so if you've already tracked it down, that
> would save a little time.)

Don't know myself exactly, but I assume this is referring to the
module-obarray.

Andy
-- 
http://wingolog.org/

Re: Using define in multiple threads?

[Ludovic Courtès]

Andy Wingo <wingo@pobox.com> writes:

> On Thu 20 Nov 2008 00:36, "Neil Jerram" <neiljerram@googlemail.com> writes:
>
>> 2008/11/18 Ludovic Courtès <ludo@gnu.org>:
>>>
>>> I submitted a bug: https://savannah.gnu.org/bugs/index.php?24867 .  I'm
>>> not sure whether this can easily be fixed in 1.8, since that would
>>> require adding a mutex to `module-type' and access it from C, which
>>> would break the ABI.  Ideas?
>>
>> Where exactly is the troublesome hash table?  (There's a lot of code
>> underneath scm_m_define(), so if you've already tracked it down, that
>> would save a little time.)
>
> Don't know myself exactly, but I assume this is referring to the
> module-obarray.

Exactly.

Ludo'.