Re: guile threading deadlock

["Linas Vepstas" <linasvepstas@gmail.com>]

2008/11/9 Ludovic Courtès <ludo@gnu.org>:
> Hi,
>
> "Linas Vepstas" <linasvepstas@gmail.com> writes:
>
>> scm_without_guile is also clearly not an option [...].  Its rather
>> ludicrous to even suggest that such wrappers be used -- typically,
>> such things are not ever within reach of the developer.
>
> [...]
>
>> I found a simple solution that works for me: just wrap
>> calls to scm_c_eval_string()  with calls to scm_with_guile().
>
> Looks like you changed your mind by the end of the message!  ;-)

?? What did I change my mind about?  I don't understand
what you are saying here.

I still think that its absurd to ask the select call, or the
pthread calls, to be wrapped, since these calls will
typically be in modules outside of the programmers
control.  ("Mechanism, not policy" is what one usually
says)

However, there is a simple solution: wrap every call
to scm_c_eval_string() with a call to scm_with_guile().
Doesn't seem very optimal or pretty, but it works.

>> Yes, well, I eventually figured it out. The very first sentence
>> of http://www.gnu.org/software/guile/manual/html_node/Blocking.html
>>
>> says it all: "A thread must not block outside of a libguile
>> function while it is in guile mode. "  Unfortunately, this
>> sentence is completely missing from the section called
>> "5.3 Initializing guile", which offers no hint of this behaviour.
>
> Hmm, I don't think it's directly related to Guile initialization.  How
> would you like to change Section 5.3?

I'd change it to indicate that there are multi-threading issues
to consider.  Guile does a highly "non-standard" thing, that
would come as a surprise to anyone who has ever done
multi-threaded., multi-processing work, or any kernel work:
it returns to the user while holding a lock.  This is such a
strange and unusual thing to do, that it should be clearly documented.
 (I also think that this should probably be
changed/fixed in some future version of guile, but that
is another matter).

In particular, any thread that uses scm_init_guile cannot
sleep -- this should be stated up front. Any thread that
is "in guile mode" cannot sleep. This too should be stated
up front.

>> Similarly, there is no hint of this in section 4.3.5
>> "Multi-threading".  Instead, I see sentences that say
>> things like "a thread promises to reach a safe point
>> reasonably frequently (see Asynchronous Signals)"
>>
>> What the heck is "a safe point"? Who is making the
>> "promise"? Guile or the user app?  What the heck do
>> "safe points" have to do with signals?   So I think that
>> section is trying to warn about the deadlock, but fails
>> utterly to actually come out and say it .... the
>> documentation on this topic needs an overhaul.
>
> I reread that part and the rest of the paragraph makes it clearer, IMO:

Clear as mud!   It comes off as word salad to me; I don't
want to be insulting, in case you are the one who wrote it
-- but it does need a total re-write.

Here's my point of view, ... I've got a PhD, I've been coding
for decades; my last job was as a linux kernel programmer,
doing close-to-the-hardware work, where there are many,
many issues with sync points and locking and races and
order-of-initialization & etc.  Yet, when I tried to read that
paragraph, I could not make sense out of it. If I can't figure
it out, who can?

> Note also that one of the first sentences of that section is:
>
>     Each thread that wants to use functions from libguile must put
>  itself into _guile mode_ and must then follow a few rules.

Whereupon it utterly *fails* to even begin to enumerate
any rules at all, lapsing into some incomprehensible
gobbldy-gook.

> IMO this indicates that "reaching a safe point frequently" is one of
> these rules.

What the heck is a "safe point"?  Whose job is it to reach
this "safe point"? Who is making the promises? Guile?
the use application? Etc. These sort of things need to be
made clear.

> Do you have any suggestions to clarify that?

Well, yes.  But it would perhaps be easiest for me
to write the text myself, rather than explain how the
text could be written.

>> I suppose I could volunteer to provide updated documentation,
>
> Yes!  :-)

Well, if I provided patch files, would that work? Where would
I send these?  Who maintains this stuff?

--linas