guile threading deadlock

guile threading deadlock

[Linas Vepstas]

I've got a little deadlock problem w/ guile.  Here's the pseudocode:

for (i=0; i<10 i++)  {
   pthread_create (load_scm, expr[i]);
}

void load_scm (char * expr) {
   scm_init_guile();
   scm_c_eval(expr);
   /* and pthread exit now that we're done */
}

The expr's are entirely scm lambda defines of one sort or another,
and do not do any actual computation.

The above deadlocks; (using guile-1.8.5) .

In the stack traces below,
Only three threads are deadlocked, in total.
The first is stuck in some sort of scheme thread exit code, the
second is stuck in gc, and the third is stuck in scm_init.

This is one of the cleanest set of stack traces I've seen; other
ones have one or two threads stuck in gc, many in scm_init.

I'm guessing there's some bug in the thread-exit code. I'm
thinking the guile thread-exit handler leaves some kind of
lock behind.

#0  0xffffe425 in __kernel_vsyscall ()
#1  0xf7e43589 in __lll_lock_wait () from /lib/tls/i686/cmov/libpthread.so.0
#2  0xf7e3eba6 in _L_lock_95 () from /lib/tls/i686/cmov/libpthread.so.0
#3  0xf7e3e58a in pthread_mutex_lock () from /lib/tls/i686/cmov/libpthread.so.0
#4  0xf782729c in scm_pthread_mutex_lock (mutex=0xf785c568) at threads.c:1444
#5  0xf7828ad5 in do_thread_exit (v=0x9179e18) at threads.c:483
#6  0xf77b4842 in c_body (d=0xf6657328) at continuations.c:350
#7  0xf7829b19 in scm_c_catch (tag=0x104, body=0xf77b4830 <c_body>,
    body_data=0xf6657328, handler=0xf77b4850 <c_handler>,
    handler_data=0xf6657328,
    pre_unwind_handler=0xf78293e0 <scm_handle_by_message_noexit>,
    pre_unwind_handler_data=0x0) at throw.c:200
#8  0xf77b4cf2 in scm_i_with_continuation_barrier (body=0xf77b4830 <c_body>,
    body_data=0xf6657328, handler=0xf77b4850 <c_handler>,
    handler_data=0xf6657328,
    pre_unwind_handler=0xf78293e0 <scm_handle_by_message_noexit>,
    pre_unwind_handler_data=0x0) at continuations.c:326
#9  0xf77b4dd3 in scm_c_with_continuation_barrier (
    func=0xf7828ab0 <do_thread_exit>, data=0x9179e18) at continuations.c:368
#10 0xf78288f9 in scm_i_with_guile_and_parent (
    func=0xf7828ab0 <do_thread_exit>, data=0x9179e18, parent=0xf7693e70)
    at threads.c:695
#11 0xf78289ee in scm_with_guile (func=0xf7828ab0 <do_thread_exit>,
    data=0x9179e18) at threads.c:683
#12 0xf7828a43 in on_thread_exit (v=0x9179e18) at threads.c:505
#13 0xf7e3bbb0 in __nptl_deallocate_tsd ()
   from /lib/tls/i686/cmov/libpthread.so.0
#14 0xf7e3c509 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#15 0xf7c3ae5e in clone () from /lib/tls/i686/cmov/libc.so.6

Not sure who called clone ... not my code ... this is confusing. ...

#0  0xffffe425 in __kernel_vsyscall ()
#1  0xf7e43589 in __lll_lock_wait () from /lib/tls/i686/cmov/libpthread.so.0
#2  0xf7e3eba6 in _L_lock_95 () from /lib/tls/i686/cmov/libpthread.so.0
#3  0xf7e3e58a in pthread_mutex_lock () from /lib/tls/i686/cmov/libpthread.so.0
#4  0xf782720b in scm_i_thread_put_to_sleep () at threads.c:1588
#5  0xf77d1c09 in scm_i_gc (what=0xf784702e "cells") at gc.c:552
#6  0xf77d1f4d in scm_gc_for_newcell (freelist=0xf785c7ac,
    free_cells=0xf5500664) at gc.c:509
#7  0xf77e6114 in scm_list_2 (e1=0xf769fab0, e2=0x911b230)
    at ../libguile/inline.h:115
#8  0xf77cb518 in ceval (x=0x404, env=0xf768e7f0) at eval.c:4459
#9  0xf77cb856 in ceval (x=0xf7687ac0, env=0xf768e7f0) at eval.c:3399
#10 0xf77cb856 in ceval (x=0xf768ea20, env=0xf768eb08) at eval.c:3399
#11 0xf77cbb93 in ceval (x=0xf76d93d0, env=0xf768ce68) at eval.c:3665
#12 0xf77cb856 in ceval (x=0xf76dac68, env=0xf768ce50) at eval.c:3399
#13 0xf77cc169 in ceval (x=0xf76d89d8, env=0xf768c8d8) at eval.c:3650
#14 0xf77cbb93 in ceval (x=0xf7673ca8, env=0xf768d638) at eval.c:3665
#15 0xf77cd2e6 in call_closure_1 (proc=0xf768d620, arg1=0xf768d5a0)
    at eval.c:5263
#16 0xf77cdedd in scm_map (proc=0xf768d620, arg1=0xf768d5b0, args=0x404)
    at eval.c:5491
#17 0xf77cb5ba in ceval (x=0x404, env=0xf768d5c8) at eval.c:4369
#18 0xf77cc169 in ceval (x=0xf768d5f8, env=0xf768d5c8) at eval.c:3650
#19 0xf77cc179 in ceval (x=0xf768d468, env=0xf768d0a8) at eval.c:3370
#20 0xf77cd00a in scm_primitive_eval_x (exp=0xf768d070) at eval.c:5923
#21 0xf7823bb8 in inner_eval_string (data=0xf768d040) at strports.c:500
#22 0xf77cefde in scm_c_with_fluid (fluid=0x9119900, value=0xf769ae80,
    cproc=0xf7823b90 <inner_eval_string>, cdata=0xf768d040) at fluids.c:459
#23 0xf77e8335 in scm_c_call_with_current_module (module=0xf769ae80,
    func=0xf7823b90 <inner_eval_string>, data=0xf768d040) at modules.c:104
#24 0xf7823e21 in scm_eval_string_in_module (string=0xf769f860,
    module=0xf769ae80) at strports.c:527
#25 0xf7823e55 in scm_eval_string (string=0xf769f860) at strports.c:535
#26 0xf7823e85 in scm_c_eval_string (
    expr=0xf5500d04 "(use-modules (ice-9 streams))\n") at strports.c:481
#27 0xf7829b19 in scm_c_catch (tag=0x104, body=0xf7823e60 <scm_c_eval_string>,
    body_data=0xf5500d04,
    handler=0xf79522f8
<opencog::SchemeEval::catch_handler_wrapper(void*, scm_unused_struct*,
scm_unused_struct*)>, handler_data=0x9115c38,
    pre_unwind_handler=0xf795236a
<opencog::SchemeEval::preunwind_handler_wrapper(void*,
scm_unused_struct*, scm_unused_struct*)>,
    pre_unwind_handler_data=0x9115c38) at throw.c:200
#28 0xf79525f2 in opencog::SchemeEval::eval (this=0x9115c38, expr=@0xf5e55ab0)
    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/guile/SchemeEval.cc:320

The stack continues into my code, a la pseudocode at top.

#0  0xffffe425 in __kernel_vsyscall ()
#1  0xf7e43589 in __lll_lock_wait () from /lib/tls/i686/cmov/libpthread.so.0
#2  0xf7e3eba6 in _L_lock_95 () from /lib/tls/i686/cmov/libpthread.so.0
#3  0xf7e3e58a in pthread_mutex_lock () from /lib/tls/i686/cmov/libpthread.so.0
#4  0xf7826fd7 in guilify_self_1 (base=<value optimized out>) at threads.c:448
#5  0xf78288ab in scm_i_init_thread_for_guile (base=0xf4308000,
    parent=0xf7693e70) at threads.c:570
#6  0xf7828b82 in scm_init_guile () at threads.c:674

stack continues in my code, a la pseudocode at top


--linas

Re: guile threading deadlock

[Ludovic Courtès]

Hello!

"Linas Vepstas" <linasvepstas@gmail.com> writes:

> I've got a little deadlock problem w/ guile.  Here's the pseudocode:
>
> for (i=0; i<10 i++)  {
>    pthread_create (load_scm, expr[i]);
> }
>
> void load_scm (char * expr) {
>    scm_init_guile();
>    scm_c_eval(expr);
>    /* and pthread exit now that we're done */
> }
>
> The expr's are entirely scm lambda defines of one sort or another,
> and do not do any actual computation.

Can you try to provide actual code to reproduce the problem?  :-)
That would be great.

Did you compile Guile with thread support (Debian's package doesn't have
thread support)?  If not, that may be the problem.  See
http://thread.gmane.org/gmane.lisp.guile.devel/7754 for details.

> #12 0xf7828a43 in on_thread_exit (v=0x9179e18) at threads.c:505
> #13 0xf7e3bbb0 in __nptl_deallocate_tsd ()
>    from /lib/tls/i686/cmov/libpthread.so.0
> #14 0xf7e3c509 in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
> #15 0xf7c3ae5e in clone () from /lib/tls/i686/cmov/libc.so.6
>
> Not sure who called clone ... not my code ... this is confusing. ...

The C library.

Hope this helps,
Ludo'.

Re: guile threading deadlock

[Linas Vepstas]

Hi,

2008/11/8 Ludovic Courtès <ludo@gnu.org>:
> Hello!
>
> "Linas Vepstas" <linasvepstas@gmail.com> writes:
>
>> I've got a little deadlock problem w/ guile.  Here's the pseudocode:

I've got a much, much simpler case, see below.

> Can you try to provide actual code to reproduce the problem?  :-)
> That would be great.

Sure, but you won't enjoy debugging it.  Or even building it.
Go to https://code.launchpad.net/~opencog-dev
check out the branch called "staging". Build it.
then cd to directory opencog/scm and run load.sh

> Did you compile Guile with thread support

Yes. You will observe that the stack traces I sent were
deadlocked in garbage collection?   Without thread
support, how could things possibly deadlock?

Anyway, I have an even simpler variant, with only *one*
thread deadlocked in gc.    Here's the scenario:

thread A:
   scm_init_guile();
   does some other stuff, then
   goes to sleep in select, waiting on socket input
   (as expected).

thread B:
   scm_init_guile() -- hangs here.

B deadlocks with the stack trace below:

#0  0xffffe425 in __kernel_vsyscall ()
#1  0xf7e60589 in __lll_lock_wait () from
/lib/tls/i686/cmov/libpthread.so.0
#2  0xf7e5bba6 in _L_lock_95 () from /lib/tls/i686/cmov/libpthread.so.0
#3  0xf7e5b58a in pthread_mutex_lock () from
/lib/tls/i686/cmov/libpthread.so.0
#4  0xf7844464 in scm_i_thread_put_to_sleep () at threads.c:1615
#5  0xf77eeca9 in scm_i_gc (what=0xf786422e "cells") at gc.c:552
#6  0xf77eefed in scm_gc_for_newcell (freelist=0xf787984c,
free_cells=0x99fa25c)
    at gc.c:509
#7  0xf7843bff in guilify_self_2 (parent=0xf76b0e70) at
../libguile/inline.h:115
#8  0xf7845a9b in scm_i_init_thread_for_guile (base=0xf3b8a000,
    parent=0xf76b0e70) at threads.c:578
#9  0xf7845d82 in scm_init_guile () at threads.c:682
#10 0xf796f928 in opencog::SchemeEval::thread_init (this=0x995bc38)
    at ...

I built guile, and added debug prints:  one to
scm_enter_guile, which takes a lock, and one
to scm_leave_guile, with drops a lock.  I also
put prints into scm_i_thread_put_to_sleep ().

The behaviour is very clear, and every simple:
when scm_init_guile is called in thread A, the result
is that it is in "guile mode" i.e. holding the lock --
it is created holding the lock.  There's a series of
pairs of calls to leave..enter which are always
paired up.  Anyway, when thread A finally goes to
sleep waiting on input, it does so with its lock held.
Read libguile/threads.c:scm_enter_guile() to see
what I mean:

static void
scm_enter_guile (scm_t_guile_ticket ticket)
{
  scm_i_thread *t = (scm_i_thread *)ticket;
  if (t)
    {
      scm_i_pthread_mutex_lock (&t->heap_mutex);
      resume (t);
    }
}

while scm_leave_guile() does symmetrically the opposite.

Anyway, at this point, thread A is sleeping, holding the
above lock, because the last guile-thing it did was to
call scm_enter_guile().

Then *later on*, thread B calls scm_init_guile(), and
hangs, very clearly in scm_i_thread_put_to_sleep().
The printf reveal the hangs happen here:

     /* Signal all threads to go to sleep
       */
      scm_i_thread_go_to_sleep = 1;
      for (t = all_threads; t; t = t->next_thread)
      {
           scm_i_pthread_mutex_lock (&t->heap_mutex);
      }

Well, the for loop then gets stuck, waiting for the lock.
But the lock will never be granted, because thread A
is holding it, and is in permanent sleep. As a result,
thread B is blocked, forever, thus a deadlock.

I'm somewhat stumped, because I can't imagine how
this code *ever* could have worked in the first place.
The deadlock seems really blatent to me.  It seems
criminal for guile to *ever* return to the caller, while
still holding a lock of any sort. But every clearly,
scm_init_guile() (and I guess most other calls) return
to C code,  with a lock held.  This is just begging for
deadlocks!

--linas

Re: guile threading deadlock

[Ludovic Courtès]

Hi,

"Linas Vepstas" <linasvepstas@gmail.com> writes:

> Anyway, I have an even simpler variant, with only *one*
> thread deadlocked in gc.    Here's the scenario:
>
> thread A:
>    scm_init_guile();
>    does some other stuff, then
>    goes to sleep in select, waiting on socket input
>    (as expected).
>
> thread B:
>    scm_init_guile() -- hangs here.

Just to be sure: do you use raw select(2), `scm_std_select ()', or
select(2) within `scm_without_guile ()'?

The first option would lead to the situation you're observing because
Guile's GC requires thread to cooperate, which is why
`scm_without_guile' exists (info "(guile) Blocking").  You should use
one of the other options.

Ludo'.

Re: guile threading deadlock

[Linas Vepstas]

2008/11/9 Ludovic Courtès <ludo@gnu.org>:
> Hi,
>
> "Linas Vepstas" <linasvepstas@gmail.com> writes:
>
>> Anyway, I have an even simpler variant, with only *one*
>> thread deadlocked in gc.    Here's the scenario:
>>
>> thread A:
>>    scm_init_guile();
>>    does some other stuff, then
>>    goes to sleep in select, waiting on socket input
>>    (as expected).
>>
>> thread B:
>>    scm_init_guile() -- hangs here.
>
> Just to be sure: do you use raw select(2), `scm_std_select ()', or
> select(2) within `scm_without_guile ()'?

Raw select. I don't actually use it, its in some other library
that the project uses; wrapping it with a guile function is
not an option; its not code that we (the project) can even
control.

scm_without_guile is also clearly not an option, as it requires
an inversion of control.  In my app. guile is a bit player, just
another small guy inside a much larger framework; and so
anything invasive is just not possible. Similar remarks for
scm_pthread_mutex_lock(), etc.  Its rather ludicrous to even
suggest that such wrappers be used -- typically, such things
are not ever within reach of the developer.

> The first option would lead to the situation you're observing because
> Guile's GC requires thread to cooperate, which is why
> `scm_without_guile' exists (info "(guile) Blocking").

Yes, well, I eventually figured it out. The very first sentence
of http://www.gnu.org/software/guile/manual/html_node/Blocking.html

says it all: "A thread must not block outside of a libguile
function while it is in guile mode. "  Unfortunately, this
sentence is completely missing from the section called
"5.3 Initializing guile", which offers no hint of this behaviour.

Similarly, there is no hint of this in section 4.3.5
"Multi-threading".  Instead, I see sentences that say
things like "a thread promises to reach a safe point
reasonably frequently (see Asynchronous Signals)"

What the heck is "a safe point"? Who is making the
"promise"? Guile or the user app?  What the heck do
"safe points" have to do with signals?   So I think that
section is trying to warn about the deadlock, but fails
utterly to actually come out and say it .... the
documentation on this topic needs an overhaul.

I found a simple solution that works for me: just wrap
calls to scm_c_eval_string()  with calls to scm_with_guile().

That is, I basically call
scm_with_guile(scm_c_eval_string, expr)
and so am *never* in "guile mode", except while evaluating
an expression.  This solution is "obvious", once one
understands it ... but again, the documentation doesn't
even hint this: scm_c_eval_string does not even hint that
wrapping it with scm_with_guile is going to be the "typical"
use case.  (well, I actually wrap it scm_c_catch first.. which
is also a whole under-documented area).

I suppose I could volunteer to provide updated documentation,
if someone can hold my hand during its creation.

--linas

Re: guile threading deadlock

[Ludovic Courtès]

Hi,

"Linas Vepstas" <linasvepstas@gmail.com> writes:

> scm_without_guile is also clearly not an option [...].  Its rather
> ludicrous to even suggest that such wrappers be used -- typically,
> such things are not ever within reach of the developer.

[...]

> I found a simple solution that works for me: just wrap
> calls to scm_c_eval_string()  with calls to scm_with_guile().

Looks like you changed your mind by the end of the message!  ;-)

> Yes, well, I eventually figured it out. The very first sentence
> of http://www.gnu.org/software/guile/manual/html_node/Blocking.html
>
> says it all: "A thread must not block outside of a libguile
> function while it is in guile mode. "  Unfortunately, this
> sentence is completely missing from the section called
> "5.3 Initializing guile", which offers no hint of this behaviour.

Hmm, I don't think it's directly related to Guile initialization.  How
would you like to change Section 5.3?

> Similarly, there is no hint of this in section 4.3.5
> "Multi-threading".  Instead, I see sentences that say
> things like "a thread promises to reach a safe point
> reasonably frequently (see Asynchronous Signals)"
>
> What the heck is "a safe point"? Who is making the
> "promise"? Guile or the user app?  What the heck do
> "safe points" have to do with signals?   So I think that
> section is trying to warn about the deadlock, but fails
> utterly to actually come out and say it .... the
> documentation on this topic needs an overhaul.

I reread that part and the rest of the paragraph makes it clearer, IMO:

     While in guile mode, a thread promises to reach a safe point
  reasonably frequently (*note Asynchronous Signals::).  In addition to
  running signal handlers, these points are also potential rendezvous
  points of all guile mode threads where Guile can orchestrate global
  things like garbage collection.  Consequently, when a thread in guile
  mode blocks and does no longer frequent safe points, it might cause
  all other guile mode threads to block as well.  To prevent this from
  happening, a guile mode thread should either only block in libguile
  functions (who know how to do it right), or should temporarily leave
  guile mode with `scm_without_guile'.

Note also that one of the first sentences of that section is:

     Each thread that wants to use functions from libguile must put
  itself into _guile mode_ and must then follow a few rules.

IMO this indicates that "reaching a safe point frequently" is one of
these rules.

Nonetheless, I agree that "safe point" should be defined, or a different
wording should be used.

Do you have any suggestions to clarify that?

> I suppose I could volunteer to provide updated documentation,

Yes!  :-)

Thanks,
Ludo'.

Re: guile threading deadlock

[Linas Vepstas]

2008/11/9 Ludovic Courtès <ludo@gnu.org>:
> Hi,
>
> "Linas Vepstas" <linasvepstas@gmail.com> writes:
>
>> scm_without_guile is also clearly not an option [...].  Its rather
>> ludicrous to even suggest that such wrappers be used -- typically,
>> such things are not ever within reach of the developer.
>
> [...]
>
>> I found a simple solution that works for me: just wrap
>> calls to scm_c_eval_string()  with calls to scm_with_guile().
>
> Looks like you changed your mind by the end of the message!  ;-)

?? What did I change my mind about?  I don't understand
what you are saying here.

I still think that its absurd to ask the select call, or the
pthread calls, to be wrapped, since these calls will
typically be in modules outside of the programmers
control.  ("Mechanism, not policy" is what one usually
says)

However, there is a simple solution: wrap every call
to scm_c_eval_string() with a call to scm_with_guile().
Doesn't seem very optimal or pretty, but it works.

>> Yes, well, I eventually figured it out. The very first sentence
>> of http://www.gnu.org/software/guile/manual/html_node/Blocking.html
>>
>> says it all: "A thread must not block outside of a libguile
>> function while it is in guile mode. "  Unfortunately, this
>> sentence is completely missing from the section called
>> "5.3 Initializing guile", which offers no hint of this behaviour.
>
> Hmm, I don't think it's directly related to Guile initialization.  How
> would you like to change Section 5.3?

I'd change it to indicate that there are multi-threading issues
to consider.  Guile does a highly "non-standard" thing, that
would come as a surprise to anyone who has ever done
multi-threaded., multi-processing work, or any kernel work:
it returns to the user while holding a lock.  This is such a
strange and unusual thing to do, that it should be clearly documented.
 (I also think that this should probably be
changed/fixed in some future version of guile, but that
is another matter).

In particular, any thread that uses scm_init_guile cannot
sleep -- this should be stated up front. Any thread that
is "in guile mode" cannot sleep. This too should be stated
up front.

>> Similarly, there is no hint of this in section 4.3.5
>> "Multi-threading".  Instead, I see sentences that say
>> things like "a thread promises to reach a safe point
>> reasonably frequently (see Asynchronous Signals)"
>>
>> What the heck is "a safe point"? Who is making the
>> "promise"? Guile or the user app?  What the heck do
>> "safe points" have to do with signals?   So I think that
>> section is trying to warn about the deadlock, but fails
>> utterly to actually come out and say it .... the
>> documentation on this topic needs an overhaul.
>
> I reread that part and the rest of the paragraph makes it clearer, IMO:

Clear as mud!   It comes off as word salad to me; I don't
want to be insulting, in case you are the one who wrote it
-- but it does need a total re-write.

Here's my point of view, ... I've got a PhD, I've been coding
for decades; my last job was as a linux kernel programmer,
doing close-to-the-hardware work, where there are many,
many issues with sync points and locking and races and
order-of-initialization & etc.  Yet, when I tried to read that
paragraph, I could not make sense out of it. If I can't figure
it out, who can?

> Note also that one of the first sentences of that section is:
>
>     Each thread that wants to use functions from libguile must put
>  itself into _guile mode_ and must then follow a few rules.

Whereupon it utterly *fails* to even begin to enumerate
any rules at all, lapsing into some incomprehensible
gobbldy-gook.

> IMO this indicates that "reaching a safe point frequently" is one of
> these rules.

What the heck is a "safe point"?  Whose job is it to reach
this "safe point"? Who is making the promises? Guile?
the use application? Etc. These sort of things need to be
made clear.

> Do you have any suggestions to clarify that?

Well, yes.  But it would perhaps be easiest for me
to write the text myself, rather than explain how the
text could be written.

>> I suppose I could volunteer to provide updated documentation,
>
> Yes!  :-)

Well, if I provided patch files, would that work? Where would
I send these?  Who maintains this stuff?

--linas

Re: guile threading deadlock

[Ludovic Courtès]

Hello Linas,

"Linas Vepstas" <linasvepstas@gmail.com> writes:

> However, there is a simple solution: wrap every call
> to scm_c_eval_string() with a call to scm_with_guile().
> Doesn't seem very optimal or pretty, but it works.

Cool.

> In particular, any thread that uses scm_init_guile cannot
> sleep -- this should be stated up front. Any thread that
> is "in guile mode" cannot sleep. This too should be stated
> up front.

Again, it *is* documented, but maybe not sufficiently prominently, or
maybe it could be repeated in other places of the manual.  Can you say
specifically how you'd like to change that?

> Here's my point of view, ... I've got a PhD, I've been coding
> for decades; my last job was as a linux kernel programmer,
> doing close-to-the-hardware work

Great, congratulations!  :-)

>> Do you have any suggestions to clarify that?
>
> Well, yes.  But it would perhaps be easiest for me
> to write the text myself, rather than explain how the
> text could be written.

Good, that's what I was suggesting.

> Well, if I provided patch files, would that work? Where would
> I send these?  Who maintains this stuff?

Please send patches to <bug-guile@gnu.org>.  If they are non-trivial,
you'll need to assign copyright to the FSF, but we can work this out
off-line in due time.

Thanks,
Ludo'.

Re: guile threading deadlock

[Linas Vepstas]

Hi,

2008/11/9 Ludovic Courtès <ludo@gnu.org>:
> "Linas Vepstas" <linasvepstas@gmail.com> writes:
>
>> However, there is a simple solution: wrap every call
>> to scm_c_eval_string() with a call to scm_with_guile().
>> Doesn't seem very optimal or pretty, but it works.
>
> Cool.

Well, not quite. I also discovered that I have to re-define
smob-based functions in every new thread. That is,
I have to call scm_c_define_gsubr() in each thread
that I plan to use; simply calling these once  in some
parent thread results in

ERROR: Unbound variable: whatever-xxx
ABORT: unbound-variable

in the other threads.  I find this behaviour "surprising";
is there another spot of documentation that I should have
seen, that explained this, or is this a bug, or is this
working as designed?

> Again, it *is* documented, but maybe not sufficiently prominently, or
> maybe it could be repeated in other places of the manual.  Can you say
> specifically how you'd like to change that?

Haven't I answered this question three times now?

I feel that this conversation is not moving forward, I am
not sure why, or what i can do to be more helpful.

> Please send patches to <bug-guile@gnu.org>.

I'll try to do so in the next few says.

> If they are non-trivial,
> you'll need to assign copyright to the FSF, but we can work this out
> off-line in due time.

I have a generic assignment form with the FSF.

--linas