Re: Does anyone actually use threads with guile?

Re: Does anyone actually use threads with guile?

[Linas Vepstas]

2008/11/12 Linas Vepstas <linasvepstas@gmail.com>:

> Today, I got a new crash. I have multiple threads, which
> are doing nothing but a bunch of define's, in parallel.
> (They're loading scheme code from various files).

Studying the code just a little bit more, this looks like
a dopey and pointless error check within guile,
specifically, at libguile/throw.c line 695.
Removing it seems to result in a runable system.

Basically, at any given time, some thread might be
in a critical section. Some other thread may be
throwing an error for some utterly unrelated reason.
Yet, when the error is thrown, this "critical section"
check will trip, and it will do so for an utterly bogus
reason.  At least, that describes my case.

Is there any reason at all not to remove this check
entirely? (at  libguile/throw.c line 695.)

Should I be posting this sort of stuff to guile-devel,
instead of guile-user?  I've cross-posted to bug-guile too,
if this is driving you nuts, please tell me to stop;
I wanted to make sure the right folks saw this.

--linas

>
> The stack trace is below. This is on guile-1.8.5
>
> --linas
>
> throw from within critical section.
> Program received signal SIGABRT, Aborted.
> [Switching to Thread 0xf5e56b90 (LWP 8655)]
> 0xffffe425 in __kernel_vsyscall ()
> (gdb) bt
> #0  0xffffe425 in __kernel_vsyscall ()
> #1  0xf7aee085 in raise () from /lib/tls/i686/cmov/libc.so.6
> #2  0xf7aefa01 in abort () from /lib/tls/i686/cmov/libc.so.6
> #3  0xf7789208 in scm_ithrow (key=0xf460f190, args=0xf3c8da88, noreturn=1)
>    at throw.c:695
> #4  0xf771eeb2 in scm_error_scm (key=0xf460f190, subr=0xf3d4e360,
>    message=0xf3d4e340, args=0x404, data=0x4) at error.c:92
> #5  0xf775ef9b in scm_i_input_error (function=0xf77aaf38 "scm_i_lreadparen",
>    port=0xf3c8d970, message=0xf77aae7c "end of file", arg=0x404) at read.c:110
> #6  0xf775f2e9 in flush_ws (port=0xf3c8d970,
>    eoferr=0xf77aaf38 "scm_i_lreadparen") at read.c:261
> #7  0xf776204c in scm_read_sexp (chr=<value optimized out>, port=0xf3c8d970)
>    at read.c:357
> #8  0xf7760b91 in scm_read_expression (port=0xf3c8d970) at read.c:1079
> #9  0xf7761fde in scm_read_sexp (chr=<value optimized out>, port=0xf3c8d970)
>    at read.c:362
> #10 0xf7760b91 in scm_read_expression (port=0xf3c8d970) at read.c:1079
> #11 0xf7761fde in scm_read_sexp (chr=<value optimized out>, port=0xf3c8d970)
>    at read.c:362
> ---Type <return> to continue, or q <return> to quit---
> #12 0xf7760b91 in scm_read_expression (port=0xf3c8d970) at read.c:1079
> #13 0xf7782bc2 in inner_eval_string (data=0xf3c8d970) at strports.c:499
> #14 0xf772dfde in scm_c_with_fluid (fluid=0x8aa7900, value=0xf45e8e80,
>    cproc=0xf7782b90 <inner_eval_string>, cdata=0xf3c8d970) at fluids.c:459
> #15 0xf7747335 in scm_c_call_with_current_module (module=0xf45e8e80,
>    func=0xf7782b90 <inner_eval_string>, data=0xf3c8d970) at modules.c:104
> #16 0xf7782e21 in scm_eval_string_in_module (string=0xf3d4e230,
>    module=0xf45e8e80) at strports.c:527
> #17 0xf7782e55 in scm_eval_string (string=0xf3d4e230) at strports.c:535
> #18 0xf7782e85 in scm_c_eval_string (
>    expr=0xf3a05a44 "(define (wire-bidi a-wire b-wire
> uni-device)\n\t(let ((device (wire-null-device))\n\t\t\t(do-connect
> #t)\n\t\t\t(myname \"\")\n\t\t)\n\t\t(define (connect-me)\n\t\t\t; Two
> distinct checks of 'do-connect' are made, because "...) at
> strports.c:481
> #19 0xf7788b79 in scm_c_catch (tag=0x104, body=0xf7782e60 <scm_c_eval_string>,
>    body_data=0xf3a05a44,
>    handler=0xf78b05d8
> <opencog::SchemeEval::catch_handler_wrapper(void*, scm_unused_struct*,
> scm_unused_struct*)>, handler_data=0x8ac32d0,
>    pre_unwind_handler=0xf78b064a
> <opencog::SchemeEval::preunwind_handler_wrapper(---Type <return> to
> continue, or q <return> to quit---
> void*, scm_unused_struct*, scm_unused_struct*)>,
>    pre_unwind_handler_data=0x8ac32d0) at throw.c:200
> #20 0xf78b0a1d in opencog::SchemeEval::do_eval (this=0x8ac32d0,
> expr=@0xf5e55ab0)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/guile/SchemeEval.cc:364
> #21 0xf78b0cd7 in opencog::SchemeEval::c_wrap_eval (p=0x8ac32d0)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/guile/SchemeEval.cc:341
> #22 0xf7713842 in c_body (d=0xf5e55a08) at continuations.c:350
> #23 0xf7788b79 in scm_c_catch (tag=0x104, body=0xf7713830 <c_body>,
>    body_data=0xf5e55a08, handler=0xf7713850 <c_handler>,
>    handler_data=0xf5e55a08,
>    pre_unwind_handler=0xf7788440 <scm_handle_by_message_noexit>,
>    pre_unwind_handler_data=0x0) at throw.c:200
> #24 0xf7713cf2 in scm_i_with_continuation_barrier (body=0xf7713830 <c_body>,
>    body_data=0xf5e55a08, handler=0xf7713850 <c_handler>,
>    handler_data=0xf5e55a08,
>    pre_unwind_handler=0xf7788440 <scm_handle_by_message_noexit>,
>    pre_unwind_handler_data=0x0) at continuations.c:326
> ---Type <return> to continue, or q <return> to quit---
> #25 0xf7713dd3 in scm_c_with_continuation_barrier (
>    func=0xf78b0ca2 <opencog::SchemeEval::c_wrap_eval(void*)>, data=0x8ac32d0)
>    at continuations.c:368
> #26 0xf7787959 in scm_i_with_guile_and_parent (
>    func=0xf78b0ca2 <opencog::SchemeEval::c_wrap_eval(void*)>, data=0x8ac32d0,
>    parent=0xf45e1e70) at threads.c:695
> #27 0xf7787a4e in scm_with_guile (
>    func=0xf78b0ca2 <opencog::SchemeEval::c_wrap_eval(void*)>, data=0x8ac32d0)
>    at threads.c:683
> #28 0xf78b034c in opencog::SchemeEval::eval (this=0x8ac32d0, expr=@0xf5e55ab0)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/guile/SchemeEval.cc:334
> #29 0xf78bedd2 in opencog::SchemeShell::do_eval (this=0x8a90a18,
> expr=@0x8ad9868)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/shell/SchemeShell.cc:232
> #30 0xf78beffd in opencog::SchemeShell::eval (this=0x8a90a18, expr=@0x8ad9868,
>    h=0x8ad9978)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/shell/SchemeShell.cc:143
> ---Type <return> to continue, or q <return> to quit---
> #31 0xf7f02faf in opencog::ConsoleSocket::OnLine (this=0x8ad97d8,
>    line=@0x8ad9868)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/server/ConsoleSocket.cc:88
> #32 0x0806a738 in TcpSocket::OnRead ()
> #33 0x0806e68d in TcpSocket::OnRead ()
> #34 0x08066c0d in SocketHandler::Select ()
> #35 0x080631c4 in SocketHandler::Select ()
> #36 0x0805b5fa in Socket::SocketThread::Run ()
> #37 0x08072073 in Thread::StartThread ()
> #38 0xf7d9b4fb in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
> #39 0xf7b99e5e in clone () from /lib/tls/i686/cmov/libc.so.6
>

Re: Does anyone actually use threads with guile?

[Andy Wingo]

Hi Linas,

[cc'ing them all, but followups to just one please -- bug-guile is
probably best]

For my part I apologize for not having the cycles to poke this more
thoroughly. Fortunately you are a good programmer and can figure things
out :)

History: pthread support was new with 1.8, as you probably know. A
number of people have used it, but it seems that perhaps you are
pounding it more than most.

On Thu 13 Nov 2008 05:56, "Linas Vepstas" <linasvepstas@gmail.com> writes:

> Basically, at any given time, some thread might be
> in a critical section. Some other thread may be
> throwing an error for some utterly unrelated reason.
> Yet, when the error is thrown, this "critical section"
> check will trip, and it will do so for an utterly bogus
> reason.  At least, that describes my case.
>
> Is there any reason at all not to remove this check
> entirely? (at  libguile/throw.c line 695.)

I think the idea behind the check sounds good -- it is incorrect to
throw from within a critical section, and the check detects this.

But the check is incorrect as you noticed, it should be checking if the
current thread is in a critical section.

So we have two options, to remove the check or to fix the critical
section counter (possiblity to be thread-local). My instinct would be
that removal is the correct solution in 1.8, and that in master the
decision depends on whether we merge the BDW GC or not, and what the
impact that GC has on the idea of critical sections.

More input appreciated.

Cheers,

Andy
-- 
http://wingolog.org/

Critical sections

[Ludovic Courtès]

Hi,

Andy Wingo <wingo@pobox.com> writes:

> I think the idea behind the check sounds good -- it is incorrect to
> throw from within a critical section, and the check detects this.
>
> But the check is incorrect as you noticed, it should be checking if the
> current thread is in a critical section.
>
> So we have two options, to remove the check or to fix the critical
> section counter (possiblity to be thread-local). My instinct would be
> that removal is the correct solution in 1.8, and that in master the
> decision depends on whether we merge the BDW GC or not, and what the
> impact that GC has on the idea of critical sections.

Looking at uses of critical sections, it's not always clear what the
rationale is.

In some cases, e.g., `scm_sys_modify_instance ()' and
`scm_sys_modify_class ()', it's used to make an operation atomic.

In other cases, it seems to be used as a coarse-grain mutex (e.g.,
`scm_symbol_to_keyword ()', `scm_set_object_property_x ()').

So, at first sight, I'm not sure changing GCs would lead us to change
that.

Thanks,
Ludo'.

Re: Does anyone actually use threads with guile?

[Linas Vepstas]

2008/11/13 Andy Wingo <wingo@pobox.com>:
> For my part I apologize for not having the cycles

Fine, I'm hacking around it for now, but would like to see
something for 1.8.6.

> On Thu 13 Nov 2008 05:56, "Linas Vepstas" <linasvepstas@gmail.com> writes:
>
>> Basically, at any given time, some thread might be
>> in a critical section. Some other thread may be
>> throwing an error for some utterly unrelated reason.
>> Yet, when the error is thrown, this "critical section"
>> check will trip, and it will do so for an utterly bogus
>> reason.  At least, that describes my case.
>>
>> Is there any reason at all not to remove this check
>> entirely? (at  libguile/throw.c line 695.)
>
> I think the idea behind the check sounds good -- it is incorrect to
> throw from within a critical section, and the check detects this.
>
> But the check is incorrect as you noticed, it should be checking if the
> current thread is in a critical section.

The patch below does this.

I do not understand how 'async' fits into the grand scheme
of things. From what I can tell, though, there won't be any
cases where the thrower will be in a different thread than
where scm_ithrow() will run.  So the patch should be good.

--linas

---
 libguile/throw.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: guile-1.8.5/libguile/throw.c
===================================================================
--- guile-1.8.5.orig/libguile/throw.c  2008-11-13 16:02:26.000000000 -0600
+++ guile-1.8.5/libguile/throw.c 2008-11-13 16:29:46.000000000 -0600
@@ -689,7 +689,7 @@ scm_ithrow (SCM key, SCM args, int noret
   SCM dynpair = SCM_UNDEFINED;
   SCM winds;

-  if (scm_i_critical_section_level)
+  if (SCM_I_CURRENT_THREAD->block_asyncs)
     {
       fprintf (stderr, "throw from within critical section.\n");
       abort ();

Re: Does anyone actually use threads with guile?

[Han-Wen Nienhuys]

Linas Vepstas escreveu:
> 2008/11/12 Linas Vepstas <linasvepstas@gmail.com>:
> 
>> Today, I got a new crash. I have multiple threads, which
>> are doing nothing but a bunch of define's, in parallel.
>> (They're loading scheme code from various files).
> 
> Studying the code just a little bit more, this looks like
> a dopey and pointless error check within guile,
> specifically, at libguile/throw.c line 695.
> Removing it seems to result in a runable system.


Re -the subject- : I looked at the evaluator code from a threading
perspective a few months back, and come to the conclusion that it 
is totally broken.   The memoization code rewrites expressions 
without locking or atomicity, and if you have two threads memoizing
(ie. executing the same code for the 1st time), you will have 
unpredictable effects.

-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen