Does anyone actually use threads with guile?

Does anyone actually use threads with guile?

[Linas Vepstas]

Does anyone actually use threads with guile?
Or am I the first to do so?

Today, I got a new crash. I have multiple threads, which
are doing nothing but a bunch of define's, in parallel.
(They're loading scheme code from various files).

The stack trace is below. This is on guile-1.8.5

--linas

throw from within critical section.
Program received signal SIGABRT, Aborted.
[Switching to Thread 0xf5e56b90 (LWP 8655)]
0xffffe425 in __kernel_vsyscall ()
(gdb) bt
#0  0xffffe425 in __kernel_vsyscall ()
#1  0xf7aee085 in raise () from /lib/tls/i686/cmov/libc.so.6
#2  0xf7aefa01 in abort () from /lib/tls/i686/cmov/libc.so.6
#3  0xf7789208 in scm_ithrow (key=0xf460f190, args=0xf3c8da88, noreturn=1)
    at throw.c:695
#4  0xf771eeb2 in scm_error_scm (key=0xf460f190, subr=0xf3d4e360,
    message=0xf3d4e340, args=0x404, data=0x4) at error.c:92
#5  0xf775ef9b in scm_i_input_error (function=0xf77aaf38 "scm_i_lreadparen",
    port=0xf3c8d970, message=0xf77aae7c "end of file", arg=0x404) at read.c:110
#6  0xf775f2e9 in flush_ws (port=0xf3c8d970,
    eoferr=0xf77aaf38 "scm_i_lreadparen") at read.c:261
#7  0xf776204c in scm_read_sexp (chr=<value optimized out>, port=0xf3c8d970)
    at read.c:357
#8  0xf7760b91 in scm_read_expression (port=0xf3c8d970) at read.c:1079
#9  0xf7761fde in scm_read_sexp (chr=<value optimized out>, port=0xf3c8d970)
    at read.c:362
#10 0xf7760b91 in scm_read_expression (port=0xf3c8d970) at read.c:1079
#11 0xf7761fde in scm_read_sexp (chr=<value optimized out>, port=0xf3c8d970)
    at read.c:362
---Type <return> to continue, or q <return> to quit---
#12 0xf7760b91 in scm_read_expression (port=0xf3c8d970) at read.c:1079
#13 0xf7782bc2 in inner_eval_string (data=0xf3c8d970) at strports.c:499
#14 0xf772dfde in scm_c_with_fluid (fluid=0x8aa7900, value=0xf45e8e80,
    cproc=0xf7782b90 <inner_eval_string>, cdata=0xf3c8d970) at fluids.c:459
#15 0xf7747335 in scm_c_call_with_current_module (module=0xf45e8e80,
    func=0xf7782b90 <inner_eval_string>, data=0xf3c8d970) at modules.c:104
#16 0xf7782e21 in scm_eval_string_in_module (string=0xf3d4e230,
    module=0xf45e8e80) at strports.c:527
#17 0xf7782e55 in scm_eval_string (string=0xf3d4e230) at strports.c:535
#18 0xf7782e85 in scm_c_eval_string (
    expr=0xf3a05a44 "(define (wire-bidi a-wire b-wire
uni-device)\n\t(let ((device (wire-null-device))\n\t\t\t(do-connect
#t)\n\t\t\t(myname \"\")\n\t\t)\n\t\t(define (connect-me)\n\t\t\t; Two
distinct checks of 'do-connect' are made, because "...) at
strports.c:481
#19 0xf7788b79 in scm_c_catch (tag=0x104, body=0xf7782e60 <scm_c_eval_string>,
    body_data=0xf3a05a44,
    handler=0xf78b05d8
<opencog::SchemeEval::catch_handler_wrapper(void*, scm_unused_struct*,
scm_unused_struct*)>, handler_data=0x8ac32d0,
    pre_unwind_handler=0xf78b064a
<opencog::SchemeEval::preunwind_handler_wrapper(---Type <return> to
continue, or q <return> to quit---
void*, scm_unused_struct*, scm_unused_struct*)>,
    pre_unwind_handler_data=0x8ac32d0) at throw.c:200
#20 0xf78b0a1d in opencog::SchemeEval::do_eval (this=0x8ac32d0,
expr=@0xf5e55ab0)
    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/guile/SchemeEval.cc:364
#21 0xf78b0cd7 in opencog::SchemeEval::c_wrap_eval (p=0x8ac32d0)
    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/guile/SchemeEval.cc:341
#22 0xf7713842 in c_body (d=0xf5e55a08) at continuations.c:350
#23 0xf7788b79 in scm_c_catch (tag=0x104, body=0xf7713830 <c_body>,
    body_data=0xf5e55a08, handler=0xf7713850 <c_handler>,
    handler_data=0xf5e55a08,
    pre_unwind_handler=0xf7788440 <scm_handle_by_message_noexit>,
    pre_unwind_handler_data=0x0) at throw.c:200
#24 0xf7713cf2 in scm_i_with_continuation_barrier (body=0xf7713830 <c_body>,
    body_data=0xf5e55a08, handler=0xf7713850 <c_handler>,
    handler_data=0xf5e55a08,
    pre_unwind_handler=0xf7788440 <scm_handle_by_message_noexit>,
    pre_unwind_handler_data=0x0) at continuations.c:326
---Type <return> to continue, or q <return> to quit---
#25 0xf7713dd3 in scm_c_with_continuation_barrier (
    func=0xf78b0ca2 <opencog::SchemeEval::c_wrap_eval(void*)>, data=0x8ac32d0)
    at continuations.c:368
#26 0xf7787959 in scm_i_with_guile_and_parent (
    func=0xf78b0ca2 <opencog::SchemeEval::c_wrap_eval(void*)>, data=0x8ac32d0,
    parent=0xf45e1e70) at threads.c:695
#27 0xf7787a4e in scm_with_guile (
    func=0xf78b0ca2 <opencog::SchemeEval::c_wrap_eval(void*)>, data=0x8ac32d0)
    at threads.c:683
#28 0xf78b034c in opencog::SchemeEval::eval (this=0x8ac32d0, expr=@0xf5e55ab0)
    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/guile/SchemeEval.cc:334
#29 0xf78bedd2 in opencog::SchemeShell::do_eval (this=0x8a90a18,
expr=@0x8ad9868)
    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/shell/SchemeShell.cc:232
#30 0xf78beffd in opencog::SchemeShell::eval (this=0x8a90a18, expr=@0x8ad9868,
    h=0x8ad9978)
    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/shell/SchemeShell.cc:143
---Type <return> to continue, or q <return> to quit---
#31 0xf7f02faf in opencog::ConsoleSocket::OnLine (this=0x8ad97d8,
    line=@0x8ad9868)
    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/server/ConsoleSocket.cc:88
#32 0x0806a738 in TcpSocket::OnRead ()
#33 0x0806e68d in TcpSocket::OnRead ()
#34 0x08066c0d in SocketHandler::Select ()
#35 0x080631c4 in SocketHandler::Select ()
#36 0x0805b5fa in Socket::SocketThread::Run ()
#37 0x08072073 in Thread::StartThread ()
#38 0xf7d9b4fb in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#39 0xf7b99e5e in clone () from /lib/tls/i686/cmov/libc.so.6

Re: Does anyone actually use threads with guile?

[Linas Vepstas]

2008/11/12 Linas Vepstas <linasvepstas@gmail.com>:

> Today, I got a new crash. I have multiple threads, which
> are doing nothing but a bunch of define's, in parallel.
> (They're loading scheme code from various files).

Studying the code just a little bit more, this looks like
a dopey and pointless error check within guile,
specifically, at libguile/throw.c line 695.
Removing it seems to result in a runable system.

Basically, at any given time, some thread might be
in a critical section. Some other thread may be
throwing an error for some utterly unrelated reason.
Yet, when the error is thrown, this "critical section"
check will trip, and it will do so for an utterly bogus
reason.  At least, that describes my case.

Is there any reason at all not to remove this check
entirely? (at  libguile/throw.c line 695.)

Should I be posting this sort of stuff to guile-devel,
instead of guile-user?  I've cross-posted to bug-guile too,
if this is driving you nuts, please tell me to stop;
I wanted to make sure the right folks saw this.

--linas

>
> The stack trace is below. This is on guile-1.8.5
>
> --linas
>
> throw from within critical section.
> Program received signal SIGABRT, Aborted.
> [Switching to Thread 0xf5e56b90 (LWP 8655)]
> 0xffffe425 in __kernel_vsyscall ()
> (gdb) bt
> #0  0xffffe425 in __kernel_vsyscall ()
> #1  0xf7aee085 in raise () from /lib/tls/i686/cmov/libc.so.6
> #2  0xf7aefa01 in abort () from /lib/tls/i686/cmov/libc.so.6
> #3  0xf7789208 in scm_ithrow (key=0xf460f190, args=0xf3c8da88, noreturn=1)
>    at throw.c:695
> #4  0xf771eeb2 in scm_error_scm (key=0xf460f190, subr=0xf3d4e360,
>    message=0xf3d4e340, args=0x404, data=0x4) at error.c:92
> #5  0xf775ef9b in scm_i_input_error (function=0xf77aaf38 "scm_i_lreadparen",
>    port=0xf3c8d970, message=0xf77aae7c "end of file", arg=0x404) at read.c:110
> #6  0xf775f2e9 in flush_ws (port=0xf3c8d970,
>    eoferr=0xf77aaf38 "scm_i_lreadparen") at read.c:261
> #7  0xf776204c in scm_read_sexp (chr=<value optimized out>, port=0xf3c8d970)
>    at read.c:357
> #8  0xf7760b91 in scm_read_expression (port=0xf3c8d970) at read.c:1079
> #9  0xf7761fde in scm_read_sexp (chr=<value optimized out>, port=0xf3c8d970)
>    at read.c:362
> #10 0xf7760b91 in scm_read_expression (port=0xf3c8d970) at read.c:1079
> #11 0xf7761fde in scm_read_sexp (chr=<value optimized out>, port=0xf3c8d970)
>    at read.c:362
> ---Type <return> to continue, or q <return> to quit---
> #12 0xf7760b91 in scm_read_expression (port=0xf3c8d970) at read.c:1079
> #13 0xf7782bc2 in inner_eval_string (data=0xf3c8d970) at strports.c:499
> #14 0xf772dfde in scm_c_with_fluid (fluid=0x8aa7900, value=0xf45e8e80,
>    cproc=0xf7782b90 <inner_eval_string>, cdata=0xf3c8d970) at fluids.c:459
> #15 0xf7747335 in scm_c_call_with_current_module (module=0xf45e8e80,
>    func=0xf7782b90 <inner_eval_string>, data=0xf3c8d970) at modules.c:104
> #16 0xf7782e21 in scm_eval_string_in_module (string=0xf3d4e230,
>    module=0xf45e8e80) at strports.c:527
> #17 0xf7782e55 in scm_eval_string (string=0xf3d4e230) at strports.c:535
> #18 0xf7782e85 in scm_c_eval_string (
>    expr=0xf3a05a44 "(define (wire-bidi a-wire b-wire
> uni-device)\n\t(let ((device (wire-null-device))\n\t\t\t(do-connect
> #t)\n\t\t\t(myname \"\")\n\t\t)\n\t\t(define (connect-me)\n\t\t\t; Two
> distinct checks of 'do-connect' are made, because "...) at
> strports.c:481
> #19 0xf7788b79 in scm_c_catch (tag=0x104, body=0xf7782e60 <scm_c_eval_string>,
>    body_data=0xf3a05a44,
>    handler=0xf78b05d8
> <opencog::SchemeEval::catch_handler_wrapper(void*, scm_unused_struct*,
> scm_unused_struct*)>, handler_data=0x8ac32d0,
>    pre_unwind_handler=0xf78b064a
> <opencog::SchemeEval::preunwind_handler_wrapper(---Type <return> to
> continue, or q <return> to quit---
> void*, scm_unused_struct*, scm_unused_struct*)>,
>    pre_unwind_handler_data=0x8ac32d0) at throw.c:200
> #20 0xf78b0a1d in opencog::SchemeEval::do_eval (this=0x8ac32d0,
> expr=@0xf5e55ab0)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/guile/SchemeEval.cc:364
> #21 0xf78b0cd7 in opencog::SchemeEval::c_wrap_eval (p=0x8ac32d0)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/guile/SchemeEval.cc:341
> #22 0xf7713842 in c_body (d=0xf5e55a08) at continuations.c:350
> #23 0xf7788b79 in scm_c_catch (tag=0x104, body=0xf7713830 <c_body>,
>    body_data=0xf5e55a08, handler=0xf7713850 <c_handler>,
>    handler_data=0xf5e55a08,
>    pre_unwind_handler=0xf7788440 <scm_handle_by_message_noexit>,
>    pre_unwind_handler_data=0x0) at throw.c:200
> #24 0xf7713cf2 in scm_i_with_continuation_barrier (body=0xf7713830 <c_body>,
>    body_data=0xf5e55a08, handler=0xf7713850 <c_handler>,
>    handler_data=0xf5e55a08,
>    pre_unwind_handler=0xf7788440 <scm_handle_by_message_noexit>,
>    pre_unwind_handler_data=0x0) at continuations.c:326
> ---Type <return> to continue, or q <return> to quit---
> #25 0xf7713dd3 in scm_c_with_continuation_barrier (
>    func=0xf78b0ca2 <opencog::SchemeEval::c_wrap_eval(void*)>, data=0x8ac32d0)
>    at continuations.c:368
> #26 0xf7787959 in scm_i_with_guile_and_parent (
>    func=0xf78b0ca2 <opencog::SchemeEval::c_wrap_eval(void*)>, data=0x8ac32d0,
>    parent=0xf45e1e70) at threads.c:695
> #27 0xf7787a4e in scm_with_guile (
>    func=0xf78b0ca2 <opencog::SchemeEval::c_wrap_eval(void*)>, data=0x8ac32d0)
>    at threads.c:683
> #28 0xf78b034c in opencog::SchemeEval::eval (this=0x8ac32d0, expr=@0xf5e55ab0)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/guile/SchemeEval.cc:334
> #29 0xf78bedd2 in opencog::SchemeShell::do_eval (this=0x8a90a18,
> expr=@0x8ad9868)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/shell/SchemeShell.cc:232
> #30 0xf78beffd in opencog::SchemeShell::eval (this=0x8a90a18, expr=@0x8ad9868,
>    h=0x8ad9978)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/shell/SchemeShell.cc:143
> ---Type <return> to continue, or q <return> to quit---
> #31 0xf7f02faf in opencog::ConsoleSocket::OnLine (this=0x8ad97d8,
>    line=@0x8ad9868)
>    at /home/linas/src/novamente/src/opencog-stage4/staging/opencog/server/ConsoleSocket.cc:88
> #32 0x0806a738 in TcpSocket::OnRead ()
> #33 0x0806e68d in TcpSocket::OnRead ()
> #34 0x08066c0d in SocketHandler::Select ()
> #35 0x080631c4 in SocketHandler::Select ()
> #36 0x0805b5fa in Socket::SocketThread::Run ()
> #37 0x08072073 in Thread::StartThread ()
> #38 0xf7d9b4fb in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
> #39 0xf7b99e5e in clone () from /lib/tls/i686/cmov/libc.so.6
>

Re: Does anyone actually use threads with guile?

[Andy Wingo]

Hi Linas,

[cc'ing them all, but followups to just one please -- bug-guile is
probably best]

For my part I apologize for not having the cycles to poke this more
thoroughly. Fortunately you are a good programmer and can figure things
out :)

History: pthread support was new with 1.8, as you probably know. A
number of people have used it, but it seems that perhaps you are
pounding it more than most.

On Thu 13 Nov 2008 05:56, "Linas Vepstas" <linasvepstas@gmail.com> writes:

> Basically, at any given time, some thread might be
> in a critical section. Some other thread may be
> throwing an error for some utterly unrelated reason.
> Yet, when the error is thrown, this "critical section"
> check will trip, and it will do so for an utterly bogus
> reason.  At least, that describes my case.
>
> Is there any reason at all not to remove this check
> entirely? (at  libguile/throw.c line 695.)

I think the idea behind the check sounds good -- it is incorrect to
throw from within a critical section, and the check detects this.

But the check is incorrect as you noticed, it should be checking if the
current thread is in a critical section.

So we have two options, to remove the check or to fix the critical
section counter (possiblity to be thread-local). My instinct would be
that removal is the correct solution in 1.8, and that in master the
decision depends on whether we merge the BDW GC or not, and what the
impact that GC has on the idea of critical sections.

More input appreciated.

Cheers,

Andy
-- 
http://wingolog.org/

Re: Does anyone actually use threads with guile?

[Jon Wilson]

Hi Linas,

Linas Vepstas wrote:
> Does anyone actually use threads with guile?
> Or am I the first to do so?

I once wrote an irc chat bot in guile 1.8 (a pretty crappy one, though) 
that used threads.  I never had any trouble with it, but I didn't do any 
really advanced stuff (nothing that needed synchronization or critical 
sections) either.  Also, it was entirely a guile program, I didn't do 
any coding in C for it, which of course opens up a whole nother can of 
worms.

I also have a kinda slick stream implementation which I found 
particularly useful for multithreaded programs, if you're interested. 
You can at any time split off a new reference to the stream, and the new 
reference will only contain objects that have not yet been examined by 
any code.  This is useful for when several threads all need to read from 
the same input, and input-processors may be added at any time (you don't 
want them to have to slog through all the old input, nor keep it around 
in memory, of course).
Regards,
Jon

Re: Does anyone actually use threads with guile?

[Han-Wen Nienhuys]

Linas Vepstas escreveu:
> 2008/11/12 Linas Vepstas <linasvepstas@gmail.com>:
> 
>> Today, I got a new crash. I have multiple threads, which
>> are doing nothing but a bunch of define's, in parallel.
>> (They're loading scheme code from various files).
> 
> Studying the code just a little bit more, this looks like
> a dopey and pointless error check within guile,
> specifically, at libguile/throw.c line 695.
> Removing it seems to result in a runable system.


Re -the subject- : I looked at the evaluator code from a threading
perspective a few months back, and come to the conclusion that it 
is totally broken.   The memoization code rewrites expressions 
without locking or atomicity, and if you have two threads memoizing
(ie. executing the same code for the 1st time), you will have 
unpredictable effects.

-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen