Re: multi-threading

["Neil Jerram" <neiljerram@googlemail.com>]

2008/7/6 Henri Häkkinen <henri.hakkinen@pp2.inet.fi>:
> On Sat, 2008-07-05 at 23:34 +0100, Neil Jerram wrote:
>
>> Your program looks OK to me.  There is a known stack overflow problem
>> on some OSs, when just trying to load boot-9.scm, and when libguile is
>> compiled without much optimization.  What OS are you running on?
>> Also, if you could manage to test the same program with the latest
>> release (1.8.5), please do that and let us know the result.
>>
>>          Neil
>
> Thank you for your reply. I resolved the problem. I am using Debian
> Linux server and guile was installed from a Debian package. The problem
> was that it was not compiled with pthreads support. Running '*features*'
> from guile shell did not list "threads". I recompiled guile from sources
> with threads support, and my program went okay. That ERROR: Stack
> overflow was very confusing for using threads in non-threads supported
> guile. I suggest fixing it.

The following is mostly for the record, because I'm not sure if
there's anything specific we can do.  If anyone has any comments or
ideas, please say.

The scenario here was:
- Guile configured and built without threads support (because coming
form Debian, which doesn't do --with-threads).
- A program which
  - calls scm_boot_guile() on the main thread
  - inside the scm_boot_guile(), creates another thread (pthread_create)
  - in that other thread, do scm_with_guile(thread_main, ...)
  - in thread_main, call scm_c_eval_string(...).
- Guile reports a Stack overflow error.

Explanation:
- When configured without threads, there is only one scm_i_thread
structure, because Guile assumes that it will only be used on one
thread.  This structure's "base" field is set, within the
implementation of scm_boot_guile(), to the base address of the main
thread.
- The base address of the other thread will be very different from
that of the main thread.
- Stack overflow checking works by subtracting &p from
scm_i_thread->base, and comparing this to the allowed stack depth
(where p is some variable on the stack in the function where the check
is being applied).
- It is easy for (scm_i_thread->base - &p) to exceed any stack depth
limit, because scm_i_thread->base and &p point into two different
stacks.

A plausible-sounding solution would be to use calls to pthread_self()
so that we catch any attempt by more than one thread to use Guile.  As
the code stands, however, (it appears that) the whole point of
--without-threads is to completely avoid using the pthreads API; this
is reflected both in the code and in configure.in checks.  So using
pthread_self() would be contrary to that point.

Another possibility would be a global counter, counting the number of
calls to scm_with_guile() that haven't yet completed.  But that
doesn't work either, because

- scm_with_guile() is deliberately designed to cope with being called
by a thread that is already in Guile mode, because there are contexts
where it isn't clear if a thread is already in Guile mode - hence
there are cases where the global counter could validly be >1

- the global counter wouldn't catch the case where one thread left
Guile mode, and then another thread entered it.


In practice, maybe the most useful thing we can do is to encourage
configuration --with-threads, and in particular to work through
whatever the reason is why Debian doesn't do this.

Regards,
     Neil