Re: wip-threads-and-fork

[Nala Ginrut <nalaginrut@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3386 bytes --]

On Thu, Mar 22, 2012 at 5:26 AM, Ludovic Courtès <ludo@gnu.org> wrote:

> Hello!
>
> (With delay...)
>
> Andy Wingo <wingo@pobox.com> skribis:
>
> > I would have preferred this, but I came to the conclusion that this
> > approach is not sound.
>
> Without exposing ‘pthread_atfork’, how would you suggest making user
> code “fork-safe”?  A use case would be reviving the futures thread pool
> after ‘fork’.
>
> > Did you see that I merged the atfork bits into master?
> > (wip-threads-and-fork also had some CLOEXEC bits that needed more
> > baking).  They worked... sorta.  They had a few problems:
> >
> >   1) It's impossible to work around the lack of atfork() in libraries
> >      that you depend on.
> >
> >      For example, wip-threads-and-fork called fork() within the GC alloc
> >      lock, to get around the lack of a pthread_atfork() in libgc.  But
> >      then I submitted a patch to make libgc do this itself:
> >
> >
> http://thread.gmane.org/gmane.comp.programming.garbage-collection.boehmgc/4940
> >
> >      It's pretty difficult to tell which version of libgc you would
> >      have.  There is no workaround that is sufficient.
>
> Indeed, good point.
>
> >   2) POSIX explicitly disclaims the result of calling non-signal-safe
> >      primitives after a fork() of a multithreaded program.
>
> Right, though reality seems to be more pleasant than POSIX. ;-)
>
> >   3) Nobody cares about these bugs.  See e.g. the lack of response at
> >      http://sourceware.org/bugzilla/show_bug.cgi?id=13725.  Even Bruno
> >      didn't reply to the Cc.  See point (2).
> >
> >   4) The atfork mechanism imposes a total ordering on locks.  This is
> >      possible for static locks, but difficult for locks on collectable
> >      Scheme objects.
> >
> >   5) Relatedly, just to be able to lock all weak tables at a fork, we
> >      had to create a new weak table-of-tables and add the tables to it.
> >      This is needless complication and overhead.
> >
> >   6) scm_c_atfork() is a broken interface.  Because it hangs its hooks
> >      off of one pthread_atfork() invocation, it can cause newer locks to
> >      insert themselves in the wrong position relative to
> >      pthread_atfork() calls made between when Guile installed the
> >      scm_c_atfork handler, and the call to scm_c_atfork.
> >
> >      There can be only one pthread_atfork() list, in a correct program.
>
> OK, thanks for the nice summary.  Indeed, this is a complex story.
>
> > In the end I reverted those patches because they were just complication
> > that didn't solve any fundamental problems.
>
> OK.
>
> > I came to the opinion, having run a threaded, forking program, that we
> > would be much better off if we provided good abstractions to spawn
> > processes, but that expecting fork() to work in a multithreaded program
> > is not realistic.
>
> Yes, things like ‘open-process’ make sense.
>
> What about adding a big fat warning in the doc about use of
> ‘primitive-fork’ in a multi-threaded program?
>

Please do it! My 5 cents.


>
> > Still, there is one other thing that perhaps we could do to shut down
> > the signal handling thread around a fork().  Dunno, perhaps it is worth
> > looking into.
>
> What would be the expected benefit?
>
> Thanks,
> Ludo’.
>
>

[-- Attachment #2: Type: text/html, Size: 4563 bytes --]