From: ludo@gnu.org (Ludovic Courtès)
To: Eelco Dolstra <eelco.dolstra@logicblox.com>
Cc: guix-devel <guix-devel@gnu.org>, nix-dev <nix-dev@lists.science.uu.nl>
Subject: Avoiding threads in the daemon
Date: Thu, 18 Dec 2014 17:32:52 +0100 [thread overview]
Message-ID: <87mw6lym95.fsf@gnu.org> (raw)
Nix commit 524f89 changed libstore to use fork + unshare instead of
clone(2). The problem is that, in doing so, it also removed use of
CLONE_NEWPID and thus, (1) the build process no longer has PID 1, and
(2) build processes end up in the global PID space.
Adding CLONE_NEWPID to the unshare(2) call appears to break things (for
instance, future calls to pthread_create by that process fail with
EINVAL, other calls to clone(2) fail with ENOMEN) which may be why
CLONE_NEWPID isn’t used here.
The stated reason for this commit is this:
commit 524f89f1399724e596f61faba2c6861b1bb7b9c5
Author: Eelco Dolstra <eelco.dolstra@logicblox.com>
Date: Thu Aug 21 14:08:09 2014 +0200
Use unshare() instead of clone()
It turns out that using clone() to start a child process is unsafe in
a multithreaded program. It can cause the initialisation of a build
child process to hang in setgroups(), as seen several times in the
build farm:
The reason is that Glibc thinks that the other threads of the parent
exist in the child, so in setxid_mark_thread() it tries to get a futex
that has been acquired by another thread just before the clone(). With
fork(), Glibc runs pthread_atfork() handlers that take care of this
(in particular, __reclaim_stacks()). But clone() doesn't do that.
Fortunately, we can use fork()+unshare() instead of clone() to set up
private namespaces.
See also https://www.mail-archive.com/lxc-devel@lists.linuxcontainers.org/msg03434.html.
The more general issue is that fork should not be used in a
multi-threaded process, unless the child immediately calls exec* after
fork (POSIX clearly specifies that if a multi-threaded program forks,
the child must only call functions that are async-signal-safe.) IOW,
the daemon should not use threads in the first place.
Thus, I think Nix commit 49fe95 (which introduces monitor-fd.hh, which
uses std::thread just for convenience) should be reverted, along with
the subsequent commits to that file; then commit 524f89 can be reverted.
WDYT?
Thanks,
Ludo’.
_______________________________________________
nix-dev mailing list
nix-dev@lists.science.uu.nl
http://lists.science.uu.nl/mailman/listinfo/nix-dev
next reply other threads:[~2014-12-18 16:32 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-12-18 16:32 Ludovic Courtès [this message]
2014-12-19 18:20 ` Avoiding threads in the daemon Eelco Dolstra
2014-12-19 18:41 ` Shea Levy
2014-12-19 18:46 ` Eelco Dolstra
2014-12-19 21:31 ` Ludovic Courtès
2014-12-19 21:36 ` Luca Bruno
2014-12-20 0:11 ` Alexander Kjeldaas
2014-12-23 16:26 ` Eelco Dolstra
2014-12-26 22:02 ` Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87mw6lym95.fsf@gnu.org \
--to=ludo@gnu.org \
--cc=eelco.dolstra@logicblox.com \
--cc=guix-devel@gnu.org \
--cc=nix-dev@lists.science.uu.nl \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).