From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id AKxHIOPbll/JagAA0tVLHw (envelope-from ) for ; Mon, 26 Oct 2020 14:23:31 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 6KwhHOPbll8jTAAAbx9fmQ (envelope-from ) for ; Mon, 26 Oct 2020 14:23:31 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 15E4B9403A6 for ; Mon, 26 Oct 2020 14:23:31 +0000 (UTC) Received: from localhost ([::1]:57206 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kX3PF-0005qH-FP for larch@yhetil.org; Mon, 26 Oct 2020 10:23:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48624) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kX3Op-0005p0-3Y for bug-guix@gnu.org; Mon, 26 Oct 2020 10:23:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:57382) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kX3On-0006kl-Un for bug-guix@gnu.org; Mon, 26 Oct 2020 10:23:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kX3On-00042h-Rz for bug-guix@gnu.org; Mon, 26 Oct 2020 10:23:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#43565: cuirass: Fibers scheduling blocked. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 26 Oct 2020 14:23:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43565 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 43565-submit@debbugs.gnu.org id=B43565.160372215015497 (code B ref 43565); Mon, 26 Oct 2020 14:23:01 +0000 Received: (at 43565) by debbugs.gnu.org; 26 Oct 2020 14:22:30 +0000 Received: from localhost ([127.0.0.1]:40695 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kX3OH-00041t-TN for submit@debbugs.gnu.org; Mon, 26 Oct 2020 10:22:30 -0400 Received: from eggs.gnu.org ([209.51.188.92]:48384) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kX3OF-00041h-T9 for 43565@debbugs.gnu.org; Mon, 26 Oct 2020 10:22:28 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:55404) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kX3OA-0006fR-K1; Mon, 26 Oct 2020 10:22:22 -0400 Received: from [2a01:e0a:19b:d9a0:50c1:4a69:300e:4531] (port=54364 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kX3O9-0000yp-FK; Mon, 26 Oct 2020 10:22:21 -0400 From: Mathieu Othacehe References: <87eemtzr1q.fsf@gnu.org> <87r1qc27mo.fsf@gnu.org> <874kmmzd92.fsf@gnu.org> <871rhpqgjy.fsf@gnu.org> Date: Mon, 26 Oct 2020 15:22:19 +0100 In-Reply-To: <871rhpqgjy.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Fri, 23 Oct 2020 14:21:05 +0200") Message-ID: <87v9excbj8.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 43565@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: -1.51 X-TUID: IgrMgblW80qZ Hey! Many thanks for your help, you rock! > But does Cuirass create file descriptors as O_NONBLOCK? This has to be > done explicitly, Fibers won=E2=80=99t do it for us. As it turns out, the= answer > is no, in at least one important case: the connection to the daemon > (untested patch below). > > While GC is running, Cuirass typically sends =E2=80=98build-derivations= =E2=80=99 RPCs > and they block until the GC lock is released. That can lead to the > situation above: a bunch of threads blocked in =E2=80=98read=E2=80=99 fro= m their daemon > socket, waiting for the RPC reply. OTOH, =E2=80=98build-derivations=E2= =80=99 RPCs are > made from a fresh thread created by =E2=80=98build-derivations&=E2=80=99. While I agree not opening file descriptors with O_NONBLOCK is an issue, build-derivations is called in a separate thread. Blocking this separate thread should not block the fibers. For instance, the following program: --8<---------------cut here---------------start------------->8--- (use-modules (fibers) (ice-9 threads)) (run-fibers (lambda () (spawn-fiber (lambda () (call-with-new-thread (lambda () (read (car (pipe))))))) (spawn-fiber (lambda () (while #t (format #t "alive~%") (sleep 1))))) #:hz 10 #:drain? #t) --8<---------------cut here---------------end--------------->8--- keeps displaying "alive" even if the spawned thread is blocking. I guess that's also what's happening in Cuirass because the log shows that some fibers are scheduled while the GC is running. Now the question is why there's no fetching while the GC is running? The answer is that "latest-repository-commit" called by "fetch-input" will block the only fiber dedicated to fetching. Having multiple fibers trying to fetch wouldn't solve anything because fetching requires some building from the daemon. Long story short, I think we can apply your patch that can be useful to prevent fibers talking directly to the daemon to block, even though it won't help for this particular hang, that will only be fixed the GC time will be reduced to something more acceptable. Thanks, Mathieu