From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id iOkKKfjJgWKxJgAAbAwnHQ (envelope-from ) for ; Mon, 16 May 2022 05:50:16 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id KKYDKfjJgWIEUQAA9RJhRA (envelope-from ) for ; Mon, 16 May 2022 05:50:16 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3ADE533118 for ; Mon, 16 May 2022 05:50:16 +0200 (CEST) Received: from localhost ([::1]:42050 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqRkM-0003jS-TV for larch@yhetil.org; Sun, 15 May 2022 23:50:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51228) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqRkB-0003jE-Ev for bug-guix@gnu.org; Sun, 15 May 2022 23:50:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:57393) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nqRkB-0006Qd-5T for bug-guix@gnu.org; Sun, 15 May 2022 23:50:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nqRkB-0001wA-0M for bug-guix@gnu.org; Sun, 15 May 2022 23:50:03 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#55441: [cuirass] hang in "In progress..."; runs out of pgsql connections Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 16 May 2022 03:50:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 55441 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 55441@debbugs.gnu.org Cc: Mathieu Othacehe , guix-sysadmin X-Debbugs-Original-To: bug-guix Received: via spool by submit@debbugs.gnu.org id=B.16526729937401 (code B ref -1); Mon, 16 May 2022 03:50:02 +0000 Received: (at submit) by debbugs.gnu.org; 16 May 2022 03:49:53 +0000 Received: from localhost ([127.0.0.1]:51285 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqRk1-0001vJ-2x for submit@debbugs.gnu.org; Sun, 15 May 2022 23:49:53 -0400 Received: from lists.gnu.org ([209.51.188.17]:51084) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nqRjz-0001vA-D4 for submit@debbugs.gnu.org; Sun, 15 May 2022 23:49:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51124) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqRjy-0003gz-VV; Sun, 15 May 2022 23:49:51 -0400 Received: from mail-qv1-xf35.google.com ([2607:f8b0:4864:20::f35]:33342) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nqRjx-0006Nu-9x; Sun, 15 May 2022 23:49:50 -0400 Received: by mail-qv1-xf35.google.com with SMTP id j3so2024763qvn.0; Sun, 15 May 2022 20:49:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:cc:subject:references:to:cc:date:in-reply-to:message-id :user-agent:mime-version; bh=4zH24G1P6Yk80tSfIENeVF+uUHk44/HZVEZBTQ/Zauc=; b=Jm0AC9XDtG0wf7Lzolh6Hn+C8/piVr8qMDb5VFHBf9BbBoaUeo0I5McYG2JJXQak9A 6lM/w+stXqvQW8FbDVRZiRZxdGCOxWE/8yZDIFK1AWbP2RIoYA99IYuZ5/6sE+MgmNtC TPu4oyWhIwiadgVYecqlGkiUJdLCGu5Ak8JjDm+n5x8QMCxFGUHkF6VOQxRQr5GiAN4h 2cEV1yvZiHTOggU8wITLIYqFuTgCTSksaNNAjwF6QDIt6RRqJh6StqdjgVm++l0L34Y4 H3Gdf6VFiWs4aiwhCLyAF2BZYMcbCr6K99CgCzc4wNt2PBC8peNV1/yL+MMXIwI4nxwX Novg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:cc:subject:references:to:cc:date :in-reply-to:message-id:user-agent:mime-version; bh=4zH24G1P6Yk80tSfIENeVF+uUHk44/HZVEZBTQ/Zauc=; b=A/ALxKyDSV0Jj6gIFwvYQtq6KMufRn8PhN2apI+kAoEtI45i8+DdaS3x/3y5MwsLDb BLwgXG/SpSdHD0Oj5vhk/nMogu88qtEZ7VWGTTuOoQbSbttz9Rp8eqANQyC/VMXAxJve K5gip4ddZ4hBIlaMyWyRZBDQiyyzpbMnu7ULsb+fNISoDnGBvsLeHQBRPc/SD5Y7Aj6N 2bwK4PPIWWXyoZ+K5ff999MX41W0UfMBOmF67JU127L8t/IvqGS4BRkDgotp1f5TUSy5 2O4QZ+7SlZCHHdjvhL+nkFbOJnZn3T5gqIvkYMLMwtzsWxCFZXi/C0SYXALISnMYX1RM MD0Q== X-Gm-Message-State: AOAM531+AVrsGq+iO5m6xGeJpn7B6K57KY4BIR/+FtCTlbhSggrmTdVs YZ7ZP3g2Yo++/5rkuUe2h6+VQ213wKQf7A== X-Google-Smtp-Source: ABdhPJy2Q3CRMjcwusHCQezPyBb3zxY+JT3HYF2vBW/dggA26BKjnquLmSdcX4GUosrv+WAynbdB9w== X-Received: by 2002:ad4:4b65:0:b0:456:4582:597c with SMTP id m5-20020ad44b65000000b004564582597cmr13591936qvx.33.1652672987282; Sun, 15 May 2022 20:49:47 -0700 (PDT) Received: from hurd (dsl-10-136-209.b2b2c.ca. [72.10.136.209]) by smtp.gmail.com with ESMTPSA id n188-20020a37bdc5000000b0069fc13ce1desm5224154qkf.15.2022.05.15.20.49.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 15 May 2022 20:49:46 -0700 (PDT) From: Maxim Cournoyer References: <87fslcaznn.fsf@gmail.com> <87mtfj174l.fsf@gnu.org> Date: Sun, 15 May 2022 23:49:45 -0400 In-Reply-To: <87mtfj174l.fsf@gnu.org> (Mathieu Othacehe's message of "Sun, 15 May 2022 11:23:54 +0200") Message-ID: <878rr2kug6.fsf_-_@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::f35; envelope-from=maxim.cournoyer@gmail.com; helo=mail-qv1-xf35.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1652673016; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=4zH24G1P6Yk80tSfIENeVF+uUHk44/HZVEZBTQ/Zauc=; b=B3cM/5ShUMz5uqFybgMOAwgOferGbftPm8e4jKxuG7i9U/2gLHi51auMpGFFR5o+LLUCHq Q9PNDOaqrcpfkcp+X+/uCKEUx7pR9dDeVTuBPJO5qKDwoEFOlR1DoOZ6rrsToCfZrAVGeU VW3ac/2ZbvGmz1iQyCJZwYtRV3pF1ZbUimVPtHQUtzI4Qyk+Z47PWOr0dSI94vyDLuVW7/ Ur6z4d2kQ3OM6a0aNBo8wHT5Fk9BLuU0Per2RHXDgU367bYlSBpWYdCoD8WLdsRZnga3C3 lk3ZnJ72jNLpJnL7N+YqbNG0beelVPU0fF+WbOp6EPDh9WmwkqJX/Xm33xWqog== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1652673016; a=rsa-sha256; cv=none; b=H3g2j15+LZx/W7pR6BgAEWzx2N60j0NoVEz/Hv1xC8U/tQTk0lLzM/ZEJtQPVkxf4gi5FQ 5oocuvyISsLLkbd+XXDxqU+yMT7GO8t6aRKvjD7oWPEuNfK/KFK8+6DpotAXk5YIkHW5Hv B1FHQVl5GkximzZG9xsRHmqvCYlvmTbPS9JdFkAD9ZOxaKw3xKU6RySpm4YX7Ve1u1QPH4 q62B2l+RQht8ojtL4h9K3n50GBcoT2sgifaPY8YXi96DNeXvNaaYCyfXuZv+OQ3H6HclBB gv9VzKb0EOx+27wZn7WDcftQeeWgOLiECqHEt8f+LbE1scsussRMCbwoGJ4igQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=Jm0AC9XD; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 4.47 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=Jm0AC9XD; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 3ADE533118 X-Spam-Score: 4.47 X-Migadu-Scanner: scn0.migadu.com X-TUID: 7ER1re+6LyBq Hello Guix, Since a couple days, Craps exhibits a new problem, where the evaluations on master appear stalled, showing as "In progress..." forever. Eventually, it seems the maximum number of PostgreSQL connections allowed (300, if what Ricardo wrote on #guix is accurate :-)) is reached, and we get the following error: --8<---------------cut here---------------start------------->8--- Backtrace: In ice-9/boot-9.scm: 1752:10 11 (with-exception-handler _ _ #:unwind? _ # _) In unknown file: 10 (apply-smob/0 #) In ice-9/boot-9.scm: 724:2 9 (call-with-prompt _ _ #) In ice-9/eval.scm: 619:8 8 (_ #(#(#))) In cuirass/ui.scm: 104:10 7 (run-cuirass-command _ . _) In cuirass/scripts/evaluate.scm: 101:7 6 (cuirass-evaluate _) In cuirass/utils.scm: 123:6 5 (make-worker-thread-channel _ #:parallelism _) In srfi/srfi-1.scm: 634:9 4 (for-each # ?) In cuirass/utils.scm: 125:21 3 (_ . _) In cuirass/scripts/evaluate.scm: 101:7 2 (_) In cuirass/database.scm: 375:13 1 (db-open #:database _) In squee.scm: 257:10 0 (connect-to-postgres-paramstring _) squee.scm:257:10: In procedure connect-to-postgres-paramstring: Throw to key `psql-connect-error' with args `(connection-bad "FATAL: remaining connection slots are reserved for non-replication superuser connections\n")'. --8<---------------cut here---------------end--------------->8--- (see: https://ci.guix.gnu.org/eval/323648/log/raw) Restarting postgresql workarounds the above error, but it doesn't fix the hang, and the problem connection problem recurs. I've logged at the Cuirass and messages logs, but I haven't spot anything abnormal. Thanks, Maxim