From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id yLMtKVKueV9ETwAA0tVLHw (envelope-from ) for ; Sun, 04 Oct 2020 11:13:22 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id uCvtJFKueV8rewAAbx9fmQ (envelope-from ) for ; Sun, 04 Oct 2020 11:13:22 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id EFC3B9401E3 for ; Sun, 4 Oct 2020 11:13:21 +0000 (UTC) Received: from localhost ([::1]:37024 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kP1xA-0007hC-Hi for larch@yhetil.org; Sun, 04 Oct 2020 07:13:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48980) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kP1ws-0007h0-42 for bug-guix@gnu.org; Sun, 04 Oct 2020 07:13:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:33476) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kP1wr-0004JI-RD for bug-guix@gnu.org; Sun, 04 Oct 2020 07:13:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kP1wr-0006dT-KP for bug-guix@gnu.org; Sun, 04 Oct 2020 07:13:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#43564: cuirass: Contention while registering new builds. Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 04 Oct 2020 11:13:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43564 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 43564@debbugs.gnu.org Received: via spool by 43564-submit@debbugs.gnu.org id=B43564.160180997725498 (code B ref 43564); Sun, 04 Oct 2020 11:13:01 +0000 Received: (at 43564) by debbugs.gnu.org; 4 Oct 2020 11:12:57 +0000 Received: from localhost ([127.0.0.1]:45022 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kP1wn-0006dC-Ad for submit@debbugs.gnu.org; Sun, 04 Oct 2020 07:12:57 -0400 Received: from eggs.gnu.org ([209.51.188.92]:42160) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kP1wl-0006cz-7V for 43564@debbugs.gnu.org; Sun, 04 Oct 2020 07:12:55 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40571) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kP1wg-0004Hs-13 for 43564@debbugs.gnu.org; Sun, 04 Oct 2020 07:12:50 -0400 Received: from [2a01:e0a:19b:d9a0:48f7:411c:55d8:6418] (port=51220 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kP1we-0003mj-4L for 43564@debbugs.gnu.org; Sun, 04 Oct 2020 07:12:49 -0400 From: Mathieu Othacehe References: <87imc5zrc8.fsf@gnu.org> <87k0wee3ud.fsf@gnu.org> <87h7rglj7t.fsf@gnu.org> Date: Sun, 04 Oct 2020 13:12:46 +0200 In-Reply-To: <87h7rglj7t.fsf@gnu.org> (Mathieu Othacehe's message of "Tue, 29 Sep 2020 17:02:14 +0200") Message-ID: <87r1qep7m9.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: -1.51 X-TUID: nB4ajZ8wgyMi Hello, > I'm trying locally to spawn database workers dedicated to registration, > that execute the evaluation queries in a single batch. Let's see if it > helps. I added 4 registration dedicated database workers and removed from registration all accesses to the store. Now the evaluation returns objects that contain all required information. This seem to improve the situation quite a lot. For example, those three registrations were done simultaneously in ~200 seconds: --8<---------------cut here---------------start------------->8--- 2020-10-04T12:46:19 registration took 205.429772 seconds 2020-10-04T12:46:19 evaluation 17099 registered 156 new derivations 2020-10-04T12:46:19 building 156 derivations in batches of 200 2020-10-04T12:46:19 building batch of 200 derivations (0/156) 2020-10-04T12:46:19 registration took 212.892482 seconds 2020-10-04T12:46:19 evaluation 17105 registered 164 new derivations 2020-10-04T12:46:19 building 164 derivations in batches of 200 2020-10-04T12:46:19 building batch of 200 derivations (0/164) 2020-10-04T12:46:19 registration took 205.009416 seconds 2020-10-04T12:46:19 evaluation 17102 registered 160 new derivations 2020-10-04T12:46:19 building 160 derivations in batches of 200 2020-10-04T12:46:19 building batch of 200 derivations (0/160) --8<---------------cut here---------------end--------------->8--- Thanks, Mathieu