From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id gFkJHUHX8F52JwAA0tVLHw (envelope-from ) for ; Mon, 22 Jun 2020 16:07:29 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id WPHtGEHX8F6fRAAA1q6Kng (envelope-from ) for ; Mon, 22 Jun 2020 16:07:29 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E6A449400B7 for ; Mon, 22 Jun 2020 16:07:28 +0000 (UTC) Received: from localhost ([::1]:37826 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jnOyk-000195-Jp for larch@yhetil.org; Mon, 22 Jun 2020 12:07:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36318) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jnOtW-0003uX-9X for bug-guix@gnu.org; Mon, 22 Jun 2020 12:02:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51054) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jnOtV-0007lz-Uv for bug-guix@gnu.org; Mon, 22 Jun 2020 12:02:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jnOtV-0003Ak-RT for bug-guix@gnu.org; Mon, 22 Jun 2020 12:02:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#42001: =?UTF-8?Q?=E2=80=9CSQLite?= database is =?UTF-8?Q?busy=E2=80=9D:?= contention on the store database Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 22 Jun 2020 16:02:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 42001 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 42001-submit@debbugs.gnu.org id=B42001.159284169812151 (code B ref 42001); Mon, 22 Jun 2020 16:02:01 +0000 Received: (at 42001) by debbugs.gnu.org; 22 Jun 2020 16:01:38 +0000 Received: from localhost ([127.0.0.1]:34367 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jnOt8-00039v-Is for submit@debbugs.gnu.org; Mon, 22 Jun 2020 12:01:38 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50072) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jnOt5-00039f-MX for 42001@debbugs.gnu.org; Mon, 22 Jun 2020 12:01:37 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44247) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jnOsz-0007hA-I3; Mon, 22 Jun 2020 12:01:29 -0400 Received: from lfbn-ann-1-136-86.w86-200.abo.wanadoo.fr ([86.200.104.86]:40856 helo=meru) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jnOsy-0001dh-GQ; Mon, 22 Jun 2020 12:01:29 -0400 From: Mathieu Othacehe References: <877dvz7228.fsf@gnu.org> Date: Mon, 22 Jun 2020 18:01:24 +0200 In-Reply-To: <877dvz7228.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 22 Jun 2020 13:49:35 +0200") Message-ID: <87pn9rdr8r.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 42001@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: -1.01 X-TUID: KuiVLgdGlvRH Hey Ludo, > utimensat(AT_FDCWD, "/gnu/store/whhc80cx1zpi9cj3bil3sp8wwzcf8p9p-image-root/gnu/store/n6l40ygclxfysnc2nxwrslnn5ab6wv3v-profile/lib/modules/5.4.47-gnu/kernel/net/netfilter", [{tv_sec=1592814617, tv_nsec=396083395} /* 2020-06-22T10:30:17.396083395+0200 */, {tv_sec=1, tv_nsec=0} /* 1970-01-01T01:00:01+0100 */], 0) = 0 > chmod("/gnu/store/whhc80cx1zpi9cj3bil3sp8wwzcf8p9p-image-root/gnu/store/n6l40ygclxfysnc2nxwrslnn5ab6wv3v-profile/lib/modules/5.4.47-gnu/kernel/net/netfilter", 040555) = 0 > getrusage(RUSAGE_SELF, {ru_utime={tv_sec=1306, tv_usec=334062}, ru_stime={tv_sec=143, tv_usec=819675}, ...}) = 0 > clock_gettime(CLOCK_PROCESS_CPUTIME_ID, {tv_sec=1450, tv_nsec=153748295}) = 0 > lstat("/gnu/store/whhc80cx1zpi9cj3bil3sp8wwzcf8p9p-image-root/gnu/store/n6l40ygclxfysnc2nxwrslnn5ab6wv3v-profile/lib/modules/5.4.47-gnu/kernel/net/netfilter/xt_connlimit.ko", {st_mode=S_IFLNK|0777, st_size=125, ...}) = 0 As discussed on IRC, those "image-root" derivations do exercise store deduplication a lot, are could participate to the observed database contention. Turns out I used those derivations as "staging" directories that are passed to GNU Xorriso when producing ISO images and "mke2fs" and "mkdosfs" when producing raw disk-images. Using the store was also convenient to ensure root ownership of the image files. With 1cb9effc3e875ec7bae31bb06069d16ac89f7e1d and 7f75a7ec08975eb6d6e01db61bd6b91f447f655e, I remove those "image-root" derivations, and proceed to image creation in a single derivation. The later commit is a bit more tricky because "mke2fs" is not able to override file permissions. I used "fakeroot" to make sure that the raw disk-image files are owned by "root" and not "guixbuilder". Hoping it will help, Mathieu