From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id TGAOIgp1XF/uBgAA0tVLHw (envelope-from ) for ; Sat, 12 Sep 2020 07:13:14 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id EAcEHAp1XF/tQQAAB5/wlQ (envelope-from ) for ; Sat, 12 Sep 2020 07:13:14 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id EC5E894051F for ; Sat, 12 Sep 2020 07:13:13 +0000 (UTC) Received: from localhost ([::1]:35520 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kGzih-0000UY-Ql for larch@yhetil.org; Sat, 12 Sep 2020 03:13:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:32814) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kGzib-0000UA-In for guix-patches@gnu.org; Sat, 12 Sep 2020 03:13:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:34548) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kGziY-0001pe-8f for guix-patches@gnu.org; Sat, 12 Sep 2020 03:13:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kGziY-0000vl-4m for guix-patches@gnu.org; Sat, 12 Sep 2020 03:13:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43340] [PATCH 0/5] Speed up archive export/import Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 12 Sep 2020 07:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43340 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 43340@debbugs.gnu.org Received: via spool by 43340-submit@debbugs.gnu.org id=B43340.15998947403528 (code B ref 43340); Sat, 12 Sep 2020 07:13:02 +0000 Received: (at 43340) by debbugs.gnu.org; 12 Sep 2020 07:12:20 +0000 Received: from localhost ([127.0.0.1]:46094 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGzhs-0000up-Gf for submit@debbugs.gnu.org; Sat, 12 Sep 2020 03:12:20 -0400 Received: from eggs.gnu.org ([209.51.188.92]:57042) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGzhq-0000ua-FP for 43340@debbugs.gnu.org; Sat, 12 Sep 2020 03:12:18 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:58936) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kGzhk-0001fS-Ii; Sat, 12 Sep 2020 03:12:12 -0400 Received: from [2a01:e0a:19b:d9a0:9d9d:97cc:d92a:8ac0] (port=48100 helo=cervin) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kGzhj-0001AX-St; Sat, 12 Sep 2020 03:12:12 -0400 From: Mathieu Othacehe References: <20200911144049.14632-1-ludo@gnu.org> <87een8tl0s.fsf@gnu.org> Date: Sat, 12 Sep 2020 09:12:09 +0200 In-Reply-To: <87een8tl0s.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Fri, 11 Sep 2020 17:01:39 +0200") Message-ID: <87v9gjh3jq.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: -1.01 X-TUID: wLsHUccab4n2 Hey Ludo, >> Specifically, it implements option #4 (spawning =E2=80=98guix authentica= te=E2=80=99 >> once for the whole session, instead of spawning it every time a >> store item needs to be signed or authenticated), achieving a ~15x >> speedup, which is not bad. :-) Woo, congrats! > Below is the new Gantt chart for: > > perf timechart record guix archive --export -r $(guix build coreutils -= d) -v3 >/tmp/dump > > Most of the work happens in =E2=80=98guix authenticate=E2=80=99. I never used the "timechart" sub-command but it sounds really nice. Regarding the option you chose, I think it's the more appropriate right now. It's very delicate to dedicate time and effort to tweak guix-daemon and how we use it, having in mind that we'd like to get rid of it. However, the potential short term gains can be so huge, that for now it's the best thing to do. I should just do like you, and dive into it to see what can be done for contention and locking when using 'build-paths' RPC via Cuirass. In the meantime, a short review that I hope to complete next week. Thanks for your efforts, Mathieu