From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id gDCtKb+MW184OQAA0tVLHw (envelope-from ) for ; Fri, 11 Sep 2020 14:42:07 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id +Aj+I7+MW18dewAAB5/wlQ (envelope-from ) for ; Fri, 11 Sep 2020 14:42:07 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 4FCD494050F for ; Fri, 11 Sep 2020 14:42:07 +0000 (UTC) Received: from localhost ([::1]:33182 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kGkFa-0000Q1-8h for larch@yhetil.org; Fri, 11 Sep 2020 10:42:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58190) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kGkFW-0000NI-Am for guix-patches@gnu.org; Fri, 11 Sep 2020 10:42:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:33506) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kGkFV-0006GQ-UG for guix-patches@gnu.org; Fri, 11 Sep 2020 10:42:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kGkFV-00048M-Q3 for guix-patches@gnu.org; Fri, 11 Sep 2020 10:42:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43340] [PATCH 0/5] Speed up archive export/import Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 11 Sep 2020 14:42:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 43340 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43340@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.159983527415809 (code B ref -1); Fri, 11 Sep 2020 14:42:01 +0000 Received: (at submit) by debbugs.gnu.org; 11 Sep 2020 14:41:14 +0000 Received: from localhost ([127.0.0.1]:45048 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGkEf-00046i-5l for submit@debbugs.gnu.org; Fri, 11 Sep 2020 10:41:14 -0400 Received: from lists.gnu.org ([209.51.188.17]:53504) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kGkEa-00046W-9Y for submit@debbugs.gnu.org; Fri, 11 Sep 2020 10:41:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57724) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kGkEZ-0007ls-VC for guix-patches@gnu.org; Fri, 11 Sep 2020 10:41:03 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:40154) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kGkEZ-00067O-9T; Fri, 11 Sep 2020 10:41:03 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=39260 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kGkEW-0003Bj-Dd; Fri, 11 Sep 2020 10:41:01 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Fri, 11 Sep 2020 16:40:49 +0200 Message-Id: <20200911144049.14632-1-ludo@gnu.org> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: -0.01 X-TUID: jUaa0nKP4cRu Hi! This patch series goes on top of . It addresses the performance issue described at: https://lists.gnu.org/archive/html/guix-devel/2020-09/msg00073.html Specifically, it implements option #4 (spawning ‘guix authenticate’ once for the whole session, instead of spawning it every time a store item needs to be signed or authenticated), achieving a ~15x speedup, which is not bad. :-) There’s way more C++ code than I would like, and it’s probably not pretty code (I always end up typing things like “C++ list append” in a search engine to find the obscure incantation that does that). There’s now a query/reply protocol between the daemon and ‘guix authenticate’. Nothing fancy, but it allows for strings that contain whitespace or newlines, which is necessary here. That’s it! Ludo’. Ludovic Courtès (5): daemon: Generalize 'HookInstance' to 'Agent'. daemon: Isolate signing and signature verification functions. daemon: Move 'Agent' to libutil. daemon: Spawn 'guix authenticate' once for all. authenticate: Cache the ACL and key pairs. guix/scripts/authenticate.scm | 163 ++++++++++++++++++++++++++-------- nix/libstore/build.cc | 146 +++++------------------------- nix/libstore/local-store.cc | 103 +++++++++++++++++---- nix/libutil/util.cc | 84 ++++++++++++++++++ nix/libutil/util.hh | 25 ++++++ tests/guix-authenticate.sh | 45 ++++++---- tests/store.scm | 8 +- 7 files changed, 372 insertions(+), 202 deletions(-) -- 2.28.0