From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id KNeBJKK18WHzZwEAgWs5BA (envelope-from ) for ; Wed, 26 Jan 2022 21:57:06 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id 4FtrIaK18WGHLgAAauVa8A (envelope-from ) for ; Wed, 26 Jan 2022 21:57:06 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1FB7F148B8 for ; Wed, 26 Jan 2022 21:57:06 +0100 (CET) Received: from localhost ([::1]:39010 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nCpLk-0002so-Vg for larch@yhetil.org; Wed, 26 Jan 2022 15:57:04 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37556) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nCpGs-0006Sj-Id for bug-guix@gnu.org; Wed, 26 Jan 2022 15:52:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:60764) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nCpGs-00034T-9g for bug-guix@gnu.org; Wed, 26 Jan 2022 15:52:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nCpGs-0003zz-2m for bug-guix@gnu.org; Wed, 26 Jan 2022 15:52:02 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#48007: computing derivations through inferior takes twice as long Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 26 Jan 2022 20:52:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 48007 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ricardo Wurmus Received: via spool by 48007-submit@debbugs.gnu.org id=B48007.164323029015335 (code B ref 48007); Wed, 26 Jan 2022 20:52:02 +0000 Received: (at 48007) by debbugs.gnu.org; 26 Jan 2022 20:51:30 +0000 Received: from localhost ([127.0.0.1]:53667 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCpGM-0003zH-6V for submit@debbugs.gnu.org; Wed, 26 Jan 2022 15:51:30 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:50610) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCpGJ-0003z1-LK for 48007@debbugs.gnu.org; Wed, 26 Jan 2022 15:51:28 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id B01D0316; Wed, 26 Jan 2022 21:51:20 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id l5c4frjKKyMO; Wed, 26 Jan 2022 21:51:19 +0100 (CET) Received: from ribbon (91-160-117-201.subs.proxad.net [91.160.117.201]) by hera.aquilenet.fr (Postfix) with ESMTPSA id ACA502A2; Wed, 26 Jan 2022 21:51:18 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87wnsrpf25.fsf@elephly.net> Date: Wed, 26 Jan 2022 21:51:18 +0100 In-Reply-To: <87wnsrpf25.fsf@elephly.net> (Ricardo Wurmus's message of "Sat, 24 Apr 2021 23:07:14 +0200") Message-ID: <87r18ufcft.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spamd-Bar: / X-Rspamd-Server: hera X-Rspamd-Queue-Id: B01D0316 X-Spamd-Result: default: False [-0.10 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/mixed,text/plain,text/x-patch]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 48007@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1643230626; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post; bh=7KREWvcQqskmf6aOXHuSLk1mEr1U5n3eayl1rhb5jPo=; b=qe2aoIlwDXt0Oi6dgVrnwOouv25aED5E19VMZRB/Ro26HjCBloEXZhOr9MhvgWv8TMoilc lsV4zeVdeEc8nDiGcwLDBm6yK+wclM1KJ/Di4jGCp6almSd6XZx8GwxpWXEMbsPZOzWa5u /mVxpwnkPAZqyWlTL413bceKnUa2YoK5VtSXGmvWuOGzfHsgBLuFZ3T5a3AlRTrBY0HTWa HBOYH7E5AHtYg2ZPpO8WlCsGMU5VcTuIkcgU/wH5/PP094/75H2VifLOkVorze7rrnbkst NOA6aexDSvAcykoN73vC7Hy3j/DVl8FQlBGsRpO8HmYtGE8ca0cqJCp/NBIeqQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1643230626; a=rsa-sha256; cv=none; b=MEmHkdSbKTgZkSrizzE6Y1HxgWUt5rmlddbnCUjUQg9m2tdaaRTa8FJFKFEk1RW/oaHf7U daNxdt9Va2GVycoY+5dgMLSiFsMjbrqLCvdLbNV+fjWDJMXwQAOV85/GCyP1BBgSbNOgzk tTqBzwzAUIhs/LZ5tQbZXaQw8ADcNsmQs1DeE93duVkKkULerVnAg7x8D9AXcd8qbJtMI9 qw1cFMzxFzCa/sTWakkf4VsnUwZiPoZUqEXmg9o9O0MobMvlJNE/0m/ZJU1wR7iHOnzGcC O9/eyDz9orifJtZOPcqr+6YgCVXDIjOSEL3BmNBfbUenDGUwo89NtRCO6W6miw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.13 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 1FB7F148B8 X-Spam-Score: -3.13 X-Migadu-Scanner: scn0.migadu.com X-TUID: PG+vnnsDcmJt --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi! Ricardo Wurmus skribis: > When INFERIOR is set to =E2=80=9Cy=E2=80=9D, each package specification w= ill be=20 > looked up in the current Guix via an inferior. For any other=20 > values of INFERIOR the specifications are resolved with the=20 > current Guix (the very same Guix) directly. > > Here are the timings: > > $ [env] export GUIX_PROFILING=3D"object-cache=20 > add-data-to-store-cache rpc" > $ [env] time INFERIOR=3Dn guix build -m manifest-test.scm -d > /gnu/store/mwg47gbmi98bbrywk07y5l2h9p6d1hz5-bash-minimal-5.0.16.drv > /gnu/store/kcjk6z128fa07pzp8irp6lbbyl3g16nr-r-corrplot-0.84.drv [...] > $ [env] time INFERIOR=3Dy guix build -m manifest-test.scm -d With the manifest you gave in this message, I get roughly these wall-clock times as of 3993d33d1c0129b1ca6f0fd122fe2bbe48e4f093 for: guix build -m the-manifest.scm -n INFERIOR=3Dn 4.1s INFERIOR=3Dy 36.9s With the patch below, it=E2=80=99s down to: INFERIOR=3Dy 9.3s The trick is to ensure the inferior maintains its object cache across calls. The patch needs to be cleaned up because it peeks into internals, but we should be able to do something along these lines and optimize a couple of other things. If you can give it a spin on a more representative example, that=E2=80=99s great! Thanks, Ludo=E2=80=99. --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/guix/inferior.scm b/guix/inferior.scm index 572114f626..f6866d2083 100644 --- a/guix/inferior.scm +++ b/guix/inferior.scm @@ -188,6 +188,8 @@ (define* (port->inferior pipe #:optional (close close-port)) (inferior-eval '(use-modules (srfi srfi-34)) result) (inferior-eval '(define %package-table (make-hash-table)) result) + (inferior-eval '(define %previous-object-cache #f) + result) result)) (_ #f))) @@ -559,6 +561,10 @@ (define (inferior-eval-with-store inferior store code) (let ((store (if (defined? 'port->connection) (port->connection socket #:version ,proto) (open-connection)))) + (when %previous-object-cache + (set-store-connection-cache! store (@@ (guix store) %object-cache-id) + %previous-object-cache)) + (dynamic-wind (const #t) (lambda () @@ -570,6 +576,9 @@ (define (inferior-eval-with-store inferior store code) `(store-protocol-error ,(error-message c)))) `(result ,(proc store)))) (lambda () + (set! %previous-object-cache + (store-connection-cache store + (@@ (guix store) %object-cache-id))) (close-connection store) (close-port socket))))) inferior) --=-=-=--