From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id XiNVIBJJzl+EUwAA0tVLHw (envelope-from ) for ; Mon, 07 Dec 2020 15:24:02 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id IEnBGxJJzl8vJAAAB5/wlQ (envelope-from ) for ; Mon, 07 Dec 2020 15:24:02 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id EF50F94043A for ; Mon, 7 Dec 2020 15:24:01 +0000 (UTC) Received: from localhost ([::1]:36640 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmIMq-0006dO-V2 for larch@yhetil.org; Mon, 07 Dec 2020 10:24:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39582) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kmILu-0005Wt-SL for guix-patches@gnu.org; Mon, 07 Dec 2020 10:23:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:43352) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kmILt-0003H7-TS for guix-patches@gnu.org; Mon, 07 Dec 2020 10:23:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kmILt-0003iv-Og for guix-patches@gnu.org; Mon, 07 Dec 2020 10:23:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#45102] [PATCH 0/4] Making fewer 'stat' calls during startup Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 07 Dec 2020 15:23:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 45102 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 45102@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.160735453414242 (code B ref -1); Mon, 07 Dec 2020 15:23:01 +0000 Received: (at submit) by debbugs.gnu.org; 7 Dec 2020 15:22:14 +0000 Received: from localhost ([127.0.0.1]:54898 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIL8-0003hd-Gv for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:22:14 -0500 Received: from lists.gnu.org ([209.51.188.17]:54886) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kmIL4-0003hT-Vh for submit@debbugs.gnu.org; Mon, 07 Dec 2020 10:22:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39324) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kmIL4-0004o7-Ok for guix-patches@gnu.org; Mon, 07 Dec 2020 10:22:10 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:56893) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kmIL3-00030z-LW; Mon, 07 Dec 2020 10:22:10 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=48406 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kmIL1-0001wk-Nv; Mon, 07 Dec 2020 10:22:08 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Mon, 7 Dec 2020 16:21:59 +0100 Message-Id: <20201207152159.28551-1-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -1.80 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: EF50F94043A X-Spam-Score: -1.80 X-Migadu-Scanner: ns3122888.ip-94-23-21.eu X-TUID: foG1IN7GQQ1+ Hi Guix! On Guix System, with 6 entries in $GUILE_LOAD_PATH (which should just be 2 entries, one for /run/current-system and one for ~/.guix-profile), I see this: --8<---------------cut here---------------start------------->8--- $ strace -c -e stat,openat guix guix: mankas komanda nomo Provu 'guix --help' por pli da informo. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 90.94 0.002207 0 2283 2057 stat 9.06 0.000220 0 334 177 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.002427 0 2617 2234 total --8<---------------cut here---------------end--------------->8--- This is a lot of ‘stat’ calls for nothing, 4 times more than when GUILE_LOAD_PATH is unset: --8<---------------cut here---------------start------------->8--- $ env -i $(type -P strace) -c -e stat,openat $(type -P guix) guix: missing command name Try `guix --help' for more information. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 57.89 0.000187 0 495 272 stat 42.11 0.000136 0 321 175 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000323 0 816 447 total --8<---------------cut here---------------end--------------->8--- This patch series takes a sledgehammer approach to always have as few ‘stat’ calls as possible during startup. After the change, I get: --8<---------------cut here---------------start------------->8--- $ strace -e stat,openat -c /gnu/store/g9gylj723si8i2cp8ia57a7kr4i8b1m9-guix-20201207.14/bin/guix guix: mankas komanda nomo Provu 'guix --help' por pli da informo. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 60.53 0.000322 0 454 231 stat 39.47 0.000210 0 334 177 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000532 0 788 408 total $ env -i $(type -P strace) -e stat,openat -c /gnu/store/g9gylj723si8i2cp8ia57a7kr4i8b1m9-guix-20201207.14/bin/guix guix: missing command name Try `guix --help' for more information. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 69.92 0.000272 0 442 219 stat 30.08 0.000117 0 321 175 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000389 0 763 394 total --8<---------------cut here---------------end--------------->8--- What impact does it have on startup time? Hard to tell. On my 4-year old x86_64 laptop with an SSD, it seems to be negligible, even on a cold cache. I suspect things are different on spinning disks and on NFS. The semantic difference should be invisible to users: their modules are still visible and usable in manifests, in ‘guix repl’, etc.; they just cannot take precedence over modules from Guile and from the channels. For instance, one cannot define a (gnu packages base) module or (ice-9 rdelim), drop it in $GUILE_LOAD_PATH, and have that module picked up in lieu of the original one. I think that’s a reasonable tradeoff. Thoughts? Ludo’. Ludovic Courtès (4): build: 'script/guix' uses our own 'guile' executable. self: Move Guile early in the module search path. guix: 'guile' executable ignores GUILE_LOAD_PATH during startup. self: Remove the empty string from '%load-extensions'. Makefile.am | 1 + gnu/packages/aux-files/guile-launcher.c | 46 +++++++++++++++++++++++-- guix/self.scm | 26 +++++++++----- scripts/guix.in | 2 +- 4 files changed, 63 insertions(+), 12 deletions(-) -- 2.29.2