From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 8JNNKQa6wF+nPwAA0tVLHw (envelope-from ) for ; Fri, 27 Nov 2020 08:34:14 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id kDUkJQa6wF+uGwAA1q6Kng (envelope-from ) for ; Fri, 27 Nov 2020 08:34:14 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 477D69401C0 for ; Fri, 27 Nov 2020 08:34:14 +0000 (UTC) Received: from localhost ([::1]:50434 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kiZCn-00065u-93 for larch@yhetil.org; Fri, 27 Nov 2020 03:34:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:51228) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kiZCc-000659-HI for guix-patches@gnu.org; Fri, 27 Nov 2020 03:34:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:60549) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kiZCc-0002Hu-A3 for guix-patches@gnu.org; Fri, 27 Nov 2020 03:34:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kiZCc-0004Mw-7o for guix-patches@gnu.org; Fri, 27 Nov 2020 03:34:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#44899] [PATCH 0/3] Using 'ld.so.cache' to speed up application startup Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 27 Nov 2020 08:34:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 44899 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 44899@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.160646600116723 (code B ref -1); Fri, 27 Nov 2020 08:34:02 +0000 Received: (at submit) by debbugs.gnu.org; 27 Nov 2020 08:33:21 +0000 Received: from localhost ([127.0.0.1]:43859 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kiZBv-0004Ld-LY for submit@debbugs.gnu.org; Fri, 27 Nov 2020 03:33:21 -0500 Received: from lists.gnu.org ([209.51.188.17]:49514) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kiZBq-0004LS-Mh for submit@debbugs.gnu.org; Fri, 27 Nov 2020 03:33:18 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:50928) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kiZBq-0005u3-Gk for guix-patches@gnu.org; Fri, 27 Nov 2020 03:33:14 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:33418) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kiZBq-00020u-8b; Fri, 27 Nov 2020 03:33:14 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=46266 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kiZBn-00084Q-Kb; Fri, 27 Nov 2020 03:33:12 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Fri, 27 Nov 2020 09:33:02 +0100 Message-Id: <20201127083302.2578-1-ludo@gnu.org> X-Mailer: git-send-email 2.29.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -1.76 X-Scanner: ns3122888.ip-94-23-21.eu Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-TUID: /HATiHRpAV+d Hello Guix! The other day on IRC Ricardo had the brilliant idea of using the ld.so cache to avoid the “stat storm” stemming from our long RUNPATHs, and thus to speed up application startup. As an example, Guile has 9 entries in its RUNPATH and Inkscape has 44 entries. The first patch changes the loader (1) to look for the cache in $ORIGIN/../etc/ld.so.cache, and (2) to look for the cache before looking at RUNPATH entries. The second patch adds a build phase that creates ‘etc/ld.so.cache’. It passes ‘ldconfig’ a config file that contains the union of all the RUNPATH entries of all the executables found in the output at hand. (It cannot be done in a profile hook because $ORIGIN is determined by looking at /proc/self/exe, which is the canonical file name in the store.) You can see it in action with LD_DEBUG=libs: --8<---------------cut here---------------start------------->8--- $ LD_DEBUG=libs /gnu/store/3dfv892jq8081xnsl9gfylbgv1fdicvd-guile-3.0.4/bin/guile --version 11150: find library=libguile-3.0.so.1 [0]; searching 11150: search cache=/gnu/store/3dfv892jq8081xnsl9gfylbgv1fdicvd-guile-3.0.4/bin/../etc/ld.so.cache 11150: trying file=/gnu/store/3dfv892jq8081xnsl9gfylbgv1fdicvd-guile-3.0.4/lib/libguile-3.0.so.1 11150: 11150: find library=libgc.so.1 [0]; searching 11150: search cache=/gnu/store/3dfv892jq8081xnsl9gfylbgv1fdicvd-guile-3.0.4/bin/../etc/ld.so.cache 11150: trying file=/gnu/store/hy88vf2ynlica0wj0ppi0d3b11gi2b2h-libgc-8.0.4/lib/libgc.so.1 [...] --8<---------------cut here---------------end--------------->8--- Here’s the after/before for Guile: --8<---------------cut here---------------start------------->8--- $ strace -c /gnu/store/3dfv892jq8081xnsl9gfylbgv1fdicvd-guile-3.0.4/bin/guile --version [...] 2.70 0.000259 5 46 6 openat 1.87 0.000180 1 130 88 stat 1.66 0.000159 4 36 rt_sigprocmask 0.74 0.000071 1 40 close 0.64 0.000061 3 18 fstat [...] 100.00 0.009604 600 105 total $ strace -c guile --version [...] 13.82 0.000723 4 165 114 openat 13.46 0.000704 3 190 144 stat [...] 1.43 0.000075 2 29 fstat [...] 100.00 0.005232 773 268 total --8<---------------cut here---------------end--------------->8--- Erroneous syscalls are divided by 2.5; total syscalls reduced by 22%. For Bash: --8<---------------cut here---------------start------------->8--- $ strace -c /gnu/store/qs33sf58502v1wx77va092y14sbspv4f-bash-minimal-5.0.16/bin/bash --version GNU bash, version 5.0.16(1)-release (x86_64-unknown-linux-gnu) Copyright (C) 2019 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 0.00 0.000000 0 4 read 0.00 0.000000 0 6 write 0.00 0.000000 0 5 close 0.00 0.000000 0 5 fstat 0.00 0.000000 0 12 mmap 0.00 0.000000 0 5 mprotect 0.00 0.000000 0 1 munmap 0.00 0.000000 0 3 brk 0.00 0.000000 0 1 rt_sigprocmask 0.00 0.000000 0 1 1 access 0.00 0.000000 0 1 execve 0.00 0.000000 0 1 readlink 0.00 0.000000 0 1 getuid 0.00 0.000000 0 1 getgid 0.00 0.000000 0 1 geteuid 0.00 0.000000 0 1 getegid 0.00 0.000000 0 1 arch_prctl 0.00 0.000000 0 9 4 openat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.000000 59 5 total $ strace -c /gnu/store/fvhj74pghapbjvsvj27skvkra1by1965-bash-minimal-5.0.16/bin/bash --version GNU bash, version 5.0.16(1)-release (x86_64-unknown-linux-gnu) Copyright (C) 2019 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software; you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 34.15 0.000420 12 35 16 openat 18.29 0.000225 8 27 mmap 8.13 0.000100 25 4 brk 7.72 0.000095 6 14 14 stat 7.24 0.000089 89 1 1 access 6.18 0.000076 4 19 fstat 5.45 0.000067 3 19 close 4.47 0.000055 2 20 read 3.82 0.000047 9 5 mprotect 2.36 0.000029 4 6 write 0.65 0.000008 8 1 execve 0.41 0.000005 5 1 arch_prctl 0.33 0.000004 4 1 getuid 0.24 0.000003 3 1 rt_sigprocmask 0.24 0.000003 3 1 getegid 0.16 0.000002 2 1 getgid 0.16 0.000002 2 1 geteuid ------ ----------- ----------- --------- --------- ---------------- 100.00 0.001230 157 31 total --8<---------------cut here---------------end--------------->8--- Erroneous syscalls are divided by 6; total syscalls divided by 2.7. As always, this is probably not that big a deal on warm-cache SSD, but it probably makes a difference on a cold cache, on spinning disks, and on network file systems. * Possible improvements The hard-coded ‘../etc/ld.so.cache’ means that it can only be used with first-level sub-directories like bin/ and sbin/; it won’t be used for libexec/guix/guile, for instance, which is a bummer. Perhaps we should compute the ‘ld.so.cache’ file name “lexically” instead. We should also think hard about ways users could be tricked into loading a malicious ‘ld.so.cache’. That’s also another reason why “lexical dot-dot” would be safer: we could ensure that only pre-computed ‘ld.so.cache’ that live in the store are ever loaded. The ‘ld.so.conf’ file passed to ‘ldconfig’ should ideally contains the RUNPATH entries _recursively_, such that even indirect dependencies can be found in cache. Thoughts? Ludo’. Ludovic Courtès (3): gnu: glibc: Load ${ORIGIN}/../etc/ld.so.cache when available. gremlin: Fix typo in docstring. build-system/gnu: Add 'make-dynamic-linker-cache' phase. gnu/local.mk | 1 + gnu/packages/base.scm | 11 +- gnu/packages/patches/glibc-dl-cache.patch | 122 ++++++++++++++++++++++ guix/build-system/gnu.scm | 4 + guix/build/gnu-build-system.scm | 73 +++++++++++++ guix/build/gremlin.scm | 2 +- 6 files changed, 202 insertions(+), 11 deletions(-) create mode 100644 gnu/packages/patches/glibc-dl-cache.patch -- 2.29.2