From: "Ludovic Courtès" <ludo@gnu.org>
To: 34060@debbugs.gnu.org
Subject: [bug#34060] [PATCH 00/10] Add a cache for package lookups
Date: Sun, 13 Jan 2019 16:45:32 +0100 [thread overview]
Message-ID: <20190113154532.29606-1-ludo@gnu.org> (raw)
Hello!
This patch adjust ‘guix pull’ & co. such that Guix comes with a
cache to speed up package lookups. The end result is that:
guix build coreutils
becomes equivalent to:
guix build -e '(@ (gnu packages base) coreutils)'
That means we have fewer files to open (only the closure of
(gnu packages base) instead of all the (gnu packages …) modules),
and thus less I/O and a smaller memory footprint. Without cache,
we have:
--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=gc time ./pre-inst-env guix build coreutils -nd
/gnu/store/ddlgwpishzb8985gwg5fdrydvllg254a-coreutils-8.30.drv
Garbage collection statistics:
heap size: 46.18 MiB
allocated: 102.57 MiB
GC times: 14
time spent in GC: 0.25 seconds (25% of user time)
1.01user 0.06system 0:00.96elapsed 112%CPU (0avgtext+0avgdata 158916maxresident)k
0inputs+0outputs (0major+14602minor)pagefaults 0swaps
--8<---------------cut here---------------end--------------->8---
With the cache, we get:
--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=gc time ./foo/bin/guix build coreutils -nd
/gnu/store/ddlgwpishzb8985gwg5fdrydvllg254a-coreutils-8.30.drv
Garbage collection statistics:
heap size: 36.52 MiB
allocated: 87.30 MiB
GC times: 14
time spent in GC: 0.24 seconds (27% of user time)
0.89user 0.06system 0:00.84elapsed 112%CPU (0avgtext+0avgdata 124168maxresident)k
0inputs+0outputs (0major+11423minor)pagefaults 0swaps
--8<---------------cut here---------------end--------------->8---
As a bonus, we take advantage of the cache to speed up ‘guix
package -A’ (which now runs in ~0.5s instead of ~1.7s) and
‘guix edit’.
The cache is populated by packages of all the channels specified
during ‘guix pull’.
The cache is *not* used at all when GUIX_PACKAGE_PATH is set or
a ‘-L’ flag is passed, or ./pre-inst-env is used.
Feedback welcome!
Ludo’.
Ludovic Courtès (10):
profiling: Add a "gc" profiling component.
guix package: Avoid 'find-newest-available-packages'.
packages: Remove 'find-newest-available-packages'.
inferior: Add 'gexp->derivation-in-inferior'.
discovery: Add 'fold-module-public-variables*'.
pull: Build profile with 'channel-instances->derivation'.
channels: Compute a package cache and use it.
edit: Use 'specification->location' to read information from the
cache.
guix package: '--list-available' can use data from the cache.
status: Distinguish 'package-cache' profile hook.
gnu/packages.scm | 253 ++++++++++++++++++++++++++++++++++-----
guix/channels.scm | 32 ++++-
guix/discovery.scm | 28 ++++-
guix/inferior.scm | 26 ++++
guix/profiling.scm | 25 +++-
guix/scripts/edit.scm | 29 ++---
guix/scripts/package.scm | 137 +++++++++++----------
guix/scripts/pull.scm | 4 +-
guix/status.scm | 2 +
tests/packages.scm | 77 ++++++++++--
10 files changed, 491 insertions(+), 122 deletions(-)
--
2.20.1
next reply other threads:[~2019-01-13 15:46 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-01-13 15:45 Ludovic Courtès [this message]
2019-01-13 15:47 ` [bug#34060] [PATCH 01/10] profiling: Add a "gc" profiling component Ludovic Courtès
2019-01-13 15:47 ` [bug#34060] [PATCH 02/10] guix package: Avoid 'find-newest-available-packages' Ludovic Courtès
2019-01-13 15:47 ` [bug#34060] [PATCH 03/10] packages: Remove 'find-newest-available-packages' Ludovic Courtès
2019-01-13 15:47 ` [bug#34060] [PATCH 04/10] inferior: Add 'gexp->derivation-in-inferior' Ludovic Courtès
2019-01-13 15:47 ` [bug#34060] [PATCH 05/10] discovery: Add 'fold-module-public-variables*' Ludovic Courtès
2019-01-13 15:47 ` [bug#34060] [PATCH 06/10] pull: Build profile with 'channel-instances->derivation' Ludovic Courtès
2019-01-15 19:27 ` Ludovic Courtès
2019-01-13 15:47 ` [bug#34060] [PATCH 07/10] channels: Compute a package cache and use it Ludovic Courtès
2019-01-13 15:47 ` [bug#34060] [PATCH 08/10] edit: Use 'specification->location' to read information from the cache Ludovic Courtès
2019-01-13 15:47 ` [bug#34060] [PATCH 09/10] guix package: '--list-available' can use data " Ludovic Courtès
2019-01-13 15:47 ` [bug#34060] [PATCH 10/10] status: Distinguish 'package-cache' profile hook Ludovic Courtès
2019-01-15 19:26 ` bug#34060: [PATCH 00/10] Add a cache for package lookups Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190113154532.29606-1-ludo@gnu.org \
--to=ludo@gnu.org \
--cc=34060@debbugs.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.