From: "Ludovic Courtès" <ludo@gnu.org>
To: 55398@debbugs.gnu.org
Cc: "Ludovic Courtès" <ludo@gnu.org>
Subject: [bug#55398] [PATCH 3/3] store: Use a decaying cutoff in 'map/accumulate-builds'.
Date: Fri, 13 May 2022 17:00:44 +0200 [thread overview]
Message-ID: <20220513150044.11991-3-ludo@gnu.org> (raw)
In-Reply-To: <20220513150044.11991-1-ludo@gnu.org>
This reduces the wall-clock time of:
./pre-inst-env guix system vm gnu/system/examples/desktop.tmpl -n
from 2m13s to 53s (the timings depend on which derivations have already
been built and are in store; in this case, many were missing).
* guix/store.scm (default-cutoff): New variable.
(map/accumulate-builds): Use it. Parameterize it in recursive calls to
have decaying cutoff.
---
guix/store.scm | 39 +++++++++++++++++++++++----------------
1 file changed, 23 insertions(+), 16 deletions(-)
diff --git a/guix/store.scm b/guix/store.scm
index 220901f6ce..a3240eb2e0 100644
--- a/guix/store.scm
+++ b/guix/store.scm
@@ -1362,8 +1362,12 @@ (define (build-accumulator expected-store)
(unresolved things continue)
(continue #t))))
+(define default-cutoff
+ ;; Default cutoff parameter for 'map/accumulate-builds'.
+ (make-parameter 32))
+
(define* (map/accumulate-builds store proc lst
- #:key (cutoff 30))
+ #:key (cutoff (default-cutoff)))
"Apply PROC over each element of LST, accumulating 'build-things' calls and
coalescing them into a single call.
@@ -1377,21 +1381,24 @@ (define accumulator
(build-accumulator store))
(define-values (result rest)
- (let loop ((lst lst)
- (result '())
- (unresolved 0))
- (match lst
- ((head . tail)
- (match (with-build-handler accumulator
- (proc head))
- ((? unresolved? obj)
- (if (>= unresolved cutoff)
- (values (reverse (cons obj result)) tail)
- (loop tail (cons obj result) (+ 1 unresolved))))
- (obj
- (loop tail (cons obj result) unresolved))))
- (()
- (values (reverse result) lst)))))
+ ;; Have the default cutoff decay as we go deeper in the call stack to
+ ;; avoid pessimal behavior.
+ (parameterize ((default-cutoff (quotient cutoff 2)))
+ (let loop ((lst lst)
+ (result '())
+ (unresolved 0))
+ (match lst
+ ((head . tail)
+ (match (with-build-handler accumulator
+ (proc head))
+ ((? unresolved? obj)
+ (if (>= unresolved cutoff)
+ (values (reverse (cons obj result)) tail)
+ (loop tail (cons obj result) (+ 1 unresolved))))
+ (obj
+ (loop tail (cons obj result) unresolved))))
+ (()
+ (values (reverse result) lst))))))
(match (append-map (lambda (obj)
(if (unresolved? obj)
--
2.36.0
next prev parent reply other threads:[~2022-05-13 15:02 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-13 14:59 [bug#55398] [PATCH 0/3] Improve store caching; improve 'map/accumulate-builds' performance Ludovic Courtès
2022-05-13 15:00 ` [bug#55398] [PATCH 1/3] store: 'mcached' users can specify a cache ID Ludovic Courtès
2022-05-13 15:00 ` [bug#55398] [PATCH 2/3] packages: Use separate package/graft cache Ludovic Courtès
2022-05-13 15:00 ` Ludovic Courtès [this message]
2022-05-18 22:10 ` bug#55398: [PATCH 0/3] Improve store caching; improve 'map/accumulate-builds' performance Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220513150044.11991-3-ludo@gnu.org \
--to=ludo@gnu.org \
--cc=55398@debbugs.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).