From: ludovic.courtes@inria.fr (Ludovic Courtès)
To: Ricardo Wurmus <rekado@elephly.net>
Cc: guix-devel@gnu.org
Subject: RPC performance
Date: Mon, 19 Jun 2017 10:01:01 +0200 [thread overview]
Message-ID: <87vans1k5e.fsf_-_@inria.fr> (raw)
In-Reply-To: <87k2497kk7.fsf@elephly.net> (Ricardo Wurmus's message of "Sun, 18 Jun 2017 10:43:20 +0200")
Hello,
Ricardo Wurmus <rekado@elephly.net> skribis:
> Performance differs a lot for different packages. “guix build bwa” (~ 7
> seconds) is a lot faster than “guix build python2-numpy” (> 40 seconds),
> for example.
My guess is that this is mostly due to RPC performance:
--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=rpc time ./pre-inst-env guix build bwa -n
Remote procedure call summary: 266 RPCs
built-in-builders ... 1
query-substitutable-path-infos ... 1
valid-path? ... 3
add-text-to-store ... 261
0.88user 0.04system 0:00.83elapsed 110%CPU (0avgtext+0avgdata 497792maxresident)k
0inputs+0outputs (0major+11771minor)pagefaults 0swaps
$ GUIX_PROFILING=rpc time ./pre-inst-env guix build python2-numpy -n
Remote procedure call summary: 668 RPCs
built-in-builders ... 1
query-substitutable-path-infos ... 1
valid-path? ... 3
add-text-to-store ... 663
1.22user 0.06system 0:01.33elapsed 95%CPU (0avgtext+0avgdata 508288maxresident)k
1432inputs+0outputs (0major+10752minor)pagefaults 0swaps
--8<---------------cut here---------------end--------------->8---
There are several ways we can improve it, and over time we should try to
implement all of them:
1. Buffer writes to the server socket (currently it’s terrible if you
look at the ‘write’ calls in ‘strace’).
2. Make fewer ‘add-text-to-store’ calls. This can be done by, for
instance, always using the same ‘-guile-builder’ script for
everything that uses ‘gnu-build-system’, and parameterize the
script through environment variables or similar.
3. Replace N ‘add-text-to-store’ calls with a single ‘import-paths’
call. It’s not possible yet because ‘import-paths’ requires a
signature, which is pointless for content-address items.
4. Alternatively, change the protocol to allow for RPC pipelining
(send N requests, then read N responses). Currently this isn’t
possible because all the RPCs call ‘process-stderr’ (which makes
one or more round trips) because reading their response.
I’ll look into #1 soon and we can investigate #2 on the next
‘core-updates’ cycle.
Thanks,
Ludo’.
next prev parent reply other threads:[~2017-06-19 8:01 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-27 10:56 bug#27097: [PATCH] gnu: Add emacs-direnv Christopher Baines
2017-05-27 11:31 ` Combining Guix, direnv and Emacs for environment customisation Christopher Baines
2017-05-30 15:03 ` Ludovic Courtès
2017-05-30 15:03 ` bug#27097: " Ludovic Courtès
2017-06-01 13:17 ` Roel Janssen
2017-06-03 13:59 ` Ludovic Courtès
2017-06-03 13:59 ` bug#27097: " Ludovic Courtès
2017-06-03 21:08 ` Roel Janssen
2017-06-04 21:15 ` Ludovic Courtès
2017-06-04 21:15 ` bug#27097: " Ludovic Courtès
2017-06-05 22:37 ` Roel Janssen
2017-06-07 10:35 ` Performance on NFS Ludovic Courtès
2017-06-07 13:06 ` Roel Janssen
2017-06-09 13:46 ` Ludovic Courtès
2017-06-12 8:45 ` Roel Janssen
2017-06-12 8:45 ` [bug#27097] " Roel Janssen
2017-06-12 15:58 ` Ludovic Courtès
2017-06-16 15:23 ` Ludovic Courtès
2017-06-17 7:36 ` Roel Janssen
2017-06-17 22:40 ` Ludovic Courtès
2017-06-17 23:15 ` Roel Janssen
2017-06-18 8:43 ` Ricardo Wurmus
2017-06-19 8:01 ` Ludovic Courtès [this message]
2017-06-19 8:15 ` RPC performance Ludovic Courtès
2017-06-19 14:25 ` Ricardo Wurmus
2017-06-22 14:03 ` Andy Wingo
2017-06-22 16:05 ` Ludovic Courtès
2017-06-23 9:09 ` Andy Wingo
2017-06-23 9:24 ` Ludovic Courtès
2017-06-23 9:46 ` Andy Wingo
2017-06-26 11:54 ` Ludovic Courtès
2017-06-26 14:19 ` Andy Wingo
2017-06-19 21:25 ` Ludovic Courtès
2017-06-22 8:04 ` Ricardo Wurmus
2017-06-07 11:01 ` Combining Guix, direnv and Emacs for environment customisation Ricardo Wurmus
2017-06-07 12:25 ` Performance on NFS Ludovic Courtès
2017-06-07 12:59 ` Ricardo Wurmus
2017-06-07 12:59 ` bug#27097: " Ricardo Wurmus
2017-08-29 19:02 ` Combining Guix, direnv and Emacs for environment customisation Thompson, David
2017-08-29 22:30 ` Maxim Cournoyer
2017-08-30 5:52 ` Carlo Zancanaro
2017-08-30 10:09 ` Ludovic Courtès
2017-08-30 10:16 ` Ludovic Courtès
2017-05-28 20:09 ` bug#27097: [PATCH] gnu: Add emacs-direnv Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87vans1k5e.fsf_-_@inria.fr \
--to=ludovic.courtes@inria.fr \
--cc=guix-devel@gnu.org \
--cc=rekado@elephly.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.