From: ludovic.courtes@inria.fr (Ludovic Courtès)
To: Roel Janssen <roel@gnu.org>
Cc: guix-devel@gnu.org, 27097@debbugs.gnu.org
Subject: Re: Performance on NFS
Date: Mon, 12 Jun 2017 17:58:46 +0200 [thread overview]
Message-ID: <87tw3lyz7t.fsf@inria.fr> (raw)
In-Reply-To: <rbu60g1k314.fsf@gnu.org> (Roel Janssen's message of "Mon, 12 Jun 2017 10:45:27 +0200")
Hi!
Roel Janssen <roel@gnu.org> skribis:
> From these timings, I don't think it has a big impact.
Wait, wait. If we take the best timings of each series of runs, we get:
hpc-submit1: 26.4s -> 18.0s (-30%)
hpc-guix: 26.2s -> 22.7s (-13%)
This is arguably insufficient but still non-negligible. I’ve committed
it as b46712159c15f72fc28b71d17d5a7c74fcb64ed0.
With commit 015f17e8b9eff97f656852180ac51c75438d7f9d, the number of
open(2) calls for that same command drops from 991 to 795 (including 122
errors). I suspect we can’t reduce it further:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env strace -o ,,s guix build coreutils -n
$ grep '^open.*' < ,,s |wc -l
795
$ grep '^open.*\.go"' < ,,s |wc -l
563
$ grep '^open.*\.patch"' < ,,s |wc -l
29
$ grep '^open.*\.scm"' < ,,s |wc -l
6
--8<---------------cut here---------------end--------------->8---
Could you check how this affects performance on your NFS system?
There’s possibly another low-hanging fruit, which is to disable file
name canonicalization (via ‘%file-port-name-canonicalization’.) It
special care though, so I’ll try that later.
> This makes me wonder, can't we replace the disk-intensive stuff with a
> database? If we only have to read the files on disk once, after which
> we extracted the information (the hashes?) needed to compute which
> links have to be created to make an environment, then actually
> creating the environment can be as fast as only creating those links.
Essentially we need to compute derivations as a function of local files
(sent to the daemon with ‘add-to-store’) and other derivations. We
cannot avoid that.
In the case of a remote server, communications with the daemon play an
important role too. Have you tried setting ‘GUIX_DAEMON_SOCKET’ as
suggested before instead of using the “socat hack”? I think this should
be faster (see commit 950d51c9d9a5107c5dac279da7d2e431134b5f43.)
HTH,
Ludo’.
next prev parent reply other threads:[~2017-06-12 15:58 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-27 10:56 bug#27097: [PATCH] gnu: Add emacs-direnv Christopher Baines
2017-05-27 11:31 ` Combining Guix, direnv and Emacs for environment customisation Christopher Baines
2017-05-30 15:03 ` Ludovic Courtès
2017-05-30 15:03 ` bug#27097: " Ludovic Courtès
2017-06-01 13:17 ` Roel Janssen
2017-06-03 13:59 ` Ludovic Courtès
2017-06-03 13:59 ` bug#27097: " Ludovic Courtès
2017-06-03 21:08 ` Roel Janssen
2017-06-04 21:15 ` Ludovic Courtès
2017-06-04 21:15 ` bug#27097: " Ludovic Courtès
2017-06-05 22:37 ` Roel Janssen
2017-06-07 10:35 ` Performance on NFS Ludovic Courtès
2017-06-07 13:06 ` Roel Janssen
2017-06-09 13:46 ` Ludovic Courtès
2017-06-12 8:45 ` Roel Janssen
2017-06-12 8:45 ` [bug#27097] " Roel Janssen
2017-06-12 15:58 ` Ludovic Courtès [this message]
2017-06-16 15:23 ` Ludovic Courtès
2017-06-17 7:36 ` Roel Janssen
2017-06-17 22:40 ` Ludovic Courtès
2017-06-17 23:15 ` Roel Janssen
2017-06-18 8:43 ` Ricardo Wurmus
2017-06-19 8:01 ` RPC performance Ludovic Courtès
2017-06-19 8:15 ` Ludovic Courtès
2017-06-19 14:25 ` Ricardo Wurmus
2017-06-22 14:03 ` Andy Wingo
2017-06-22 16:05 ` Ludovic Courtès
2017-06-23 9:09 ` Andy Wingo
2017-06-23 9:24 ` Ludovic Courtès
2017-06-23 9:46 ` Andy Wingo
2017-06-26 11:54 ` Ludovic Courtès
2017-06-26 14:19 ` Andy Wingo
2017-06-19 21:25 ` Ludovic Courtès
2017-06-22 8:04 ` Ricardo Wurmus
2017-06-07 11:01 ` Combining Guix, direnv and Emacs for environment customisation Ricardo Wurmus
2017-06-07 12:25 ` Performance on NFS Ludovic Courtès
2017-06-07 12:59 ` Ricardo Wurmus
2017-06-07 12:59 ` bug#27097: " Ricardo Wurmus
2017-08-29 19:02 ` Combining Guix, direnv and Emacs for environment customisation Thompson, David
2017-08-29 22:30 ` Maxim Cournoyer
2017-08-30 5:52 ` Carlo Zancanaro
2017-08-30 10:09 ` Ludovic Courtès
2017-08-30 10:16 ` Ludovic Courtès
2017-05-28 20:09 ` bug#27097: [PATCH] gnu: Add emacs-direnv Ludovic Courtès
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87tw3lyz7t.fsf@inria.fr \
--to=ludovic.courtes@inria.fr \
--cc=27097@debbugs.gnu.org \
--cc=guix-devel@gnu.org \
--cc=roel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.