From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludovic.courtes@inria.fr (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: Performance on NFS Date: Mon, 12 Jun 2017 17:58:46 +0200 Message-ID: <87tw3lyz7t.fsf@inria.fr> References: <20170527105641.9426-1-mail@cbaines.net> <20170527123113.1ca668e7@cbaines.net> <87tw424cap.fsf@gnu.org> <87fufhkw85.fsf@gnu.org> <871sr0ok2h.fsf@gnu.org> <8760gbh2th.fsf@gnu.org> <87efuym57c.fsf@gnu.org> <87a85kt7ad.fsf_-_@gnu.org> <87a85hnvqm.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:48565) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dKRjx-0000C6-3l for guix-devel@gnu.org; Mon, 12 Jun 2017 11:58:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dKRjt-0006rx-Rh for guix-devel@gnu.org; Mon, 12 Jun 2017 11:58:53 -0400 In-Reply-To: (Roel Janssen's message of "Mon, 12 Jun 2017 10:45:27 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Roel Janssen Cc: guix-devel@gnu.org, 27097@debbugs.gnu.org Hi! Roel Janssen skribis: > From these timings, I don't think it has a big impact. Wait, wait. If we take the best timings of each series of runs, we get: hpc-submit1: 26.4s -> 18.0s (-30%) hpc-guix: 26.2s -> 22.7s (-13%) This is arguably insufficient but still non-negligible. I=E2=80=99ve commi= tted it as b46712159c15f72fc28b71d17d5a7c74fcb64ed0. With commit 015f17e8b9eff97f656852180ac51c75438d7f9d, the number of open(2) calls for that same command drops from 991 to 795 (including 122 errors). I suspect we can=E2=80=99t reduce it further: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env strace -o ,,s guix build coreutils -n $ grep '^open.*' < ,,s |wc -l 795 $ grep '^open.*\.go"' < ,,s |wc -l 563 $ grep '^open.*\.patch"' < ,,s |wc -l 29 $ grep '^open.*\.scm"' < ,,s |wc -l 6 --8<---------------cut here---------------end--------------->8--- Could you check how this affects performance on your NFS system? There=E2=80=99s possibly another low-hanging fruit, which is to disable file name canonicalization (via =E2=80=98%file-port-name-canonicalization=E2=80= =99.) It special care though, so I=E2=80=99ll try that later. > This makes me wonder, can't we replace the disk-intensive stuff with a > database? If we only have to read the files on disk once, after which > we extracted the information (the hashes?) needed to compute which > links have to be created to make an environment, then actually > creating the environment can be as fast as only creating those links. Essentially we need to compute derivations as a function of local files (sent to the daemon with =E2=80=98add-to-store=E2=80=99) and other derivati= ons. We cannot avoid that. In the case of a remote server, communications with the daemon play an important role too. Have you tried setting =E2=80=98GUIX_DAEMON_SOCKET=E2= =80=99 as suggested before instead of using the =E2=80=9Csocat hack=E2=80=9D? I thin= k this should be faster (see commit 950d51c9d9a5107c5dac279da7d2e431134b5f43.) HTH, Ludo=E2=80=99.