From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id cSxPHmhT3173JgAA0tVLHw (envelope-from ) for ; Tue, 09 Jun 2020 09:16:24 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id iNWXGWhT3175MwAAB5/wlQ (envelope-from ) for ; Tue, 09 Jun 2020 09:16:24 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id AF792940982 for ; Tue, 9 Jun 2020 09:16:23 +0000 (UTC) Received: from localhost ([::1]:48794 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jiaMo-0007TI-MN for larch@yhetil.org; Tue, 09 Jun 2020 05:16:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51826) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jiaMU-0007SJ-2L for bug-guix@gnu.org; Tue, 09 Jun 2020 05:16:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:47391) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jiaMT-0007wg-NA for bug-guix@gnu.org; Tue, 09 Jun 2020 05:16:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jiaMT-0000Or-Iz for bug-guix@gnu.org; Tue, 09 Jun 2020 05:16:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#41702: `guix environment` performance issues Resent-From: Lars-Dominik Braun Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 09 Jun 2020 09:16:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41702 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 41702-submit@debbugs.gnu.org id=B41702.15916941321498 (code B ref 41702); Tue, 09 Jun 2020 09:16:01 +0000 Received: (at 41702) by debbugs.gnu.org; 9 Jun 2020 09:15:32 +0000 Received: from localhost ([127.0.0.1]:58937 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jiaM0-0000O6-6n for submit@debbugs.gnu.org; Tue, 09 Jun 2020 05:15:32 -0400 Received: from mail-wm1-f49.google.com ([209.85.128.49]:53192) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jiaLy-0000Ns-98 for 41702@debbugs.gnu.org; Tue, 09 Jun 2020 05:15:30 -0400 Received: by mail-wm1-f49.google.com with SMTP id r9so2095910wmh.2 for <41702@debbugs.gnu.org>; Tue, 09 Jun 2020 02:15:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=qz4Fu3nZaZG2CLuasY5u1szhQ0Dyk5hOv98/XWZ5f1I=; b=jP3zCMyf+vrHZiMZ0et32GFF4IeY7qtzwFGaS9kzDVwzKpR4C4Pq5LBBZ1tIX1Ltq0 3/IGb9m26WjiOYTax6fw6rXH4bCF+Wv29imYKoV6bIWbqOe7daYNYLdgQgq7yi2VJKLR Jgj6BUGa3+z6fMUlZ7AagJkFFVi4A0ujK++jyr73jnTLqgjWCepC4K3bDoth2OBMSZLc BKPL68T+ZAgWyH5uG+nhwPtFZqVeXz13SblFnba+XAMmmelUGrhIz0a/TTDeAF2SoDpY ruTRpOYZvAQfdPoX6ZukaPCx4Wol7ZE0KGwskRAP33YlUC/yi/etozTE4D7vPRMN+Kfz h4hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=qz4Fu3nZaZG2CLuasY5u1szhQ0Dyk5hOv98/XWZ5f1I=; b=rzIzad18CtNbriiEpVqMr2comC4pEnNQXWoJ0WLb8LYX+8F905YeILSe/IxJ5cgv+t k2fecfMbAyDuyjTV+q/bE3nkuV2Vn7zwYKMgmtQwJFjyi7HGS/oOBXQj5vM+yIZXdm5t TxnIe54klIl04SxZnbavdQEWBu0ZJMkkCw92NB8xHWJ56P1L5mf6+IChxp7LaKcRYsbQ TuzV1SE/SRHmnAzIBJxo6lvQ9gqZQUcoeGiiulog7gCxbWrCVRzcCkYiWTCiVC/sczXu ZNzKqPa1aoDT6E0Wa4Wvb26l5uaM91Ff2CW3ADg7/zFAckl4ikDb/762ZD7Y9RLUFi5+ MrgA== X-Gm-Message-State: AOAM530kEJIIvRddXlfoJgHjhNiknSDKlO50mjQ1vOkwoUyEJRSGjYFM bK2kapAObljXTagm/AE2j5+nygtBZNeZLWHdzZ6MJ7Ak1OefOUgxfIzwzlRWfMADvC+xupUkQ6C ZxLmP3fL/Hp9+VOwFqs68mwwpN+8J91r4QBBF20Ujj+icrrRDhbSVOp2t1fuAk3XLKOuZ9+BsE9 WGAbo= X-Google-Smtp-Source: ABdhPJzlQ3huw0HzeLj0aBtVws0x62rRoHHH8RdbCo+jm94YcF1nhx9VjXOLOa8enzfdEKbhAMTIVw== X-Received: by 2002:a1c:98cc:: with SMTP id a195mr2967897wme.89.1591694124024; Tue, 09 Jun 2020 02:15:24 -0700 (PDT) Received: from localhost (dynamic-2a01-0c23-7c5e-9e00-3d61-c8ee-9cd9-ef49.c23.pool.telefonica.de. [2a01:c23:7c5e:9e00:3d61:c8ee:9cd9:ef49]) by smtp.gmail.com with ESMTPSA id o18sm2276486wme.19.2020.06.09.02.15.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Jun 2020 02:15:23 -0700 (PDT) Date: Tue, 9 Jun 2020 11:15:22 +0200 From: Lars-Dominik Braun Message-ID: <20200609091522.GB8597@zpidnp36> References: <20200604082316.GA3146@zpidnp36> <87mu5gtbwg.fsf@gnu.org> <20200608090453.GC3166@zpidnp36> <87k10hp6ba.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Pk6IbRAofICFmK5e" Content-Disposition: inline In-Reply-To: <87k10hp6ba.fsf@gnu.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 41702@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=leibniz-psychology-org.20150623.gappssmtp.com header.s=20150623 header.b=jP3zCMyf; dmarc=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: -1.61 X-TUID: 3DS4MU5St6q2 --Pk6IbRAofICFmK5e Content-Type: multipart/mixed; boundary="XF85m9dhOBO43t/C" Content-Disposition: inline --XF85m9dhOBO43t/C Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hey, > That=E2=80=99s over SSH, right? correct, the worst possible case: Inside two VM=E2=80=99s on a Laptop, SSH = transport between them and /gnu+/var/guix on an NFS share (nfsd is in the same VM as guix-daemon). > Probably what=E2=80=99s killing us is the round-trip time for all these s= mall > RPCs. We would need pipelining but the RPC protocol is not designed to > make that easy. That would have been my best guess too, but it does not seem to be the bigg= est problem right now. Looking at the numbers again (both patches applied) with= the attached manifest[1], I see that: ---snip--- Local UNIX socket with and without --no-grafts: N Min Max Median Avg Stddev x 10 6.07 6.35 6.145 6.16 0.08232726 + 10 17.47 17.89 17.545 17.602 0.14351152 Difference at 99.0% confidence 11.442 +/- 0.150576 185.747% +/- 4.07133% Local UNIX socket vs. guix://localhost transport: N Min Max Median Avg Stddev x 10 17.47 17.89 17.545 17.602 0.14351152 + 10 17.43 18.1 17.61 17.642 0.20131788 No difference proven at 99.0% confidence Local UNIX socket vs ssh://localhost transport: N Min Max Median Avg Stddev x 10 17.47 17.89 17.545 17.602 0.14351152 + 10 33.46 35.27 34.315 34.359 0.53873205 Difference at 99.0% confidence 16.757 +/- 0.5074 95.1994% +/- 3.13957% ---snap--- So I would conclude: 1) Grafting still takes a lot of time and needs more work 2) Linux optimizes localhost networking pretty well 3) Our SSH transport is terribly slow Moving to non-localhost communication between two VM=E2=80=99s: ---snip--- guix://localhost vs. guix://remote-host transport: N Min Max Median Avg Stddev x 10 17.43 18.1 17.61 17.642 0.20131788 + 10 20.88 22.58 21.095 21.222 0.49689704 Difference at 99.0% confidence 3.58 +/- 0.487934 20.2925% +/- 2.85159% guix://remote-host vs. ssh://remote-host: N Min Max Median Avg Stddev x 10 20.88 22.58 21.095 21.222 0.49689704 + 10 30.1 32.56 31.005 31.093 0.70740606 Difference at 99.0% confidence 9.871 +/- 0.786769 46.5131% +/- 4.35326% ---snap--- Conclusion here is the same: Not alot of impact of networking/NFS and SSH transport is still terribly slow. (Confusingly faster than localhost though= =2E) > Perhaps you could =E2=80=9Cstrace -Tt=E2=80=9D the thing to check whether= this > hypothesis is correct by looking at the time we spend waiting for > replies? I=E2=80=99m not sure this will yield meaningful data for SSH, so I analyzet= it for guix://localhost vs. guix://remote-host. Takeaway is, yes, of course there = is a statistically significant difference and it=E2=80=99s about 40%=C2=B150%, w= hich means this method is pretty useless, because we can=E2=80=99t bin RPC=E2=80=99s by typ= e. So, I guess it would make sense for me to look at the SSH transport itself again and see if there are any other low-hanging fruit. Not sure how much I= can help with profiling guile/guix itself. A different/better RPC protocol is probably GSoC/v2.0-worthy? Sorry for all the lengthy emails, Lars [1] You=E2=80=99ll need this channel: https://github.com/leibniz-psychology= /guix-zpid --XF85m9dhOBO43t/C Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="manifest.scm" (specifications->manifest '( "coreutils" "findutils" "procps" "strace" "openssh" "mit-krb5" "bash" "which" "zip" "geeqie" "util-linux" "grep" "glibc" ;; for locales "glibc-locales" ;; front-end software "jupyter-zpid" "python-jupyterlab" ;; available notebook kernel ; provided by jupyter-zpid ;"python-ipykernel" "r-irkernel" "rstudio-server-zpid" "r" ;; for RMarkdown "r-knitr" "r-yaml" "r-markdown" "r-rmarkdown" "texlive" ;; commonly used r packages "r-psych" "r-ggplot2" "r-lattice" "r-foreign" "r-readr" "r-haven" "r-dplyr" "r-tidyr" "r-stringr" "r-forecast" "r-lme4" "r-nlme" "r-nnet" "r-glmnet" "r-caret" "r-xmisc" "r-splitstackshape" "r-tm" "r-quanteda" "r-topicmodels" "r-stm" ;;"r-parallel" "r-dt" "r-nlp" "r-data-table" "r-hmisc" "r-learnr" "r-metafor" ;; for rmarkdown "ghc-pandoc" )) --XF85m9dhOBO43t/C-- --Pk6IbRAofICFmK5e Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl7fUycACgkQQhN3ARo3 hEbGwgv9G/Jy0viE4XGqrmhCfmnQSKMvxgGZg7WPUBNYaOFcUr3fjpR13egOlcqS mvebo/JZ8cs+nRNQB7/v+FTYFPTmk/xFIN3jLNdHnqUmHHBmhLsirbyQRdMXYr6F sD7eho57lQ/nlnTsEU6NZQwO3WA3kd4aFjvNxSBpaVeO1HPaxyWTPEQVqpWGN7u3 nFzw4866BBBNP1R1nSdbUKsAzilHKTWMJ0B6O34dNK1/fdaVSJ/yNaAzehN+cykF SgYCzl1eUYquNUtZvyHjNptYm9BX54YQPA7HrMmpsd8qOaeTe6R4HHrV6+JPy/oK rrr2RPsVnI1bY5o58mRKzk8up75FsTlN6tWBwuempr00dIsjny/gjBHuL6J5v3vC GqmHOMhjF66PNmiwheqCGHNae8tq7Li4diFmSo9uV+kFpwgmGQ13ywZeKex2l6xV g0vpX0bGkr1gxK/XWy2dYRtgkNYf5ycAhUBBLOhTlN6/+nWI6QLEgKiAmZR16Ju5 AVtbp5Gm =zWIs -----END PGP SIGNATURE----- --Pk6IbRAofICFmK5e--