From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id +CNVNK6v2F6GMgAA0tVLHw (envelope-from ) for ; Thu, 04 Jun 2020 08:24:14 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id TdUoMK6v2F6BXwAA1q6Kng (envelope-from ) for ; Thu, 04 Jun 2020 08:24:14 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DF2FC9404CB for ; Thu, 4 Jun 2020 08:24:13 +0000 (UTC) Received: from localhost ([::1]:33646 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jglAY-00023J-Sp for larch@yhetil.org; Thu, 04 Jun 2020 04:24:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48492) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jglAQ-000214-BH for bug-guix@gnu.org; Thu, 04 Jun 2020 04:24:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:33511) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jglAQ-0003zu-2j for bug-guix@gnu.org; Thu, 04 Jun 2020 04:24:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jglAP-0001sl-U5 for bug-guix@gnu.org; Thu, 04 Jun 2020 04:24:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#41702: `guix environment` performance issues Resent-From: Lars-Dominik Braun Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 04 Jun 2020 08:24:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 41702 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 41702@debbugs.gnu.org X-Debbugs-Original-To: bug-guix@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.15912590057177 (code B ref -1); Thu, 04 Jun 2020 08:24:01 +0000 Received: (at submit) by debbugs.gnu.org; 4 Jun 2020 08:23:25 +0000 Received: from localhost ([127.0.0.1]:45057 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jgl9o-0001rh-TY for submit@debbugs.gnu.org; Thu, 04 Jun 2020 04:23:25 -0400 Received: from lists.gnu.org ([209.51.188.17]:35836) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jgl9m-0001rY-TE for submit@debbugs.gnu.org; Thu, 04 Jun 2020 04:23:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48448) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgl9m-0001xr-N6 for bug-guix@gnu.org; Thu, 04 Jun 2020 04:23:22 -0400 Received: from mail-ed1-x532.google.com ([2a00:1450:4864:20::532]:41745) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgl9k-0003uP-Mo for bug-guix@gnu.org; Thu, 04 Jun 2020 04:23:22 -0400 Received: by mail-ed1-x532.google.com with SMTP id ce8so3952525edb.8 for ; Thu, 04 Jun 2020 01:23:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leibniz-psychology-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:subject:message-id:mime-version:content-disposition :user-agent; bh=GUsCJnysXURzM+btb8b0onoT10tehoeNWU2ITasfoqo=; b=F0ioEYnIrxb2P6DUvkTx5mjIHZmeaSfhuZZpxVV+WXLHmcg7zdsl35OvXazTvREdDQ h8qi9pnGsC77d/A1Ftm5bcCYJ7Yl5RoS/fZOmjAD6rPwpJndVZBEFeL5OLNu3NWNGyoj z+uvUNiK/KBI7c35dM/L5FTizzo9mGq7Qk+O/akSmovt6fTblQN8chVceJumA6B2hdxF WgWF6zZmT5EAVxJ29H+C1fYSES4dNLcOqxypFFfvh5HR2vvvLQSVdDtn3uyo2N385lb/ UZ/aK1u2yc0l+Wi0yt2+/46Zup4uOGFPvmIcL/0NVQ9zoXcuNIzFD42z12D+mQa1du2o ulIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-disposition:user-agent; bh=GUsCJnysXURzM+btb8b0onoT10tehoeNWU2ITasfoqo=; b=GXXbcDTL/5s43ExdeNp8ZxO1UxnhrZdI0NfNU3JFuAy0vagLfCQ5YFIl6PORxkUMjb ZQiiB34tRld2rB9BkSa7E14C4U8iHMg37x0VYbxQbCtS22muUi0OnWewzPiqMIq8HhUP OPl/FQ8UqemTo+pM52rHUJq2gAQK+5cf0ygdMNu79tXbBiUvZ9/rAIInwZ5EFMKtCJxV 3Bvbn6U65p5HWLQyf2aZDl0kzjjXKhJWx7am//ikth8eIlEOmZuvcCutxwh9H0vaDEFK jYPAJukqVPOyZ7s37FglCc1kW84KvQYXDkvXXhHVlUdSR+KEeCOjD97VWVPNtCCvkm8x hFdg== X-Gm-Message-State: AOAM530c6T6z+gu6dTEOUpRwEeeO1cLqZW2J98GFOM3YbAqd1Ez5/Gme x0p21YRfL2V47kC476u8pU1aAETxfkbDSx8BTRF4HVNQW8PE9yeMxoLOzIMKVTWdcr8nhlEwLMQ X9Jp/ZD8k5acUtiec84pL+d/kiQSTMHegKDvunPNILi0roG3YkPPFH372feW8rsyfcoWC2LRQ/I oCKw== X-Google-Smtp-Source: ABdhPJx985fJ7VBpf5/94I81LK81huj7+wGLOedoy4RDvfx8qH+XfFSfQdRQqlJ9NK8tgL7SSTwQBw== X-Received: by 2002:a05:6402:3ca:: with SMTP id t10mr3339066edw.128.1591258998105; Thu, 04 Jun 2020 01:23:18 -0700 (PDT) Received: from localhost (dynamic-2a01-0c23-8458-f500-08f2-7cdd-6690-abd3.c23.pool.telefonica.de. [2a01:c23:8458:f500:8f2:7cdd:6690:abd3]) by smtp.gmail.com with ESMTPSA id v12sm1687474eda.39.2020.06.04.01.23.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Jun 2020 01:23:17 -0700 (PDT) Date: Thu, 4 Jun 2020 10:23:16 +0200 From: Lars-Dominik Braun Message-ID: <20200604082316.GA3146@zpidnp36> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="UugvWAfsgieZRqgk" Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) Received-SPF: none client-ip=2a00:1450:4864:20::532; envelope-from=ldb@leibniz-psychology.org; helo=mail-ed1-x532.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=leibniz-psychology-org.20150623.gappssmtp.com header.s=20150623 header.b=F0ioEYnI; dmarc=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: -1.61 X-TUID: CbQsZRE1tnAk --UugvWAfsgieZRqgk Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, I=E2=80=99ve noticed that `guix environment` can be very very slow for some= packages. Whereas usually a call like time guix environment --ad-hoc -- true takes 600ms to 1.5s, it takes 8.4s for the package r-learnr on my Ryzen 5 3= 600 with NVMe SSD, 32G of RAM and a warm cache. This seems to accumulate when trying to install multiple unrelated =E2=80=9Cslow=E2=80=9D packages into t= he same environment, i.e. jupyter (2.9s) + r-rlearnr (8.4s) + r-stm (2.9s) =3D 12.1s. What these packages seem to have in common is a deep chain of propagated inputs. And in fact, if I create artificial test packages where test-999 propagates test-998, test-998=E2=86=92test-997 and so on the command above takes 1m 9s= =2E A single package propagating 999 packages takes 4.5s. Instrumenting guix-environment=E2=80=99s with-store call with statprof I se= e a lot of time is spent in the garbage collector: Total time: 24.672604202 seconds (19.431122691 seconds in GC) =2E/pre-inst-env guix environment --ad-hoc r-learnr -- true 25,18s user 0,= 24s system 308% cpu 8,248 total More specifically in an anonymous function and reap-pipes, which is a gc ho= ok, I believe: % cumulative self time seconds seconds calls procedure 33.41 14.49 8.24 anon #xbb8480 27.95 6.90 6.90 ice-9/popen.scm:145:0= :reap-pipes 4.37 1.08 1.08 anon #xbbdcd8 3.28 0.86 0.81 ice-9/vlist.scm:539:0= :vhash-assq 2.40 2.37 0.59 guix/grafts.scm:202:22 Not sure if that helps tracking the issue down though. Cheers, Lars PS: Here=E2=80=99s `guix describe`: Generation 1 Jun 03 2020 10:38:17 (current) guix 9ab2790 repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 9ab2790e2acf872c0f2cf41ec180de80d1eee3f3 zpid bab33c1 repository URL: https://github.com/leibniz-psychology/guix-zpid.git branch: master commit: bab33c19e0f0304d7aa82751653eb85d8cad8c0a --UugvWAfsgieZRqgk Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGzBAABCgAdFiEEyk+M9DfXR4/aBV/UQhN3ARo3hEYFAl7Yr3EACgkQQhN3ARo3 hEZvXgwAortYSBFszMEb4QyWF4EtfFQ8GMqhj9Av2r72HXGKtqnu+ZoY4VLK3eZx Ntt0HIn50+BLQ/0TkWRCy5znN5t4NPVg7Qdn46DlCM9rOL6kkr61xrVGmmi5yzva rir2cUIgLFHEwHQPP3AoKZAVrkzIoKxHhHPzFGyLZPglf0MER82X30pj5Yq14h3r WlA+u6ktDsnHGYc9dD25mTSvuhITT8NAp4viNBC328FEZ+b/ct2/lpeln/khZYmU HSG+bYI9R+fwg6isCPtKuZB+B7T5703KOyzvwbiasnCToMr1/PCV91z3Arba8MsC eYZUmv8FfsrENz7iFs7le9J3csH39bhrZn7DNADwSt3HTILGHrrbEikgD3R3Lyvq RcYRA9NZLZHSRdYntPbktFvK4i4H7UIBHx/b2MmiGAgrhqYvAPPFgTU8D0Y/w4Rd rq6uXckgl3WlGnbmsVO7CEefanSv3RwWg7+m2LJ4sziXpT5fmyp+qru/l58yCUsZ MiyfjkrB =lFqp -----END PGP SIGNATURE----- --UugvWAfsgieZRqgk--