From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id UM9pAwnkTWcRgwAAqHPOHw:P1 (envelope-from ) for ; Mon, 02 Dec 2024 16:44:57 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id UM9pAwnkTWcRgwAAqHPOHw (envelope-from ) for ; Mon, 02 Dec 2024 17:44:57 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=inria.fr header.s=dc header.b=C9UDexhj; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=inria.fr ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1733157896; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=lbeatsHcq2i1C+9pg2cBNBdJ3n4pB2PQTSCPBJsYKWE=; b=qAzztRBycROn5jiAAjQfbCRK0LG+J0rO8k4l8mIa2QBegTugFhuQ8/F6oCmXUKRCY8owKa pVRPvuib5+VLW97S3juiHn5rayW7r7P6Ai1z2XNkl8i73aY2Lq5oknOq3x7EagH2TZ9xte wK1TOMc/ZVHSmANvt4k0kczWMWLW5ETGlj1Z1mEdwxF4fdsE/HXoVhwSHSLNYlFEGK+fx5 OJddFzIlythjNtc8/1TV4keoxi3IHLXDbh3ol5IjafdjCVnGsF+0uLiaA99HHdFjmj34yM oZRxAWyiqXIqRsMTlyB/alLLr4/YBD4Sljqs9E5UnkSYAAjjYc6Oj87RgkJF+A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=inria.fr header.s=dc header.b=C9UDexhj; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=inria.fr ARC-Seal: i=1; s=key1; d=yhetil.org; t=1733157896; a=rsa-sha256; cv=none; b=Mr2YcWub3op/5+G0kp77nAQ4pTWJz9aPuirZLKwViWKH4/x9Up13bUjWOiTPpZVLVLyI2o KRR+CU/YgnHwMquMD7IfngnpdeKwLUh/cda3zB42/MgDibQCTS8UXqNZd0VRCAW1p58nb+ 3Cv3p+HQ+D7DOu2D8OkNdR5HseGmeFL1o72wWRKvYlJ3wh4kUYkluR0ny9mIJ7fiU3wHH6 DwJwLr2xMBXiaM/e5dv5fhQc28UhAfXlZtICA9lRVXyRXn1yt2cyARtmnKnkls993xnJ6E F9+oFrJLQzxyn3VlqSeP2R43d7iaUGOZs963cJ6bAcKnmWx8UrUu+LUMawK9Gg== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C46CB789BB for ; Mon, 02 Dec 2024 17:44:56 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tI9X8-0000jH-Mu; Mon, 02 Dec 2024 11:44:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI9X5-0000iF-R4 for guix-devel@gnu.org; Mon, 02 Dec 2024 11:44:23 -0500 Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tI9X3-0004ep-Df for guix-devel@gnu.org; Mon, 02 Dec 2024 11:44:23 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=inria.fr; s=dc; h=from:to:subject:date:message-id:mime-version; bh=lbeatsHcq2i1C+9pg2cBNBdJ3n4pB2PQTSCPBJsYKWE=; b=C9UDexhjgfQD/xyHZZU4RSHC+JyRcpgJ8cAz6SAgUxFu84w0Fr6U2Eap eSJVQOqJSFC7TlpdcFRA5hNL78TqcfLAQ5YS0hhtXhc/145uxUJ9vUAki 79kGOf903GHB06AXShsdKS78HAhr9gsgjXlbINeFomhCnsRVswD1SuGno E=; X-IronPort-AV: E=Sophos;i="6.12,202,1728943200"; d="scm'?scan'208";a="196922606" Received: from unknown (HELO ribbon) ([193.50.110.52]) by mail2-relais-roc.national.inria.fr with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Dec 2024 17:44:16 +0100 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= To: guix-devel Subject: =?utf-8?Q?=E2=80=98cargo-build-system=E2=80=99?= makes everything slow X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: Duodi 12 Frimaire an 233 de la =?utf-8?Q?R=C3=A9volu?= =?utf-8?Q?tion=2C?= jour du Raifort X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Mon, 02 Dec 2024 17:44:16 +0100 Message-ID: <87ldwy3uhr.fsf@inria.fr> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=192.134.164.83; envelope-from=ludovic.courtes@inria.fr; helo=mail2-relais-roc.national.inria.fr X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: mx11.migadu.com X-Migadu-Spam-Score: -4.81 X-Spam-Score: -4.81 X-Migadu-Queue-Id: C46CB789BB X-TUID: O9PwFy0tcQ3I --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, I was profiling the =E2=80=98dependents=E2=80=99 procedure use by the new = =E2=80=98guix build --dependents=E2=80=99 option in (guix scripts build). Like =E2=80=98guix r= efresh -l=E2=80=99, it lowers all the packages to a bag, constructs the graph, and traverses all the nodes backward. And it=E2=80=99s slow (~25s). The profile is clear (test program below): --8<---------------cut here---------------start------------->8--- 100.0% with-exception-handler at ice-9/boot-9.scm:1689:4 100.0% thunk at guix/store.scm:686:4 100.0% run-with-store at guix/store.scm:2196:0 100.0% #x7f1cb0ccbeb0 at guix/scripts/build.scm:596:6 100.0% #x7f1c9c353ef0 at guix/graph.scm:98:7 99.6% #x7f1cb1b61e00 at guix/store.scm:1925:0 99.6% loop at guix/store.scm:1925:0 99.6% add-edge at guix/graph.scm:92:7 99.6% #x7f1c9c3340bc at guix/scripts/graph.scm:187:10 96.3% #x15d3b3bc 95.8% bag-node-edges at guix/scripts/graph.scm:159:0 93.1% package->bag at guix/packages.scm:1755:0 92.5% thunk at guix/packages.scm:1763:4 89.3% lower at guix/build-system/cargo.scm:305:0 33.5% vhash-assq at ice-9/vlist.scm:539:0 9.4% #x15d3cf98 21.0% #x15d3de90 3.6% vhash-cons at ice-9/vlist.scm:449:0 0.9% #x15d3cf98 0.1% #x15d3b430 0.1% #x15d3daf4 0.1% reap-pipes at ice-9/popen.scm:183:0 2.5% filter-map at srfi/srfi-1.scm:681:0 2.0% #x7f1cb0a997f4 at guix/build-system/carg= o.scm:298:4 --8<---------------cut here---------------end--------------->8--- 89% of the time is spent lowering =E2=80=98cargo-build-system=E2=80=99 pack= ages, specifically in =E2=80=98expand-crate-sources=E2=80=99, which computes more= or less the same thing over and over again. (It also returns more than just transitive sources of Crates, AFAICS.) Problem is little can be done about it, at least not without triggering a world rebuild. But even then, it would help a lot to just not do that. Thoughts? Ludo=E2=80=99. --=-=-= Content-Type: text/plain Content-Disposition: inline; filename=prof-dependents.scm Content-Description: test program (use-modules (statprof) (ice-9 time) (guix) ((gnu packages) #:select (all-packages)) ((guix scripts build) #:select (dependents)) (gnu packages version-control)) (time (->bool (all-packages))) ;warm up (with-store store (statprof (lambda () (dependents store (list libgit2) 1)) #:full-stacks? #t #:display-style 'tree)) --=-=-=--