From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:1008:1e59::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id cK73LlqinmWSBwEAkFu2QA (envelope-from ) for ; Wed, 10 Jan 2024 14:57:46 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id QAJ3KFqinmV0ugAA62LTzQ (envelope-from ) for ; Wed, 10 Jan 2024 14:57:46 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1704895066; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post; bh=B9X7IHQnZZqBlQbf46EFkgo5NIizOP7oD489kzysM0M=; b=KZJMdPKJZ+WL83w9Byee4mg58uIq9mkO/iSSAi6OIn38kTK4jUsySQeU2UD2NQHiWhL4X4 KLa9JCsjRSrstjFs905Xm8Jmt6kdM+uVoDwCgk21IKnBou16FzXMCvkhrmnfMbc1zfZYyT gB9Z1wka/NQMuFngwQ90vvtudVdqZi0+hPr59U6YHWUVa5dNs1nUOSso5PR17FClncCfCF Wzhs4eN0fBEJIykVxJTrOPWdXXClxhBv6hKLU7BEekzzlGrE+3rRPr1349ZLCzBfiN2Oqn m9MqVOGieHFW8IZvy/wKsMyH36f+yiithuUTuXzSsNsKxKd9Jd47MdgEcsGauQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1704895066; a=rsa-sha256; cv=none; b=ok8JnEBPCL4PeGvKzW7ArZrKL6wE6/t2sG2zJmBUl9DPn9swkO4ERBxdz3bmjqgMpbUJ2b lo8TbdoFk/pITDTD2bGe8jVo2+7N6wi1hKoacn+lKBrr4daLLuwlCsgocY/X8dVAb4gDtc +KGOtTJ0aib911zgAdlyzWN9Ez2ZDqiagp/oOcv7Zf70Vo7Em7SBpU9YOoiQMPbHx2Ht1P 9VdTfuP78E4/LmUnyViU7qNs60NKKODeG0PnEJq2qxB/bYoWjKTP11z4+djtqmZp1NXhEK WFTznj7ixB75rLO0JylfR3fSG1wBGwn+dFC/3el75yWNRchUeYUuH3OkIyW/xg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 51ED046067 for ; Wed, 10 Jan 2024 14:57:45 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rNZ5J-00046l-4D; Wed, 10 Jan 2024 08:57:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rNZ4y-00042g-Eb for guix-patches@gnu.org; Wed, 10 Jan 2024 08:57:12 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rNZ4q-00007N-5B for guix-patches@gnu.org; Wed, 10 Jan 2024 08:57:08 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rNZ4o-0003Og-OP for guix-patches@gnu.org; Wed, 10 Jan 2024 08:57:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#68266] [PATCH 1/7] gnu: Memozise make-ld-wrapper results. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 10 Jan 2024 13:57:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 68266 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 68266@debbugs.gnu.org Received: via spool by 68266-submit@debbugs.gnu.org id=B68266.170489496212923 (code B ref 68266); Wed, 10 Jan 2024 13:57:02 +0000 Received: (at 68266) by debbugs.gnu.org; 10 Jan 2024 13:56:02 +0000 Received: from localhost ([127.0.0.1]:39296 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rNZ3q-0003ML-65 for submit@debbugs.gnu.org; Wed, 10 Jan 2024 08:56:02 -0500 Received: from mira.cbaines.net ([212.71.252.8]:43094) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rNZ3n-0003M3-W9 for 68266@debbugs.gnu.org; Wed, 10 Jan 2024 08:56:00 -0500 Received: from localhost (unknown [217.155.61.229]) by mira.cbaines.net (Postfix) with ESMTPSA id 26BCD27BBE2; Wed, 10 Jan 2024 12:38:31 +0000 (GMT) Received: from felis (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 47b23c40; Wed, 10 Jan 2024 12:38:30 +0000 (UTC) References: <87plyfrb2x.fsf@cbaines.net> <13f83a0db585e81572240e35dfef473aa73fe996.1704472849.git.mail@cbaines.net> <87v8833fl7.fsf@gnu.org> <878r4zr6a0.fsf@cbaines.net> <87plyaw1bn.fsf@gnu.org> User-agent: mu4e 1.10.7; emacs 29.1 From: Christopher Baines Date: Wed, 10 Jan 2024 12:28:31 +0000 In-reply-to: <87plyaw1bn.fsf@gnu.org> Message-ID: <87cyu9gy7w.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -8.39 X-Spam-Score: -8.39 X-Migadu-Queue-Id: 51ED046067 X-Migadu-Scanner: mx11.migadu.com X-TUID: A0r3SNiXamLd --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Christopher Baines skribis: > >> I believe the reason packages from make-ld-wrapper were showing up >> multiple times in the cache for me is linked to it's use in the >> cross-base module, as part of the cross-gcc procedure. >> >> A later commit does change cross-gcc to return a single package record >> for some given arguments, so that probably resolves the biggest misuse >> of make-ld-wrapper. > > Oh, I see. > > Before resorting to memoization (which doesn=E2=80=99t come for free), we= can > for instance make sure we do not create new package records from inputs > fields (since they are thunked, a fresh record would be created each > time the field is accessed). Here=E2=80=99s an example: > > diff --git a/gnu/packages/cross-base.scm b/gnu/packages/cross-base.scm > index 6ee7b315d8..ca3b41a4c7 100644 > --- a/gnu/packages/cross-base.scm > +++ b/gnu/packages/cross-base.scm > @@ -306,6 +306,11 @@ (define* (cross-gcc target > XGCC as the base compiler. Use XBINUTILS as the associated cross-Binuti= ls. > If LIBC is false, then build a GCC that does not target a libc; otherwis= e, > target that libc." > + (define ld-wrapper > + (make-ld-wrapper (string-append "ld-wrapper-" target) > + #:target (const target) > + #:binutils xbinutils)) > + > (package > (inherit xgcc) > (name (string-append "gcc-cross-" > @@ -313,8 +318,7 @@ (define* (cross-gcc target > target)) > (source > (origin > - (inherit > - (package-source xgcc)) > + (inherit (package-source xgcc)) > (patches > (append > (origin-patches (package-source xgcc)) > @@ -353,10 +357,7 @@ (define* (cross-gcc target > ,@(cross-gcc-arguments target xgcc libc))) >=20=20 > (native-inputs > - `(("ld-wrapper-cross" ,(make-ld-wrapper > - (string-append "ld-wrapper-" target) > - #:target (const target) > - #:binutils xbinutils)) > + `(("ld-wrapper-cross" ,ld-wrapper) > ("binutils-cross" ,xbinutils) >=20=20 > ,@(let ((inputs (append (package-inputs xgcc) > > > This particular one doesn=E2=80=99t have a huge impact on simple cases bu= t it > might pay off for bigger graphs: > > $ GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build hello --target= =3Daarch64-linux-gnu --no-grafts -d > /gnu/store/i8x1gg73x3fvxn4lrrgbcb97sz4qq1yx-hello-2.12.1.drv > Object Cache: > fresh caches: 20 > lookups: 5514 > hits: 5008 (90.8%) > cache size: 505 entries > $ # before: > $ GUIX_PROFILING=3Dobject-cache ./pre-inst-env guix build hello --target= =3Daarch64-linux-gnu --no-grafts -d > /gnu/store/kdnh9bwg874aq8bvvw9y2vkl7z94icqa-hello-2.12.1.drv > Object Cache: > fresh caches: 20 > lookups: 5406 > hits: 4907 (90.8%) > cache size: 498 entries > >> I think there's other cases (in the llvm and mold modules) where it >> looks like it's called multiple times with the same arguments, so maybe >> that's an argument for having memoization around make-ld-wrapper even >> though it's not needed for all uses. > > On guix-devel, Efraim mentioned cargo-build-system packages. Is that > related? Looking at the various changes, while rust-sysroot doesn't appear that many times in the cache, I think it's responsible for most of the slowness. Maybe that's due in part to what you say above, using the cross- procedures in the native-inputs field. Maybe the transformation you suggest above could be applied, but I'd be worried about how that would work if you set one %current-system/%current-target-system when you call the procedure, then another when you compute the derivation. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmWej8NfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9Xc/1g/7B3RzVXQfEw1WTkDK9Ia3PJfA8EhN4cRr HVwtVs2SFCjffaM5HTa3LyGzoZnrFV9yItJ1m+2SBZOPdAi26euXnBTEnA72RELg MB2KK6bpdTZ42iV7ZxWT/RGDrRXhSd0YvzbtpRtMbfdk8wFXBpN7886ubFiOQemX WDlL7loxI7Ncfqv3XJim+DMkBeeCS3dLiVHGRnBBuWMl8X93atOY4y6/kys/rebc /yO2cGwwOjZrjVPddFjs0FsAQFov00ESz9f387n5YPU2aEsRSqqYAcyu16oMg+4B gx029IIIbtFiwCyja5qbpfGuXTIKXNsvxZR9dvV4E2BAllJvHbRaarI+ENzvAdD4 0fddw8KVV5+D2MPUzC+O7INcF+b4yO3IR52Bo9OjyvCCgkWDCc0QBVYXkgCYr3ZN dOnQqxNoA/oVt3SzTIi1QjL0i57aLjrEXdbDs0goWmxoe7pOEAkUVFfcqCqjfGHg taawfZhFI+PgosLgnye+Tfe/tRqpfSI+ODPjS3CLfT4mpekOlbzHf91AF6S2oo/L dDbtLRbNmeobKk+2tIUS99O8GPYoRyQKhTx1EE0sz2meEuohaHCmdWi5pqbcYMlR aeASq/syYUso3XJdpEK+clYPcpts7mDYMxYHMeQs2kfHFOyI2eB6VMBmfz+wFMDq PAShmn5JWaU= =7avI -----END PGP SIGNATURE----- --=-=-=--