From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:403:4ea1::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id OAEPA8xWymHlnAAAgWs5BA (envelope-from ) for ; Tue, 28 Dec 2021 01:14:04 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id KKeOOMtWymHpZQAAG6o9tA (envelope-from ) for ; Tue, 28 Dec 2021 01:14:03 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 980C853BE for ; Tue, 28 Dec 2021 01:14:03 +0100 (CET) Received: from localhost ([::1]:47146 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n207u-0001oD-Qp for larch@yhetil.org; Mon, 27 Dec 2021 19:14:02 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43172) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n207b-0001o5-Uo for help-guix@gnu.org; Mon, 27 Dec 2021 19:13:44 -0500 Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:36201) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n207Z-0006EL-L9 for help-guix@gnu.org; Mon, 27 Dec 2021 19:13:43 -0500 Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 869343200D18; Mon, 27 Dec 2021 19:13:39 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Mon, 27 Dec 2021 19:13:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=famulari.name; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=mesmtp; bh=i2D0ZplQGoAxf2JvJDKzdpLk GUUB/RfbsOPCdUFT20k=; b=SJNzpsBtO5Jyg0haXJ3nOqOe7wkw6mgMGxaO7xd2 J0oIRoqo1UnuZkRvbPivaut68phzx6Fub6wCw2iTA/Ifo+W3XWr7WIefSxyN9ndD e8b0p1jOIS89bdyD5ogGkWe1jPbzaZKsVuNX7rp+uFBegfIXvm3EvRQV1M9DN8Sx 1PY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=i2D0Zp lQGoAxf2JvJDKzdpLkGUUB/RfbsOPCdUFT20k=; b=d9Jf3yIg4iePOLjSwtAc8c ipdosvPkQwixCCvW9uKut9ZUyHhGeHyR23TzCBzFxEHIZyTC98vqkYkR5iORIakW MtOLiu7NyMvUjQVdT3UI3JlfSWypx+/PNkyl5bjJJlY4hY+ofXSAq6eHszQDB6GA 9VZBxxMQP8TM2/xFsUr6m6xlCa5T4bEtZ78gzAcOJHa6SgnKi/9l3ycU544h39ax ghcbWzt53WIFXz5ABLmc8ECs0vMyuAq3XiGzqiKbBjs4AOYZoVeMpbis6Sr0xJsm Ad7O3aI0Ij2wgKwVipx5l6V06hZOU4FW/Ft408kvv2Aq3xnyDGeei/XgxwSVdQZA == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvuddruddukedgtddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvffukfhfgggtuggjsehttdortddttddvnecuhfhrohhmpefnvghoucfh rghmuhhlrghrihcuoehlvghosehfrghmuhhlrghrihdrnhgrmhgvqeenucggtffrrghtth gvrhhnpeetgffgueeuudejleelvdejudejledtvdfgleffgfefueetieevtdekleehuedu ueenucffohhmrghinhepghhnuhdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurf grrhgrmhepmhgrihhlfhhrohhmpehlvghosehfrghmuhhlrghrihdrnhgrmhgv X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 27 Dec 2021 19:13:38 -0500 (EST) Date: Mon, 27 Dec 2021 19:13:36 -0500 From: Leo Famulari To: Matthew Brooks Subject: Re: Avoiding rebuilds (e.g. of mariadb, the entire Rust chain, etc.)? Message-ID: References: <20211226234130.5ad3d9f2@mailbox.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20211226234130.5ad3d9f2@mailbox.org> Received-SPF: pass client-ip=64.147.123.25; envelope-from=leo@famulari.name; helo=wout2-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "help-guix@gnu.org" Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1640650443; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=i2D0ZplQGoAxf2JvJDKzdpLkGUUB/RfbsOPCdUFT20k=; b=CzgeBE2fDn/aGFCKJpIGiLyB5my3f5+PwZwnuxsysFqasZ2AyDK4EXS6rqTZIrOAblPutX Jg71va3WPO4iTOw9uiHX93FYOPMwo/4ajvsk2X3YTNIepPmgBh5Y9zCtIleuPTabkElzQ7 fk4vKHjCAbaGVZy87z1C/uWGmO1Es7POrfR41qkJNDm3DOCNXcJX2bcHrXEnfkTlCOdErw zeHIUuRAoejQIOmCdjFovABLq8e9LPz4LuYg0fSG5S9mCPZQDMJevFm0pCEhb1k7N6GAci fd6vuFtRWeDLMYseK686vxdT6bU5YXZlL+6Thg5n6PTtwZzmcBamAVAzgRCiug== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1640650443; a=rsa-sha256; cv=none; b=NQPgjQIxGPeFpbmfwXAErD9+ZJ8bO7CURG0KAsnhvAR6+a1gDaVPQTODoqBdDAiGECHsjT 6GA3fzBMpkC6002CnAk5Iz46M0D6f28D1BU1QxBq7m+3/YeDCMlhuTqIFgcbIy9/SNmDtq +puen4MeeQAo9vJXwzk1lWBJfh9fQCFqeXS2JWjcCFZja16DQDNRFR/PwMpt1lNnTq0rIp no+6rZ97P7kWCgEu/X/o1r5/xosIZ1GJyB7fp0DYSq587FdMLre5VZngFlKQ2t/LujsT1Y mqZK3sppx5L01lxF72lVaBs84raxQFxT1ITEJemRnhPhM2aICwnnNZNcgPcwEw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=famulari.name header.s=mesmtp header.b=SJNzpsBt; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=d9Jf3yIg; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.87 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=famulari.name header.s=mesmtp header.b=SJNzpsBt; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=d9Jf3yIg; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "help-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="help-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 980C853BE X-Spam-Score: -3.87 X-Migadu-Scanner: scn1.migadu.com X-TUID: HslmNJ4iv/YE On Sun, Dec 26, 2021 at 11:41:30PM -0600, Matthew Brooks wrote: > Is there any way to avoid rebuilding stuff like mariadb, the entire Rust chain, etc. unless one of those packages *actually* changes? It seems like every few days every single package needs to rebuild for some reason, including many packages that spend unbelievably long times running tests that will never actually be of use to me, so I'm usually only able to update every couple of weeks since so much constantly needs to be rebuilt and everything takes so long. In Guix, packages are considered to have changed when any part of their dependency graph changes. So, no, there is not a way to avoid rebuilding a package when one of its transitive dependencies has changed. We have guidelines about how to make changes to the distro that are based on how many rebuilds they cause. See item 8: https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html#Submitting-Patches Both MariaDB and Rust have several thousand packages that depend on them, so we change these packages seldomly, and use the build farm to rebuild everything in advance so that users don't have to. If users choose to avoid substitutes, that's fine too, but there's no way to avoid doing the work. Recently, we deployed the results of a "core-updates" project, which updated core packages, and caused every single package to be rebuilt. So, that's why you are having to rebuild more than usual. > It seems to have gotten worse over time as well, as more and more base packages pull in extremely computation-hungry dependency chains. At this very moment, for example, I'm waiting for Rust & co. to compile simply to delete old system generations from the bootloader. Yes, Rust continues to grow in importance and has become a relatively low-level dependency of the distro on the x86_64 platform. We are continually finding ways to shorten the "bootstrap chain" of Rust, but it's still fairly long. > I've got substitutes turned off (primarily because I like the idea of every package being "home built" as it were), but I can't imagine that even the official substitiute servers would be able to keep up with the constant rebuilds my system seems to want unless I've got something horribly misconfigured somewhere. The build farm has ~2800 powerful CPU cores for the x86_64 architecture, so it can keep up. Other architectures are less well-resourced. And of course very few users have access to those kinds of resources. > (extra-options '("--gc-keep-derivations=yes" "--gc-keep-outputs=yes" "--no-substitutes")) I would have suggested adding those first two options. Beyond that, maybe others can suggest something that will help.