From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id aO3HIyDvlWKmbwAAbAwnHQ (envelope-from ) for ; Tue, 31 May 2022 12:34:08 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id OHLOIiDvlWJ5FgAAG6o9tA (envelope-from ) for ; Tue, 31 May 2022 12:34:08 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7DA53A4A2 for ; Tue, 31 May 2022 12:34:07 +0200 (CEST) Received: from localhost ([::1]:45098 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nvzCQ-0007BI-Lh for larch@yhetil.org; Tue, 31 May 2022 06:34:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42324) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvz6W-0002Ml-0g for guix-devel@gnu.org; Tue, 31 May 2022 06:28:00 -0400 Received: from sender4-of-o51.zoho.com ([136.143.188.51]:21108) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvz6O-0001zq-A0; Tue, 31 May 2022 06:27:55 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1653992864; cv=none; d=zohomail.com; s=zohoarc; b=OxPuilhNoTn2qqtFIHmRrr/UmXtKKcWIM+7Vx3XgewacIm7eucO7peb/FtmLz+LKsgwn8oiks57wvedlYr0XF36REj6j+X/kAjxl3c56BxSUGrmaRwo5eKau2a3Q/ZmVmv69xlEgMnWhQyiAJpRq/MvRYsMevFBZG1wgPCAk1x8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1653992864; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=i2/5R4n8T+8e2+h3VjDBsPX/sW4rkn7BRTCyDCQPOZ8=; b=Q+EeAg4b8TT1lGp/8oo/kPPaE9Oicd7Yc8O4+P0DGQVyA4U9MYBB2VVp9nhpp1mZDdVQy4M2hRQK4wKu72pC26Uaa23kNkNyRXmEREiifmb7ZqMO/gL1niTvYCRcSb4RSZMS8wxfONysnri/Gws1MIXZty1AIunYEKvcqRo9+I4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1653992864; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:In-reply-to:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To; bh=i2/5R4n8T+8e2+h3VjDBsPX/sW4rkn7BRTCyDCQPOZ8=; b=exBEP/SCgHxOJrMGMhwcMMVo1VKjfW2QWXsARJPJcGc7lFCVb2uFWpGc/2m7M9Qp TOlYaYZAX/Q0j6yeAYGRaIb2JzxnQVlBf4TNxWsQ+D43UbVB8FSUejMQJYLMsYupOGB BGLSib+bjo6OCk3vHpP+MDHRzpLDj3i8fSjNwgG4= Received: from localhost (46-108-142-46.pool.kielnet.net [46.142.108.46]) by mx.zohomail.com with SMTPS id 1653992863329129.2197726373821; Tue, 31 May 2022 03:27:43 -0700 (PDT) References: <2067ba1e606855eace261fd0b0ae9721b369bbd5.camel@telenet.be> <878ru5epyv.fsf@elephly.net> <753ba5897ed397b5e95175cd139137975245945b.camel@telenet.be> <875yozdta6.fsf@gnu.org> <752f9901f6c3c813b5553534aefc2e4e8f5aad63.camel@telenet.be> User-agent: mu4e 1.6.10; emacs 28.1 From: Ricardo Wurmus To: Maxime Devos Cc: =?utf-8?Q?G=C3=A1bor?= Boskovits , Ludovic =?utf-8?Q?Court=C3=A8s?= , Guix Devel Subject: Re: Faster "guix pull" by incremental compilation and non-circular modules? Date: Tue, 31 May 2022 12:23:42 +0200 In-reply-to: X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Message-ID: <878rqi0zfo.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.51; envelope-from=rekado@elephly.net; helo=sender4-of-o51.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1653993248; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=i2/5R4n8T+8e2+h3VjDBsPX/sW4rkn7BRTCyDCQPOZ8=; b=ppL2DAcXNudqwmYbtp9sYe2j9ICfQjkpGajyjmwmgUWo82Szg6pfwwiZqEXOhbiuOVvzVT XlDqjfFNYTZYXQ2u1J0FabAexjxIjW5L8wpv800xuSuVTI2DXU/he3f/b8TksskxhKUlo+ JFqIkR7mtDLOuFfsEbeBtQncM898woXko4dl5YO7XY3w+wOpgatQqguXWP46/L1Knkb42v 5U3PaTyoMOuznddfxnADju5/GU5/GCoLcYPCuHPVHxkZu8qPd1deJukNJCkgXpUnjWWLqY HxLp+vzHIHvsGgjTJRqCiln5+r0JAvF+dOSxtRTqOa8eZ5oo7witlWgZ19MK8g== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1653993248; a=rsa-sha256; cv=pass; b=QEfU+QLCGW0QvfWmWstQFniV/NPDdd8EuLhbJzpleANgZaVLSCyypnPitda37cGH+zQYz9 vopqAQX7qtVIJga8i3p+pFI4mXl0BaKm668OsNdnX3bVQ3m2hsynA52QWxYT6UlmL4ZnQG 9i94t29Y5wkNvir/f2rBt5mDEDxcc+YPfA6SABxlREcLKJA/lrUSx35OPFiwamt8Mhg5Nr 0NGL+4tmaKn06qN0AOl2rojgdgp/kcsOatouKeReRLspV4JhsUTULiX2KqhAk8Me9ryWgP Ufxjt2GhPdROs74YztbGyflOQyfa0q4qxsov6DZ9L9Kc89bTa3fCwm1FYKTZzw== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b="exBEP/SC"; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.53 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b="exBEP/SC"; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 7DA53A4A2 X-Spam-Score: -3.53 X-Migadu-Scanner: scn1.migadu.com X-TUID: vV9mFuDnnqsQ Maxime Devos writes: > [[PGP Signed Part:Undecided]] > G=C3=A1bor Boskovits schreef op di 31-05-2022 om 06:54 [+0200]: >> I was thinking about a bit of a different structure that can also be >> automated. My original idea was to use the already existing tree >> structure of the derivations, and split it based on depth. I think >> that gives a bit more structure, but might require splitting things >> that now are together (for example iirc sometimes we are defining >> bootstrap packages inheriting from the fully fledged ones, which >> introduces a syntactic dependency on something that is=C2=A0 higher up t= he >> tree). Wdyt? > > The package modules could be split and reorganised a bit to roughly > follow the derivation DAG, if that's what you mean (*)? Then the "guix" > package would depend on less -> compute-guix-derivation etc has less to > do, also good for lowering "guix $do_something" memory footprint.=20 This seems like a thing we=E2=80=99d have to do repeatedly as module cycles= can appear due to seemingly innocent package upgrades. I get the appeal of untangling the module graph, but I suspect it will turn out to be a lot of busywork leading to repeated disruption (because packages keep moving to different modules, breaking third party channels for no good reason) for little gain. I do hope we can reduce the amount of work that compute-guix-derivation has to perform, even if it causes some disruption once. --=20 Ricardo