From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms1.migadu.com with LMTPS id OOz8EA22QmZSkAAAqHPOHw:P1 (envelope-from ) for ; Tue, 14 May 2024 02:53:33 +0200 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id OOz8EA22QmZSkAAAqHPOHw (envelope-from ) for ; Tue, 14 May 2024 02:53:33 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none ("invalid DKIM record") header.d=freakingpenguin.com header.s=x header.b=o5eboK42; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1715648013; a=rsa-sha256; cv=none; b=lLxaqWzRCBrLjvlHAXavLrtZT7raZsJFSDIiyxOyV4NJNwdmwEg+jHZSswLlZIWRW+/5oF /s+/8NPCCC678tdCahwWFB1npS2NyTOkf1pTH71NyB7n4iI/PLe74QSCt7vLP/NK99jZPH Bl7dZEwqeYwC/O/KvXLWyJMAKtlFOPZ8lRTP8JqmC5FNwNj5xcP8ZXWqiHNYN9pT61+CKk SwS/D4Tl1GE/IdG9HOZd4kHdWnd6AaF6dG74U2BIcDTakihrKK+djIRwTZecKmI3KdZb5P aXM7Oq5Pml+JHD2Lo8sdXOkPopygcy2AtQk1u021rm4dovNfLMamUBHBsXjL7A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none ("invalid DKIM record") header.d=freakingpenguin.com header.s=x header.b=o5eboK42; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1715648013; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=gr6f39gx4huytPR8Lt0nHex7WRB3Meqzca4PBRdNAd4=; b=QCKLCUV8/ikEdrL3Ku1PfeSorl1b4APbRjOogSzBq7Y/u5E/1qckBsaqgrNAvHHCz7adhW Tcpr6OiupiHHlCFUQq2cdGAM34QtMNkJ9b9G6socoNvsOranUqxeG2zkmGMAVDa9/dMNIH xSBdGwegQxxM1+B9GHMti51NmY6lnCHaMq9CYY2xyNrPYRh4+2o7dwdBjPQJ23dgLBUale m80s9t4/YkjWgekmOhSu8OxedytrCDT/pFAtvA56+KiznAL8e/eYV1nRHFpzOmmgae5Lsm mh/KEBJgpfcsTGmWWhmSOiBsy1HAP5Mz/JSGq3LYmPhYezwoscVwY4XuuQAdIw== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E2B58314CD for ; Tue, 14 May 2024 02:53:32 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s6gPQ-0007Qq-Re; Mon, 13 May 2024 20:52:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s6gPO-0007Qa-I5 for guix-devel@gnu.org; Mon, 13 May 2024 20:52:46 -0400 Received: from mail-108-mta32.mxroute.com ([136.175.108.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s6gPJ-000836-21 for guix-devel@gnu.org; Mon, 13 May 2024 20:52:46 -0400 Received: from filter006.mxroute.com ([136.175.111.2] filter006.mxroute.com) (Authenticated sender: mN4UYu2MZsgR) by mail-108-mta32.mxroute.com (ZoneMTA) with ESMTPSA id 18f749636ee0008ca2.002 for (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384); Tue, 14 May 2024 00:52:32 +0000 X-Zone-Loop: 320f34ea550405d783a2cb64d06e26863d965ffab043 X-Originating-IP: [136.175.111.2] DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=freakingpenguin.com; s=x; h=Content-Transfer-Encoding:Content-Type: MIME-Version:Message-ID:Date:References:In-Reply-To:Subject:Cc:To:From:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=gr6f39gx4huytPR8Lt0nHex7WRB3Meqzca4PBRdNAd4=; b=o5eboK42oVhxdUZYsUpD43Tp0b 2ulfWR+bxRBsdl5sVKa4dhgWtoehfcf4orgEN885dKr9ewXh9g4I6Sv1W71vRXucco+89ygKKod31 DzkhiMjy3mw7ypOLzrwb18LDWvV7kggBjTaHWnv7HQ48/4o/nzTqdAOmgihqO06mwf9sq6tbDAX9/ xjk5jAdz8fkPDwmp/BjNWOBC3lPZd2GF/+zev2lZhmnvGaNXJP9XBDV6WPf3Pg41t//59uydHhtpZ 0Di57WzGYsp9sJ6E351zMjDq/z8VtpSV0H57m9OUTUD0xBGPyKjMDu5tKfwnMuwC5UMK/SdTOqxmR lhqPRvwg==; From: Richard Sent To: Simon Tournier Cc: Edouard Klein , guix-devel@gnu.org Subject: Re: Guix pull: avoiding "Computing Guix derivation" In-Reply-To: <875xvh1e9y.fsf@gmail.com> (Simon Tournier's message of "Tue, 14 May 2024 01:30:01 +0200") References: <874jb16993.fsf@rdklein.fr> <87ikzh340q.fsf@gmail.com> <87y18d1koi.fsf@freakingpenguin.com> <875xvh1e9y.fsf@gmail.com> Date: Mon, 13 May 2024 20:52:23 -0400 Message-ID: <87jzjx1ago.fsf@freakingpenguin.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Authenticated-Id: richard@freakingpenguin.com Received-SPF: pass client-ip=136.175.108.32; envelope-from=richard@freakingpenguin.com; helo=mail-108-mta32.mxroute.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, SPF_PASS=-0.001, T_SPF_HELO_TEMPERROR=0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Queue-Id: E2B58314CD X-Migadu-Scanner: mx12.migadu.com X-Migadu-Spam-Score: -6.37 X-Spam-Score: -6.37 X-TUID: STwrIQQ9Z/gC Simon Tournier writes: > Hi, > > On lun., 13 mai 2024 at 17:11, Richard Sent = wrote: > >> Instead of A and B building C directly, A and B download the >> substitutable Guix package D, then use D to build C. Because D is a >> reproducible package, it should be substitutable from both A and B. >> Then, because D->C is the same for everyone, we could substitute the >> Guix derivation computation for C. > > Maybe I am missing some details. From my understanding, your D is the > result of the =E2=80=9CComputing derivation=E2=80=9D dance. And it is a = minimalist > build because it must work for both cases: with or without substitutes. > > Somehow, my understanding of the current process is: > > A -> D -> C > B -> D* -> C > > And, D or D* can be the same script. Here, the property is a kind of > idempotency. Hence, C is substitutable. > > IIUC, your proposal is to fix D (the intermediary step). You propose > the package named =E2=80=99guix=E2=80=99 that changes barely, but it coul= d be something > more minimalist. The requirements is: susbtitutable. The problem is > transferred to the first arrow, no? > > How can we be sure that A and B points to the same D? > > Other said, A lives in the past. Later, we changed D becoming D* > because some bugs. The other revision B lives after this change. > Therefore, we have the same picture: > > A -> D > B -> D* > > But then how do we go to C since D and D* does not have a kind of > idempotent property. > > From my understanding, the current situation/process is very similar > with the one for compiling compiler: you have your new Source of > compiler and an OldCompiler binary, so you run: > > OldCompiler(Source) -> Compiler > and again Compiler(Source) -> NewCompiler > > Somehow, whatever the choice of some OldCompiler binary, you get the > same NewCompiler binary (aside trusting trust attack and friends, > obviously ;-)) You're correct. This solution wouldn't be sufficient to avoid "Computing Guix Derivation" for every possible A or B. To my understanding it could reduce the frequency this occurs. (To change letters yet again, I'll use Z to refer to the output Guix.) As a package (either "guix" or a minimalist subset), D, D*, D**, etc. should all be substitutable. If we consider input Guix's A, B, C, E, F, ..., our original "Compute Guix Derivation" graph would look like: --8<---------------cut here---------------start------------->8--- Z / \ /| |\ /| | | |\ / | | | | \ A B C E F G --8<---------------cut here---------------end--------------->8--- i.e. for every possible input Guix (and output Z), we'd need to store a substitutable intermediate step. Not feasible. By having A, B, C, ..., use a substitutable Guix package D, our graph would look like this instead: --8<---------------cut here---------------start------------->8--- Z /| \ / | \ D D* D** /|\ |\ | A B C E F G --8<---------------cut here---------------end--------------->8--- Assuming D changes significantly less frequently than A, B, C..., I would think this should be something we could feasibly substitute (at least for recent D and Z). Perhaps this isn't viable. I'll confess to not being super familiar with how build-self.scm operates. At a naive level, $ guix build guix doesn't need to run "Computing Guix derivation", so if A->D doesn't need to compute the derivation, and D->Z is substitutable for some subset of D and Z, I'd think we can skip "Computing Guix derivation" as long as someone isn't trying to either a) Use an A that refers to a D********** that's since been GCd b) Try to get a Z that hasn't been built by D, D*, D**, etc. Does that sound correct? --=20 Take it easy, Richard Sent Making my computer weirder one commit at a time.