From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms1.migadu.com with LMTPS id CDeyLEKCQmas9AAAqHPOHw:P1 (envelope-from ) for ; Mon, 13 May 2024 23:12:34 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id CDeyLEKCQmas9AAAqHPOHw (envelope-from ) for ; Mon, 13 May 2024 23:12:34 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none ("invalid DKIM record") header.d=freakingpenguin.com header.s=x header.b=JdPRgQpL; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1715634754; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=lvTGxKLzltDchk0V0pGC0DzjS74K5UV7NGbSITT81JU=; b=kOgRrIExNG8GPe1v5O8xn/dpM0ntexNXcxOuCX6RbYw2Zoelb5wB4tigaJ4/lNTxn+YCgv 8yKuNE2ZsEd+A6P3+e6w/fV/f4MBZQV4RGUxUMY31RMAAYthQoWGg9SjWgPMihBgcMwrQb EeTIHS/D58/Cl87ozd9QYswUDrIhkH68qD0q0vzgZjLyPKVm2cibx5s3yFdOIjNxqUUa/q altts0/CtZoAtGKpeKZ45iBA2G1wiUsmLKHpImyyCB5siFkxAxNBL6ddDmr5CUC9SJeJDL WQWcGUn4ocM23u/SSLcP1uIlb9EryqJnZfc9pnWpcxsau/yalB1xmHw9jtq0qw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none ("invalid DKIM record") header.d=freakingpenguin.com header.s=x header.b=JdPRgQpL; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Seal: i=1; s=key1; d=yhetil.org; t=1715634754; a=rsa-sha256; cv=none; b=c0kDoB3/4sQLfhPm+hAnuaBrmug0uZ3xVdeC/EUg8OH+99R/qdorNEmsHkPDM2VsCgAMXm 3bv+KbtuVJvg+3Ga9dLjF+gjUA9yCh8/6q/1wyn3hrdATCceEGxxrz5yzlLUvjeaKcS8hA 5eKLDhhCL1ikXTNzeyZINwmW0Z38yxG7VQcoaXD13vMODEc/kgTnGyO8o2Rxt+UNSkyMCZ mjdHYDIWvtUU/sabXTjaug70W2XffKEFjvmSYXH53yoWhjQ6LCnuL07Ws7APDwvZ/Kja2G 3l2jOumZNt5uEPhWxuGORk1MIIxyGy4P8q8JlsFo7oUeFq8vRYRODDIMvbBVdQ== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 2BE4471986 for ; Mon, 13 May 2024 23:12:34 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s6cxf-0004pf-LW; Mon, 13 May 2024 17:11:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s6cxe-0004pH-D4 for guix-devel@gnu.org; Mon, 13 May 2024 17:11:54 -0400 Received: from mail-108-mta0.mxroute.com ([136.175.108.0]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s6cxc-0004cK-86 for guix-devel@gnu.org; Mon, 13 May 2024 17:11:54 -0400 Received: from filter006.mxroute.com ([136.175.111.2] filter006.mxroute.com) (Authenticated sender: mN4UYu2MZsgR) by mail-108-mta0.mxroute.com (ZoneMTA) with ESMTPSA id 18f73cc24e70008ca2.002 for (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384); Mon, 13 May 2024 21:11:49 +0000 X-Zone-Loop: 3abd75f9e69200d687d3a61da99704173151516be2cc X-Originating-IP: [136.175.111.2] DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=freakingpenguin.com; s=x; h=Content-Transfer-Encoding:Content-Type: MIME-Version:Message-ID:Date:References:In-Reply-To:Subject:Cc:To:From:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=lvTGxKLzltDchk0V0pGC0DzjS74K5UV7NGbSITT81JU=; b=JdPRgQpLlU2KK+PkAS5lPeYlB4 Z/YbhQL/i8BLq3ao7P7qoj/7mwEqE/+rp2/ZGI/wWFq2J9Vvyt/1DPHJGWow7pVYnab5KlGpQeT57 KOcowN5t1df5XUr8/8h1gm4XIJXlCrv+HfZGBFCOD5muDxmRX7+6qVB5FQOTOol6y9r6HhLoqmGjU 6fNQxPV1xfC4j9ryjba1cER1tUvGQq4gm+4kWF2ZcFltcwgs+LGWKLyOWcfq0zP076LWpZ1mRLFgr 2t1O6XKeIo1+ht6FDd3+1/B7LczCk6JWR/Bt1SWUj143ZNbjMk/9ycYwWY7oicII5N3EAIet1cUaZ rLaGq3yQ==; From: Richard Sent To: Simon Tournier Cc: Edouard Klein , guix-devel@gnu.org Subject: Re: Guix pull: avoiding "Computing Guix derivation" In-Reply-To: <87ikzh340q.fsf@gmail.com> (Simon Tournier's message of "Mon, 13 May 2024 21:28:37 +0200") References: <874jb16993.fsf@rdklein.fr> <87ikzh340q.fsf@gmail.com> Date: Mon, 13 May 2024 17:11:41 -0400 Message-ID: <87y18d1koi.fsf@freakingpenguin.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Authenticated-Id: richard@freakingpenguin.com Received-SPF: pass client-ip=136.175.108.0; envelope-from=richard@freakingpenguin.com; helo=mail-108-mta0.mxroute.com X-Spam_score_int: -4 X-Spam_score: -0.5 X-Spam_bar: / X-Spam_report: (-0.5 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, NUMERIC_HTTP_ADDR=1.242, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Spam-Score: -6.37 X-Migadu-Queue-Id: 2BE4471986 X-Migadu-Scanner: mx10.migadu.com X-Migadu-Spam-Score: -6.37 X-TUID: H+yUpK2KU/dV Simon Tournier writes: > Hi, > > On lun., 13 mai 2024 at 17:04, Edouard Klein wrote: > >> - Why is this step not substitutable ? The inputs are known, a hash can >> be derived, a substitute server could be queried for an output of that >> hash ? What am I missing ? Does the guix derivation not end up in the >> store ? What makes it so special that it can't be served by a substitute >> server ? > > Assume we are running two different Guix revisions, say A and B. And at > the end of our respective =E2=80=9Cguix pull=E2=80=9D, we expect to have = the same > revision, say C. We expect to then run the same Guix. > > Other said, how can we =E2=80=9Ccompile=E2=80=9D the code of C using one = machinery from > A and another potentially different from B and expect to have the same > result? > > Somehow, we need an intermediary step: something minimal that is > independent of A and B but produces the same C. And it=E2=80=99s the aim= of > =E2=80=9CComputing derivation=E2=80=9D with the script build-aux/build-se= lf.scm. > > The inputs are known, indeed. However, the ones from A and from B are > not necessary the sames. For instance, Guile of A might be different of > Guile of B. Somehow, that =E2=80=9CComputing derivation=E2=80=9D is what= allows to time > travel. > ... > All the point is to build the same result starting from two different > machinery (different Guile, modules, etc) and ending to the same output. > Considering the combinatorial, it would not be possible to substitutes; > it would mean compute all the couples A-C and B-C. > > That=E2=80=99s an hard topic and speed it up would be very nice. Bah it = would > require to redesign stuff and perhaps find a process to build the same > output using two different toolchains, somehow. Thanks for the detailed explanation Simon! Given that Guix packages itself as a package, I wonder if there is room to utilize the "Guix package" Guix as a uniform base to build "Guix, the collection of channels" and skip "computing Guix derivation". For instance, --8<---------------cut here---------------start------------->8--- ~ $ guix shell guix -- guix describe The following derivation will be built: /gnu/store/lk0qkgsz33qzhi6c83bw4nbc3sbzjb3s-profile.drv building CA certificate bundle... listing Emacs sub-directories... building fonts directory... building directory of Info manuals... building profile with 1 package... guix 4c94b9e repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 4c94b9e983bc51d9504655f1e7727c4f6d14b6b7 --8<---------------cut here---------------end--------------->8--- Instead of A and B building C directly, A and B download the substitutable Guix package D, then use D to build C. Because D is a reproducible package, it should be substitutable from both A and B. Then, because D->C is the same for everyone, we could substitute the Guix derivation computation for C. My understanding is this would resolve the question of "how to build C given that A and B are different without a long derivation computation?" --8<---------------cut here---------------start------------->8--- ~ $ guix shell guix -- guix time-machine -q --branch=3Dmaster -- describe Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.or= g/git/guix.git'... Computing Guix derivation for 'x86_64-linux'... / substitute: updating substitutes from 'http://10.1.1.101:80'... 100.0% substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 10= 0.0% substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% The following derivations will be built: /gnu/store/yp694ahf5fb8sm06r9llmgciax8yq20j-profile.drv /gnu/store/dw2gzlz8n93dmvkpb9wbr45p7awsixal-profile.drv /gnu/store/x59ywl8rncs0aqjqnxmjyhyzcznnm643-inferior-script.scm.drv building CA certificate bundle... listing Emacs sub-directories... building fonts directory... building directory of Info manuals... building profile with 1 package... building /gnu/store/x59ywl8rncs0aqjqnxmjyhyzcznnm643-inferior-script.scm.dr= v... building package cache... building profile with 1 package... guix 0edbb93 repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 0edbb93130651f29dc59fe4ca546a5065670ac8a --8<---------------cut here---------------end--------------->8--- I imagine there's a large amount of behind the scenes work that would need to be done for this to work, but if nothing else it's an interesting thought experiment! --=20 Take it easy, Richard Sent Making my computer weirder one commit at a time.