Re: Guix pull: avoiding "Computing Guix derivation"

[Richard Sent <richard@freakingpenguin.com>]

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi,
>
> On lun., 13 mai 2024 at 17:04, Edouard Klein <edou@rdklein.fr> wrote:
>
>> - Why is this step not substitutable ? The inputs are known, a hash can
>> be derived, a substitute server could be queried for an output of that
>> hash ? What am I missing ? Does the guix derivation not end up in the
>> store ? What makes it so special that it can't be served by a substitute
>> server ?
>
> Assume we are running two different Guix revisions, say A and B.  And at
> the end of our respective “guix pull”, we expect to have the same
> revision, say C.  We expect to then run the same Guix.
>
> Other said, how can we “compile” the code of C using one machinery from
> A and another potentially different from B and expect to have the same
> result?
>
> Somehow, we need an intermediary step: something minimal that is
> independent of A and B but produces the same C.  And it’s the aim of
> “Computing derivation” with the script build-aux/build-self.scm.
>
> The inputs are known, indeed.  However, the ones from A and from B are
> not necessary the sames.  For instance, Guile of A might be different of
> Guile of B.  Somehow, that “Computing derivation” is what allows to time
> travel.
> ...
> All the point is to build the same result starting from two different
> machinery (different Guile, modules, etc) and ending to the same output.
> Considering the combinatorial, it would not be possible to substitutes;
> it would mean compute all the couples A-C and B-C.
>
> That’s an hard topic and speed it up would be very nice.  Bah it would
> require to redesign stuff and perhaps find a process to build the same
> output using two different toolchains, somehow.

Thanks for the detailed explanation Simon!

Given that Guix packages itself as a package, I wonder if there is room
to utilize the "Guix package" Guix as a uniform base to build "Guix, the
collection of channels" and skip "computing Guix derivation".

For instance,

--8<---------------cut here---------------start------------->8---
~ $ guix shell guix -- guix describe
The following derivation will be built:
  /gnu/store/lk0qkgsz33qzhi6c83bw4nbc3sbzjb3s-profile.drv

building CA certificate bundle...
listing Emacs sub-directories...
building fonts directory...
building directory of Info manuals...
building profile with 1 package...
  guix 4c94b9e
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 4c94b9e983bc51d9504655f1e7727c4f6d14b6b7
--8<---------------cut here---------------end--------------->8---

Instead of A and B building C directly, A and B download the
substitutable Guix package D, then use D to build C. Because D is a
reproducible package, it should be substitutable from both A and B.
Then, because D->C is the same for everyone, we could substitute the
Guix derivation computation for C.

My understanding is this would resolve the question of "how to build C
given that A and B are different without a long derivation computation?"

--8<---------------cut here---------------start------------->8---
~ $ guix shell guix -- guix time-machine -q --branch=master -- describe
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Computing Guix derivation for 'x86_64-linux'... /
substitute: updating substitutes from 'http://10.1.1.101:80'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
The following derivations will be built:
  /gnu/store/yp694ahf5fb8sm06r9llmgciax8yq20j-profile.drv
  /gnu/store/dw2gzlz8n93dmvkpb9wbr45p7awsixal-profile.drv
  /gnu/store/x59ywl8rncs0aqjqnxmjyhyzcznnm643-inferior-script.scm.drv

building CA certificate bundle...
listing Emacs sub-directories...
building fonts directory...
building directory of Info manuals...
building profile with 1 package...
building /gnu/store/x59ywl8rncs0aqjqnxmjyhyzcznnm643-inferior-script.scm.drv...
building package cache...
building profile with 1 package...
  guix 0edbb93
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 0edbb93130651f29dc59fe4ca546a5065670ac8a
--8<---------------cut here---------------end--------------->8---

I imagine there's a large amount of behind the scenes work that would
need to be done for this to work, but if nothing else it's an
interesting thought experiment!

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.