all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* A real-life test of long-term reproducibility
@ 2022-08-04  8:43 Konrad Hinsen
  2022-08-04 15:35 ` blake
  2022-08-07 20:53 ` Ludovic Courtès
  0 siblings, 2 replies; 6+ messages in thread
From: Konrad Hinsen @ 2022-08-04  8:43 UTC (permalink / raw)
  To: Guix Devel

Hi everyone,

One of our claims is that Guix can rebuild code identically as long as
we have a machine with a Linux kernel and a POSIX filesystem. This week
I had an occasion to put this to a real-life test. So far it's a
failure. I can guess reasons for my failed attempts, but I don't think
they were unreasonable to try. So I'd like to document something that
works, to avoid others falling into the same traps. I just don't know
yet what the Right Way To Do It is!

The package I want to rebuild and use is "nmoldyn" from Guix commit
f250a868d8c687df08559682fa68fb4ea2a1ea69. That's the commit referenced
in my notes, obtained via "guix describe" in early 2018. I am pretty
sure it worked fine back then.

First attempt:

   $ guix time-machine --commit=f250a868d8c687df08559682fa68fb4ea2a1ea69 -- build nmoldyn
   Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
   Backtrace:
   In guix/store.scm:
       672:3 19 (_)
   In ice-9/boot-9.scm:
     1752:10 18 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
   In guix/store.scm:
      659:37 17 (thunk)
   In guix/status.scm:
       815:4 16 (call-with-status-report _ _)
   In guix/store.scm:
      1298:8 15 (call-with-build-handler #<procedure 7ff03a85b960 at guix/ui.scm:1…> …)
     2168:25 14 (run-with-store #<store-connection 256.99 7ff03b715c30> _ # _ # _ # _)
   In guix/inferior.scm:
       903:8 13 (_ _)
   In guix/channels.scm:
       944:2 12 (_ _)
       891:2 11 (_ _)
   In ./guix/monads.scm:
       487:9 10 (_ _)
   In guix/store.scm:
      1996:8  9 (_ _)
   In guix/channels.scm:
      642:36  8 (_ #<store-connection 256.99 7ff03b715c30>)
      703:11  7 (_)
   In ice-9/eval.scm:
       619:8  6 (_ #(#(#(#<directory (build-self) 7ff03a8678c0>) "/gnu/store/…" …) …))
      626:19  5 (_ #(#(#(#<directory (build-self) 7ff03a8678c0>) "/gnu/store/…" …) …))
       155:9  4 (_ #(#(#(#<directory (build-self) 7ff03a8678c0>) "/gnu/store/…" …) …))
      223:20  3 (proc #(#(#(#<directory (build-self) 7ff03a8678c0>) "/gnu/sto…" …) …))
   In unknown file:
              2 (%resolve-variable (7 . %guix-register-program) #<directory (build-…>)
   In ice-9/boot-9.scm:
     1685:16  1 (raise-exception _ #:continuable? _)
     1685:16  0 (raise-exception _ #:continuable? _)

   ice-9/boot-9.scm:1685:16: In procedure raise-exception:
   error: %guix-register-program: unbound variable

I don't understand what is going wrong here, but it may be related to
the fact that the commit I am trying to go back to is older than "guix
time-machine". If that's the explanation, it would help if Guix showed
some clear error message instead of crashing.

Next I tried checking out the source code for commit
f250a868d8c687df08559682fa68fb4ea2a1ea69, and building it from
source. This is a bit tricky because 2018 Guix cannot be built in
today's Guix build environment. For example, today we have Guile 3, but
back then we had Guile 2.2. So I need to do "guix environment guix" in
an older Guix, before the Guile 3 transition, but later than the
introduction of time-machine. I picked one somewhat at random:

   $ guix time-machine --commit=e2293cbbe0cd20ddeb932e6f5616565ab468c087
   -- environment –pure guix

Then I did "bootstrap", "configure –localstatedir=/var", "make
check". The latter shows 15 failures, some of which look important:

   FAIL: tests/builders.scm
   FAIL: tests/derivations.scm
   FAIL: tests/packages.scm
   FAIL: tests/guix-environment.sh
   FAIL: tests/guix-daemon.sh

And indeed I cannot build much with my compiled guix:

   $ ./pre-inst-env guix build nmoldyn

hangs after a while, running a binary called "test-lock" for hours.

Given the time lapse, I suppose there have been incompatible changes in
the build daemon, making the old Guix incompatible with the rather
recent build daemon running on my machine. But is there a way around
this, other than installing an old Guix in a fully isolated VM?

And if installing the old Guix in a VM is the only solution, what would
be the best way to do that? I can't think of much else than starting
from another distribution (e.g. Debian) and following the installation
instructions. That's already a lot of work, but it's made worse by the
installation instructions being hidden inside the manual of that old
commit, which I cannot easily consult.

I'd be grateful for any suggestions!

Cheers,
  Konrad


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: A real-life test of long-term reproducibility
  2022-08-04  8:43 A real-life test of long-term reproducibility Konrad Hinsen
@ 2022-08-04 15:35 ` blake
  2022-08-07 20:53 ` Ludovic Courtès
  1 sibling, 0 replies; 6+ messages in thread
From: blake @ 2022-08-04 15:35 UTC (permalink / raw)
  To: Konrad Hinsen, Guix Devel

Hi Konrad, 

Just one quick comment:

August 4, 2022 8:43 AM, "Konrad Hinsen" <konrad.hinsen@fastmail.net> wrote:

> Hi everyone,
> 
> One of our claims is that Guix can rebuild code identically as long as
> we have a machine with a Linux kernel and a POSIX filesystem.

This actually isn't the claim. Reproducibility is only guaranteed on
Guix systems. It's important that you reproduce the entire system, which
means you will have needed to have saved the commit of your version of
the Guix package manager in order to return to that system.

See the section of the Guix manual "Replicating Guix" for more info.
https://guix.gnu.org/en/manual/devel/en/html_node/Replicating-Guix.html

Happy hacking!

> This week
> I had an occasion to put this to a real-life test. So far it's a
> failure. I can guess reasons for my failed attempts, but I don't think
> they were unreasonable to try. So I'd like to document something that
> works, to avoid others falling into the same traps. I just don't know
> yet what the Right Way To Do It is!
> 
> The package I want to rebuild and use is "nmoldyn" from Guix commit
> f250a868d8c687df08559682fa68fb4ea2a1ea69. That's the commit referenced
> in my notes, obtained via "guix describe" in early 2018. I am pretty
> sure it worked fine back then.
> 
> First attempt:
> 
> $ guix time-machine --commit=f250a868d8c687df08559682fa68fb4ea2a1ea69 -- build nmoldyn
> Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
> Backtrace:
> In guix/store.scm:
> 672:3 19 (_)
> In ice-9/boot-9.scm:
> 1752:10 18 (with-exception-handler _ _ #:unwind? _ #:unwind-for-type _)
> In guix/store.scm:
> 659:37 17 (thunk)
> In guix/status.scm:
> 815:4 16 (call-with-status-report _ _)
> In guix/store.scm:
> 1298:8 15 (call-with-build-handler #<procedure 7ff03a85b960 at guix/ui.scm:1…> …)
> 2168:25 14 (run-with-store #<store-connection 256.99 7ff03b715c30> _ # _ # _ # _)
> In guix/inferior.scm:
> 903:8 13 (_ _)
> In guix/channels.scm:
> 944:2 12 (_ _)
> 891:2 11 (_ _)
> In ./guix/monads.scm:
> 487:9 10 (_ _)
> In guix/store.scm:
> 1996:8 9 (_ _)
> In guix/channels.scm:
> 642:36 8 (_ #<store-connection 256.99 7ff03b715c30>)
> 703:11 7 (_)
> In ice-9/eval.scm:
> 619:8 6 (_ #(#(#(#<directory (build-self) 7ff03a8678c0>) "/gnu/store/…" …) …))
> 626:19 5 (_ #(#(#(#<directory (build-self) 7ff03a8678c0>) "/gnu/store/…" …) …))
> 155:9 4 (_ #(#(#(#<directory (build-self) 7ff03a8678c0>) "/gnu/store/…" …) …))
> 223:20 3 (proc #(#(#(#<directory (build-self) 7ff03a8678c0>) "/gnu/sto…" …) …))
> In unknown file:
> 2 (%resolve-variable (7 . %guix-register-program) #<directory (build-…>)
> In ice-9/boot-9.scm:
> 1685:16 1 (raise-exception _ #:continuable? _)
> 1685:16 0 (raise-exception _ #:continuable? _)
> 
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> error: %guix-register-program: unbound variable
> 
> I don't understand what is going wrong here, but it may be related to
> the fact that the commit I am trying to go back to is older than "guix
> time-machine". If that's the explanation, it would help if Guix showed
> some clear error message instead of crashing.
> 
> Next I tried checking out the source code for commit
> f250a868d8c687df08559682fa68fb4ea2a1ea69, and building it from
> source. This is a bit tricky because 2018 Guix cannot be built in
> today's Guix build environment. For example, today we have Guile 3, but
> back then we had Guile 2.2. So I need to do "guix environment guix" in
> an older Guix, before the Guile 3 transition, but later than the
> introduction of time-machine. I picked one somewhat at random:
> 
> $ guix time-machine --commit=e2293cbbe0cd20ddeb932e6f5616565ab468c087
> -- environment –pure guix
> 
> Then I did "bootstrap", "configure –localstatedir=/var", "make
> check". The latter shows 15 failures, some of which look important:
> 
> FAIL: tests/builders.scm
> FAIL: tests/derivations.scm
> FAIL: tests/packages.scm
> FAIL: tests/guix-environment.sh
> FAIL: tests/guix-daemon.sh
> 
> And indeed I cannot build much with my compiled guix:
> 
> $ ./pre-inst-env guix build nmoldyn
> 
> hangs after a while, running a binary called "test-lock" for hours.
> 
> Given the time lapse, I suppose there have been incompatible changes in
> the build daemon, making the old Guix incompatible with the rather
> recent build daemon running on my machine. But is there a way around
> this, other than installing an old Guix in a fully isolated VM?
> 
> And if installing the old Guix in a VM is the only solution, what would
> be the best way to do that? I can't think of much else than starting
> from another distribution (e.g. Debian) and following the installation
> instructions. That's already a lot of work, but it's made worse by the
> installation instructions being hidden inside the manual of that old
> commit, which I cannot easily consult.
> 
> I'd be grateful for any suggestions!
> 
> Cheers,
> Konrad


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: A real-life test of long-term reproducibility
  2022-08-04  8:43 A real-life test of long-term reproducibility Konrad Hinsen
  2022-08-04 15:35 ` blake
@ 2022-08-07 20:53 ` Ludovic Courtès
  2022-08-08  5:21   ` Timothy Sample
                     ` (2 more replies)
  1 sibling, 3 replies; 6+ messages in thread
From: Ludovic Courtès @ 2022-08-07 20:53 UTC (permalink / raw)
  To: Konrad Hinsen; +Cc: Guix Devel

Hi Konrad!

Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:

> The package I want to rebuild and use is "nmoldyn" from Guix commit
> f250a868d8c687df08559682fa68fb4ea2a1ea69. That's the commit referenced
> in my notes, obtained via "guix describe" in early 2018. I am pretty
> sure it worked fine back then.

That’s a commit from January 2018, which is a few months before the
release of 0.15.0, the first release with proper ‘guix pull’ support:

  https://guix.gnu.org/en/blog/2018/gnu-guix-and-guixsd-0.15.0-released/

In other words: warranty void.  :-)

We can go back to 1.0.0, and presumably to 0.15.0, but anything older
than this is unknown territory.

> First attempt:
>
>    $ guix time-machine --commit=f250a868d8c687df08559682fa68fb4ea2a1ea69 -- build nmoldyn
>    Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...

[...]

>    ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>    error: %guix-register-program: unbound variable

It’s a variable that used to exist in (guix config) but was removed long
ago (in ea0a06cee2ba05451f94714a4f913db02efbe92c, shortly before
0.15.0.)

> I don't understand what is going wrong here, but it may be related to
> the fact that the commit I am trying to go back to is older than "guix
> time-machine". If that's the explanation, it would help if Guix showed
> some clear error message instead of crashing.

It could check whether the target commit is in the closure of the
v0.15.0 commit, but then that would give special treatment to the
existing commit history.  Maybe that’s OK though?

> Next I tried checking out the source code for commit
> f250a868d8c687df08559682fa68fb4ea2a1ea69, and building it from
> source. This is a bit tricky because 2018 Guix cannot be built in
> today's Guix build environment. For example, today we have Guile 3, but
> back then we had Guile 2.2. So I need to do "guix environment guix" in
> an older Guix, before the Guile 3 transition, but later than the
> introduction of time-machine. I picked one somewhat at random:
>
>    $ guix time-machine --commit=e2293cbbe0cd20ddeb932e6f5616565ab468c087
>    -- environment –pure guix
>
> Then I did "bootstrap", "configure –localstatedir=/var", "make
> check". The latter shows 15 failures, some of which look important:
>
>    FAIL: tests/builders.scm
>    FAIL: tests/derivations.scm
>    FAIL: tests/packages.scm
>    FAIL: tests/guix-environment.sh
>    FAIL: tests/guix-daemon.sh

We’d have to check what’s in ‘test-suite.log’; it might as well be an
environment issue more than a “real” problem.

> And indeed I cannot build much with my compiled guix:
>
>    $ ./pre-inst-env guix build nmoldyn
>
> hangs after a while, running a binary called "test-lock" for hours.

IIRC that was a bug in a Gnulib test (bundled in several GNU packages)
that would hang on machines with maybe more than 4 cores.  (See commit
acc2dab7f2f50c9169d6388007c770878eae4a9c for example.)  There might be
hints on how to work around it in the mailing list archive…

> Given the time lapse, I suppose there have been incompatible changes in
> the build daemon, making the old Guix incompatible with the rather
> recent build daemon running on my machine. But is there a way around
> this, other than installing an old Guix in a fully isolated VM?
>
> And if installing the old Guix in a VM is the only solution, what would
> be the best way to do that? I can't think of much else than starting
> from another distribution (e.g. Debian) and following the installation
> instructions. That's already a lot of work, but it's made worse by the
> installation instructions being hidden inside the manual of that old
> commit, which I cannot easily consult.

Again, none of this should be necessary as long as the target commit is
after 0.15.0 (or 1.0.0).

That said, it may be possible to build that Jan. 2018 Guix using an
environment created from Guix 0.15.0 or 1.0.0; it’s likely to have the
right versions of dependencies, and it should be reachable with
time-machine.

I’d be curious to hear how it goes!

Besides, I recently added ‘etc/time-travel-manifest.scm’ and added a
corresponding jobset at <https://ci.guix.gnu.org/jobset/time-travel>
(the jobset is not working well, but that’s because there are several
processes concurrently accessing the same cached checkout; we need a
trick to work around that…).  I’d like to consolidate this so we can
make sure that traveling back in time (after 0.15.0 or 1.0.0) remains a
possibility.

Thanks,
Ludo’.


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: A real-life test of long-term reproducibility
  2022-08-07 20:53 ` Ludovic Courtès
@ 2022-08-08  5:21   ` Timothy Sample
  2022-08-08  8:44   ` Konrad Hinsen
  2022-08-08  8:49   ` Konrad Hinsen
  2 siblings, 0 replies; 6+ messages in thread
From: Timothy Sample @ 2022-08-08  5:21 UTC (permalink / raw)
  To: Konrad Hinsen; +Cc: Ludovic Courtès, Guix Devel

[-- Attachment #1: Type: text/plain, Size: 5385 bytes --]

Hi Konrad,

Ludovic Courtès <ludo@gnu.org> writes:

> Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:
>
>> The package I want to rebuild and use is "nmoldyn" from Guix commit
>> f250a868d8c687df08559682fa68fb4ea2a1ea69. That's the commit referenced
>> in my notes, obtained via "guix describe" in early 2018. I am pretty
>> sure it worked fine back then.
>
> That’s a commit from January 2018, which is a few months before the
> release of 0.15.0, the first release with proper ‘guix pull’ support:
>
>   https://guix.gnu.org/en/blog/2018/gnu-guix-and-guixsd-0.15.0-released/
>
> In other words: warranty void.  :-)

Agreed.  That’s probably too early in Guix’s history to get good
results.  Keep in mind that the Software Heritage crawlers only started
indexing Guix packages in 2019.  That being said, I was too curious to
leave this alone, so I tried my hand at building that package from that
commit.

> [...]

> Again, none of this should be necessary as long as the target commit is
> after 0.15.0 (or 1.0.0).
>
> That said, it may be possible to build that Jan. 2018 Guix using an
> environment created from Guix 0.15.0 or 1.0.0; it’s likely to have the
> right versions of dependencies, and it should be reachable with
> time-machine.
>
> I’d be curious to hear how it goes!

It turns out it’s pretty easy, apart from having to boil the ocean.
Here’s what I did.

I decided to try the easiest thing first, which is checkout f250a86 and
try to build it with an environment from modern Guix.  We still have
Guile 2.2 packages, after all.  Here’s the environment that I used:

    $ guix shell --pure guile@2.2 guile2.2-gnutls guile2.2-gcrypt \
          gcc-toolchain coreutils bash autoconf automake guile2.2-git \
          sqlite bzip2 gettext libtool pkg-config grep sed patch \
          diffutils m4 findutils tar xz gzip libgcrypt gawk make\
          texinfo help2man

From there, I was able to build Guix normally.  Some of those packages
were guesses, and some were added to placate “configure”, I’m not sure
that all of them are necessary, but they are sufficient.

After that, I tried

    $ ./pre-inst-env guix build nmoldyn

It hummed along for a long time, and then my desktop crashed due to the
kernel’s out-of-memory process killer.  For whatever reason, and even
with 16G of RAM and 16G of swap, the Python 3 build gets smitten by the
OOM killer.  It happens when running the test suite, so I disabled it.
(From what I read, the test suite is used to generate profiles to train
the optimizer, so this is probably a bad idea.  I decided to forge ahead
instead of worry about this too much.)

Next, it couldn’t find the source code for ImageMagick 6.9.9-30.  It
took a minute, but I tracked it down hidden in the following RPM:

    http://mirror.speedpartner.de/remi-archive/SRPMS/ImageMagick6-6.9.9.30-1.remi.src.rpm

After extracting the tarball and adding it to the store with
“guix download”, the build continued.

There was one other source code issue, which was Mesa 17.2.1.  It was
easy to solve, as they just moved it to the “older-versions” directory
on their server.

The only other two issues were test failures due to certificate expiry.
It happened for Python 2 and NSS.  I remember there was a discussion
about this problem in general recently, but I couldn’t track it down.
For Python 2, I disabled the offending tests, and for NSS, I just
disabled the whole test suite.

After that, I got:

    @ build-succeeded /gnu/store/40x598cvpp16m87z52sj98c62dfbmmka-nmoldyn-3.0.11.drv -
    /gnu/store/khf2pf29vfq6b3dcca9p9mvz387jvdl7-nmoldyn-3.0.11

I’m honestly shocked that it worked so well.  I wish I had a better way
to keep track of where the sources came from.  For example, I’m curious
how many came from the build farm or other fallback options.

Overall, I give Guix two thumbs up!  Other than the Python 3 optimizer
bit (which might be solvable), nothing substantive had to be changed to
make this happen.

For best practices, I do have one suggestion.  The Guix package
collection is not uniformly reproducible or archived.  The best thing
you can do to ensure the long-term prospects of your projects is to
actually check how much of the source code is archived and how many of
the builds are reproducible.  There is no turn-key solution for this
(though maybe there should be), but you can check the source code
against the Preservation of Guix database [1], and you can check
reproducibility using the Guix Data Service or with “guix check”.
Ideally everything would be archived and reproducible, but it’s a work
in progress.  By checking, you might be able to fix some of the issues
now while it’s easy instead of in five or ten years when it’s
impossible!

Thanks Konrad for the interesting experiment.  While testing this out, I
came to really appreciate how hackable Guix is.  Even if I couldn’t find
Mesa 17.2.1, say, I could proceed with a similar version or try to build
it with Git.  It’s not ideal to have to make changes, but it’s nice to
know that Guix fails gracefully.

In case it’s useful, I’ve attached the changes I made.


-- Tim

[1] It’s at <https://ngyro.com/pog-reports/latest/>, and yes, I know I
ought to update it!  :)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch.diff --]
[-- Type: text/x-patch, Size: 2727 bytes --]

diff --git a/gnu/packages/gl.scm b/gnu/packages/gl.scm
index c42b4a00cb..83629d30e4 100644
--- a/gnu/packages/gl.scm
+++ b/gnu/packages/gl.scm
@@ -223,7 +223,10 @@ also known as DXTn or DXTC) for Mesa.")
     (source
       (origin
         (method url-fetch)
-        (uri (list (string-append "https://mesa.freedesktop.org/archive/"
+        (uri (list (string-append "https://archive.mesa3d.org/"
+                                  "older-versions/17.x/"
+                                  "mesa-" version ".tar.xz")
+                   (string-append "https://mesa.freedesktop.org/archive/"
                                   "mesa-" version ".tar.xz")
                    (string-append "ftp://ftp.freedesktop.org/pub/mesa/"
                                   "mesa-" version ".tar.xz")
diff --git a/gnu/packages/gnuzilla.scm b/gnu/packages/gnuzilla.scm
index a31e93d499..da9317cb42 100644
--- a/gnu/packages/gnuzilla.scm
+++ b/gnu/packages/gnuzilla.scm
@@ -336,7 +336,8 @@ in the Mozilla clients.")
              (setenv "DOMSUF" "(none)")
              (setenv "USE_IP" "TRUE")
              (setenv "IP_ADDRESS" "127.0.0.1")
-             (zero? (system* "./nss/tests/all.sh"))))
+             ;;(zero? (system* "./nss/tests/all.sh"))
+             #t))
            (replace 'install
              (lambda* (#:key outputs #:allow-other-keys)
                (let* ((out (assoc-ref outputs "out"))
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 29fa2809ab..f11f405331 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -162,7 +162,10 @@
                       "Lib/test/test_import.py"
                       "Lib/test/test_shutil.py"
                       "Lib/test/test_socket.py"
-                      "Lib/test/test_subprocess.py"))
+                      "Lib/test/test_subprocess.py"
+                      "Lib/test/test_httplib.py"
+                      "Lib/test/test_ssl.py"
+                      "Lib/test/test_urllib2_localnet.py"))
           #t))))
     (outputs '("out"
                "tk"))                     ;tkinter; adds 50 MiB to the closure
@@ -355,7 +358,12 @@ data types.")
                               "Lib/test/test_fcntl.py"))
                   #t))))
     (arguments (substitute-keyword-arguments (package-arguments python-2)
-                 ((#:tests? _) #t)))
+                 ((#:tests? _) #t)
+                 ((#:phases phases)
+                  `(modify-phases ,phases
+                     (add-before 'build 'set-extratestopts
+                       (lambda _
+                         (setenv "EXTRATESTOPTS" "--help")))))))
     (native-search-paths
      (list (search-path-specification
             (variable "PYTHONPATH")

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: A real-life test of long-term reproducibility
  2022-08-07 20:53 ` Ludovic Courtès
  2022-08-08  5:21   ` Timothy Sample
@ 2022-08-08  8:44   ` Konrad Hinsen
  2022-08-08  8:49   ` Konrad Hinsen
  2 siblings, 0 replies; 6+ messages in thread
From: Konrad Hinsen @ 2022-08-08  8:44 UTC (permalink / raw)
  To: Ludovic Courtès, Timothy Sample; +Cc: Guix Devel

Hi Ludo and Tim,

Thanks for your comments and experiments!


Ludovic Courtès <ludo@gnu.org> writes:

> That’s a commit from January 2018, which is a few months before the
> release of 0.15.0, the first release with proper ‘guix pull’ support:

Ouch. I wasn't aware that even "guix pull" happened later!

>> I don't understand what is going wrong here, but it may be related to
>> the fact that the commit I am trying to go back to is older than "guix
>> time-machine". If that's the explanation, it would help if Guix showed
>> some clear error message instead of crashing.
>
> It could check whether the target commit is in the closure of the
> v0.15.0 commit, but then that would give special treatment to the
> existing commit history.  Maybe that’s OK though?

I'd say yes, because the only problem is the existing commit history.
At least until we extend time-machine to actually change the past ;-)

> IIRC that was a bug in a Gnulib test (bundled in several GNU packages)
> that would hang on machines with maybe more than 4 cores.  (See commit
> acc2dab7f2f50c9169d6388007c770878eae4a9c for example.)  There might be
> hints on how to work around it in the mailing list archive…

Hardware dependencies...

> That said, it may be possible to build that Jan. 2018 Guix using an
> environment created from Guix 0.15.0 or 1.0.0; it’s likely to have the
> right versions of dependencies, and it should be reachable with
> time-machine.

I'll try with 1.0.0, that looks nicer than 0.15.0 ;-)

> Besides, I recently added ‘etc/time-travel-manifest.scm’ and added a
> corresponding jobset at <https://ci.guix.gnu.org/jobset/time-travel>

Nice! Guix will be certified for time travel!


Timothy Sample <samplet@ngyro.com> writes:

> It turns out it’s pretty easy, apart from having to boil the ocean.
> Here’s what I did.

Very interesting, thanks!

> I’m honestly shocked that it worked so well.  I wish I had a better way
> to keep track of where the sources came from.  For example, I’m curious
> how many came from the build farm or other fallback options.
>
> Overall, I give Guix two thumbs up!  Other than the Python 3 optimizer
> bit (which might be solvable), nothing substantive had to be changed to
> make this happen.

Indeed. And yet, from the point of view of a non-expert user, even the
slightest fix required is a show stopper.

> For best practices, I do have one suggestion.  The Guix package
> collection is not uniformly reproducible or archived.  The best thing
> you can do to ensure the long-term prospects of your projects is to
> actually check how much of the source code is archived and how many of
> the builds are reproducible.  There is no turn-key solution for this

Yes, that's a good idea, and I have done it for my most recent packages.
Time will tell if this is enough.

> Thanks Konrad for the interesting experiment.  While testing this out, I
> came to really appreciate how hackable Guix is.  Even if I couldn’t find
> Mesa 17.2.1, say, I could proceed with a similar version or try to build
> it with Git.  It’s not ideal to have to make changes, but it’s nice to
> know that Guix fails gracefully.

Definitely!

Cheers,
  Konrad.


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: A real-life test of long-term reproducibility
  2022-08-07 20:53 ` Ludovic Courtès
  2022-08-08  5:21   ` Timothy Sample
  2022-08-08  8:44   ` Konrad Hinsen
@ 2022-08-08  8:49   ` Konrad Hinsen
  2 siblings, 0 replies; 6+ messages in thread
From: Konrad Hinsen @ 2022-08-08  8:49 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Guix Devel

Hi Ludo,

> We can go back to 1.0.0, and presumably to 0.15.0, but anything older
> than this is unknown territory.

Even 1.0.0 isn't obvious:

  $ guix time-machine --commit=version-1.0.0 -- environment guix
  guix time-machine: error: Git error: unable to parse OID - contains invalid characters

OK, so let's try the commit hash:

  $ guix time-machine --commit=48aa30ce73d45dc5f126f42f01e65f1be4a9b578 -- environment guix
  Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
  Authenticating channel 'guix', commits 9edb3f6 to 48aa30c (6 new commits)...
  guix time-machine: error: commit
  48aa30ce73d45dc5f126f42f01e65f1be4a9b578 is not a descendant of
  introductory commit 9edb3f66fd807b096b48283debdcddccfea34bad

Cheers,
  Konrad.


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2022-08-08  9:29 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-04  8:43 A real-life test of long-term reproducibility Konrad Hinsen
2022-08-04 15:35 ` blake
2022-08-07 20:53 ` Ludovic Courtès
2022-08-08  5:21   ` Timothy Sample
2022-08-08  8:44   ` Konrad Hinsen
2022-08-08  8:49   ` Konrad Hinsen

Code repositories for project(s) associated with this inbox:

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.