Problem bootstrapping Guix - "make update-guix-package" result: no code for module (gcrypt hash)

Problem bootstrapping Guix - "make update-guix-package" result: no code for module (gcrypt hash)

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 4037 bytes --]

Hi,

I'm trying to bootstrap current Guix (master) from Guix past (1.1.0 binary
tarball).

The goal is: I want to have only guix-the-package-manager at a specific guix
commit (!) available inside a Docker image.

Because the package "guix" in guix is always behind a little bit, that
means I have to first check out that commit from source code, then use
make update-guix-package in order to update the guix package definition
in gnu/packages/package-management.scm to that commit and then use guix pack
in order to pack guix into a tarball, which I then extract and then copy into
a previously-empty docker image.

Or so I thought.

What I have currently is on https://gitlab.com/daym/guix-on-docker/ .
Especially see Dockerfile.

You can build it on your machine using

$ docker build --build-arg BOOTSTRAP_GUIX_COMMIT_ID=0b21bf6245b36ea21d2eeaf2340a99aa4d5894db .

(or a newer commit--it doesn't matter) and see it fail.

(it will first download https://ftp.gnu.org/gnu/guix/guix-binary-1.1.0.x86_64-linux.tar.xz
and then check out guix anonymously from https://git.savannah.gnu.org/git/guix.git at
the specified commit)

Or you can look at the CI logs on
https://gitlab.com/daym/guix-on-docker/-/pipelines and see it fail there.

The failure is when it does that:

    && ENV="guix environment --pure guix --ad-hoc git guile-readline guile-json guile-zlib guile
-lzlib bash which --" \
    && ${ENV} make update-guix-package \
    && ./pre-inst-env guix pack --verbosity=2 -f tarball --localstatedir -r /tmp/guix.tar.gz --profile-name=current-guix guix \

and it prints the following:

(see https://gitlab.com/daym/guix-on-docker/-/jobs/755694619 )

make[1]: Leaving directory '/master'
accepted connection from pid 17760, user root
git rev-parse HEAD
ff43f128b7c142ae3f758d34137e562e6f7ef0e0
./pre-inst-env "/gnu/store/s08nkx9dzpvn41s4k277p9b27abpcjvp-profile/bin/guile"			\
   ./build-aux/update-guix-package.scm	\
   "`git rev-parse HEAD`"
accepted connection from pid 17766, user root
source code for commit ff43f128b7c142ae3f758d34137e562e6f7ef0e0: /gnu/store/3bz5q9g2p11pazy4g4vq8x8qp86c7ngh-guix-1.1.0-28.ff43f12-checkout (GC root: guix-1.1.0-28.ff43f12-checkout)
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /master/./build-aux/update-guix-package.scm
;;; compiled /root/.cache/guile/ccache/3.0-LE-8-4.3/master/build-aux/update-guix-package.scm.go
;;; compiling /master/gnu/packages/package-management.scm
;;; compiled /root/.cache/guile/ccache/3.0-LE-8-4.3/master/gnu/packages/package-management.scm.go
Backtrace:
In ice-9/boot-9.scm:
  3223:13 19 (_)
In ice-9/threads.scm:
    390:8 18 (_ _)
In ice-9/boot-9.scm:
  3507:20 17 (_)
   2806:4 16 (save-module-excursion _)
  3527:26 15 (_)
In unknown file:
          14 (primitive-load-path "guix/store" #<procedure 7f29ec5a9?>)
In guix/store.scm:
     23:0 13 (_)
In ice-9/boot-9.scm:
   3380:4 12 (define-module* _ #:filename _ #:pure _ #:version _ # _ ?)
  3393:24 11 (_)
   222:29 10 (map1 (((guix utils)) ((guix config)) ((guix #)) ((?)) ?))
   222:29  9 (map1 (((guix config)) ((guix deprecation)) ((guix ?)) ?))
   222:29  8 (map1 (((guix deprecation)) ((guix memoization)) ((?)) ?))
   222:29  7 (map1 (((guix memoization)) ((guix serialization)) (#) ?))
   222:29  6 (map1 (((guix serialization)) ((guix monads)) ((# #)) ?))
   222:29  5 (map1 (((guix monads)) ((guix records)) ((guix #)) (#) ?))
   222:29  4 (map1 (((guix records)) ((guix base16)) ((guix #)) (#) ?))
   222:29  3 (map1 (((guix base16)) ((guix base32)) ((gcrypt #)) # ?))
   222:29  2 (map1 (((guix base32)) ((gcrypt hash)) ((guix #)) (#) ?))
   222:17  1 (map1 (((gcrypt hash)) ((guix profiling)) ((rnrs #)) # ?))
   3300:6  0 (resolve-interface (gcrypt hash) #:select _ #:hide _ # _ ?)

ice-9/boot-9.scm:3300:6: In procedure resolve-interface:
no code for module (gcrypt hash).

What now?

Am I doing it wrong?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Problem bootstrapping Guix - "make update-guix-package" result: no code for module (gcrypt hash)

[Ludovic Courtès]

Hi,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> I'm trying to bootstrap current Guix (master) from Guix past (1.1.0 binary
> tarball).
>
> The goal is: I want to have only guix-the-package-manager at a specific guix
> commit (!) available inside a Docker image.

Why build Guix from source?  I guess it’s enough to do:

  guix pull --commit=XYZ

if all you want is Guix at commit XYZ.  Or am I missing something?

Thanks,
Ludo’.

Re: Problem bootstrapping Guix - "make update-guix-package" result: no code for module (gcrypt hash)

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 2114 bytes --]

Hi Ludo,

On Mon, 05 Oct 2020 14:20:08 +0200
Ludovic Courtès <ludo@gnu.org> wrote:

> Danny Milosavljevic <dannym@scratchpost.org> skribis:
> 
> > I'm trying to bootstrap current Guix (master) from Guix past (1.1.0 binary
> > tarball).
> >
> > The goal is: I want to have only guix-the-package-manager at a specific guix
> > commit (!) available inside a Docker image.  
> 
> Why build Guix from source?  I guess it’s enough to do:
> 
>   guix pull --commit=XYZ
> 
> if all you want is Guix at commit XYZ.  Or am I missing something?

I'm doing a project for Heads where we are trying to switch over their build
system to something that makes their builds more reproducible (for example
Guix).

They are using github and gitlab test runners for a lot of things, so one of
the ways we are trying to do continuous integration is to do the following:

(1) Have guix-the-package-manager be built and published on
repository.gitlab.com.  It eventually does "./pre-inst-env guix pack guix"
and then puts the result into a new docker container.  I can't see how to do that
after a guix pull.  Note that I don't want to also carry garbage (this entire
thing has to be verified for security eventually, so...).  Currently, guix
is being bootstrapped from Alpine, and I don't want Alpine to remain in there.

(2) Use the result in order to build boards using tiny Dockerfiles
which would just say

  FROM repository.gitlab.com/guix-on-docker
  RUN guix build heads-kgpe-d16

and throw away the derivation (or publish it, too?)--but keep the log file
and exit status.

Note that (1) should pin a specific Guix commit for a long time since Heads
does not want to build on a moving target since they do hash verification
on bootup, and firmware is hard to keep working (i.e. someone has to
manually verify, on real hardware, whether stuff still works after an
update of the toolchain).  And Heads basically is ONLY security-relevant
stuff.

But you are right--I'll now instead just guix gc and then copy /gnu and
/var/guix and /etc/guix into a "FROM scratch" Docker image.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Problem bootstrapping Guix - "make update-guix-package" result: no code for module (gcrypt hash)

[Ludovic Courtès]

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> I'm doing a project for Heads where we are trying to switch over their build
> system to something that makes their builds more reproducible (for example
> Guix).
>
> They are using github and gitlab test runners for a lot of things, so one of
> the ways we are trying to do continuous integration is to do the following:
>
> (1) Have guix-the-package-manager be built and published on
> repository.gitlab.com.  It eventually does "./pre-inst-env guix pack guix"
> and then puts the result into a new docker container.  I can't see how to do that
> after a guix pull.  Note that I don't want to also carry garbage (this entire
> thing has to be verified for security eventually, so...).  Currently, guix
> is being bootstrapped from Alpine, and I don't want Alpine to remain in there.

Why not just run “guix pack guix” with a “guix pull”-provided Guix?
You’d benefit from transparency and provenance tracking, the reduced
binary seeds, etc., which is very different from what you get by
building on top of Alpine.

If you need a specific Guix commit, you can also run:

  guix pack guix \
    --with-commit=guix=a2ed00f79fd5bf69c6cca3fa7bdc62726bf848fa \
    --with-git-url=guix=https://git.savannah.gnu.org/git/guix.git

You can still get test failures if there’s a problem on that commit, as
we’ve seen before, but apart from that it looks like what you need no?

(The ‘--with-git-url’ is only necessary because the default URL uses the
“dumb” transparent, which libgit2 apparently dislikes.)

> (2) Use the result in order to build boards using tiny Dockerfiles
> which would just say
>
>   FROM repository.gitlab.com/guix-on-docker
>   RUN guix build heads-kgpe-d16
>
> and throw away the derivation (or publish it, too?)--but keep the log file
> and exit status.

Then again, why even go through Docker?  You could just as well in one
go do:

  guix time-machine --commit=a2ed00f79fd5bf69c6cca3fa7bdc62726bf848fa -- \
    build heads-kgpe-d16

I’ve used Guix with GitLab-CI for instance, and it makes absolutely no
sense to me to resort to Docker if you already have Guix running.

> Note that (1) should pin a specific Guix commit for a long time since Heads
> does not want to build on a moving target since they do hash verification
> on bootup, and firmware is hard to keep working (i.e. someone has to
> manually verify, on real hardware, whether stuff still works after an
> update of the toolchain).  And Heads basically is ONLY security-relevant
> stuff.

An additional reason to avoid hopping through Alpine…

HTH!

Ludo’.