Re: Verification Builds for Guix

[Jan Nieuwenhuizen <janneke@gnu.org>]

Paul Spooren writes:

[cc: guix-devel]

Hello Paul,

> I've used the last week a bit to work on some kind of verification build
> collector. Essentially a scraper of rebuild results. Meaning, an official binary
> provided by a project is tried to rebuild.

Okay...

> This works already for Archlinux and OpenWrt[0], now I wanted to know if the
> same would make sense for Guix.

Possibly...I'm not sure, cc'ing guix-devel :-)

> Checking out the packages part of the website I found that there is at
> least an CI creating binaries. Are those also offered to users?

Yes, Guix (like Nix) uses a mechanism that enables (encourages?) users
to always build everything from source.  The result of a package build,
a "binary package" is uniquely identified by a hash of its inputs,
including its build recipe, e.g.,
"18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2".

This hash includes the transitive dependencies and can be calculated
without compiling anything.  The "binary package", once built, will be
installed using this hash, e.g.:
"/gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2".

A user can opt-in to using "binary substitutes".  In that case when a
installing a package, before the package build actually starts, Guix
checks if a binary is already available from a substitute server using
this hash.  If a binary is available it simply (well...keys/trust etc.)
downloads the build result.  In my case it checks for these in order

    http://banaan.local:8080/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2
    http://janneke.lilypond.org:8080/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2
    http://kluit.dezyne.org:8181/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2
    https://ci.guix.gnu.org/nar/lzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2

> If so a rebuilder of exactly these images would be interesting to
> integrate in the rebuild-collector.

Okay...so be clear on this; Guix (nor Nix) have the concept of
REbuilders; all builders are equal: the central CI builder is not "more
equal" than any other ;-)

Guix does have the "guix challenge" command

--8<---------------cut here---------------start------------->8---
guix challenge --help
Usage: guix challenge [PACKAGE...]
Challenge the substitutes for PACKAGE... provided by one or more servers.

      --substitute-urls=URLS
                         compare build results with those at URLS
--8<---------------cut here---------------end--------------->8---

now the trick is, to get "someone" to run that on an interesting portion
of the archive...and to report the results in some common format.

Looking at

> [0]: https://rebuild.aparcar.org/

I think it could work, although for Guix I think it would be more
natural to consider the local build to be the the "original" to compare
binary substitute servers against?  Maybe the format could be more
symmetrical?

Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com