* Re: Verification Builds for Guix [not found] <492ca854398724be49d45bcf253358c17694084f.camel@aparcar.org> @ 2020-05-13 6:53 ` Jan Nieuwenhuizen 2020-05-13 10:55 ` zimoun 0 siblings, 1 reply; 2+ messages in thread From: Jan Nieuwenhuizen @ 2020-05-13 6:53 UTC (permalink / raw) To: Paul Spooren; +Cc: guix-devel Paul Spooren writes: [cc: guix-devel] Hello Paul, > I've used the last week a bit to work on some kind of verification build > collector. Essentially a scraper of rebuild results. Meaning, an official binary > provided by a project is tried to rebuild. Okay... > This works already for Archlinux and OpenWrt[0], now I wanted to know if the > same would make sense for Guix. Possibly...I'm not sure, cc'ing guix-devel :-) > Checking out the packages part of the website I found that there is at > least an CI creating binaries. Are those also offered to users? Yes, Guix (like Nix) uses a mechanism that enables (encourages?) users to always build everything from source. The result of a package build, a "binary package" is uniquely identified by a hash of its inputs, including its build recipe, e.g., "18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2". This hash includes the transitive dependencies and can be calculated without compiling anything. The "binary package", once built, will be installed using this hash, e.g.: "/gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2". A user can opt-in to using "binary substitutes". In that case when a installing a package, before the package build actually starts, Guix checks if a binary is already available from a substitute server using this hash. If a binary is available it simply (well...keys/trust etc.) downloads the build result. In my case it checks for these in order http://banaan.local:8080/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2 http://janneke.lilypond.org:8080/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2 http://kluit.dezyne.org:8181/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2 https://ci.guix.gnu.org/nar/lzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2 > If so a rebuilder of exactly these images would be interesting to > integrate in the rebuild-collector. Okay...so be clear on this; Guix (nor Nix) have the concept of REbuilders; all builders are equal: the central CI builder is not "more equal" than any other ;-) Guix does have the "guix challenge" command --8<---------------cut here---------------start------------->8--- guix challenge --help Usage: guix challenge [PACKAGE...] Challenge the substitutes for PACKAGE... provided by one or more servers. --substitute-urls=URLS compare build results with those at URLS --8<---------------cut here---------------end--------------->8--- now the trick is, to get "someone" to run that on an interesting portion of the archive...and to report the results in some common format. Looking at > [0]: https://rebuild.aparcar.org/ I think it could work, although for Guix I think it would be more natural to consider the local build to be the the "original" to compare binary substitute servers against? Maybe the format could be more symmetrical? Greetings, janneke -- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com ^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: Verification Builds for Guix 2020-05-13 6:53 ` Verification Builds for Guix Jan Nieuwenhuizen @ 2020-05-13 10:55 ` zimoun 0 siblings, 0 replies; 2+ messages in thread From: zimoun @ 2020-05-13 10:55 UTC (permalink / raw) To: Jan Nieuwenhuizen; +Cc: Guix Devel, Paul Spooren Dear, I do not know if it is relevant and if it completes what janneke already said. On Wed, 13 May 2020 at 08:54, Jan Nieuwenhuizen <janneke@gnu.org> wrote: > > Checking out the packages part of the website I found that there is at > > least an CI creating binaries. Are those also offered to users? The build farm (by default) is reachable at [1] and it offers binary substitutes. Another entry point is Guix Data Service [1] which collects data about packages, builds, branches, etc.; for example the history of versions [3] or the status of builds [4]. Moreover, there is a recent proposal: build-coordinator [5]. [1] https://ci.guix.gnu.org/ [2] http://data.guix.gnu.org/ [3] https://data.guix.gnu.org/repository/1/branch/master/package/git [4] https://data.guix.gnu.org/repository/1/branch/master/package/git/output-history?output=out&system=armhf-linux&target=none [5] https://lists.gnu.org/archive/html/guix-devel/2020-04/msg00323.html > This hash includes the transitive dependencies and can be calculated > without compiling anything. The "binary package", once built, will be > installed using this hash, e.g.: > "/gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2". This hash identifies the package and the transitive dependencies, so the hash is an unique identifier (modulo hash collisions which are another story). The same hash should produce the same binary or there is source of non-determinism. That's why "guix build --check" rebuilds and compare the two builds. I do not know what is the policy on the build farm ci.guix.gnu.org about checking the rebuild. > Guix does have the "guix challenge" command > > --8<---------------cut here---------------start------------->8--- > guix challenge --help > Usage: guix challenge [PACKAGE...] > Challenge the substitutes for PACKAGE... provided by one or more servers. > > --substitute-urls=URLS > compare build results with those at URLS > --8<---------------cut here---------------end--------------->8--- > > now the trick is, to get "someone" to run that on an interesting portion > of the archive...and to report the results in some common format. I am not aware of such initiative. Which should be really cool. On the other hand, using this information of challenging the local builds against remote builds would allow to share /gnu/store/; other said full distributed substitutes mechanism. Best regards, simon ^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2020-05-13 10:56 UTC | newest] Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <492ca854398724be49d45bcf253358c17694084f.camel@aparcar.org> 2020-05-13 6:53 ` Verification Builds for Guix Jan Nieuwenhuizen 2020-05-13 10:55 ` zimoun
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).