From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id mLlXKYSZu14TfAAA0tVLHw (envelope-from ) for ; Wed, 13 May 2020 06:53:56 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id uMqDEpOZu16nSAAAB5/wlQ (envelope-from ) for ; Wed, 13 May 2020 06:54:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D8439941094 for ; Wed, 13 May 2020 06:54:08 +0000 (UTC) Received: from localhost ([::1]:60016 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYlHN-0001GH-P9 for larch@yhetil.org; Wed, 13 May 2020 02:54:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49538) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jYlHG-0001Fr-Gs for guix-devel@gnu.org; Wed, 13 May 2020 02:54:02 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:59069) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jYlHF-0004TA-RP; Wed, 13 May 2020 02:54:01 -0400 Received: from [2001:980:1b4f:1:42d2:832d:bb59:862] (port=41320 helo=dundal.peder.onsbrabantnet.nl) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1jYlHF-0002SB-9V; Wed, 13 May 2020 02:54:01 -0400 From: Jan Nieuwenhuizen To: Paul Spooren Subject: Re: Verification Builds for Guix Organization: AvatarAcademy.nl References: <492ca854398724be49d45bcf253358c17694084f.camel@aparcar.org> X-Url: http://AvatarAcademy.nl Date: Wed, 13 May 2020 08:53:58 +0200 In-Reply-To: <492ca854398724be49d45bcf253358c17694084f.camel@aparcar.org> (Paul Spooren's message of "Tue, 12 May 2020 10:08:54 -1000") Message-ID: <87ftc4qq9l.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Scanner: scn0 X-Spam-Score: -1.01 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Scan-Result: default: False [-1.01 / 13.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; GENERIC_REPUTATION(0.00)[-0.53968487655623]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.51.188.0/24:c]; IP_REPUTATION_HAM(0.00)[asn: 22989(0.06), country: US(-0.00), ip: 209.51.188.17(-0.54)]; DWL_DNSWL_BLOCKED(0.00)[209.51.188.17:from]; HAS_ORG_HEADER(0.00)[]; MX_GOOD(-0.50)[cached: eggs.gnu.org]; RCPT_COUNT_TWO(0.00)[2]; MAILLIST(-0.20)[mailman]; FORGED_RECIPIENTS_MAILLIST(0.00)[]; RCVD_IN_DNSWL_FAIL(0.00)[209.51.188.17:server fail]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:22989, ipnet:209.51.188.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[larch=yhetil.org]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; FROM_NEQ_ENVFROM(0.00)[janneke@gnu.org,guix-devel-bounces@gnu.org]; FROM_HAS_DN(0.00)[]; URIBL_BLOCKED(0.00)[avataracademy.com:url,dezyne.org:url,gnu.org:email,aparcar.org:url,joyofsource.com:url,lilypond.org:url]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[gnu.org]; HAS_LIST_UNSUB(-0.01)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.51.188.17:from]; FORGED_SENDER_MAILLIST(0.00)[] X-TUID: 3Y+EoBF4SPF+ Paul Spooren writes: [cc: guix-devel] Hello Paul, > I've used the last week a bit to work on some kind of verification build > collector. Essentially a scraper of rebuild results. Meaning, an official= binary > provided by a project is tried to rebuild. Okay... > This works already for Archlinux and OpenWrt[0], now I wanted to know if = the > same would make sense for Guix. Possibly...I'm not sure, cc'ing guix-devel :-) > Checking out the packages part of the website I found that there is at > least an CI creating binaries. Are those also offered to users? Yes, Guix (like Nix) uses a mechanism that enables (encourages?) users to always build everything from source. The result of a package build, a "binary package" is uniquely identified by a hash of its inputs, including its build recipe, e.g., "18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2". This hash includes the transitive dependencies and can be calculated without compiling anything. The "binary package", once built, will be installed using this hash, e.g.: "/gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2". A user can opt-in to using "binary substitutes". In that case when a installing a package, before the package build actually starts, Guix checks if a binary is already available from a substitute server using this hash. If a binary is available it simply (well...keys/trust etc.) downloads the build result. In my case it checks for these in order http://banaan.local:8080/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guil= e-3.0.2 http://janneke.lilypond.org:8080/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5= l1n-guile-3.0.2 http://kluit.dezyne.org:8181/nar/gzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-= guile-3.0.2 https://ci.guix.gnu.org/nar/lzip/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile= -3.0.2 > If so a rebuilder of exactly these images would be interesting to > integrate in the rebuild-collector. Okay...so be clear on this; Guix (nor Nix) have the concept of REbuilders; all builders are equal: the central CI builder is not "more equal" than any other ;-) Guix does have the "guix challenge" command --8<---------------cut here---------------start------------->8--- guix challenge --help Usage: guix challenge [PACKAGE...] Challenge the substitutes for PACKAGE... provided by one or more servers. --substitute-urls=3DURLS compare build results with those at URLS --8<---------------cut here---------------end--------------->8--- now the trick is, to get "someone" to run that on an interesting portion of the archive...and to report the results in some common format. Looking at > [0]: https://rebuild.aparcar.org/ I think it could work, although for Guix I think it would be more natural to consider the local build to be the the "original" to compare binary substitute servers against? Maybe the format could be more symmetrical? Greetings, janneke --=20 Jan Nieuwenhuizen | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar=C2=AE http://AvatarAcademy.com