From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christopher Baines Subject: Re: Automatically building packages affected by submitted patches Date: Mon, 04 Mar 2019 23:12:51 +0000 Message-ID: <87ef7mfd3g.fsf@cbaines.net> References: <87tvgmfgkw.fsf@cbaines.net> <87ef7nvsrf.fsf@elephly.net> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([209.51.188.92]:55592) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h0wla-0001KN-KM for guix-devel@gnu.org; Mon, 04 Mar 2019 18:13:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h0wlY-0002cj-Mz for guix-devel@gnu.org; Mon, 04 Mar 2019 18:13:02 -0500 Received: from li622-129.members.linode.com ([212.71.249.129]:59090 helo=mira.cbaines.net) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h0wlV-0002TH-P8 for guix-devel@gnu.org; Mon, 04 Mar 2019 18:12:58 -0500 In-reply-to: <87ef7nvsrf.fsf@elephly.net> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ricardo Wurmus Cc: guix-devel@gnu.org --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Ricardo Wurmus writes: >> In short, following on from some previous emails about Patchwork [1] and >> tracking and inspecting how Guix changes over time [2], I've got to the >> point where I have a very rough setup for building packages changed by >> patches sent to the guix-patches mailing list. > > This is great! I'm glad you think so :) >> With the goal of displaying a check in Patchwork about the >> build status of the affected packages, I need to compare what's been >> build by my Cuirass instance, with what https://ci.guix.info/ has >> built. To do this, my current plan is to have the Guix Data Service >> monitor a number of Cuirass instances somehow, extract information from >> them and store it. > > Would it not make sense to have the build farm perform the builds > instead of having a separate Cuirass instance? I'm using my Cuirass instance because it's easy to test with. But yes, I can see advantages in using the main build farm to perform the builds, especially in potentially providing substitutes more promptly. Were you thinking of anything in particular? >> - Looks at patch series in the Patchwork database that have been >> processed through the patchwork-test-series job in Laminar. > > Is the Laminar job something that could become part of Cuirass itself? The patchwork-test-series job [1] running within Laminar is doing a few things: - It takes the patches from patchwork.cbaines.net and applies them to the master branch. - (part 1) It pushes the resulting Git commits/branch up to [2]. - (part 2) It instructs the Guix Data Service to fetch the base and tip of the branch, and load the data from those commits. - (part 3) It also stores the hashes of the commits in the same database used by Patchwork. The above 3 parts of the process are a bit complicated, so I'll attempt to explain how they fit together. If you want to instruct Cuirass to build all the packages that are new or that have been changed, you need to know what the state was before the patches, and what the state with the patches looks like. This is where part 2 comes in. Once the Guix Data Service has processed the two commits, it can provide this information. This is where part 1 comes in providing the patches in a more machine readable form to the Guix Data Service and Cuirass, as they both fetch from the Git branch created in part 1. The Guix Data Service takes time to work out what's changed between the two revisions. This is where storing the commit hashes against the Patchwork series in the same database as Patchwork (part 3) comes in. That allows regularly checking if there are any Patchwork series that don't have a specification in Cuirass, and attempting to create one using the information from the Guix Data Service. 1: https://laminar.cbaines.net/cfg/jobs/patchwork-test-series.run 2: https://git.cbaines.net/guix/patches So back to your question, I don't think any of the above would fit neatly within Cuirass. I'm still thinking of Cuirass as a build service, you describe what you want it to build, and it goes away and does that. One thing I've become aware of recently is how similar Cuirass and the Guix Data Service actually are. The evaluations part of Cuirass and storing the derivations is very similar to the Guix Data Service. Do let me know if you have more thoughts or questions... Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAlx9sPNfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XfVlA//QOMLs/sndOm4BNN45lZiobfxfZ+rU1RdNm8JX/SMKBlwdh77OxGSxfo6 mieNCF9rEmVSW3fd3ytCE3DyYYC/64/Gqi5BXDZaPBFZNvjRB+PFHxo/OB2BpNq/ SSuc0cd7SjrHuBpoiwihVFaZy5UsIV4UHd5XuUVymPB3EYeAEIbCNYb/hnLB/oYC tcxpM9AVyysPTUxz01l+y9idm7mnDXgeqtyTRwpNI13BwIfBwWxyFVjjBpiHT6Xz fmwWSloDhc2UGAgXlnknfU7ggBFvM5EAokLssZx1sdJf1uAL+2gNBdTh1w9ZT13m oBjpi271KkFBigxuYJ4ppiceDXqHsh7/tbNvZvviOwFlMrrpYcJgeUHNOHEDgEgn gt+G+cMO50V1RQ+riGlZozN5hCllP/BCHFCvdQrVMvCgKmlWN3cTbLWML9nD98RD MFL6TLs50GFVHTgdRiq+AxwVFqJ96q+Em53Kfz4PYqN/n9izDwg/6tmPlqC1bX2d hkIMxrCCImI9Nq/8sx8gd5wa8qmA0vIAwlJ9mwWpp+cEmwaYjVQbrsAPjRoIWHmb 8ff6DLvtW9lfCG6y2DXnS4UHIHKFfcFXV07QYhC4fSAHxJpWOka4t8+t2C4rZOLY rCc/Nn09COsP4Z81fxc4q6CKfE4xUxor+lA5paQlnkrTD6w5LL8= =yhjL -----END PGP SIGNATURE----- --=-=-=--