From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christopher Baines Subject: Re: February update on data.guix.gnu.org and the Guix Data Service Date: Thu, 12 Mar 2020 19:39:41 +0000 Message-ID: <87zhclcqgy.fsf@cbaines.net> References: <87pnedyqdb.fsf@cbaines.net> <87imj9u2s0.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:44580) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jCTgQ-00071h-87 for guix-devel@gnu.org; Thu, 12 Mar 2020 15:39:55 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jCTgO-0000dd-CI for guix-devel@gnu.org; Thu, 12 Mar 2020 15:39:53 -0400 In-reply-to: <87imj9u2s0.fsf@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane-mx.org@gnu.org Sender: "Guix-devel" To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-devel@gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Hi! > > Christopher Baines skribis: > >> Derivations for system tests [3], as well as channel instances [4] >> (which relate to guix pull) are now captured. This is still a work in >> progress, I think only the x86_64-linux derivations for the system tests >> are captured, and the systems for the channel instances are limited by >> what's available in the qemu-binfmt service. >> >> 3: http://data.guix.gnu.org/revision/3dd311e3a059131ef245417106d4fb65922= 2ef3c/system-tests >> 4: http://data.guix.gnu.org/revision/3dd311e3a059131ef245417106d4fb65922= 2ef3c/channel-instances > > Neat! One small thing to add here is that I've noticed the system test derivations recorded in the Guix Data Service differ from what Cuirass is building. I think this is because they're tweaked a bit in system-test-job (guix ci): https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/ci.scm#n280 >> Related to this, I've also added some code to enable removing data for a >> branch, and removing unreferenced derivations. This is both for cleaning >> up the data.guix.gnu.org database now I only want data for master, but >> it should also be useful when using the Guix Data Service in an >> environment where branches come and go, or be the basis of setting a >> retention period for the data. > > Nice. Were you concerned specifically of unbounded growth on the > data.guix.gnu.org instance? Partially, although I was more trying to reduce the time between the master branch changing and the information being available in the guix-data-service. Even when the channel-instance derivation was being computed just to x86-64_linux, it could take up to ~24 hours for some core-updates changes. Now that the channel-instance derivations are computed for multiple architectures, I think having those jobs take a long time would have impacted how fast the jobs for the master branch were processed. > It would be interesting for data.guix.gnu.org to keep as much data as > possible, including perhaps for branches that have been deleted. Then > the problem becomes more of a UI problem: how to display, for instance, > only the =E2=80=9Cactive=E2=80=9D branches on the main page. > > WDYT? So, after I sent this email, I setup the guix-patches-data.cbaines.net instance of the Guix Data Service to process data for the main Guix git repository, but for all branches. The index page for the Guix Data Service doesn't list deleted branches, but they are shown on the page for each repository, for example: https://guix-patches-data.cbaines.net/repository/2/ >> I forget exactly when, but recently I've been trying to revive the patch >> review setup I was working on around a year ago [6]. I've setup an >> instance of the Guix Data Service for this [7] (separate to the >> data.guix.gnu.org one). I might try and have that instance of the Guix >> Data Service process all the branches in the Guix git repository, now >> that data.guix.gnu.org doesn't do that. >> >> 6: https://lists.gnu.org/archive/html/guix-devel/2019-02/msg00089.html >> 7: https://guix-patches-data.cbaines.net/ > > Yay! > > At the Guix Days, you also shown very concretely how people could take > advantage of data.guix.gnu.org in their patch review workflow. Perhaps > it=E2=80=99d be useful to share that info here or in the manual, even if = it=E2=80=99s > still evolving. Indeed, hopefully I can get around to looking at it again soon :) >> Back to features though, the output from inferior processes used when >> loading data for a revision is now captured and stored in the >> database. This means you can see more of what's going on, like the >> building of libgit2 here for example [8]. Previously you got less >> information [9]. >> >> 8: http://data.guix.gnu.org/job/14657 >> 9: http://data.guix.gnu.org/job/14610 > > Great. > > Could you make a support request on Savannah to enable commit > notifications on the repo? That=E2=80=99d be awesome. :-) Sure, I've created a ticket now: https://savannah.nongnu.org/support/index.php?110208 > Thanks for the great update, as always! You're welcome :) --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl5qj/1fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9Xd63hAAlGIVCx1BGY43SABnCoKDE3kLUmHoByXx7yubWS3EPDcITo7qVrpbqnjX SYrZf0SqKQoMKmsM3iHUe0atYzcvY156r/kv8n6CbA0z2NyRJvfS6ocy8Hj9WsIt tsDQZoD7pQjOMGS5zVcs2tA3Um0lPcVFCNW73k7PNYccaumrXgUkDdA5+BGVwvAx B3r4aMU8vSoCvcKRGJ518Uzc7+OPyx1RpVo6k/QJiNGJLzPl3Dq42wTphcVGNSMd p3GbCirayUMhDIpeuVPQYPqiW09SYBj/YahA8Cz8560xSefMkwZ7dNnPsY1B2cL3 SB87d2+RQwwTIE17gndBrqpt2DApippIEnj3Fl5nBmM/XP9aaveRjpkYQOzCKcxk 3cgNMtEXc8s+oVGUg3f0pU15Ts8tlYpb+r2+ypyLIixSQCJ7Ot4xPgOzIGct4Jhu ovMNs+tXAcFqiBjiXQBC8pE6jNypN6jAT/BAdXfoq5Ord4iBr6z8CPYcJXmWCjm4 ra53Hlps8c5QJtKI5IAtvnZm66s8mtqQ88V6yHGRzqiVuLG2wBAjXMGph9EVz146 UdFJL8qk2VVsd1u/oAoMUwG6QjplxALl8NPymfqcHyBoTD5cjePLrJzAqdRAblb8 N+0eErpsipvjZu3Nz8Wpvy893v+79yoWyfaTi+NYBhQCi4fB02k= =WA3o -----END PGP SIGNATURE----- --=-=-=--