From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christopher Baines Subject: February update on data.guix.gnu.org and the Guix Data Service Date: Mon, 17 Feb 2020 19:18:24 +0000 Message-ID: <87pnedyqdb.fsf@cbaines.net> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:53543) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j3lua-0000SC-6u for guix-devel@gnu.org; Mon, 17 Feb 2020 14:18:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j3luY-0004lf-Tt for guix-devel@gnu.org; Mon, 17 Feb 2020 14:18:32 -0500 Received: from mira.cbaines.net ([212.71.252.8]:57040) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1j3luY-0004kK-Ng for guix-devel@gnu.org; Mon, 17 Feb 2020 14:18:30 -0500 Received: from localhost (unknown [46.237.162.11]) by mira.cbaines.net (Postfix) with ESMTPSA id 06F9317B91 for ; Mon, 17 Feb 2020 19:18:29 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 741654c9 for ; Mon, 17 Feb 2020 19:18:26 +0000 (UTC) List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane-mx.org@gnu.org Sender: "Guix-devel" To: guix-devel@gnu.org --=-=-= Content-Type: text/plain Hey, Another update on the Guix Data Service, I sent out the last update on the 5th of January [1]. 1: https://lists.gnu.org/archive/html/guix-devel/2020-01/msg00073.html Derivations for system tests [3], as well as channel instances [4] (which relate to guix pull) are now captured. This is still a work in progress, I think only the x86_64-linux derivations for the system tests are captured, and the systems for the channel instances are limited by what's available in the qemu-binfmt service. 3: http://data.guix.gnu.org/revision/3dd311e3a059131ef245417106d4fb659222ef3c/system-tests 4: http://data.guix.gnu.org/revision/3dd311e3a059131ef245417106d4fb659222ef3c/channel-instances The way cross-built derivations are handled has changed. Previously the system values were used, but I've now tried to move in the direction of using GNU triplets. This can be seen on the revision pages in the derivations table [5]. 5: http://data.guix.gnu.org/revision/3dd311e3a059131ef245417106d4fb659222ef3c It's now possible to configure the Guix Data Service to only process certain branches by listing branches to explicitly include or exclude. Using this feature, I've configured data.guix.gnu.org to only look at master as this is what I want this instance of the Guix Data Service to focus on. Previously, the jobs for core-updates, staging and other branches could block processing revisions on the master branch, as often they would take longer to process due to having to build lots of packages just to build Guix. This issue would have only been amplified now that data.guix.gnu.org is emulating other architectures and building Guix for those as well. Related to this, I've also added some code to enable removing data for a branch, and removing unreferenced derivations. This is both for cleaning up the data.guix.gnu.org database now I only want data for master, but it should also be useful when using the Guix Data Service in an environment where branches come and go, or be the basis of setting a retention period for the data. I forget exactly when, but recently I've been trying to revive the patch review setup I was working on around a year ago [6]. I've setup an instance of the Guix Data Service for this [7] (separate to the data.guix.gnu.org one). I might try and have that instance of the Guix Data Service process all the branches in the Guix git repository, now that data.guix.gnu.org doesn't do that. 6: https://lists.gnu.org/archive/html/guix-devel/2019-02/msg00089.html 7: https://guix-patches-data.cbaines.net/ Back to features though, the output from inferior processes used when loading data for a revision is now captured and stored in the database. This means you can see more of what's going on, like the building of libgit2 here for example [8]. Previously you got less information [9]. 8: http://data.guix.gnu.org/job/14657 9: http://data.guix.gnu.org/job/14610 As always, let me know if you have any comments or questions! Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl5K5wBfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9XfZ2Q/+Mo8bFbLvl/0dIJPQQA5FZKrgk5k//YdM1NudRrfIlNne8KiEq1k0D9U2 HcYiSFot5AIDnOjZNRTIB0MrhkYgRxCbn+kqoxFw6gHYQqVaR0DBUUSCrbLBjTvp nQbJOn5eZnxc1mFHdBaFl/A9bm73MaqdhIj3dMSEM8Qi1mabS4EipG+5OkQ3EQb+ n+i1Y0oDweh8ru1/JOrevezLLJMacoel7KBDt7aLtUaLSKMaMWZEKUB6S4EZ5Zhf y+xa9twoxqrjuRmToNKiPdfvdz4Jw+hgRuEwL5mQMxNpv3Dq0galcmZacBm9dphc AHlaDPJsUaL+htx0cVAllkWgagsJzFn5ftRZ4VQoKPhdNnOyOOYSYCvquextWrc/ XHgKEuD93f+sfiC+yCwExptUlvStwMlozaWfdTjCFPLVFGWBJ0gvvAZ0yCbWoRsU ujtMbCUySWDtLejAo0LjIR8gH0vLFPtZjuZToptQ0krqFwbN76Ncmp1+rttOsi/F grhpafSQMIwLFGaMKaXL3cv+35UREn0O2Ia7cEbrmchw5ZAKGtHzKoMRQTlWAFxS AyRJ502zB9bGips3eL8+mm/HBVPX5SsSDRsc7n1c9c9W5pWVJhxzTd6l+l7IAKXI snov28dUfVy8Bujk4M4/GEDp5xWxTXvwlTEqPd3rHw6C7qXbi+8= =9CaL -----END PGP SIGNATURE----- --=-=-=--