From mboxrd@z Thu Jan 1 00:00:00 1970 From: Amirouche Subject: Re: More progress with the Guix Data Service Date: Fri, 17 May 2019 12:07:37 +0200 Message-ID: <9b0ce280d338340ea0621f46cb7d0377@hyper.dev> References: <87pnohms3t.fsf@cbaines.net> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Return-path: Received: from eggs.gnu.org ([209.51.188.92]:44218) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hRZmL-0005kz-0D for guix-devel@gnu.org; Fri, 17 May 2019 06:07:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hRZmF-0002gV-AA for guix-devel@gnu.org; Fri, 17 May 2019 06:07:48 -0400 In-Reply-To: <87pnohms3t.fsf@cbaines.net> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Christopher Baines Cc: guix-devel@gnu.org, Guix-devel On 2019-05-17 09:56, Christopher Baines wrote: > Hey, > > In summary, I think the Guix Data Service might be getting useful > enough > that setting it up properly might be a good next step, and I'd be > interested in what others think? Yeah. And it looks good :) > A bit over a month ago, I sent out an update about one of the things > I've been working on, something I've been calling the "Guix Data > Service" [1]. > > 1: https://lists.gnu.org/archive/html/guix-devel/2019-04/msg00094.html > > Since then, I've made some more progress. There's a new statistics page > [2]. I've got used to using Sqitch [3] to help manage the database > schema, and I've added some basic tests. statistic page is a 404. I am very interested to learn how you use sqitch. > > 2: https://prototype-guix-data-service.cbaines.net/statistic > 3: http://guix.gnu.org/packages/sqitch-0.9999/ > > The error handling for loading new revisions is also more resilient > now. > > As well as listening to the Guix Commits mailing list for emails about > new revisions, more of the information in these emails is now stored, > in > particular, the time they were sent, and the branch the email applies > to. This can be seen on the new Branches page [4]. > > 4: https://prototype-guix-data-service.cbaines.net/branches Here when I click on a commit, I would expect the message of the commit and the build status of the impacted softwares, e.g. https://prototype-guix-data-service.cbaines.net/revision/d59c90a5bfdd6b723bea939b8538c7c9b3c1b2a6 > > The content negotiation a little bit, at least in terms of the code, > and > JSON output support has been added to more pages. > > There's now a basic search function on the packages page [5], and the > location, and the licenses for packages is now being stored (which can > be seen on the page for a package, for example [6]). > > 5: > https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages?search_query=Guile&field=version&field=synopsis&after_name=&limit_results=1000 > 6: > https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/package/0ad/0.0.23b-alpha > > The location and license information is something I added specifically, > as I noticed the Repology [7] service scraped [8] these from the Guix > website. > > 7: https://repology.org/ > 8: > https://github.com/repology/repology/blob/master/repology/fetchers/fetchers/guix.py > > While the Guix Data Service started as something to enable better > understanding patches in an automated way, I think there are more uses > for it, and initially, it's probably better to focus on the simple > ones. It is not clear right now that it is related to patches, there is no patch anywhere to see. > The Repology use case is pretty simple, I think ideally there would be > some machine readable data about the current state of packages in Guix > available over the internet, and Repology would be able to download > that > on a regular basis. Repology is nice, but I would prefer wikidata support. > > The URL is a bit long, but I think that is now close to being possible > with the Guix data service. I haven't got something working yet to > easily access data for the latest revision, but for a particular > revision, you can request a JSON file containing all the information I > think Repology currently gets about all packages. For example: > > > https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages.json?field=version&field=synopsis&field=description&field=home-page&field=location&field=licenses&limit_results=99999 FWIW it is very slow. > Does anyone have any thoughts on this? It seems to me it would be useful to have that informations somewhere. Like you explain having the correct build information is a must have IMO along the logs. Where is the code? Thanks for sharing!