From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= Subject: Re: More progress with the Guix Data Service Date: Tue, 21 May 2019 10:49:14 +0200 Message-ID: <87a7fg5h11.fsf@gnu.org> References: <87pnohms3t.fsf@cbaines.net> <871s0trw43.fsf@gnu.org> <87imu4narn.fsf@cbaines.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([209.51.188.92]:49803) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hT0SV-0006Uj-8U for guix-devel@gnu.org; Tue, 21 May 2019 04:49:23 -0400 In-Reply-To: <87imu4narn.fsf@cbaines.net> (Christopher Baines's message of "Mon, 20 May 2019 21:14:52 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Christopher Baines Cc: guix-devel@gnu.org Hello! Christopher Baines skribis: > Ludovic Court=C3=A8s writes: [...] >> I would expect only patch submission info, and possibly commit >> notifications, to be grabbed from email, while the rest would be >> extracted from the repo, thereby hopefully limiting the risk of >> misinterpreting email. WDYT? > > So, currently the branch name, commit hash and date are taken from the > email. As far as I know, git branches are just pointers to commits, and > don't have any date/time associated with them. The commit date, or > author date in the commits could be stored and used, but I think these > are less interesting, and often misleading. The author date is often > quite different from the time a commit is pushed, and the commit date is > often different by some amount as well. > > Currently, if you actually want to know what was the state of a > particular branch in the Guix git repository on Savannah was, at a > particular time, I think the most reliable way of checking would > probably be to check the guix-commits mailing list. Good point. Alternately, we could get that information first-hand if we were running the Git server ourselves (which I think we=E2=80=99ll have to do at some po= int to improve our workflow, for things like commit signature checks.) > As the branch name, and commit hash both relate to the date, I don't see > that much problem with storing them. > > One thing I've also been thinking about is loading in the guix-commits > mailing list archives. That would backfill the branch information, which > might be useful/interesting... > > I did consider trying to access the clone of the Git repository that's > managed by the (guix inferiors) module, but I couldn't see an easy way > to do it, and as above, I'm not sure the date/time information is as > useful as what you can get from the mailing list. OK, makes sense. >> One thing that be great is a page similar to >> , >> but keyed by package, where you get a list of the recent package >> versions (and/or derivations) and map them to specific commits. > > Interesting, yeah, were you thinking of filtering that data for a > specific branch (like master or staging), or showing data for all > branches? As a user, I=E2=80=99d like to answer questions like =E2=80=9Cwhen was pack= age X upgraded in Guix?=E2=80=9D, =E2=80=9Cwhich commit should I pull to obtain t= hat old version of the package?=E2=80=9D, things like that. >>> https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05= b2473ea13feb2842a1330c316a00/packages.json?field=3Dversion&field=3Dsynopsis= &field=3Ddescription&field=3Dhome-page&field=3Dlocation&field=3Dlicenses&li= mit_results=3D99999 >> >> Awesome. (I advise passing =E2=80=9Climit_results=3D900=E2=80=9D though= , because the URL >> above gives a pretty big result. ;-)) > > Well, not that big? Icecat tells me it's 12MB. Also, I've recently added > a "All results" checkbox/query parameter, so you no longer have to make > up a large number. I wanted to make it possible to get all the data as a > single file, as that could simplify processing it, but there's also some > support for pagination. > > https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2= 473ea13feb2842a1330c316a00/packages.json?field=3Dversion&field=3Dsynopsis&f= ield=3Ddescription&field=3Dhome-page&field=3Dlocation&field=3Dlicenses&all_= results=3Don > > The all results option is especially important as I've now done some > work on caching. That page should be served with a max-age of a day, it > could probably be even longer as well, as the only thing that will > change the contents is software changes. NGinx is also now caching > responses, and you can see what it's doing by looking at the > X-Cache-Status header in the response. Sounds good! >> I=E2=80=99d suggest having a Guix service for the whole thing, and makin= g a >> branch in guix-maintenance.git such that bayfront (say) can run the >> service. >> >> Then we=E2=80=99ll have to reach consensus on guix-sysadmin as to which = machine >> to use depending on the resources it needs, but if you have the config, >> I=E2=80=99d argue that we can happily run it on bayfront or perhaps berl= in. And >> we can give you access to the machine so you can reconfigure once in a >> while. >> >> WDYT? > > That all sounds really good :D > > A package and service has been on my list of things to do, and I'll > hopefully sort that out in the next few weeks. Awesome. > Currently I'm running it with Guix + support for isolated inferiors [1], > but I think that's something that can be made optional in the Guix Data > Service code, as initially I'd just be thinking about processing > revisions in the Guix git repository. OK. I need to take another look at the isolated inferior patches. Thanks, Ludo=E2=80=99.