From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: [PATCH] draft addition of github updater Date: Tue, 23 Feb 2016 14:22:21 +0100 Message-ID: <87vb5fk1de.fsf@gnu.org> References: <5647D2A8.8040603@uq.edu.au> <87h9kmb8zs.fsf@gnu.org> <5675F96E.4090609@uq.edu.au> <87ziwmqtle.fsf@gnu.org> <56C92B77.3050601@uq.edu.au> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:43750) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aYCvC-0005Iy-O2 for guix-devel@gnu.org; Tue, 23 Feb 2016 08:22:37 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aYCv8-0001qR-D4 for guix-devel@gnu.org; Tue, 23 Feb 2016 08:22:34 -0500 In-Reply-To: <56C92B77.3050601@uq.edu.au> (Ben Woodcroft's message of "Sun, 21 Feb 2016 13:13:59 +1000") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Ben Woodcroft Cc: "guix-devel@gnu.org" Ben Woodcroft skribis: > Unfortunately I found a further bug - the updated URL for the new > package was actually the old URL not the updated one, and fixing this > required some refactoring. OK. > I'm afraid I'm almost out of time for this until the end of March, so > if there are any further substantive changes we might have to let this > slip the upcoming release, unless someone else can continue this > work. Soz.. No problem. It=E2=80=99s OK to leave improvements for later. We can alway= s add this version now as long as it=E2=80=99s functional and doesn=E2=80=99t bre= ak anything. > One way in which this could be improved in the future would be to > accept odd source GitHub URLs and return the newest version, but error > out when the URL needs to be guessed. That way, at least all > GitHub-sourced packages can be checked for updates even if they cannot > all be updated in place. I don't think this would be especially hard > to implement and would be quite reliable. OK. > On 04/01/16 06:46, Ludovic Court=C3=A8s wrote: >> Ben Woodcroft skribis: >> >>> It seems I miscounted before, but now it is 129 of 146 github >>> "release" packages recognised with 28 suggesting an update - see the >>> end of email for details. There is one false positive: >>> >>> gnu/packages/ocaml.scm:202:13: camlp4 would be upgraded from 4.02+6 to >>> 4.02.0+1 >>> >>> This happens because the newer versions were not made as official >>> releases just tags, so the newer versions are omitted from the API >>> response, plus there's the odd version numbering scheme. Guix is up to >>> date. >> I guess we could filter out such downgrades by adding a call to >> =E2=80=98version>?=E2=80=99, no? > > My impression is that code elsewhere (yours?) already does this, but > version>? does not work as intended for this corner case. Indeed: --8<---------------cut here---------------start------------->8--- scheme@(guile-user)> (version>? "4.02+6" "4.02.0+1") $2 =3D #f --8<---------------cut here---------------end--------------->8--- I would argue that upstream chose a confusing numbering scheme is 4.02.0+1 is supposed to be older=E2=80=A6 >> Rather use (guix http-client) and something like: >> >> (let ((port (http-fetch url))) >> (dynamic-wind >> (const #t) >> (lambda () >> (json->scm port)) >> (lambda () >> (close-port port)))) >> >> This avoids the temporary file creation etc. > > This sounds preferable but did not work as I kept getting 403 > forbidden. Displaying the URI for instance with (string-append uri > "\n") gives the below. I understand the error is trivial, but just > wanted to communicate the URI object. > > ERROR: In procedure string-append: > ERROR: In procedure string-append: Wrong type (expecting string): > #< scheme: https userinfo: #f host: "api.github.com" port: #f > path: "/repos/torognes/vsearch/releases" query: > "access_token=3D27907952ef87f3691d592b9dcd93cd4b6f20625f" fragment: #f> That=E2=80=99s because this is a URI object, not a string. >>> +;; TODO: is there some code from elsewhere in guix that can be used in= stead of >>> +;; redefining? >>> +(define (find-extension url) >>> + "Return the extension of the archive e.g. '.tar.gz' given a URL, or >>> +false if none is recognized" >>> + (find (lambda x (string-suffix? (first x) url)) >>> + (list ".tar.gz" ".tar.bz2" ".tar.xz" ".zip" ".tar"))) >> Remove this procedure and use (file-extension url) instead, from (guix u= tils). > > I figured there was something out there. The problem is file-extension > returns, for example, "gz" when we are after "tar.gz". Oh, I see. > From 29dc5a809e6d8796279911a993ef1b2237c810ca Mon Sep 17 00:00:00 2001 > From: Ben Woodcroft > Date: Sun, 15 Nov 2015 10:18:05 +1000 > Subject: [PATCH] import: Add github-updater. > > * guix/import/github.scm: New file. > * guix/scripts/refresh.scm (%updaters): Add %GITHUB-UPDATER. > * doc/guix.texi (Invoking guix refresh): Mention it. Make sure to add github.scm in Makefile.am. Otherwise LGTM! Once this is in, I=E2=80=99ll see if I can make that =E2=80=98http-fetch=E2= =80=99 change I was suggesting. Thank you! Ludo=E2=80=99.