Re: [PATCH] draft addition of github updater

[ludo@gnu.org (Ludovic Courtès)]

Ben Woodcroft <b.woodcroft@uq.edu.au> skribis:

> Unfortunately I found a further bug - the updated URL for the new
> package was actually the old URL not the updated one, and fixing this
> required some refactoring.

OK.

> I'm afraid I'm almost out of time for this until the end of March, so
> if there are any further substantive changes we might have to let this
> slip the upcoming release, unless someone else can continue this
> work. Soz..

No problem.  It’s OK to leave improvements for later.  We can always add
this version now as long as it’s functional and doesn’t break anything.

> One way in which this could be improved in the future would be to
> accept odd source GitHub URLs and return the newest version, but error
> out when the URL needs to be guessed. That way, at least all
> GitHub-sourced packages can be checked for updates even if they cannot
> all be updated in place. I don't think this would be especially hard
> to implement and would be quite reliable.

OK.

> On 04/01/16 06:46, Ludovic Courtès wrote:
>> Ben Woodcroft<b.woodcroft@uq.edu.au>  skribis:
>>
>>> It seems I miscounted before, but now it is 129 of 146 github
>>> "release" packages recognised with 28 suggesting an update - see the
>>> end of email for details. There is one false positive:
>>>
>>> gnu/packages/ocaml.scm:202:13: camlp4 would be upgraded from 4.02+6 to
>>> 4.02.0+1
>>>
>>> This happens because the newer versions were not made as official
>>> releases just tags, so the newer versions are omitted from the API
>>> response, plus there's the odd version numbering scheme. Guix is up to
>>> date.
>> I guess we could filter out such downgrades by adding a call to
>> ‘version>?’, no?
>
> My impression is that code elsewhere (yours?) already does this, but
> version>? does not work as intended for this corner case.

Indeed:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (version>? "4.02+6" "4.02.0+1")
$2 = #f
--8<---------------cut here---------------end--------------->8---

I would argue that upstream chose a confusing numbering scheme is
4.02.0+1 is supposed to be older…

>> Rather use (guix http-client) and something like:
>>
>>    (let ((port (http-fetch url)))
>>      (dynamic-wind
>>        (const #t)
>>        (lambda ()
>>          (json->scm port))
>>        (lambda ()
>>          (close-port port))))
>>
>> This avoids the temporary file creation etc.
>
> This sounds preferable but did not work as I kept getting 403
> forbidden. Displaying the URI for instance with (string-append uri
> "\n") gives the below. I understand the error is trivial, but just
> wanted to communicate the URI object.
>
> ERROR: In procedure string-append:
> ERROR: In procedure string-append: Wrong type (expecting string):
> #<<uri> scheme: https userinfo: #f host: "api.github.com" port: #f
> path: "/repos/torognes/vsearch/releases" query:
> "access_token=27907952ef87f3691d592b9dcd93cd4b6f20625f" fragment: #f>

That’s because this is a URI object, not a string.

>>> +;; TODO: is there some code from elsewhere in guix that can be used instead of
>>> +;; redefining?
>>> +(define (find-extension url)
>>> +  "Return the extension of the archive e.g. '.tar.gz' given a URL, or
>>> +false if none is recognized"
>>> +  (find (lambda x (string-suffix? (first x) url))
>>> +        (list ".tar.gz" ".tar.bz2" ".tar.xz" ".zip" ".tar")))
>> Remove this procedure and use (file-extension url) instead, from (guix utils).
>
> I figured there was something out there. The problem is file-extension
> returns, for example, "gz" when we are after "tar.gz".

Oh, I see.

> From 29dc5a809e6d8796279911a993ef1b2237c810ca Mon Sep 17 00:00:00 2001
> From: Ben Woodcroft <donttrustben@gmail.com>
> Date: Sun, 15 Nov 2015 10:18:05 +1000
> Subject: [PATCH] import: Add github-updater.
>
> * guix/import/github.scm: New file.
> * guix/scripts/refresh.scm (%updaters): Add %GITHUB-UPDATER.
> * doc/guix.texi (Invoking guix refresh): Mention it.

Make sure to add github.scm in Makefile.am.  Otherwise LGTM!

Once this is in, I’ll see if I can make that ‘http-fetch’ change I was
suggesting.

Thank you!

Ludo’.